Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for more complicated documentation examples #8287

Merged
merged 6 commits into from
Feb 20, 2023

Conversation

DanielNoord
Copy link
Collaborator

@DanielNoord DanielNoord commented Feb 13, 2023

Type of Changes

Type
βœ“ πŸ“œ Docs

Description

Refs #7897
Closes #7143

@DanielNoord DanielNoord added Documentation πŸ“— Skip news πŸ”‡ This change does not require a changelog entry labels Feb 13, 2023
Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good, thank you for tackling this !

@@ -0,0 +1,9 @@
from .bad2 import count_to_two # [cyclic-import]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we take into account the fact that if we want to do multiple examples properly displayed we're going to want to have bad.py, bad2.py, bad3.py (...) too ? Maybe the display can change if there's no good2.py, (...) matching the bad2.py (...) ? Edit: Did not notice it's in a directory, I think it's compatible.

if (message_dir / "bad.py").exists():
suite.append(
(message_dir.stem, message_dir / "bad.py"),
)
elif (message_dir / "bad").is_dir():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haa, this make sense, we can have directories too for multiple example. (bad / bad2/...)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in that case it makes more sense to just use one file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also do bad.py, neutral.py, lawful.py, chaotic.py and good.py

Comment on lines 558 to 569
if imported_module is None:

# If we can't find the import we still try and build the import graph with our
# best guess: the relative name of the import.
imported_module_name = imported_module.name if imported_module else node.modname
if imported_module_name is None:
return

for name, _ in node.names:
if name != "*":
self._add_imported_module(node, f"{imported_module.name}.{name}")
self._add_imported_module(node, f"{imported_module_name}.{name}")
else:
self._add_imported_module(node, imported_module.name)
self._add_imported_module(node, imported_module_name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like something that could be it's own PR.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah and it is breaking tests... Without it this test is very flaky so I'm not sure what to do

@DanielNoord
Copy link
Collaborator Author

DanielNoord commented Feb 13, 2023

@Pierre-Sassoulas I kept in the cyclic-import example to show this works. I'll remove it after your review as the behaviour is flaky and it seems to be because of astroid.

I think this PR also closes #7143 right?

@github-actions
Copy link
Contributor

πŸ€– Effect of this PR on checked open source code: πŸ€–

Effect on pandas:
The following messages are now emitted:

  1. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/__init__.py#L13
  2. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L39
  3. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L40
  4. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L40
  5. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L40
  6. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L40
  7. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L40
  8. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L40
  9. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L48
  10. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L48
  11. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L48
  12. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L48
  13. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L54
  14. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L54
  15. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L54
  16. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L54
  17. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L54
  18. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L61
  19. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L61
  20. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L61
  21. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L66
  22. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L66
  23. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L70
  24. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L70
  25. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L70
  26. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L75
  27. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L76
  28. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L77
  29. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L78
  30. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L78
  31. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L78
  32. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L78
  33. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L78
  34. import-self:
    Module import itself
    https://github.com/pandas-dev/pandas/blob/9764f3dbf0173892aba5da4b122851ded43c09bc/pandas/_libs/tslibs/__init__.py#L78

This comment was generated for commit f46f18b

@codecov
Copy link

codecov bot commented Feb 13, 2023

Codecov Report

Merging #8287 (bbfd103) into main (e64f043) will decrease coverage by 0.01%.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #8287      +/-   ##
==========================================
- Coverage   95.46%   95.46%   -0.01%     
==========================================
  Files         177      177              
  Lines       18704    18702       -2     
==========================================
- Hits        17856    17854       -2     
  Misses        848      848              
Impacted Files Coverage Ξ”
pylint/testutils/reporter_for_tests.py 93.02% <ΓΈ> (-0.32%) ⬇️

@Pierre-Sassoulas
Copy link
Member

I think this PR also closes #7143 right?

Yes, amazing !

)
elif (data_path / "bad").exists():
bad_one = f"""\
File one:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of File one / File two, we should have the actual name of the file (bad.py::. This would also simplify the handling code that can become generic and handle as many file as we want. We can also name it however we want (it's the directory prefix that indicate badness / goodness).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this suggestion! Will (hopefully) work on this tomorrow!

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if you agree with the good/ multiple fle and I'll do it :)

Comment on lines 79 to 98
if bad_py_path.exists():
bad_code = _get_titled_rst(
title="Problematic code", text=_get_python_code_as_rst(bad_py_path)
)
elif (data_path / "bad").exists():
bad_files: list[str] = []
for file in (data_path / "bad").iterdir():
if file.suffix == ".py":
bad_files.append(
f"""\
``{file.name}``:

.. literalinclude:: /{file.relative_to(Path.cwd())}
:language: python

"""
)
bad_code = _get_titled_rst(title="Problematic code", text="".join(bad_files))
else:
bad_code = ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should create a generic function for this as we also need to handle good/ directories with multiple file imo. I can do it, it's a lot easier with pycharm than vscode.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I didn't want to add code that we don't actively use right now, but go ahead and add it! Whatever makes it easier to complete this process the quickest!

)
elif (data_path / "bad").exists():
bad_files: list[str] = []
for file in (data_path / "bad").iterdir():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to sort here, so bad.py is before bad2.py

Suggested change
for file in (data_path / "bad").iterdir():
for file in sorted((data_path / "bad").iterdir()):

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turn out it's not that easy to sort Path πŸ˜„ took me embarrassingly long to understand

@DanielNoord
Copy link
Collaborator Author

I have commits almost ready that clean up this PR. Hopefully can finish this tomorrow.

@DanielNoord
Copy link
Collaborator Author

Not clearing that reporter after every file shows some underlying issues that we never seen. Nice to fix these as well I think πŸ˜„

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but it's impossible to check in read the doc now that the cyclic-import is removed.

Comment on lines -75 to -77
def on_set_current_module(self, module: str, filepath: str | None) -> None:
self.messages = []

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice !

@DanielNoord
Copy link
Collaborator Author

LGTM but it's impossible to check in read the doc now that the cyclic-import is removed.

Yeah, like I said; the cyclic-import message is 100% flakey πŸ˜…

If you know of another message where you want multiple files and that isn't flakey I can add it as well. Otherwise I think we should just merge this. I did test this locally with multiple files and checked that the tests still worked (which is how I found the issues with the reporter).

@Pierre-Sassoulas
Copy link
Member

Maybe too many lines ? It's a little insulting to readers though. Otherwise, duplicated code ? I can add this and test locally myself :)

@DanielNoord
Copy link
Collaborator Author

Maybe too many lines ? It's a little insulting to readers though. Otherwise, duplicated code ? I can add this and test locally myself :)

Duplicated code would be a good one! Perhaps make a PR that has this branch as target branch?

Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll let you check the example, and then let's merge. The result is pretty good, excited to make this live ! πŸ˜„

@DanielNoord
Copy link
Collaborator Author

Both https://pylint--8287.org.readthedocs.build/en/8287/user_guide/messages/warning/while-used.html and https://pylint--8287.org.readthedocs.build/en/8287/user_guide/messages/refactor/duplicate-code.html are definite improvements!

Indeed happy we have this now!

@stdedos
Copy link
Contributor

stdedos commented Feb 21, 2023

Both pylint--8287.org.readthedocs.build/en/8287/user_guide/messages/warning/while-used.html and pylint--8287.org.readthedocs.build/en/8287/user_guide/messages/refactor/duplicate-code.html are definite improvements!

Indeed happy we have this now!

btw - is the Similar lines in %s files %s bug or feature?
Since it's "technically code", should the messages have the backtick formatting?

@DanielNoord
Copy link
Collaborator Author

I think it might make sense indeed!

stdedos added a commit to stdedos/pylint that referenced this pull request Feb 22, 2023
In pylint-dev#7162, "we decreed" that
the `bad_code` goes first, then the `good_code`.

Fixup current changes, to revert "back to standards".

Additionally, tackle the comment here
pylint-dev#8287 (comment)

"Message emitted" is technically still code (`%-formatting`);
let us make it stand out properly.

Signed-off-by: Stavros Ntentos <133706+stdedos@users.noreply.github.com>
Pierre-Sassoulas pushed a commit that referenced this pull request Feb 22, 2023
In #7162, "we decreed" that
the `bad_code` goes first, then the `good_code`.

Fixup current changes, to revert "back to standards".

Additionally, tackle the comment here
#8287 (comment)

"Message emitted" is technically still code (`%-formatting`);
let us make it stand out properly.

Signed-off-by: Stavros Ntentos <133706+stdedos@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation πŸ“— Skip news πŸ”‡ This change does not require a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a configuration file in documentation example for message
4 participants