pylint vs epylint for editor usage

[pedroqueiroga]

Flycheck currently uses pylint, but there are some caveats and epylint seems to address them. In a recent discussion (flycheck/flycheck#1758), @cpitclaudel raised some questions about the usage of epylint, why isn't it integrated into pylint, for example. The questions are pasted below.

Questions

• What is the "right way" to run pylint in the following example?
• Is the algorithm implemented by epylint (which is essentially going up until there are no more init.py files) robust? If so, why isn't it used in pylint itself by default?
• Is epylint supported, and will it remain so in the future? Does it support all the same options as pylint, including support for reading files from stdin? (see Add support for reading from stdin #2746 and pylint: Add support for standard-input t (requires pylint 2.5) flycheck/flycheck#1546 — I worry about this because file names are handled differently there)
• What should Flycheck do in general: call epylint directly? Or call pylint directly from a working directory calculated as done by epylint? Or something else? We've been unsure for a while, esp. regarding which directory to run from, so it would be great to get a definitive answer. Additionally, I'm assuming that other editors run into the same issue, but the tool being called epylint suggests that there's something special about Emacs ^^

Example:

├── entry.py
└── pkg
    ├── bar.py
    ├── foo.py
    └── __init__.py

Another question

pylint.epylint, when used with runpy.run_module, returns a globals dictionary from which we might use, for example, the Run function: globals_dictionary['Run'](). This happens because epylint isn't a package. Could it be made into a package, with its own main.py?
Current version requires:

import runpy

epylint = runpy.run_module('pylint.epylint')
epylint['Run']()

If it were a package, with a main.py:

import runpy

runpy.run_module('pylint.epylint')

[twmr]

One additional comment: epylint currently doesn't support linting the code from stdin, but pylint does.

[Pierre-Sassoulas]

Is epylint supported, and will it remain so in the future?

We don't have emacs user in the contributors, we don't even know if epylint is used a lot. It seems no one read the doc on emacs at least and very few issues are opened. I can't answer any your questions for example because I never touched this part of the code or used it. So we need someone willing to maintain epylint so it's maintained. Alternatively it feel like we could simply remove it in 3.0 and not a lot would happen (I added the need decision label).

[nickdrozd]

See b2462c9

[cpitclaudel]

It's good to clarify the status of epylint, but most of the questions in the original OP were not about it, but about the right way to invoke pylint itself; maybe these can still be answered? (Though we've made some educated guesses over the last 2.5 years in Flycheck, so it's a bit less relevant now)

[Pierre-Sassoulas]

We do have an emacs user in the contributor now (@nickdrozd) it does not mean we can answer the questions without retro-engineering though. @cpitclaudel your educated guess are probably the most reliable source of knowledge we have. In particular if epylint is useful for you please tell us so we revert the decision to remove it from pylint :)

[cpitclaudel]

Thanks @Pierre-Sassoulas ! The questions I wanted help with were these, which I don't think involve epylint:

• What is the "right way" to run pylint in the following example? (See OP for the actual example)
• What should Flycheck do in general: call epylint directly? Or call pylint directly from a working directory calculated as done by epylint? Or something else? We've been unsure for a while, esp. regarding which directory to run from, so it would be great to get a definitive answer. Additionally, I'm assuming that other editors run into the same issue, but the tool being called epylint suggests that there's something special about Emacs ^^

The long story short about this question is that we have somewhat elaborate logic in Flycheck to find the working directory to use to call pylint (Flycheck is an emacs frontend for checkers that has support for flycheck among others). We don't use epylint, but we noticed that it has its own logic to do search for the right working directory to call pylint from, and we use roughly the same algorithm (reimplemented).

[Pierre-Sassoulas]

We implemented a recursive option in pylint to find the file to lint in a directory so I would use that alongside with an appropriate reporter, probably a custom json reporter (or the official one). It seems epylint was done specifically for IDE integration but the knowledge got lost and as maintainers don't know much about epylint the most well tested and maintained entry-point is definitely pylint now.

The idea is definitely that pylint will be launched at the root of the repository but if it's not the root of the repository you'll have to ask the user what it is. We're also going to search for configuration files to make it search up the tree until it hits a configuration file or a .git / .hg directory (#7143) so that might also be the way to get the root directory programmatically ? We did not plan to expose that as an API but we could if it's useful to someone.