Run flake8 without creating physical files

[nbirillo]

Run flake8 without creating physical files

Hello! Can I use flake8 in a Python project via API? The documentation says: Flake8 3.0.0 presently does not have a public, stable Python API. I would like to run flake8 on a dataset to get code quality violations for each code snippet, but I don't want to create physical files with code since I have millions of code snippets. I currently found that flake8 uses FileChecker but didn't find other checkers, e.g. InMemoryChecker. Is it true that my use case is not supported?

And if it is true, do I understand correctly that if I implement my own сhecker, this will be enough to solve my problem?

The list of the plugins which I would like to use are:

flake8==3.9.0

# flake8 plugins
flake8-bugbear==21.4.3
flake8-builtins==1.5.3
flake8-comprehensions==3.4.0
flake8-eradicate==1.0.0
flake8-import-order==0.18.1
flake8-plugin-utils==1.3.1
flake8-polyfill==1.0.2
flake8-return==1.1.2
flake8-spellcheck==0.24.0
mccabe==0.6.1
pep8-naming==0.11.1
wps-light==0.15.2
flake8-broken-line==0.3.0
flake8-string-format==0.3.0
flake8-commas==2.0.0
cohesion==1.0.0
radon==4.5.0

[asottile]

FileChecker isn't a supported api either -- the api lives in flake8.api.legacy though it's not great

your best bet is probably to call flake8 - with a subprocess and pass your file contents as stdin. or to use a NamedTemporaryFile and pass it that way

duplicate of #140

[nbirillo]

@asottile How can I pass my file content as stdin?
I found the following function:

def is_using_stdin(paths: list[str]) -> bool:
    """Determine if we're going to read from stdin.

    :param paths:
        The paths that we're going to check.
    :returns:
        True if stdin (-) is in the path, otherwise False
    """
    return "-" in paths

That means that I need to pass just "-" as the path argument, isn't. But If I do it, I got the error: FileNotFoundError: [Errno 2] No such file or directory: 'stdin'

My Python code:

p = run(['flake8', '-'], stdout=PIPE, input=code, encoding='ascii')

[asottile]

I can't see your screen. show your full output -- don't hide anything, don't edit it -- otherwise I can only guess at your problem

here's my framework for asking questions if you need some explicit pointers about how to ask a useful, helpful question

[nbirillo]

I just asked how can I use flkae8 with stdin, because if you noted this way, you probably know how I can do it. It would be great if you share this knowledge

My code:

from subprocess import run, PIPE

code = u"""\
def is_using_stdin(paths: list[str]) -> bool:
    \"\"\"Determine if we're going to read from stdin.

    :param paths:
        The paths that we're going to check.
    :returns:
        True if stdin (-) is in the path, otherwise False
    \"\"\"
    return "-" in paths
"""

p = run(['flake8', '-'], stdout=PIPE, input=code, encoding='ascii')
print(p.returncode)
print(p.stdout)

My output:

Traceback (most recent call last):
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/bin/flake8", line 8, in <module>
    sys.exit(main())
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/cli.py", line 22, in main
    app.run(argv)
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/application.py", line 363, in run
    self._run(argv)
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/application.py", line 351, in _run
    self.run_checks()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/main/application.py", line 264, in run_checks
    self.file_checker_manager.run()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 323, in run
    self.run_serial()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 307, in run_serial
    checker.run_checks()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 589, in run_checks
    self.run_ast_checks()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8/checker.py", line 494, in run_ast_checks
    for (line_number, offset, text, _) in runner:
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8_plugin_utils/plugin.py", line 75, in run
    self._load_file()
  File "/Users/Anastasiia.Birillo/PycharmProjects/tmp/main/venv/lib/python3.8/site-packages/flake8_plugin_utils/plugin.py", line 87, in _load_file
    with open(self._filename, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'stdin'
1

[asottile]

right so whatever flake8_plugin_utils is does not read files correctly -- it should use the lines parameter to get the contents -- this isn't a flake8 issue and leading with that to begin with would have helped me identify that sooner rather than an additional round of back-and-forth

[nbirillo]

Sorry for the stupid question, but what do you mean by using the lines parameter? How can I transfer it?
And sorry for wasting time on my problem

[asottile]

looks like thy already fixed it -- afonasev/flake8-plugin-utils@1e07ff4

you can pip install 'flake8-plugin-utils>=1.3.2' probably to make that error go away

[nbirillo]

It works, thank you a lot for the help!