Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in windows, xargs.py too large command line #1604

Closed
ufwtlsb opened this issue Sep 20, 2020 · 13 comments
Closed

in windows, xargs.py too large command line #1604

ufwtlsb opened this issue Sep 20, 2020 · 13 comments
Labels

Comments

@ufwtlsb
Copy link

ufwtlsb commented Sep 20, 2020

I use pre-commit to check java in windows
pre-commit run -a

get
The command line is too long.

I find xargs.py _get_platform_max_length too large size

I change it ,then can run

def _get_platform_max_length() -> int:  # pragma: no cover (platform specific)
    if os.name == 'posix':
        maximum = os.sysconf('SC_ARG_MAX') - 2048 - _environ_size()
        maximum = max(min(maximum, 2 ** 17), 2 ** 12)
        return maximum
    elif os.name == 'nt':
        return 2 ** 13 - 2048  # UNICODE_STRING max - headroom
    else:
        # posix minimum
        return 2 ** 12
@asottile
Copy link
Member

can you share your config and output and log? that value has been pretty thoroughly tested -- my guess is some underlying tool is expanding furthur in an incorrect way but I can't tell without you showing me anything

@ufwtlsb
Copy link
Author

ufwtlsb commented Sep 21, 2020

@asottile
Copy link
Member

asottile commented Sep 21, 2020

no no, show me your configuration, the command you ran, the output you get from that command, and the error log pre-commit generates

@ufwtlsb
Copy link
Author

ufwtlsb commented Sep 26, 2020

myconfig:

  - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks.git
    rev: master
    hooks:
      - id: pretty-format-java

run:
pre-commit run -a

have many java, so error message

命令行太长
mean: too command line long

@asottile
Copy link
Member

can you show the full outout please. show the whole thing I don't particularly care if it is not english

@apietrzak
Copy link

Hello @asottile, I have pretty the same issue:

Check for merge conflicts................................................Passed
prettier.................................................................Failed
- hook id: prettier                                                            
- duration: 7.17s                                                              
- exit code: 2                                                                 
- files were modified by this hook                                             
                                                                               
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                                  
The command line is too long.                                  

It was executed on Windows 10, python 3.9 and pre-commit 2.8.2

@asottile
Copy link
Member

asottile commented Nov 6, 2020

do you have a repository to link to?

that's prettier giving that result and not python, python's error looks like: [WinError 206] The filename or extension is too long (so it's probably prettier's fault)

@apietrzak
Copy link

apietrzak commented Nov 6, 2020

@asottile I can send only a config of pre-commit:

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.5.0
    hooks:
    -   id: check-merge-conflict
- repo: https://github.com/prettier/pre-commit
  rev: v2.1.2
  hooks:
    - id: prettier
      files: ^src_frontend/

Based on my invastigation there is an issue with executing a command to run the prettier hook.
He's trying to execute: .cache\pre-commit\repodion64\node_env-default\Scripts\prettier.cmd I'm not sure but there is some kind of a .CMD file command length limit.

It's easy to reproduce:

  • Make simple file a.CMD with content: ECHO "WORKS"
  • run python
  • subprocess.Popen([a_file_abs_path] + ['x'] * 4000) It works (in single line it's ~8k chars)
  • subprocess.Popen([a_file_abs_path] + ['x'] * 5000) It wasn't

image

pre-commit command executes a .CMD file with command which contains ~30k chars (of course it is greater than the limit).

@asottile
Copy link
Member

asottile commented Nov 6, 2020

hmmm so you're saying any .cmd script has an even shorter limit than the actual limit?

@apietrzak
Copy link

@asottile Yeah, from my tests it looks like that. I've tested it on two different machines with Win10, and i've got the same error when the command is larger than ~8k chars

@asottile
Copy link
Member

asottile commented Nov 6, 2020

hmmm ok, I've made #1686 to fix this, I've also figured out exactly the limit for cmd, it has to do with the full path to cmd.exe + /c

@apietrzak
Copy link

apietrzak commented Nov 9, 2020

@asottile Thanks a lot, i'm waiting for a new version.

@asottile
Copy link
Member

asottile commented Nov 9, 2020

if you don't want to wait, you can pip install git+https://github.com/pre-commit/pre-commit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

3 participants