GitHub Actions Windows: Invalid --only='""' (fixed in 2.16.5)

[jborean93]

Description

I've getting random errors when attempting to build Windows wheels using the pypa/cibuildwheel@2.16.4 action.

Run pypa/cibuildwheel@v2.16.4
Run actions/setup-python@v5
Installed versions
Run pipx run --python "C:\hostedtoolcache\windows\Python\3.12.1\x64\python.exe" --spec "D:\a\_actions\pypa\cibuildwheel\v2.16.4" cibuildwheel "." --output-dir '"wheelhouse"' --config-file '""' --only '""'
  pipx run --python "C:\hostedtoolcache\windows\Python\3.12.1\x64\python.exe" --spec "D:\a\_actions\pypa\cibuildwheel\v2.16.4" cibuildwheel "." --output-dir '"wheelhouse"' --config-file '""' --only '""'
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    CIBW_ARCHS: all
    CIBW_TEST_SKIP: *_arm64
    CIBW_BUILD: cp38-win32
    CIBW_BUILD_VERBOSITY: 1
creating virtual environment...
determining package name from 'D:\\a\\_actions\\pypa\\cibuildwheel\\v2.16.4'...
creating virtual environment...
installing cibuildwheel from spec 'D:\\a\\_actions\\pypa\\cibuildwheel\\v2.16.4'...
Invalid --only='""', must be a build selector with a known platform
Error: Process completed with exit code 1.

On a build that works I can see the same command is being run with the --only='""' parameter.

Run pipx run --python "C:\hostedtoolcache\windows\Python\3.12.1\x64\python.exe" --spec "D:\a\_actions\pypa\cibuildwheel\v2.16.4" cibuildwheel "." --output-dir '"wheelhouse"' --config-file '""' --only '""'
  pipx run --python "C:\hostedtoolcache\windows\Python\3.12.1\x64\python.exe" --spec "D:\a\_actions\pypa\cibuildwheel\v2.16.4" cibuildwheel "." --output-dir '"wheelhouse"' --config-file '""' --only '""'
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    CIBW_ARCHS: all
    CIBW_TEST_SKIP: *_arm64
    CIBW_BUILD: cp38-win_amd64
    CIBW_BUILD_VERBOSITY: 1

The weird thing is that this only happens sometimes, other times the build works just fine. I know the 2.15.0 version is also affected. I only started seeing this in the last day. For example here is an older build that failed 2 times while on the 3rd it worked again https://github.com/pythongssapi/python-gssapi/actions/runs/7697806880

I'm unsure whether it's my code or something else but this has worked for me for the past year or so whereas now it seems to be popping up. At a guess it might be

Build log

https://github.com/pythongssapi/python-gssapi/actions/runs/7703392610/job/20993685914?pr=341

CI config

https://github.com/pythongssapi/python-gssapi/blob/18f3e2bc5119a3621c7bc14e307ab70ce011e34f/.github/workflows/ci.yml#L123-L129

[jborean93]

The upgrade to PowerShell 7.4.x actions/runner-images#9115 might be the cause here. The older version is 7.2.x and PowerShell 7.4 changed how arguments are escaped. In particularly how double quotes are treated in an argument value has changed. For example the first example was run with pwsh 7.2.18 while the second example with 7.4.1.

PS C:\Users\vagrant-domain\Downloads\PowerShell-7.2.18-win-x64> C:\temp\print_argv.exe --option '""'
"C:\temp\print_argv.exe" --option ""
[0] --option
[1]


PS C:\Users\vagrant-domain\Downloads\PowerShell-7.4.1-win-x64> C:\temp\print_argv.exe --option '""'
"C:\temp\print_argv.exe" --option "\"\""
[0] --option
[1] ""

There are three things you can do to go back to the old behaviour:

• Don't use the inner double quotes, pwsh will quote the empty string for you when calling the exe
• Set Set `$PSNativeCommandArgumentPassing = 'Legacy' https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_preference_variables?view=powershell-7.4#psnativecommandargumentpassing
• Have the underlying Python code being called here handle a literal string value of "" as the --option (and other affected args) value

PS C:\Users\vagrant-domain\Downloads\PowerShell-7.4.1-win-x64> $PSNativeCommandArgumentPassing
Windows
PS C:\Users\vagrant-domain\Downloads\PowerShell-7.4.1-win-x64> C:\temp\print_argv.exe --option '""'
"C:\temp\print_argv.exe" --option "\"\""
[0] --option
[1] ""
PS C:\Users\vagrant-domain\Downloads\PowerShell-7.4.1-win-x64> C:\temp\print_argv.exe --option ''
"C:\temp\print_argv.exe" --option ""
[0] --option
[1]

PS C:\Users\vagrant-domain\Downloads\PowerShell-7.4.1-win-x64> $PSNativeCommandArgumentPassing = 'Legacy'
PS C:\Users\vagrant-domain\Downloads\PowerShell-7.4.1-win-x64> C:\temp\print_argv.exe --option '""'
"C:\temp\print_argv.exe" --option ""
[0] --option
[1]

Adding further fuel to that suggestion I can see the runner version of a working build is 20240122.1.0 and a failed build is 20240128.1.0.

  Image: windows-2022
  Version: 20240122.1.0
  Included Software: https://github.com/actions/runner-images/blob/win22/20240122.1/images/windows/Windows2022-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20240122.1

  Image: windows-2022
  Version: 20240128.1.0
  Included Software: https://github.com/actions/runner-images/blob/win22/20240128.1/images/windows/Windows2022-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20240128.1

[henryiii]

Does anyone remember/know why we double quote on powershell in the first place? I added it in #1346 but not sure it's needed, at least for non-paths. I think it might have been needed for paths with spaces?

[henryiii]

The problem with --x "" --y is that it resolves to --x, --y, rather that --x, empty string, --y. Is there a way to pass an empty string in powershell 7.3/7.4?

[jborean93]

Is there a way to pass an empty string in powershell 7.3/7.4?

Yep see my 3 workarounds in #1740 (comment).

Essentially the new quoting rules will pass through the actual value that was bound in the pwsh binder and an empty string value will be quoted. You either need to change the code (and break 7.2 compatbility), revert back to the old argument quoting rules, or handle a double quote literal value in the Python argparse code. I went with option 2 in my PR because it is the simplest one available.

[joerick]

According to the docs, we could just use Python as a shell instead...? Though I can't remember the issue with Meson that's mentioned in the comment here.

[henryiii]

You need cmd or powershell to get MSVC properly configured. If you run from other places, you get GCC instead. This affects other build systems too unless they look up MSVC via the registry.