Support for self-hosted Windows runners

[nazar-pc]

I tried this action on self-hosted runner and while Ubuntu worked, Windows did't:

##[debug]Evaluating condition for step: 'Install cargo-nextest'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Install cargo-nextest
##[debug]Loading inputs
##[debug]Loading env
Run taiki-e/install-action@0163f6cf65d9b9bb0e5d1f0e3ae19280a06be4b0
##[debug]Evaluating: inputs.tool
##[debug]Evaluating Index:
##[debug]..Evaluating inputs:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'tool'
##[debug]=> 'cargo-nextest'
##[debug]Result: 'cargo-nextest'
##[debug]Evaluating: inputs.checksum
##[debug]Evaluating Index:
##[debug]..Evaluating inputs:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'checksum'
##[debug]=> 'true'
##[debug]Result: 'true'
##[debug]Evaluating condition for step: 'run'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: run
##[debug]Loading inputs
##[debug]Loading env
Run bash --noprofile --norc "${GITHUB_ACTION_PATH:?}/main.sh"
##[debug]C:\cygwin64\bin\bash.EXE --noprofile --norc -e -o pipefail C:\actions-runner\_work\_temp\74c519f7-435e-4f1d-9e43-a0fae0148bd9.sh
info: install-action does not support cargo-nextest; fallback to cargo-binstall
info: installing cargo-binstall
info: downloading null
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
curl: (3) URL using bad/illegal format or missing URL
Error: Process completed with exit code 3.
##[debug]Finished: run
##[debug]Finishing: Install cargo-nextest

Would be convenient if it was supported in this environment as well.

[taiki-e]

Thanks for the report! There may be some issue with jq or path handling. Could you try aa0fb41?

[nazar-pc]

host platform: x86_64_windows
jq version: jq-1.6

https://github.com/subspace/subspace/actions/runs/6188492029/job/16800658877#step:7:1

[taiki-e]

host platform: x86_64_windows
jq version: jq-1.6
info: install-action does not support cargo-nextest; fallback to cargo-binstall
info: installing cargo-binstall@1.3.0
info: downloading null

Hmm. jq at least succeeds in parsing the manifest the first and second time (since "1.3.0" is parsed), but the subsequent comparison to "null" (even though the actual value is "null") appears to be evaluated as false.

install-action/main.sh

Line 209 in c33cd1d

if [[ "${url}" == "null" ]]; then

Given that the jq version is 1.6 and that a flag was added in 1.7 to fix the carriage return problem, I guess this is probably related to jqlang/jq#1854.

Could you try c86cf7e?

[nazar-pc]

I don't see anything helpful in the output: https://github.com/subspace/subspace/actions/runs/6188821326/job/16801617758#step:7:1
jq 1.7 if fairly new, I think there are plenty of environments where it is not yet present.

[taiki-e]

jqlang/jq#1854 appears to indicate that the problem exists only with a particular type of jq binary, and the curl error (unable to write to hidden directory) appears to be specific to a particular build way. (Both work fine, at least when using the binaries included in the GitHub-provided runners.)

Could you try 77363e9? (added potential workaround for curl hidden directory issue)

[nazar-pc]

Hm...

host platform: x86_64_windows
jq version: jq-1.7-dirty
info: install-action does not support cargo-nextest; fallback to cargo-binstall
info: installing cargo-binstall@1.3.0
info: downloading https://github.com/cargo-bins/cargo-binstall/releases/download/v1.3.0/cargo-binstall-x86_64-pc-windows-msvc.zip
info: verifying sha256 checksum for cargo-binstall-x86_64-pc-windows-msvc.zip
info: cargo-binstall installed at 
+ cargo binstall -V
error: no such command: `binstall`

	Did you mean `install`?

	View all installed commands with `cargo --list`
Error: Process completed with exit code 101.

https://github.com/subspace/subspace/actions/runs/6189004596/job/16802168830#step:7:1

[taiki-e]

Could you try f2f6a1f? Added a potential fix and some debug output.

[nazar-pc]

Didn't work either:

host platform: x86_64_windows
jq version: jq-1.7-dirty
info: install-action does not support cargo-nextest; fallback to cargo-binstall
error: no such command: `binstall`

	Did you mean `install`?

	View all installed commands with `cargo --list`
info: cargo-binstall already installed at /usr/local/bin/cargo-binstall.exe, but is not compatible version with install-action, upgrading
info: installing cargo-binstall@1.3.0
info: downloading https://github.com/cargo-bins/cargo-binstall/releases/download/v1.3.0/cargo-binstall-x86_64-pc-windows-msvc.zip
info: verifying sha256 checksum for cargo-binstall-x86_64-pc-windows-msvc.zip
Warning: cargo-binstall should be installed at /usr/local/bin; but cargo-binstall.exe not found in path
+ cargo binstall -V
error: no such command: `binstall`

	Did you mean `install`?

	View all installed commands with `cargo --list`

https://github.com/subspace/subspace/actions/runs/6189176189/job/16802753709#step:7:1

P.S. Every time I have an issue with CI it is never an exact science, but rather trial and error.

[taiki-e]

I found a bug related to determining the installation location that I can reproduce in other Windows environments.
I think that is the cause of the remaining problems.

Could you try cc5181c?

[nazar-pc]

Something new this time:

host platform: x86_64_windows
jq version: jq-1.7-dirty
info: install-action does not support cargo-nextest; fallback to cargo-binstall
info: installing cargo-binstall@1.3.0
info: downloading https://github.com/cargo-bins/cargo-binstall/releases/download/v1.3.0/cargo-binstall-x86_64-pc-windows-msvc.zip
info: verifying sha256 checksum for cargo-binstall-x86_64-pc-windows-msvc.zip
info: cargo-binstall installed at /home/Administrator/.install-action/bin/cargo-binstall.exe
+ cargo binstall -V
error: could not execute process `C:\cygwin64\home\Administrator\.install-action\bin\cargo-binstall.exe binstall -V` (never executed)

Caused by:
  Access is denied. (os error 5)

https://github.com/subspace/subspace/actions/runs/6189633359/job/16804165414#step:7:1

[taiki-e]

I changed to prefer $RUNNER_TOOL_CACHE over $HOME in a84bb47, so the permission issue should be fixed now.

[nazar-pc]

Nope, same error again: https://github.com/subspace/subspace/actions/runs/6190541151/job/16806980662#step:7:1
UPD: I used the old commit, let me try again 🙃

[nazar-pc]

No, updated commit didn't work either with a different error:

info: host platform: x86_64_windows
info: cargo is installed at /cygdrive/c/Users/Administrator/.cargo/bin/cargo
info: jq version: jq-1.7-dirty
info: install-action does not support cargo-nextest; fallback to cargo-binstall
info: installing cargo-binstall@1.3.0
info: downloading https://github.com/cargo-bins/cargo-binstall/releases/download/v1.3.0/cargo-binstall-x86_64-pc-windows-msvc.zip
info: verifying sha256 checksum for cargo-binstall-x86_64-pc-windows-msvc.zip
Warning: cargo-binstall should be installed at /c/actions-runner/_work/_tool/install-action/bin/cargo-binstall.exe; but cargo-binstall.exe not found in path
+ cargo binstall -V
error: no such command: `binstall`

	Did you mean `install`?

	View all installed commands with `cargo --list`
Error: Process completed with exit code 101.

https://github.com/subspace/subspace/actions/runs/6190591939/job/16807141866#step:7:1

[taiki-e]

Hmm, sorry, that did not seem to be enough to fix the problem. (On second thought, if installing on HOME was really the problem, then the jq run would have failed as well)

The difference between the way cargo-binstall and jq are executed is whether it goes through cargo or not. I have created a patch (79a868a) that calls cargo binstall without going through cargo, could you try that?

[nazar-pc]

Still permission denied:

+ cargo-binstall binstall -V
C:\actions-runner\_work\_actions\taiki-e\install-action\79a868a4c0889f8bc9ec358aec0009aa0a887db1/main.sh: line 11: /home/Administrator/.install-action/bin/cargo-binstall: Permission denied
Error: Process completed with exit code 126.

https://github.com/subspace/subspace/actions/runs/6190724569/job/16807554656

Is there something like chmod +x on Windows? I didn't use Windows for anything work-wise for ~10 years now.

[taiki-e]

Is there something like chmod +x on Windows?

Good point. I found https://cygwin.com/cygwin-ug-net/using-filemodes.html

On FAT or FAT32 filesystems, files are always readable, and Cygwin uses the DOS read-only attribute to determine if they are writable. Files are considered to be executable if the filename ends with .bat, .com or .exe, or if its content starts with #!.

Whether or not the .exe is used at the time of the call is indeed another difference between the way cargo-binstall and jq were executed...

I updated my patch to use .exe (84cf87d).

[nazar-pc]

info: host platform: x86_64_windows
info: cargo is located at /cygdrive/c/Users/Administrator/.cargo/bin/cargo
info: jq version: jq-1.7-dirty
info: install-action does not support cargo-nextest; fallback to cargo-binstall
info: installing cargo-binstall@1.3.0
info: downloading https://github.com/cargo-bins/cargo-binstall/releases/download/v1.3.0/cargo-binstall-x86_64-pc-windows-msvc.zip
info: verifying sha256 checksum for cargo-binstall-x86_64-pc-windows-msvc.zip
Warning: cargo-binstall should be installed at /home/Administrator/.install-action/bin/cargo-binstall.exe; but cargo-binstall.exe not found in path
+ cargo-binstall.exe binstall -V
C:\actions-runner\_work\_actions\taiki-e\install-action\84cf87d08640570a4867937b58e6cab8d5b8fed9/main.sh: line 11: cargo-binstall.exe: command not found
Error: Process completed with exit code 127.

https://github.com/subspace/subspace/actions/runs/6190869896/job/16808019785#step:7:1

[taiki-e]

Oh, that's probably a bug about adding PATHs, and it should be fixed in 6b495ce.

[nazar-pc]

Back to permission denied:

info: host platform: x86_64_windows
info: cargo is located at /cygdrive/c/Users/Administrator/.cargo/bin/cargo
info: jq version: jq-1.7-dirty
info: install-action does not support cargo-nextest; fallback to cargo-binstall
info: installing cargo-binstall@1.3.0
info: adding \home\Administrator\.install-action\bin to PATH
info: downloading https://github.com/cargo-bins/cargo-binstall/releases/download/v1.3.0/cargo-binstall-x86_64-pc-windows-msvc.zip
info: verifying sha256 checksum for cargo-binstall-x86_64-pc-windows-msvc.zip
info: cargo-binstall installed at /home/Administrator/.install-action/bin/cargo-binstall.exe
+ cargo-binstall.exe binstall -V
C:\actions-runner\_work\_actions\taiki-e\install-action\6b495ce909cc90247ed5baa261cf4bcf03f75eff/main.sh: line 11: /home/Administrator/.install-action/bin/cargo-binstall.exe: Permission denied

https://github.com/subspace/subspace/actions/runs/6191193015/job/16808987474#step:7:1

[taiki-e]

Hmm...

• One of the other differences between jq and cargo-binstall is whether it is called with the full path or not, so changing that (5826035) might work, but it probably would not resolve the underlying problem (except when there was a problem only with cargo-binstall).
• Another possibility is that there is a problem with the unzipping process of cargo-binstall's zip file or with the binaries themselves. It might be helpful to see if cargo-llvm-cov, which uses tar.gz, or shfmt, which installs binaries directly, also has problems. (In this case, we may want to revert Use cargo-binstall to install nextest #183)

That said, it may be better here to try taiki-e/cache-cargo-install-action, which was created to support cases where install-action does not work.

[NobodyXu]

@nazar-pc Could that be the windows defender or other defending mechanism kicking in?

[nazar-pc]

Full path shouldn't be necessary. Just tried taiki-e/cache-cargo-install-action and it worked just fine, will go with it for now.

@nazar-pc Could that be the windows defender or other defending mechanism kicking in?

Anything is possible, asked DevOps person on our team to take a look there.

[taiki-e]

Maybe related: microsoft/winget-cli#228