(WIP) Test batch_processing.sh in CI

[joshuacwnewton]

Just a quick WIP to test:

• Running batch_processing using CI
• Uploading the resulting csv files using the upload-artifact GH action

Don't review, etc. I'll write a proper description when ready. :)

[joshuacwnewton]

After some finagling, I've gotten the upload-artifact action to upload the 5 csv files produced by batch_processing.sh.

Now, in theory, I should be able to fetch these using the download-artifact action so that I can make a comparison in a test. :)

[joshuacwnewton]

Welp, that's disappointing. ☹️

Downloading files: You can only download artifacts that were uploaded during the same workflow run. When you download a file, you can reference it by name.

That conflicts with what I expected, given this sentence on the same page:

For example, you can use artifacts to save your build and test output after a workflow run has ended.

I'll have to find another way. :(

[joshuacwnewton]

The strange thing to me is, the retention period for artifacts is 90 days. Why would they store something for 90 days if you can't reuse it outside of a single workflow run?

[joshuacwnewton]

Interesting...

joshua@XPS-15-9560:~$ curl   -H "Accept: application/vnd.github.v3+json"   https://api.github.com/repos/neuropoly/spinalcordtoolbox/actions/artifacts
{
  "total_count": 1,
  "artifacts": [
    {
      "id": 36987543,
      "node_id": "MDg6QXJ0aWZhY3QzNjk4NzU0Mw==",
      "name": "batch-processing-results",
      "size_in_bytes": 6316,
      "url": "https://api.github.com/repos/neuropoly/spinalcordtoolbox/actions/artifacts/36987543",
      "archive_download_url": "https://api.github.com/repos/neuropoly/spinalcordtoolbox/actions/artifacts/36987543/zip",
      "expired": false,
      "created_at": "2021-01-25T00:52:39Z",
      "updated_at": "2021-01-25T00:52:39Z",
      "expires_at": "2021-04-25T00:52:35Z"
    }
  ]
}

So, despite download-artifact seemingly not working in my CI test, the csv files still get stored, and it looks like you can access them via the REST API. Weird. Why does download-artifact have a limitation, then?

No matter. Looks like there's still hope!

[joshuacwnewton]

Alrighty, figured out how to download the artifact using curl:

curl -u "$USER:$ACCESS_TOKEN" -H "Accept: application/vnd.github.v3+json" -L https://api.github.com/repos/neuropoly/spinalcordtoolbox/actions/artifacts/36987543/zip --output test.zip

The only quirk is the $USER:$ACCESS_TOKEN part. The documentation says "Anyone with read access to the repository can use this endpoint", but I'm hoping that the GITHUB_TOKEN secret can be substituted instead. This seems reasonable, given that they even include a REST API example.

Other than that, the steps are quite simple:

• Fetch the list of artifacts using a GET request
• Filter the response to find the most recent artifact
• Download that artifact using another GET request
• Use the resulting CSV files in a test

[joshuacwnewton]

Alrighty! I've done all the experimenting I wanted to do, and everything looks good to me! I'm going to close this, then open a separate PR with all of the commits cleaned up + an in-depth description. :)

[joshuacwnewton]

Just kidding -- there's one more test I want to do. There are a few workarounds in actions/runner#241 and pytest-dev/pytest#7443 to enable color output in pytest. I think this would be really nice for improving the readability of test results.

[joshuacwnewton]

Splendid! Pretty colour: https://github.com/neuropoly/spinalcordtoolbox/runs/1764465154?check_suite_focus=true

Now I'm closing this. Expected a much neater PR soon.