Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: optimise image building #13027

Merged
merged 5 commits into from May 20, 2024
Merged

Conversation

Joibel
Copy link
Member

@Joibel Joibel commented May 9, 2024

Build the argoexec and argocli images in a matrix github action.

Download and install the images into k3d in one action each.

This commit will require changes to the Required steps in github which I cannot do myself. I'm not sure why that one argoexec-image is listed as required at all, it's a dependency of testing rather than a proper part of the test or CI.

Also updates upload and download artifact actions to latest v4, which seem to be working OK now. v3 will deprecate by November. edit by agilgur5: this was to support the merge-multiple param per below

@Joibel Joibel marked this pull request as draft May 9, 2024 08:26
@Joibel Joibel force-pushed the optimize-image-ci branch 8 times, most recently from 8e853ca to 8ca09af Compare May 9, 2024 09:54
Build the argoexec and argocli images in a matrix github action.

Download and install the images into k3d in one action each.

This commit will probably require changes to the Required steps in
github which I cannot do myself.

Also updates upload and download artifact actions to latest v4, which
seem to be working OK now. v3 will deprecate by November.

Signed-off-by: Alan Clucas <alan@clucas.org>
@Joibel Joibel requested a review from agilgur5 May 9, 2024 10:29
@Joibel Joibel marked this pull request as ready for review May 9, 2024 10:29
with:
name: argocli
path: /tmp/argocli_image.tar
name: ${{matrix.image}}_image.tar
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name: ${{matrix.image}}_image.tar
name: ${{matrix.image}}

From the previous code, this should just be the name of the image. Not sure if this was a copy+paste typo?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's deliberate. We're pattern matching this with pattern: '*_image.tar' on line 164. This pattern would be '*' without the extra parts to the name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I thought that would suffice, no? Or were you trying to be extra careful in case at some point other artifacts were added or something?

@agilgur5 agilgur5 added the area/build Build or GithubAction/CI issues label May 11, 2024
@agilgur5
Copy link
Member

Follow-up to #13018 / #13018 (comment)

This commit will require changes to the Required steps in github which I cannot do myself.

Neither can I. Only @terrytangyuan and @sarabala1979 can change the required steps in the branch protection rules.

I'm not sure why that one argoexec-image is listed as required at all, it's a dependency of testing rather than a proper part of the test or CI.

All of them were required prior to #12006. Technically speaking, the image build could have a different set of changed files than E2Es although it is currently unconditional. Making sure it passes is also safer just in case -- PRs shouldn't break image builds.

Also updates upload and download artifact actions to latest v4, which seem to be working OK now. v3 will deprecate by November.

We have a follow-up issue for this #12455 where there were some lingering problems last we checked. There's also one more place that uses upload-artifact v3, the docs build. And if working we should remove the dependabot ignore as well.
So I'd probably suggest splitting that into a separate PR more generally

@agilgur5 agilgur5 self-assigned this May 11, 2024
Signed-off-by: Alan Clucas <alan@clucas.org>
@Joibel
Copy link
Member Author

Joibel commented May 17, 2024

I've reverted upload+download artifact actions again.

Signed-off-by: Alan Clucas <alan@clucas.org>
Signed-off-by: Alan Clucas <alan@clucas.org>
@Joibel
Copy link
Member Author

Joibel commented May 17, 2024

I've reverted upload+download artifact actions again.

merge-multiple doesn't work in v3. The docs are not versioned, so it's hard to discover how stuff should work. So that's part of why I'd updated, I remember now.

Also, I'm talking to myself again.

@agilgur5
Copy link
Member

agilgur5 commented May 17, 2024

merge-multiple doesn't work in v3. The docs are not versioned, so it's hard to discover how stuff should work. So that's part of why I'd updated, I remember now.

Ah that makes more sense. I edited your PR description to mention that.

It seems like "download all" is available if you don't specify name and use a directory.

But let's keep v4 regardless so we don't have to change that again in the future

@Joibel
Copy link
Member Author

Joibel commented May 20, 2024

@terrytangyuan or @sarabala1979, this commit changes the name of a CI step. The ci step argoexec-image is in the required list, and this replaces it with the matrix steps:
image
I assume these are called argo-images (argoexec) and argo-images (argocli), but maybe the required can just be argo-images

This will need updating. This PR doesn't actually change anything that happens in the CI, that happened in #13018

@terrytangyuan terrytangyuan merged commit ff646fe into argoproj:main May 20, 2024
30 checks passed
@terrytangyuan
Copy link
Member

Merged

@terrytangyuan
Copy link
Member

Updated settings too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build Build or GithubAction/CI issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants