Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the kola test workflow reusable #374

Merged
merged 7 commits into from Jul 13, 2022

Conversation

gabriel-samfira
Copy link
Member

  • Adds a reusable workflow that can run tests
  • Adapts the ci.yaml to use reusable workflow
  • Creates a new workflow that helps trigger tests using an arbitrary workflow run.

In this PR, the artifact download action is from my own fork, because it includes a fix that streams the artifact directly to disk. The upstream action downloads to memory and also un-archives the artifact in memory, which leads to huge memory usage in our case. I created a PR for this with the upstream project here:

dawidd6/action-download-artifact#175

Signed-off-by: Gabriel Adrian Samfira gsamfira@cloudbasesolutions.com

this initial attempt runs right after the "packages" jobs and downloads
the resulting artifacts.

The QEMU image is unzipped then the kola test is running using the
ci-automation scripts.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
@gabriel-samfira gabriel-samfira force-pushed the make-workflow-pluggable branch 2 times, most recently from af0b1e1 to e7fa8e4 Compare July 11, 2022 08:57
@gabriel-samfira gabriel-samfira changed the base branch from tormath1/kola to main July 11, 2022 09:08
Copy link
Member

@pothos pothos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!
For cherry-picking the changes in ci-automation/* should be done in a separate commit.
I'll look into the kola VM profile RAM situation - why was 8 GB swap chosen?

@gabriel-samfira
Copy link
Member Author

gabriel-samfira commented Jul 11, 2022

Looks good, thanks! For cherry-picking the changes in ci-automation/* should be done in a separate commit. I'll look into the kola VM profile RAM situation - why was 8 GB swap chosen?

A little over what 3 VMs of 2.5 GB memory would consume. It gives a chance for tests to finish before an out of memory condition is reached. Hitting swap will slow down newly spawned tests to allow already spawned tests to finish and free up memory.

Without swap, some VMs will be killed when memory is exhausted.

Will split the PR in 2 commits.

  * Add SKIP_COPY_TO_BINCACHE environment variable that will skip
    uploading test results to bincache. This is useful if we want to
    upload test results as artifacts on github.
  * make QEMU_IMAGE_NAME configurable

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
  * Adds a reusable workflow that can run tests
  * Adapts the ci.yaml to use reusable workflow
  * Creates a new workflow that helps trigger tests using an arbitrary
    workflow run.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
.github/workflows/run-kola-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/run-kola-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/run-kola-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/run-kola-tests.yaml Outdated Show resolved Hide resolved
Remove arch and parallel options. Hardcode them for now.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
@gabriel-samfira gabriel-samfira force-pushed the make-workflow-pluggable branch 3 times, most recently from ea3948a to 667a6cc Compare July 12, 2022 21:57
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
  * arm64 tests now run in LXD containers instead of VMs
  * parallelism increased for arm64
  * some setup steps are skipped on arm64

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
@pothos
Copy link
Member

pothos commented Jul 13, 2022

A fix for the failed test is prepared in flatcar/bootengine#47

I just realize that we even should backport the .github changes (and before that the previous PR), otherwise the a PR against an older scripts branch like flatcar-3227 would run without the changes done here, or?

@gabriel-samfira
Copy link
Member Author

I just realize that we even should backport the .github changes (and before that the previous PR), otherwise the a PR against an older scripts branch like flatcar-3227 would run without the changes done here, or?

Yes. The workflows would need to be backported to branches where you push PRs to as well as far as I remember.

@pothos pothos merged commit 76b47a0 into flatcar:main Jul 13, 2022
pothos added a commit that referenced this pull request Jul 13, 2022
Make the kola test workflow reusable
pothos added a commit that referenced this pull request Jul 13, 2022
Make the kola test workflow reusable
pothos added a commit that referenced this pull request Jul 13, 2022
Make the kola test workflow reusable
pothos added a commit that referenced this pull request Jul 13, 2022
Make the kola test workflow reusable
@pothos
Copy link
Member

pothos commented Jul 13, 2022

He, I was a bit late with picking #356 and it already started the workflow builds for all branches :D

@gabriel-samfira
Copy link
Member Author

He, I was a bit late with picking #356 and it already started the workflow builds for all branches :D

no worries. They can easily be canceled in the "Actions" tab of the repository. :D

t-lo pushed a commit that referenced this pull request Apr 17, 2023
….20210914

net-vpn/wireguard-tools: Move to portage-stable; Sync with Gentoo; Updates to wireguard-tools-1.0.20210914
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants