Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Builds from external contributors duplicate tests run across mutliple machines #21318

Closed
ZachJW34 opened this issue May 3, 2022 · 1 comment · Fixed by #21305
Closed
Labels
CI General issues involving running in a CI provider process: tests Related to our internal tests

Comments

@ZachJW34
Copy link
Contributor

ZachJW34 commented May 3, 2022

Current behavior

When a build is created by an external contributor, the tests are not split up across the machines due to them not having access to the record keys. We still parallelize the job (up to 7 machines) which means we are using up CI time and increasing the likelihood that the job fails due to flake.

Desired behavior

We should not be parallelizing jobs if the tests cannot be split up, or we should use our own logic for splitting the tests across machines

Test code to reproduce

Check out a build from an external contributor e.g. https://app.circleci.com/pipelines/github/cypress-io/cypress/37271/workflows/1ad10bf4-86c3-4bff-bad8-044b3952ad4b/jobs/1499238/parallel-runs/5?filterBy=ALL

Cypress Version

9.6.0

Other

No response

@flotwig
Copy link
Contributor

flotwig commented May 3, 2022

I think anywhere we use Cy parallelization we should fall back to using CircleCI test splitting, same way we do for driver-integration-tests:

TESTFILES=$(circleci tests glob "cypress/integration/**/*spec.*" | circleci tests split --total=$CIRCLE_NODE_TOTAL)

This could be factored out into a unique CircleCI step to reduce duplication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI General issues involving running in a CI provider process: tests Related to our internal tests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants