Add master branch workflow for test262 tests #10556
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR adds a nightly job for running test262 tests. The results of these tests will be stored in build artifacts which will be used to compare with test262 runs on PR (future work!). Robin has also done a lot of work to ensure all babel packages are properly linked to the test runner. Additionally, Nicolo has helped in mentorship and contributions to babel-test262-runner which is used here. Co-authored-by: Robin Ricard <rricard2@bloomberg.net>
nicolo-ribaudo
added
area: test262
area: tests
PR: Internal 🏠
Co-Authored-By: Nicolò Ribaudo <nicolo.ribaudo@gmail.com>
| node lib/download-node | ||
| - run: | ||
| name: Run Test262 | ||
| command: node lib/run-tests I_AM_SURE | tee ~/test262.tap |
There was a problem hiding this comment.
Rather than linking every Babel package, what do you think of adding a --babelMonorepo .. cli clag to lib/run-tests?
There was a problem hiding this comment.
That is something I wanted to do in a followup PR in which I want to:
- Use something like yargs: http://yargs.js.org/
- Refactor existing cli arguments
- Add path to babel argument
- Add path to whitelist files so that they can remain colocated to babel/babel repo
- Add babel plugins/config argument
nicolo-ribaudo
approved these changes
Oct 15, 2019
|
Thinking a bit more about this I am realizing that only running these tests nightly doesn’t achieve our end result. Because when I query master branch artifacts, it isn't necessary that I find it in the latest master job, if other non-nightly jobs have run. Based on that thinking, I’ll add a commit to make these tests always run on master, so that I can rely on them existing when I’m in a PR job. |
jbhoosreddy
changed the title
We're no longer giving xunit output to CircleCI because once it sees failing tests it will mark babel's master as broken. As a stopgap, it will render the test results in the mocha spec format, which is human readable.
nicolo-ribaudo
approved these changes
Oct 16, 2019
existentialism
approved these changes
Oct 17, 2019
|
Relates to: #4987 |
This was referenced Nov 28, 2019
This was referenced Dec 5, 2019
lock
bot
added
the
outdated
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #10500
This PR adds a master branch job for running test262 tests. The results of
these tests will be stored in build artifacts which will be used to
compare with test262 runs on PR (future work!). Robin has also done a
lot of work to ensure all babel packages are properly linked to the
test runner. Additionally, Nicolo has helped in mentorship and
contributions to babel-test262-runner which is used here (babel/babel-test262-runner@c80353...399abd).
Co-authored-by: Robin Ricard rricard2@bloomberg.net