-
Notifications
You must be signed in to change notification settings - Fork 319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] Enable compatibility test matrix for unit tests #1915
[CI] Enable compatibility test matrix for unit tests #1915
Conversation
jenkins run the matrix tests |
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
jenkins run the matrix tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Nice work @v1v
* master: Bump version.slf4j from 1.7.31 to 1.7.32 (elastic#1933) Add description about memory pool metrics to docs (elastic#1925) updated team membership check in action Add 1.25.0 to cloudfoundry index Update CHANGELOG.asciidoc (elastic#1929) fixed community label action [maven-release-plugin] prepare for next development iteration [maven-release-plugin] prepare release v1.25.0 Prepare release 1.25.0 (elastic#1927) synchronize json schema specs (elastic#1926) added labeling of community issues and PRs added community labeler config Upgrade Byte Buddy to 1.11.8 (elastic#1923) Bump json-schema-validator from 1.0.56 to 1.0.57 (elastic#1916) Bump log4j2-ecs-layout from 1.0.1 to 1.1.0 (elastic#1917) Added support for setting the framework using the public api (elastic#1908) (elastic#1909) Add Javalin plugin (elastic#1822)
Good job ! Do you know if there is a way to set the stage name dynamically on the matrix axis value ?
While not being something blocking, that would definitely be great and help spot failures, we have the Java version visible when the cursor is over the |
jenkins run the matrix tests |
As far as I'm aware that's a limitation with the declarative pipeline, though, the GitHub checks should provide those details: I'll include the jdk version in the script therefore the |
run the matrix tests |
The matrix testing works properly, however there are plenty of tests that fail and need investigation. Thus, if we merge it to
|
Here is the non-exhaustive list of bugs that have been found on Jdk 15 and 16:
Thus as we won't be able to fix those in the next few weeks, having a separate option than the |
Yes, I'll enable a new parameter called
Yes, I'll configure a weekly run with the Therefore, this feature will never run by default for the time being on branches/tags/PRs. But still PRs can be tested with the GitHub comment (see #1915 (comment)) or the UI parameter ( Does it sound like a good plan? |
…junit-support * upstream/master: Updated get-user-teams-membership to version 1.0.3 ensure that the socket is closed even if there is an exception from the other close (elastic#1946) Make sure trace context headers are added only once (elastic#1937) Update CHANGELOG.asciidoc Ecs reformatting more fields (elastic#1910) Use path when high-level framework method is unknown (elastic#1906) Semver parsing enhancement (elastic#1931)
Thanks @v1v , that's definitely a good plan. We might also reuse a similar strategy for the tests on Windows, which should likely be run on a weekly basis. |
…for-windows-only * upstream/master: (100 commits) [CI] Enable compatibility test matrix for unit tests (elastic#1915) Updated get-user-teams-membership to version 1.0.3 ensure that the socket is closed even if there is an exception from the other close (elastic#1946) Make sure trace context headers are added only once (elastic#1937) Update CHANGELOG.asciidoc Ecs reformatting more fields (elastic#1910) Use path when high-level framework method is unknown (elastic#1906) Semver parsing enhancement (elastic#1931) Bump version.slf4j from 1.7.31 to 1.7.32 (elastic#1933) Add description about memory pool metrics to docs (elastic#1925) updated team membership check in action Add 1.25.0 to cloudfoundry index Update CHANGELOG.asciidoc (elastic#1929) fixed community label action [maven-release-plugin] prepare for next development iteration [maven-release-plugin] prepare release v1.25.0 Prepare release 1.25.0 (elastic#1927) synchronize json schema specs (elastic#1926) added labeling of community issues and PRs added community labeler config ...
Looks like merging this might have triggered the full run because the parameter wasn't yet present. I'm going to re-run the master branch to ensure this is working as-expected now that the the parameter is in place. |
What does this PR do?
Run the compatibility unit tests in the CI on a weekly basis for the
master
branch.On a PR basis can be enabled with:
run compatibility tests
orjenkins run the compatibility tests
orrun the compatibility tests
compatibility_ci
parameter.Why
To get towards supporting more recent versions officially, we should add Java 12-16 to the test matrix.
Important
java12
is the latest java version for that flavour so, there are different ones that could be used injdk >12
:This unit test matrix will run always after the
APM ITs
, for simplicity, I didn't add more complexity in the pipeline to run more things in parallel.Pipeline UI
Checklist