Skip Firefox test using Cypress v9 to resolve flaky tests #660
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Test resultsThe results are good! SuccessNote especially the success of SkippedAs planned, Failures
In detail:
|
This was referenced Dec 11, 2022
|
Hi @jaffrepaul How do you feel about this PR? Disabling the Firefox test on v9 is a quick workaround to get the test suite to succeed running v10, instead of getting tripped up on "Error: could not find CRI target" from the v9 test. I don't expect to see a fix for this scenario however, so a more permanent solution would be to remove the complete v9 test for Firefox instead of simply skipping it. @nagash77 responded in cypress-io/cypress#23215 (comment) so I don't expect any change in the Cypress code for 9.x. If there is no identified issue on the Mozilla Firefox side with CRI (Chrome Remote Interface?) then there is also not much likelihood of Mozilla issuing a fix. So in total there is not much chance of this test becoming reliable again. |
disable unreliable Firefox test using Cypress v9 GitHub Cypress issue cypress-io/cypress#23215 describes the problem
MikeMcC399
force-pushed
the
fix/firefox-reliability
branch
from
569cab2
to
1ee906e
Compare
|
Thanks @MikeMcC399. Im ok to skip this test but leave as-is for reference and the thread which discusses things in more depth. |
jaffrepaul
approved these changes
Dec 15, 2022
|
I'm happy you agreed and it is rewarding to see that the badge
and all other workflows are succeeding! |
|
🎉 This PR is included in version 5.0.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
kodiakhq bot
added a commit
to weareinreach/InReach
that referenced
this pull request
Dec 20, 2022
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [actions/checkout](https://togithub.com/actions/checkout) | action | digest | `7dd9e2a` -> `755da8c` | | [chromaui/action](https://togithub.com/chromaui/action) | action | digest | `0ec8cd8` -> `2440d35` | | [cypress-io/github-action](https://togithub.com/cypress-io/github-action) | action | patch | `v5.0.0` -> `v5.0.2` | | [github/codeql-action](https://togithub.com/github/codeql-action) | action | digest | `a669cc5` -> `959cbb7` | | [tinkurlab/monorepo-pr-labeler-action](https://togithub.com/tinkurlab/monorepo-pr-labeler-action) | action | patch | `4.1.2` -> `4.1.3` | --- ### Release Notes <details> <summary>cypress-io/github-action</summary> ### [`v5.0.2`](https://togithub.com/cypress-io/github-action/releases/tag/v5.0.2) [Compare Source](https://togithub.com/cypress-io/github-action/compare/v5.0.1...v5.0.2) Numerous housekeeping updates to bring this action back up to speed. \*In combination with v5.0.1, which triggered without a new dist build. 👏 Special thanks to [@​MikeMcC399](https://togithub.com/MikeMcC399) for all the diligent work! 👏 **Fix Failed Tests** - [cypress-io/github-action#643 - [cypress-io/github-action#658 - [cypress-io/github-action#660 **Update Dependencies** - [cypress-io/github-action#644 - [cypress-io/github-action#650 **Resolve Warnings** - [cypress-io/github-action#651 - [cypress-io/github-action#666 - [cypress-io/github-action#670 - [cypress-io/github-action#675 **Docs/Example Updates** - [cypress-io/github-action#665 - [cypress-io/github-action#668 **Tooling Updates:** - [cypress-io/github-action#667 - [cypress-io/github-action#677 ### [`v5.0.1`](https://togithub.com/cypress-io/github-action/releases/tag/v5.0.1) [Compare Source](https://togithub.com/cypress-io/github-action/compare/v5.0.0...v5.0.1) ##### Bug Fixes - correct .gitignore for videos and screenshots ([89e59e5](https://togithub.com/cypress-io/github-action/commit/89e59e5cfc7760513fe9e9ce4c690345d94eb27a)) </details> <details> <summary>tinkurlab/monorepo-pr-labeler-action</summary> ### [`v4.1.3`](https://togithub.com/TinkurLab/monorepo-pr-labeler-action/releases/tag/4.1.3): Monorepo PR Labeler v4.1.3 [Compare Source](https://togithub.com/tinkurlab/monorepo-pr-labeler-action/compare/4.1.2...4.1.3) Changes include: Bug fix to ignore directories starting with a dot `.` but not directories that contain a dot `.`. According to the docs, this action should ignore mono repos that begin with a dot but it currently ignores also mono repo with a dot even if it is not the first character. Thanks to [@​MoLow](https://togithub.com/MoLow) for the fix! </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/weareinreach/InReach). PR-URL: #107 Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR resolves the issue #659 "CI: Sporadic failures in example-firefox".
The Firefox test under Cypress v9 in .github/workflows/example-firefox.yml is disabled, since it is unreliable and the issue has been shown to be reproducible on a local machine independently of github-action.
This allows the Cypress v10 test to then run reliably.