-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
E2E Tests: Update Multi Entity Saving test to improve reliability #37139
Merged
andrewserong
merged 3 commits into
trunk
from
update/multi-entity-savings-e2e-tests-to-improve-reliability
Dec 6, 2021
Merged
E2E Tests: Update Multi Entity Saving test to improve reliability #37139
andrewserong
merged 3 commits into
trunk
from
update/multi-entity-savings-e2e-tests-to-improve-reliability
Dec 6, 2021
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrewserong
added
[Type] Code Quality
Issues or PRs that relate to code quality
[Package] E2E Test Utils
/packages/e2e-test-utils
[Package] E2E Tests
/packages/e2e-tests
labels
Dec 6, 2021
Size Change: 0 B Total Size: 1.11 MB ℹ️ View Unchanged
|
Failing e2e tests seem unrelated. I'll try a rebase to re-run them. |
ockham
force-pushed
the
update/multi-entity-savings-e2e-tests-to-improve-reliability
branch
from
December 6, 2021 13:32
8f1b404
to
b3bbe58
Compare
ockham
approved these changes
Dec 6, 2021
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Andrew! This LGTM 👍
Thanks for the review and rebase @ockham, much appreciated! 🙇 |
andrewserong
deleted the
update/multi-entity-savings-e2e-tests-to-improve-reliability
branch
December 6, 2021 22:37
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
[Package] E2E Test Utils
/packages/e2e-test-utils
[Package] E2E Tests
/packages/e2e-tests
[Type] Code Quality
Issues or PRs that relate to code quality
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I noticed on
trunk
that we sometimes see intermittent failures of the multi entity saving e2e test. A recent example is this failure. The testSite blocks should save individually
first publishes the post with only one of the site entities selected, and then clicks "Update" to check that the deselected entity appears in the list of checkboxes.This test appears to fail intermittently due to a race condition where the test is run so quickly, that the Update panel is re-opened before the publish event has finished. This is much faster than real world usage, and means that the test runner sees two checkboxes where it expects to see only one. By running with Headless switched to
false
I could see the issue where two checkboxes are visible for a tiny moment before the publishing concludes:This results in the following error message:
The fix
The fix for the above involves the following:
getOption
andsetOption
functions from the Site Title block test that @ockham introduced, and moved them to thee2e-test-utils
package, as they're now useful in more than one place.How has this been tested?
Locally, after starting up
wp-env
and runningnpm run dev
, I ran the following:Types of changes
Code quality
Checklist:
*.native.js
files for terms that need renaming or removal).