test: clean up skipped flaky tests

[flotwig]

Additional details

• In test: change some skipped tests to use retries #24124 some "persistent" failures from skipped flake were left skipped.
• This PR attempts to resolve those, by fixing broken tests or updating issues for persistent failures.

Summary of failures looked at

1. ❌ Persistently failing. `uses cy.origin twice` in `origin/validation.cy.ts` https://dashboard.cypress.io/projects/ypt4pf/runs/40003/test-results/0c4b2400-e352-49e2-90f6-fdc91506aa9e
   1. Flaky test: Permission denied to access property "addEventListener" on cross-origin object #23451
2. ✅ Removed due to XHR removal in 12. `xhr log has response body/status code when xhr response is logged second` in `proxy_logging.cy.ts` https://dashboard.cypress.io/projects/ypt4pf/runs/40003/test-results/7d0e5b2a-c299-4ebb-8478-fdfbfcb39ef2
3. ✅ Removed due to XHR removal in 12. `xhr log has response body/status code when xhr response is logged first` in `proxy_logging.cy.ts` https://dashboard.cypress.io/projects/ypt4pf/runs/40003/test-results/d4c78b51-b525-462d-a844-a4803d263799
4. ❌ Still represents a real issue with origin. `verifies fetch requests made while a secondary origin is active eventually update with snapshots of the secondary origin` in `origin/snapshots.cy.ts`
   1. xhr requests logged from within a cy.origin block are duplicated in the log #23840
5. ❌ No longer flakes locally, but fails in CI. `send Studio usage metadata` in `record_spec` https://app.circleci.com/pipelines/github/cypress-io/cypress/44422/workflows/25bf1f35-9741-4f30-b29c-6ec11cf7e89c/jobs/1861142
   1. Flaky test: AssertionError: expected 'POST /runs/00748421-e035-4a3d-8604-8468cc48bdb5/instances' to equal 'POST /instances/e9e81b5e-cc58-4026-b2ff-8ae3161435a6/results' #23151
6. ❌ Seems to pass eventually, but still flaky, fails in CI. `passes in parallel with group` in `record_spec` https://app.circleci.com/pipelines/github/cypress-io/cypress/44422/workflows/25bf1f35-9741-4f30-b29c-6ec11cf7e89c/jobs/1861142
   1. Flaky test: expected exit code 3 but got 1: expected 1 to equal 3 #23152
7. ✅ Fixed selector. Still flaky, but converted to a retry. `makes the right command for pnpm` / `makes the right command for npm` in `project-setup.cy.ts` https://app.circleci.com/pipelines/github/cypress-io/cypress/45902/workflows/14823615-1e9c-48fd-8d1b-b75a314110dc/jobs/1932073
8. ✅ Was already fixed by another PR. `saves the filter when navigating to a spec and back` in `specs_list_e2e.cy.ts` https://app.circleci.com/pipelines/github/cypress-io/cypress/45902/workflows/14823615-1e9c-48fd-8d1b-b75a314110dc/jobs/1932099
9. ❌ Consistently fails now. `passes filtered options to Project#open and sets cli config` in `cypress_spec.js` https://app.circleci.com/pipelines/github/cypress-io/cypress/46741/workflows/a44414d3-e3ab-4614-aeb3-5c041a87400f/jobs/1960772
   1. Flaky test: must call #initializeConfig before accessing config #23149
10. ❌ Fails persistently once a flake happens with "port in use 4455". `takes the user to first step of ct setup when switching from app` in `project-setup.cy.ts` https://app.circleci.com/pipelines/github/cypress-io/cypress/46741/workflows/a44414d3-e3ab-4614-aeb3-5c041a87400f/jobs/1960739
    1. Flaky test: Port 4445 is already in use #23418
11. ❌ Seems to no longer flake: Flaky test: AssertionError: Timed out retrying after 4000ms: Expected to find content: '/(a few|[0-9]) seconds? ago/' within the element: <div.h-full.grid.grid-col-1.git-info-row.justify-start.items-center> but never did. #23474 (comment) `shows no icon and file system timestamp for files` in `specs_list_no_git_repo.cy.ts` https://app.circleci.com/pipelines/github/cypress-io/cypress/47574/workflows/c287ce0b-245b-449e-9bab-86a66f0265dd/jobs/1990772
12. ❌ No longer flakes locally, but fails in CI. gets correct status for files on linux in GitDataSource.spec.ts Flaky test: Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/root/cypress/packages/data-context/test/unit/sources/GitDataSource.spec.ts) #23317

Steps to test

How has the user experience changed?

PR Tasks

• Have tests been added/updated?
• [na] Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
• [na] Has a PR for user-facing changes been opened in cypress-documentation?
• [na] Have API changes been updated in the type definitions?

[cypress]

1 failed and 27 flaky tests on run #44310 ↗︎

1

26867

1281

0

27

Details:

Merge branch 'develop' into address-skipped-flake
Project: cypress	Commit: a4bee8ca96
Status: Failed	Duration: 19:33 💡
Started: Feb 22, 2023 3:13 PM	Ended: Feb 22, 2023 3:33 PM

  cypress/e2e/e2e/origin/commands/waiting.cy.ts • 1 failed test • 5x-driver-electron

View Output Video

Test
cy.origin waiting > alias > waits for the route alias to have a response

  commands/net_stubbing.cy.ts • 1 flaky test • 5x-driver-firefox

View Output Video

Test
network stubbing > intercepting request > can delay and throttle a StaticResponse

  e2e/origin/commands/assertions.cy.ts • 1 flaky test • 5x-driver-firefox

View Output Video

Test
cy.origin assertions > #consoleProps > .should() and .and()

  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-firefox

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry

  e2e/origin/navigation.cy.ts • 1 flaky test • 5x-driver-electron

View Output Video

Test
delayed navigation > errors > redirects to an unexpected cross-origin

  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-electron

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry

The first 5 flaky specs are shown, see all 15 specs in Cypress Cloud.

_{This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.}