fix: Suppress filesystem errors during glob search

[mike-plummer]

• Closes Cypress does not find any component testing specs when folder matching glob can not be accessed #24109

Additional details

Currently, our glob searches for project specs and components for "Create from Component" will fail if a filesystem error is encountered (such as a permission denied) and return an empty array. This can happen even if there's a deeply nested resource that has nothing to do with Cypress that is permissions-restricted or has a bad symlink. This PR updates the behavior to log and retry the glob search if an error is encountered so the user can at least get partial results even if there is a part of their specPattern that cannot be traversed.

Unfortunately there isn't an elegant way to "watch and log" in Globby. It only exposes a "suppress all errors" flag or we can roll our own filesystem adapter which felt heavy for this use case (and wrapping fs functions is tricky due to extensive use of callback args for error-handling). The best approach I could find was to try it once and, if an error is encountered, log and try again with errors suppressed. If the retry fails then we bail out with our current behavior of "log and return empty array".

Steps to test

1. Create a new Cypress CT project using React or Vue. Make sure it has at least one component and at least one spec file
2. Create a new directory within the project and remove read permission from it:
   mkdir secretDir
chmod -r secretDir
3. Launch latest Cypress version in CT open mode (npx cypress open --component), observe no specs are listed
4. Select "Create from Component", observe no components are listed
5. Close Cypress and re-open project using this branch with DEBUG=cypress:data-context:sources:FileDataSource
6. Observe that spec file is listed, retry "Create from Component" and observe component(s) are found
7. Observe that filesystem EACCES error was logged with comment regarding retry

Note: This issue also impacts e2e, instructions can be adapted to validate e2e is also impacted/fixed by creating restricted directory under cypress/e2e to match default e2e spec pattern

How has the user experience changed?

PR Tasks

• Have tests been added/updated?
• Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
• [na] Has a PR for user-facing changes been opened in cypress-documentation?
• [na] Have API changes been updated in the type definitions?

[cypress]

14 flaky tests on run #44050 ↗︎

0

9401

953

0

14

Details:

Merge branch 'develop' into mikep/24109-ignore-filesystem-errs
Project: cypress	Commit: 79e678b3d2
Status: Passed	Duration: 15:52 💡
Started: Feb 13, 2023 10:09 PM	Ended: Feb 13, 2023 10:25 PM

  e2e/origin/cookie_behavior.cy.ts • 4 flaky tests • 5x-driver-electron

View Output Video

Test
... > same site / cross origin > XMLHttpRequest > sets cookie on same-site request if withCredentials is true, and attaches to same-site request if withCredentials is true
... > same site / cross origin > fetch > sets same-site cookies if "include" credentials option is specified from request, but does not attach same-site cookies to request by default (same-origin)
... > same site / cross origin > XMLHttpRequest > sets cookie on same-site request if withCredentials is true, and attaches to same-site request if withCredentials is true
... > same site / cross origin > fetch > sets same-site cookies if "include" credentials option is specified from request, but does not attach same-site cookies to request by default (same-origin)

  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-electron

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry

  cypress/proxy-logging.cy.ts • 1 flaky test • 5x-driver-electron

View Output Video

Test
... > flags > req + res modified flagged as expected

  commands/net_stubbing.cy.ts • 3 flaky tests • 5x-driver-webkit

View Output Video

Test
network stubbing > intercepting request > can delay and throttle a StaticResponse
... > with `times` > only uses each handler N times
... > stops waiting when an xhr request is canceled

  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-webkit

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry

_{This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.}

[lmiller1990]

Should we just do this by default instead of the "if it fails, retry and suppress errors"? Any downside to that approach?

[mike-plummer]

That was my original plan, but if we do that we get zero logging of the problem. My fear is that a user will have a spec file within a restricted directory and that spec will just never show up and there will be no logging (debug or otherwise) explaining why which will make it very difficult to troubleshoot.

Taking the "retry with suppress" approach is less performant, but gives us the possibility of debug logs.

[marktnoonan]

Tested as described in the issue and this works great.

[marktnoonan]

@mike-plummer is it expected to still have problems running a spec that's now reachable? The spec is not inside the secretDir but its existence still appears to be a problem when parsing the support file...

At least this tells the user what the exact problem is so they can fix it.

[mike-plummer]

@mike-plummer is it expected to still have problems running a spec that's now reachable? The spec is not inside the secretDir but its existence still appears to be a problem when parsing the support file...

At least this tells the user what the exact problem is so they can fix it.

@marktnoonan Interesting, I must've only tested this on Webpack because I hadn't seen that issue before. This is intended only to fix the "no specs at all show up" issue in the Specs List. As you point out, the error is appropriately surfaced to the user in this scenario so I'm not concerned