expecting an array or an iterable object but got [object Null] using playwright-webkit

[JoanEsquivel]

Current behavior

I am experiencing a couple of errors just using the new browser experimental feature.

• TypeError
  expecting an array or an iterable object but got [object Null] Because this error occurred during a before each hook we are skipping all of the remaining tests.

• TypeError
  expecting an array or an iterable object but got [object Null]

Desired behavior

Should run my tests without problems.

Test code to reproduce

https://github.com/JoanEsquivel/cypress-webkit

1. Just install dependencies.
2. Open the test runner
3. Run for example:

• cypress/e2e/1-helloworld.ct.ts
• cypress/e2e/10-checkboxes.cy

Cypress Version

10.8.0

Node version

v16.13.0

Operating System

Windows 10

Debug Logs

TypeError
expecting an array or an iterable object but got [object Null]
View stack trace
 Print to console
TypeError: expecting an array or an iterable object but got [object Null]
    automation@http://uitestingplayground.com/__cypress/runner/cypress_runner.js:153782:63
    @[native code]

Other

I want to say that this could be my mistake doing a wrong usage of typescript or some library. However, it is weird that this issue is only happening for webkit.

Also I see the same error from "bestu" in the discord channel General/general.

Appreciate your help.

[flotwig]

Hi @JoanEsquivel, this is because experimentalSessionAndOrigin is not currently compatible with experimentalWebKitSupport: We are working on it: #23799

However, this user experience is not intended. You should get a cleaner error. I'll keep this issue open to track cleaning up this error.

[JoanEsquivel]

Thanks for the quick response and help.

[nagash77]

@JoanEsquivel I am going to close this ticket as expected behavior until the referenced work is complete. Feel free to follow along with #23799 for updates as to when this issue will be resolved.

[Elecash]

@JoanEsquivel @nagash77 I'm also getting the same error although I'm not using experimentalSessionAndOrigin. Could this be a different error?

I don't have a repo to reproduce the error, but if needed I might try to build one if needed.

Btw, this error is not consistent for me, sometimes it does fails, sometimes it doesn't.

[flotwig]

@nagash77 This issue should be kept open, we can give a clearer error here easily, #23799 will take more time.

[flotwig]

@Elecash if you could put together a repro, that would be most helpful. I haven't seen this error with WebKit outside of experimentalSessionAndOrigin.

[rbknajera]

I'm seeing the same error and I'm not using experimentalSessionAndOrigin
I see it intermittently and I believe it's caused by custom commands, I commented out a line containing the custom command and it worked, but then the next test fails with the same error
expecting an array or an iterable object but got [object Null] Because this error occurred during a before each hook we are skipping all of the remaining tests.

Update:
Moving the same custom command to inside the test (instead the beforeEach) works with no issues.

[Elecash]

@flotwig we're still determining if this is a flaky test or a problem in cypress, we'll provide more information as soon as we know.

[BryanEdClark]

Seeing this issue as well without having any other experimental state other than webkit.

I am seeing odd behavior when I turned off retry logic, it is working exactly 50% of the time alternating, First launch - no error - 2nd run error - 3rd no error - 4th error

Edit-

Looks like this error is based on certain commands, I created 6 tests in a super basic suite when the failure occurs 3 will pass and 3 will fail.

Note: Below will only give the 'before' hook error if one of the failing commands is in a before otherwise it will fail at the test level with the same error message 'expecting an array or an iterable object but got [object Null]'

Commands that cause the tests to give object Null error: (3 tests fail 3 tests pass)
cy.visit('/');
cy.document().then((window) => { document.cookie = 'a'; }); - Note: Empty cookie works calling cy.document() is also itself fine.

Commands that work: (All 6 tests passed)
cy.log('Something here');
cy.get(''); [note: fails to find, but doesnt give the immediate null error]

Will update this comment as I continue to find a pattern on which commands cause the error, current theory is anything that access a global variable (ex. baseURL from visit)

Edit 2 - 'AfterEach' theory

So If I run the test fresh in what should cause a failure (ex. cy.visit('/')) running only a single test it passes, if I then update the test to be in a passing state (completely blank 0 actions) the test will fail, so there is a 100% fail rate regardless of commands passed after one of the bad code sets is given in.

This makes me believe it's something happening in a hidden After method that is then putting the test into a bad state that causes the next test to fail. However I am not sure on how the above 'bad' commands are somehow causing this.

[flotwig]

This issue has been fixed in #23442. Please look out for this fix in the next release of Cypress (October 25th)

[cypress-bot]

Released in 10.11.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v10.11.0, please open a new issue.