Entire test suite exits without report when test references page object alias that does not exist in the page object

[reallymello]

Description of the bug/issue

When I run a batch of tests and one is mistakenly referencing a page object element alias that is not present in the page object being used I expect that one test to fail with error stating that, but have the rest of the run continue to completion and produce a test report, but instead the test run exits early and no final test report or output is produced

Steps to reproduce

Possibly useful background, our total test suite is over 50 tests in the one project with each test being over 600 lines. They make heavy use of the page object model

1. Run a large suite of tests
2. In one of the tests in the middle reference a page object such as myPageObject.element('@thisDoesNotExist').click() where @thisDoesNotExist does not exist in the elements collection for your page object.
3. Run the entire batch of tests in parallel using npx nightwatch --headless
4. The tests will run, but usually shortly after execution comes across the test with the above mistake the tests stop executing and you return to console without a test report and no real way of seeing what happened.

When I disabled parallel execution I was able to see the error from Nightwatch alerting me that the alias does not exist on my page object and it lists what element do exist, but then the console output after that appears to continue on with a bit more test execution before it terminates.

Sample test

No response

Command to run

npx nightwatch --headless

Verbose Output

No response

Nightwatch Configuration

No response

Nightwatch.js Version

3.3.8

Node Version

18.15.0

Browser

Chrome Latest

Operating System

Windows 11

Additional Information

No response

[garg3133]

@reallymello Thanks for this issue.

#4012 should take care of the unhandledRejection and uncaughtException in the worker threads which was causing the Nightwatch parallel test run to suddenly stop in between with no errors or any other information.

But I'm not sure if using the wrong alias is causing that uncaught exception in your case because I tried the steps you mentioned above and Nightwatch is giving me information on what went wrong, even in parallel runs:

Can you maybe try running your tests with #4012 once to see if it can help you find the actual issue? Also, please check if the HTML report is working fine after encountering such an uncaught exception in a worker thread.

[reallymello]

Hi @garg3133 any update on #4012 or this? It is causing major issues for us causing our test runs not to report results when unhandled errors happen in the suite for one reason or another.

[AutomatedTester]

I have added #4187 so I am not sure that #4012 is the reason so we will need to look into this further