Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test configuration options for running tests against multiple products #5964

Merged
merged 3 commits into from Jun 12, 2020
Merged

Add test configuration options for running tests against multiple products #5964

merged 3 commits into from Jun 12, 2020

Conversation

mjzffr
Copy link
Contributor

@mjzffr mjzffr commented Jun 2, 2020

This is in support of Mozilla's internal sync to Puppeteer 3.1 at https://bugzilla.mozilla.org/show_bug.cgi?id=1632710

Notably add itChromeOnly, itRegularInstallOnly

Also unskip more Firefox tests in Puppeteer CI.

@mjzffr
chore: Remove "Extracting..." log message
48702d6 
Fixes puppeteer#5741
@mjzffr
test: support extra Launcher options and skips
4d93527 
The extra Launcher options and skipping conditions enable
unit tests to be run more easily by third-parties, e.g.
browser vendors that are interested in Puppeteer support.

Extra Launcher options were previously removed as part of
switching away from the custom test harness.
@mjzffr mjzffr requested a review from jackfranklin June 2, 2020 20:58
mathiasbynens
mathiasbynens approved these changes Jun 4, 2020
View reviewed changes
Copy link
Member

@mathiasbynens mathiasbynens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excited about the many unskipped tests! LGTM, but @jackfranklin PTAL as well.

jackfranklin
jackfranklin reviewed Jun 4, 2020
View reviewed changes
src/BrowserFetcher.ts
@@ -415,9 +415,6 @@ function extractTar(tarPath: string, folderPath: string): Promise<unknown> {
tarStream.on('error', reject);
tarStream.on('finish', fulfill);
const readStream = fs.createReadStream(tarPath);
readStream.on('data', () => {
process.stdout.write('\rExtracting...');
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been meaning to get rid of this for so long, thank you!

jackfranklin
jackfranklin approved these changes Jun 4, 2020
View reviewed changes
Copy link
Collaborator

@jackfranklin jackfranklin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So exciting to see many more tests not skipped!

@mathiasbynens
Copy link
Member

mathiasbynens commented Jun 8, 2020 •
edited

This is failing on CI on macOS with the following:

  1) navigation
       Page.goto
         should fail when navigating to bad SSL after redirects:
     Error: expect(received).toContain(expected) // indexOf
Expected substring: "net::ERR_CERT_AUTHORITY_INVALID"
Received string:    "net::ERR_CERT_INVALID at https://localhost:8908/redirect/1.html"
      at Context.it (test/navigation.spec.js:183:31)
      at process._tickCallback (internal/process/next_tick.js:68:7)

@jackfranklin does Travis somehow get

puppeteer/test/navigation.spec.js

Lines 151 to 159 in 6cfe142

/* If you are running this on pre-Catalina versions of macOS this will fail locally.
/* Mac OSX Catalina outputs a different message than other platforms.
* See https://support.google.com/chrome/thread/18125056?hl=en for details.
* If you're running pre-Catalina Mac OSX this test will fail locally.
*/
const EXPECTED_SSL_CERT_MESSAGE =
os.platform() === 'darwin'
? 'net::ERR_CERT_INVALID'
: 'net::ERR_CERT_AUTHORITY_INVALID';
wrong? Any ideas what's going on here?

To resolve this, instead of the current check, we could just check for the presence of both the substrings 'ERR_CERT_' and '_INVALID'. WDYT?

jackfranklin
jackfranklin reviewed Jun 10, 2020
View reviewed changes
test/navigation.spec.js Outdated
.goto(httpsServer.PREFIX + '/redirect/1.html')
.catch((error_) => (error = error_));
if (isChrome)
expect(error.message).toContain('net::ERR_CERT_AUTHORITY_INVALID');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mjzffr I think you need to change the assertion here back to what it was - the EXPECTED_SSL_CERT_MESSAGE constant. See the comment further up this file about some slight oddness with Chrome + Mac versions that Travis hits vs what we get locally. Did you need to change this to get the test passing for you? There may well be a better way of dealing of this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change may just have been the result of a bad merge on my end.

@jackfranklin
Copy link
Collaborator

@mjzffr this looks good except you left the describe.only in - once that's gone I think this should be green and we can ship it.

@jackfranklin
Copy link
Collaborator

I think a webhook didn't fire here because the build is green on Travis but GH doesn't know about it...going to prod it to rebuild.

@mathiasbynens mathiasbynens merged this pull request into puppeteer:master Jun 12, 2020
mathiasbynens pushed a commit that referenced this pull request Jun 12, 2020
@mjzffr @mathiasbynens
chore: add test configuration options for running tests against multi…
3d56a9e 
…ple products (#5964)

* chore: remove "Extracting..." log message

Fixes #5741.

* test: support extra Launcher options and skips

The extra Launcher options and skipping conditions enable
unit tests to be run more easily by third-parties, e.g.
browser vendors that are interested in Puppeteer support.

Extra Launcher options were previously removed as part of
switching away from the custom test harness.

* test: enable more tests for Firefox
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jackfranklin jackfranklin jackfranklin approved these changes

@mathiasbynens mathiasbynens mathiasbynens approved these changes

Assignees
No one assigned
Labels
cla: yes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@mjzffr @mathiasbynens @jackfranklin @googlebot