Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test configuration options for running tests against multiple products #5964

Merged
merged 3 commits into from Jun 12, 2020
Merged

Add test configuration options for running tests against multiple products #5964

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
48702d6
chore: Remove "Extracting..." log message
mjzffr May 31, 2020
4d93527
test: support extra Launcher options and skips
mjzffr May 31, 2020
42897bc
test: enable more tests for Firefox
mjzffr Jun 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 0 additions & 3 deletions src/BrowserFetcher.ts
View file
Open in desktop
Expand Up @@ -415,9 +415,6 @@ function extractTar(tarPath: string, folderPath: string): Promise<unknown> {
tarStream.on('error', reject);
tarStream.on('finish', fulfill);
const readStream = fs.createReadStream(tarPath);
readStream.on('data', () => {
process.stdout.write('\rExtracting...');
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been meaning to get rid of this for so long, thank you!

readStream.pipe(bzip()).pipe(tarStream);
});
}
Expand Down
7 changes: 5 additions & 2 deletions test/README.md
View file
Open in desktop
Expand Up @@ -24,11 +24,14 @@ If your test needs a Puppeteer page and context, you can use the `setupTestPageA

The best place to look is an existing test to see how they use the helpers.

## Skipping tests for Firefox
## Skipping tests in specific conditions

Tests that are not expected to pass in Firefox can be skipped. You can skip an individual test by using `itFailsFirefox` rather than `it`. Similarly you can skip a describe block with `describeFailsFirefox`.

There is also `describeChromeOnly` which will only execute the test if running in Chromium. Note that this is different from `describeFailsFirefox`: the goal is to get any `FailsFirefox` calls passing in Firefox, whereas `describeChromeOnly` should be used to test behaviour that will only ever apply in Chromium.
There is also `describeChromeOnly` and `itChromeOnly` which will only execute the test if running in Chromium. Note that this is different from `describeFailsFirefox`: the goal is to get any `FailsFirefox` calls passing in Firefox, whereas `describeChromeOnly` should be used to test behaviour that will only ever apply in Chromium.

There are also tests that assume a normal install flow, with browser binaries ending up in `.local-<browser>`, for example. Such tests are skipped with
`itOnlyRegularInstall` which checks `BINARY` and `PUPPETEER_ALT_INSTALL` environment variables.

[Mocha]: https://mochajs.org/
[Expect]: https://www.npmjs.com/package/expect
Expand Down
33 changes: 15 additions & 18 deletions test/browsercontext.spec.js
View file
Open in desktop
Expand Up @@ -20,7 +20,7 @@ const utils = require('./utils');

describe('BrowserContext', function () {
setupTestBrowserHooks();
itFailsFirefox('should have default context', async () => {
it('should have default context', async () => {
const { browser } = getTestState();
expect(browser.browserContexts().length).toEqual(1);
const defaultContext = browser.browserContexts()[0];
Expand All @@ -30,7 +30,7 @@ describe('BrowserContext', function () {
expect(browser.defaultBrowserContext()).toBe(defaultContext);
expect(error.message).toContain('cannot be closed');
});
itFailsFirefox('should create new incognito context', async () => {
it('should create new incognito context', async () => {
const { browser } = getTestState();

expect(browser.browserContexts().length).toBe(1);
Expand All @@ -41,22 +41,19 @@ describe('BrowserContext', function () {
await context.close();
expect(browser.browserContexts().length).toBe(1);
});
itFailsFirefox(
'should close all belonging targets once closing context',
async () => {
const { browser } = getTestState();

expect((await browser.pages()).length).toBe(1);

const context = await browser.createIncognitoBrowserContext();
await context.newPage();
expect((await browser.pages()).length).toBe(2);
expect((await context.pages()).length).toBe(1);

await context.close();
expect((await browser.pages()).length).toBe(1);
}
);
it('should close all belonging targets once closing context', async () => {
const { browser } = getTestState();

expect((await browser.pages()).length).toBe(1);

const context = await browser.createIncognitoBrowserContext();
await context.newPage();
expect((await browser.pages()).length).toBe(2);
expect((await context.pages()).length).toBe(1);

await context.close();
expect((await browser.pages()).length).toBe(1);
});
itFailsFirefox('window.open should use parent tab context', async () => {
const { browser, server } = getTestState();

Expand Down
36 changes: 15 additions & 21 deletions test/click.spec.js
View file
Open in desktop
Expand Up @@ -32,7 +32,7 @@ describe('Page.click', function () {
await page.click('button');
expect(await page.evaluate(() => result)).toBe('Clicked');
});
itFailsFirefox('should click svg', async () => {
it('should click svg', async () => {
const { page } = getTestState();

await page.setContent(`
Expand All @@ -55,23 +55,20 @@ describe('Page.click', function () {
}
);
// @see https://github.com/puppeteer/puppeteer/issues/4281
itFailsFirefox(
'should click on a span with an inline element inside',
async () => {
const { page } = getTestState();
it('should click on a span with an inline element inside', async () => {
const { page } = getTestState();

await page.setContent(`
await page.setContent(`
<style>
span::before {
content: 'q';
}
</style>
<span onclick='javascript:window.CLICKED=42'></span>
`);
await page.click('span');
expect(await page.evaluate(() => window.CLICKED)).toBe(42);
}
);
await page.click('span');
expect(await page.evaluate(() => window.CLICKED)).toBe(42);
});
it('should not throw UnhandledPromiseRejection when page closes', async () => {
const { page } = getTestState();

Expand All @@ -98,12 +95,10 @@ describe('Page.click', function () {
await Promise.all([page.click('a'), page.waitForNavigation()]);
expect(page.url()).toBe(server.PREFIX + '/wrappedlink.html#clicked');
});
itFailsFirefox(
'should click when one of inline box children is outside of viewport',
async () => {
const { page } = getTestState();
it('should click when one of inline box children is outside of viewport', async () => {
const { page } = getTestState();

await page.setContent(`
await page.setContent(`
<style>
i {
position: absolute;
Expand All @@ -112,10 +107,9 @@ describe('Page.click', function () {
</style>
<span onclick='javascript:window.CLICKED = 42;'><i>woof</i><b>doggo</b></span>
`);
await page.click('span');
expect(await page.evaluate(() => window.CLICKED)).toBe(42);
}
);
await page.click('span');
expect(await page.evaluate(() => window.CLICKED)).toBe(42);
});
it('should select the text by triple clicking', async () => {
const { page, server } = getTestState();

Expand Down Expand Up @@ -244,7 +238,7 @@ describe('Page.click', function () {
)
).toBe('clicked');
});
itFailsFirefox('should double click the button', async () => {
it('should double click the button', async () => {
const { page, server } = getTestState();

await page.goto(server.PREFIX + '/input/button.html');
Expand Down Expand Up @@ -290,7 +284,7 @@ describe('Page.click', function () {
).toBe('context menu');
});
// @see https://github.com/puppeteer/puppeteer/issues/206
itFailsFirefox('should click links which cause navigation', async () => {
it('should click links which cause navigation', async () => {
const { page, server } = getTestState();

await page.setContent(`<a href="${server.EMPTY_PAGE}">empty.html</a>`);
Expand Down
52 changes: 23 additions & 29 deletions test/cookies.spec.js
View file
Open in desktop
Expand Up @@ -220,21 +220,18 @@ describe('Cookie specs', () => {
})
).toEqual(['foo=bar', 'password=123456']);
});
itFailsFirefox(
'should have |expires| set to |-1| for session cookies',
async () => {
const { page, server } = getTestState();
it('should have |expires| set to |-1| for session cookies', async () => {
const { page, server } = getTestState();

await page.goto(server.EMPTY_PAGE);
await page.setCookie({
name: 'password',
value: '123456',
});
const cookies = await page.cookies();
expect(cookies[0].session).toBe(true);
expect(cookies[0].expires).toBe(-1);
}
);
await page.goto(server.EMPTY_PAGE);
await page.setCookie({
name: 'password',
value: '123456',
});
const cookies = await page.cookies();
expect(cookies[0].session).toBe(true);
expect(cookies[0].expires).toBe(-1);
});
itFailsFirefox('should set cookie with reasonable defaults', async () => {
const { page, server } = getTestState();

Expand Down Expand Up @@ -348,22 +345,19 @@ describe('Cookie specs', () => {
expect(cookie.secure).toBe(true);
}
);
itFailsFirefox(
'should be able to set unsecure cookie for HTTP website',
async () => {
const { page, server } = getTestState();
it('should be able to set unsecure cookie for HTTP website', async () => {
const { page, server } = getTestState();

await page.goto(server.EMPTY_PAGE);
const HTTP_URL = 'http://example.com';
await page.setCookie({
url: HTTP_URL,
name: 'foo',
value: 'bar',
});
const [cookie] = await page.cookies(HTTP_URL);
expect(cookie.secure).toBe(false);
}
);
await page.goto(server.EMPTY_PAGE);
const HTTP_URL = 'http://example.com';
await page.setCookie({
url: HTTP_URL,
name: 'foo',
value: 'bar',
});
const [cookie] = await page.cookies(HTTP_URL);
expect(cookie.secure).toBe(false);
});
itFailsFirefox('should set a cookie on a different domain', async () => {
const { page, server } = getTestState();

Expand Down
2 changes: 1 addition & 1 deletion test/elementhandle.spec.js
View file
Open in desktop
Expand Up @@ -233,7 +233,7 @@ describe('ElementHandle specs', function () {
'Node is either not visible or not an HTMLElement'
);
});
itFailsFirefox('should throw for <br> elements', async () => {
it('should throw for <br> elements', async () => {
const { page } = getTestState();

await page.setContent('hello<br>goodbye');
Expand Down
2 changes: 1 addition & 1 deletion test/emulation.spec.js
View file
Open in desktop
Expand Up @@ -127,7 +127,7 @@ describe('Emulation', () => {
'iPhone'
);
});
it('should support clicking', async () => {
itFailsFirefox('should support clicking', async () => {
const { page, server } = getTestState();

await page.emulate(iPhone);
Expand Down
66 changes: 30 additions & 36 deletions test/evaluation.spec.js
View file
Open in desktop
Expand Up @@ -35,31 +35,31 @@ describe('Evaluation specs', function () {
const result = await page.evaluate(() => 7 * 3);
expect(result).toBe(21);
});
(bigint ? itFailsFirefox : xit)('should transfer BigInt', async () => {
(bigint ? it : xit)('should transfer BigInt', async () => {
const { page } = getTestState();

const result = await page.evaluate((a) => a, BigInt(42));
expect(result).toBe(BigInt(42));
});
itFailsFirefox('should transfer NaN', async () => {
it('should transfer NaN', async () => {
const { page } = getTestState();

const result = await page.evaluate((a) => a, NaN);
expect(Object.is(result, NaN)).toBe(true);
});
itFailsFirefox('should transfer -0', async () => {
it('should transfer -0', async () => {
const { page } = getTestState();

const result = await page.evaluate((a) => a, -0);
expect(Object.is(result, -0)).toBe(true);
});
itFailsFirefox('should transfer Infinity', async () => {
it('should transfer Infinity', async () => {
const { page } = getTestState();

const result = await page.evaluate((a) => a, Infinity);
expect(Object.is(result, Infinity)).toBe(true);
});
itFailsFirefox('should transfer -Infinity', async () => {
it('should transfer -Infinity', async () => {
const { page } = getTestState();

const result = await page.evaluate((a) => a, -Infinity);
Expand Down Expand Up @@ -202,31 +202,31 @@ describe('Evaluation specs', function () {
expect(result).not.toBe(object);
expect(result).toEqual(object);
});
(bigint ? itFailsFirefox : xit)('should return BigInt', async () => {
(bigint ? it : xit)('should return BigInt', async () => {
const { page } = getTestState();

const result = await page.evaluate(() => BigInt(42));
expect(result).toBe(BigInt(42));
});
itFailsFirefox('should return NaN', async () => {
it('should return NaN', async () => {
const { page } = getTestState();

const result = await page.evaluate(() => NaN);
expect(Object.is(result, NaN)).toBe(true);
});
itFailsFirefox('should return -0', async () => {
it('should return -0', async () => {
const { page } = getTestState();

const result = await page.evaluate(() => -0);
expect(Object.is(result, -0)).toBe(true);
});
itFailsFirefox('should return Infinity', async () => {
it('should return Infinity', async () => {
const { page } = getTestState();

const result = await page.evaluate(() => Infinity);
expect(Object.is(result, Infinity)).toBe(true);
});
itFailsFirefox('should return -Infinity', async () => {
it('should return -Infinity', async () => {
const { page } = getTestState();

const result = await page.evaluate(() => -Infinity);
Expand Down Expand Up @@ -298,30 +298,27 @@ describe('Evaluation specs', function () {
const result = await page.evaluate('2 + 5;\n// do some math!');
expect(result).toBe(7);
});
itFailsFirefox('should accept element handle as an argument', async () => {
it('should accept element handle as an argument', async () => {
const { page } = getTestState();

await page.setContent('<section>42</section>');
const element = await page.$('section');
const text = await page.evaluate((e) => e.textContent, element);
expect(text).toBe('42');
});
itFailsFirefox(
'should throw if underlying element was disposed',
async () => {
const { page } = getTestState();
it('should throw if underlying element was disposed', async () => {
const { page } = getTestState();

await page.setContent('<section>39</section>');
const element = await page.$('section');
expect(element).toBeTruthy();
await element.dispose();
let error = null;
await page
.evaluate((e) => e.textContent, element)
.catch((error_) => (error = error_));
expect(error.message).toContain('JSHandle is disposed');
}
);
await page.setContent('<section>39</section>');
const element = await page.$('section');
expect(element).toBeTruthy();
await element.dispose();
let error = null;
await page
.evaluate((e) => e.textContent, element)
.catch((error_) => (error = error_));
expect(error.message).toContain('JSHandle is disposed');
});
itFailsFirefox(
'should throw if elementHandles are from other frames',
async () => {
Expand Down Expand Up @@ -376,17 +373,14 @@ describe('Evaluation specs', function () {
expect(result).toEqual([42]);
}
);
itFailsFirefox(
'should transfer 100Mb of data from page to node.js',
async function () {
const { page } = getTestState();
it('should transfer 100Mb of data from page to node.js', async function () {
const { page } = getTestState();

const a = await page.evaluate(() =>
Array(100 * 1024 * 1024 + 1).join('a')
);
expect(a.length).toBe(100 * 1024 * 1024);
}
);
const a = await page.evaluate(() =>
Array(100 * 1024 * 1024 + 1).join('a')
);
expect(a.length).toBe(100 * 1024 * 1024);
});
it('should throw error with detailed information on exception inside promise ', async () => {
const { page } = getTestState();

Expand Down