puppeteer randomly fails after around 40 seconds

[NikolaiT]

Thanks for providing the most recent binaries. Hugely appreciated. Located in Neukölln by the way ;)

I have an issue with the latest version of this module and puppeteer:

    "chrome-aws-lambda": "^1.12.1",
    "puppeteer": "^1.12.2",

After around 30 to 60 seconds in my puppeteer function, let page = await browser.newPage(); fails with an error:

2019-03-03T16:25:49.045Z	53f2d87e-153c-40b9-a64b-6502e7db59f7	{ Error: Protocol error (Page.createIsolatedWorld): Could not create isolated world
at Promise (/var/task/node_modules/puppeteer/lib/Connection.js:183:56)
at new Promise (<anonymous>)
at CDPSession.send (/var/task/node_modules/puppeteer/lib/Connection.js:182:12)
at Promise.all.frames.map.frame (/var/task/node_modules/puppeteer/lib/FrameManager.js:259:63)
at Array.map (<anonymous>)
at FrameManager._ensureIsolatedWorld (/var/task/node_modules/puppeteer/lib/FrameManager.js:259:37)
at <anonymous>
at process._tickDomainCallback (internal/process/next_tick.js:228:7)
-- ASYNC --
at Target.<anonymous> (/var/task/node_modules/puppeteer/lib/helper.js:108:27)
at Browser._createPageInContext (/var/task/node_modules/puppeteer/lib/Browser.js:177:31)
at <anonymous>
at process._tickDomainCallback (internal/process/next_tick.js:228:7)
-- ASYNC --
at Browser.<anonymous> (/var/task/node_modules/puppeteer/lib/helper.js:108:27)
at Object.somefunction (/var/task/node_modules/somemodule/index.js:68:38)
at <anonymous>
at process._tickDomainCallback (internal/process/next_tick.js:228:7)
message: 'Protocol error (Page.createIsolatedWorld): Could not create isolated world' }

I use

memorySize: 1600
timeout: 200

for my AWS lambda settings. so memory and timeout cannot be an issue.

Any ideas? I think it must be puppeteer related or chrome flags related. I use the following flags to the puppeteer constructor (they are mostly what you use in this module):

            this.browser = await puppeteer.launch({
                args: pupArgs,
                defaultViewport: chromium.defaultViewport,
                executablePath: execPath,
                headless: chromium.headless,
                ignoreHTTPSErrors: true,
            });

            if (this.config.debug) {
                console.log(`browser ${execPath} successfully launched.`);
                console.log(pupArgs);
            }

let pupArgs=
[ '--disable-accelerated-2d-canvas',
     '--disable-background-timer-throttling',
     '--disable-breakpad',
     '--disable-client-side-phishing-detection',
     '--disable-cloud-import',
     '--disable-default-apps',
     '--disable-dev-shm-usage',
     '--disable-extensions',
     '--disable-gesture-typing',
     '--disable-gpu',
     '--disable-hang-monitor',
     '--disable-infobars',
     '--disable-notifications',
     '--disable-offer-store-unmasked-wallet-cards',
     '--disable-offer-upload-credit-cards',
     '--disable-popup-blocking',
     '--disable-print-preview',
     '--disable-prompt-on-repost',
     '--disable-setuid-sandbox',
     '--disable-software-rasterizer',
     '--disable-speech-api',
     '--disable-sync',
     '--disable-tab-for-desktop-share',
     '--disable-translate',
     '--disable-voice-input',
     '--disable-wake-on-wifi',
     '--enable-async-dns',
     '--enable-simple-cache-backend',
     '--enable-tcp-fast-open',
     '--hide-scrollbars',
     '--media-cache-size=33554432',
     '--metrics-recording-only',
     '--mute-audio',
     '--no-default-browser-check',
     '--no-first-run',
     '--no-pings',
     '--no-sandbox',
     '--no-zygote',
     '--password-store=basic',
     '--prerender-from-omnibox=disabled',
     '--use-mock-keychain',
     '--single-process',
     '--window-position=0,0',
     '--ignore-certifcate-errors',
     '--ignore-certifcate-errors-spki-list' ],

[NikolaiT]

Edit:

I think this issue is related to: puppeteer/puppeteer#3944

I will try with puppeteer 1.11.0

npm i puppeteer-core@1.11.0

[alixaxel]

This is really interesting @NikolaiT, seems like PPTR 1.12.x is a bit problematic... 😕

Let me know how it goes with 1.11.

[NikolaiT]

I used puppeteer-core@1.11.0 to scrape 20.000 urls in the AWS Lambda cluster and it worked fine. No random timeouts or puppeteer hanging when creating a new js world upon page.goto() or browser.newPage() calls.

This bug seems to be fixed in puppeteer 1.13.0 released 12 hours ago

I will test it and report to you boys

[alixaxel]

@NikolaiT I released 1.13.0 binary today.

[alixaxel]

Closing due to lack of feedback.