puppeteer 9.1.0 broken: "target should exist before targetInfoChanged"

[jribbens]

Steps to reproduce

Unfortunately I can't provide a test case. However I can say that 9.0.0 works every time, and 9.1.0 fails every time.

Tell us about your environment:

• Puppeteer version: 9.1.0
• Platform / OS version: Windows 10 or Ubuntu 20.04.2 or Amazon Linux 2
• URLs (if applicable):
• Node.js version: 15.14.0

What happens instead?

Puppeteer crashes with the following message:

.../node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26                                                                            
        throw new Error(message);                                               
              ^                         
                                        
Error: target should exist before targetInfoChanged
    at Object.assert (.../node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26:15)          
    at Browser._targetInfoChanged (.../node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:205:21)                                   
    at .../node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62                            
    at Array.map (<anonymous>)
    at Object.emit (.../node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)
    at Connection.emit (.../node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
    at Connection._onMessage (.../node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:115:18)
    at WebSocket.<anonymous> (.../node_modules/puppeteer/lib/cjs/puppeteer/node/NodeWebSocketTransport.js:13:32)
    at WebSocket.onMessage (.../node_modules/ws/lib/event-target.js:132:16)
    at WebSocket.emit (node:events:369:20)

[timbru31]

I can confirm this issue. Maybe that's related to the fact that the page does (via JS) a reload?

[AlonsoK28]

Same for me

....\node_modules\puppeteer\lib\cjs\puppeteer\common\assert.js:26
        throw new Error(message);
              ^
Error: target should exist before targetInfoChanged

Which version of Puppeteer works?

[matter13311]

I have the same issue. My program still works. It still successfully scrapes the urls that I inputted, however, my terminal gets spammed with the error:
UnhandledPromiseRejectionWarning: Error: target should exist before targetInfoChanged

[mathiasbynens]

@jschfflr Does this sound related to the target filter work?

[jschfflr]

Yes, this is related to my change and fixed in #7203.
I'll cut a new release now.

[jribbens]

Can confirm that 9.1.1 fixes this from my point of view.

[jschfflr]

@jribbens Thanks!

[matter13311]

Hello, sorry, I updated my puppeteer to the latest version, 9.1.1. And I still get the following error.

UnhandledPromiseRejectionWarning: Error: target should exist before targetInfoChanged

Followed by

UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 5)

I do have a try catch block though. Not sure why the error is happening. This is being run on a centOS system.

[jschfflr]

@matter13311 Thanks for your feedback! Unfortunately, I can no longer reproduce the problem on my machine. Could you share a minimal reproduction maybe so that I can make sure that your use case also gets fixed?

[matter13311]

Hello @jschfflr ,
I apologize for my extremely long response.
I am not sure how to share a minimal reproduction, but I can give you the entire error log that I see from the terminal

(node:22857) UnhandledPromiseRejectionWarning: Error: target should exist before targetInfoChanged
    at Object.assert (/home/node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26:15)
    at Browser._targetInfoChanged (/home/node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:205:21)
    at /home/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62
    at Array.map (<anonymous>)
    at Object.emit (/home/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)
    at Connection.emit (/home/node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
    at Connection._onMessage (/home/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:115:18)
    at WebSocket.<anonymous> (/home/node_modules/puppeteer/lib/cjs/puppeteer/node/NodeWebSocketTransport.js:13:32)
    at WebSocket.onMessage (/home/node_modules/ws/lib/event-target.js:132:16)
    at WebSocket.emit (events.js:314:20)
(node:22857) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 9)

I have a array of product URLS, that I want to visit, and I'm running them through a for loop.

//start of scraper
(async ()=>{
try{
    const browser = await puppeteer.launch({
      args: [
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--unhandled-rejections=strict'
      ]
    });

    const page = await browser.newPage();
    for(let i = 0; i< urlArray.length; i++){
      let randomInterval = getRndInteger(1000,5000); //generate a random number between 1000 and 5000 ms
      const url = urlArray[i];
      try{
        await page.goto(url, {waitUntil: 'networkidle0'}); //visit the URL and the networkidle2 makes sure everything is loaded before proceeding
        console.log("visiting: ", url);
        await page.waitForTimeout(randomInterval); //wait for the random time
        var data = await page.evaluate(()=>{   //evaluate the page you opened. like running code into console.
          var title = document.querySelector('h1[data-test="product-title"] > span').innerText;
          var price = document.querySelector('div[data-test="product-price"]').innerText;
          if (price != undefined){
            price = price.replace('$','');
          }
          let img = document.querySelector('div[class="slideDeckPicture"] > div > div > div > div > img').src;
          let pricePerUnit = document.querySelector('button[aria-atomic="true"]').innerText;
          if(!/\d/.test(pricePerUnit)){ //if pricePerUnit has no number, then skip it. 
            console.log("skipping: ", url);
          }else{
            if(pricePerUnit.includes("/")){//if the pricePerUnit has a /, then determine what's on the left or right of it.
              var weightSide= pricePerUnit.substr(0, pricePerUnit.indexOf('/')); 
              var unitSide = pricePerUnit.substring(pricePerUnit.indexOf("/") + 1);
              var weight = Number(weightSide.replace(/[^0-9\.]+/g,""));
              var unit = unitSide.replace(/[0-9]/g, '');

            }else{
              var weight = Number(pricePerUnit.replace(/[^0-9\.]+/g,""));
              var unit = pricePerUnit.replace(/[0-9]/g, '');
            }
            return{
              title,
              price,
              img,
              weight,
              unit
            }
          }
        }) //end up data.evaulate
        console.log("data", data);
        let sql = `UPDATE products set product_price = ${data.price}, product_image = '${data.img}', product_weight =${data.weight}, product_unit ='${data.unit}' where product_url = '${url}'`;
        connection.query(sql, (err , results)=>{
          console.log("sql query: ", sql);
        })
      }catch(e){
        console.log("Error inside loop: ", e);
      }
      
    }
  
    await browser.close();
  }catch(e){
    console.log("Error occurred: ", e);  
  }

  connection.end(function(){
    console.log("connection closed!");

});
})();

[jschfflr]

Could you verify that you have the latest version of puppeteer (>=9.1.1) installed?

[matter13311]

@jschfflr
Yeah, it is 9.1.1

"puppeteer": {
      "version": "9.1.1",
      "resolved": "https://registry.npmjs.org/puppeteer/-/puppeteer-9.1.1.tgz",
      "integrity": "sha512-W+nOulP2tYd/ZG99WuZC/I5ljjQQ7EUw/jQGcIb9eu8mDlZxNY2SgcJXTLG9h5gRvqA3uJOe4hZXYsd3EqioMw==",
      "requires": {
        "debug": "^4.1.0",
        "devtools-protocol": "0.0.869402",
        "extract-zip": "^2.0.0",
        "https-proxy-agent": "^5.0.0",
        "node-fetch": "^2.6.1",
        "pkg-dir": "^4.2.0",
        "progress": "^2.0.1",
        "proxy-from-env": "^1.1.0",
        "rimraf": "^3.0.2",
        "tar-fs": "^2.0.0",
        "unbzip2-stream": "^1.3.3",
        "ws": "^7.2.3"
      }
    },
    ```

[Anatoly-IVANOV]

@jschfflr and @matter13311 — Exact same issue here on MacOS… pops up randomly in the console / logs… in totally different contexts with different approaches to promises and try/catch (as separate blocks of awaits and as chained .then.catch ) on completely different pages… 😕

Puppeteer version 9.1.1.

(node:34839) UnhandledPromiseRejectionWarning: Error: target should exist before targetInfoChanged
    at Object.assert (node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26:15)
    at Browser._targetInfoChanged (node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:205:21)
    at node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62
    at Array.map (<anonymous>)
    at Object.emit (node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)
    at Connection.emit (node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
    at Connection._onMessage (node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:115:18)
    at WebSocket.<anonymous> (node_modules/puppeteer/lib/cjs/puppeteer/node/NodeWebSocketTransport.js:13:32)
    at WebSocket.onMessage (node_modules/puppeteernode_modules/ws/lib/event-target.js:132:16)
    at WebSocket.emit (events.js:315:20)
    at emitUnhandledRejectionWarning (internal/process/promises.js:168:15)
    at processPromiseRejections (internal/process/promises.js:224:9)
    at processTicksAndRejections (internal/process/task_queues.js:94:32)

[jschfflr]

Thanks for the reports!
Unfortunately, I still can't reproduce the problem - that makes it hard to address this problem.
Any chance you could come up with a reproduction?

[Anatoly-IVANOV]

@jschfflr — That’s precisely the problem… I can’t see a repeatable pattern either, which makes it hard to report something helpful (and I really want to help!) 🤗

I have tried to:

• force a total load of page on .goto ( waitUntil: 'load')
• not set additional user-agent attributes with .setUserAgent
• issue happens on completely different URLs, code bases… (we use Puppeteer for UI testing and automation)

No impact.

I have since downgraded to 9.0.0 and the issue disappeared. Never seen errors like that with any previous version of Puppeteer.

Is there anything I can do to backtrace this issue? Some extra params? Tests? If you have any ideas, I’d do my best.

[ucarbehlul]

I can reproduce the problem with below code:

      const puppeteer = require('puppeteer')

      const browser = await puppeteer.launch()
      const page = await browser.newPage()
      
      let html = null
      const link = "https://www.someurl.com"
      try {
        await page.setJavaScriptEnabled(true)
        await page.goto(link)
        await page.waitForFunction(`document.querySelector("script[type='application/ld+json']")`);
        html = await page.content()
      } finally {
        await page.close()
        await browser.close()
      }

note that someurl.com does a JS redirect.

[sleeper1023]

me too