Dependency issue: outdated version of puppeteer and vulnerable versions of node-fetch

[gunesacar]

Hi!

Installing Tracker Radar Collector results in several warnings and high severity vulnerabilities per npm audit (on Ubuntu 22.04, npm 8.11.0, node v16.15.1).

It appears that the outdated puppeteer is the culprit:

tracker-radar-collector/package-lock.json

Line 18 in 8e43a27

"puppeteer": "^10.2.0",

$ git clone git@github.com:duckduckgo/tracker-radar-collector.git
...

$ git log --oneline -1
8e43a27 (HEAD -> main, origin/main, origin/HEAD) Expand CH collection (#83)

$ npm i
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.0: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated puppeteer@10.2.0: Version no longer supported. Upgrade to @latest

added 299 packages, and audited 300 packages in 21s

36 packages are looking for funding
  run `npm fund` for details

2 high severity vulnerabilities

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.

$ npm audit
# npm audit report

node-fetch  <2.6.7
Severity: high
node-fetch is vulnerable to Exposure of Sensitive Information to an Unauthorized Actor - https://github.com/advisories/GHSA-r683-j2x4-v87g
fix available via `npm audit fix --force`
Will install puppeteer@19.4.1, which is a breaking change
node_modules/node-fetch
  puppeteer  10.0.0 - 13.1.1
  Depends on vulnerable versions of node-fetch
  node_modules/puppeteer

2 high severity vulnerabilities

To address all issues (including breaking changes), run:
  npm audit fix --force

Let me know if you need more logs to help with the issue. Thanks already!

[muodov]

Hey @gunesacar, thanks for reporting this!
Updating puppeteer is definitely useful, at least because it would also update the underlying Chromium version.
Theoretically, updating puppeteer should not cause issues, but tracker-radar-collector is sending raw CDP commands, and there's a chance that some of them have changed, so we'd like to check that the upgrade doesn't break the logic.
Until that, you are most welcome to upgrade to the latest puppeteer, and report any problems if you see any!

[muodov]

@gunesacar just checking in here. It turned out that our current usage of raw CDP is incompatible with the latest puppeteer after all. puppeteer started using autoAttach, which conflicts with tracker-radar-collector's usage of autoAttach and negatively affects the data collection. This is not so straightforward to fix, and we're considering different options at the moment.

In the meantime, you can still use --chromium-version to run the tracker-radar-collector with a newer browser version. That seems to work well.

[gunesacar]

Thank you so much @muodov . Now it's more clear why you kept the older versions.

Good luck with the autoAttach issue.