Dependency hygiene

[feross]

I want to go through the whole webtorrent dependency tree to:

• De-duplicate dependencies
• Ensure that we're using the latest version
• Pruning dependencies that aren't needed in modern JS environments (to reduce risk and increase install speed)

This issue is a placeholder to track this task.

I'm tracking progress on this work in this issue: brave/brave-browser#5490

[feross]

One nice side effect of this effort. So far I've been able to reduce the bundle size by 10KB!

From 98338 bytes to 88915 bytes, so far.

Should get quite a bit smaller once browserify/browserify#1916 is merged.

[RecoX]

One nice side effect of this effort. So far I've been able to reduce the bundle size by 10KB!

From 98338 bytes to 88915 bytes, so far.

Should get quite a bit smaller once browserify/browserify#1916 is merged.

Is it worth all the effort put it on this to just reduce the bundle by 10kb?

Just asking because my lack of knowledge.

[feross]

Is it worth all the effort put it on this to just reduce the bundle by 10kb?

Depends on your priorities. Small size has always been one of my priorities with WebTorrent. It's one of the best parts of WebTorrent – that it's small and carefully selects dependencies. Lots of others decentralized web projects require 2MB+ of JavaScript and don't work nearly as well as WebTorrent does. The fact that we can fit a whole torrent client into <100KB is amazing.

Also, updating dependencies gives us bug fixes, wards off potential security issues, and makes installation faster.

[ExE-Boss]

I recommend using pnpm as the package manager, as it avoids “phantom dependency” and “npm doppelganger” problems by using symlinks, hardlinks and isolated trees.

“npm doppelgangers” in particular are a common‑ish cause of exploding bundle sizes.

[feross]

“phantom dependency”

This is a valid issue that can sometimes happen but I'll be adding a rule to standard to catch this in a future version and it doesn't seem to have anything to do with bundle size?

“npm doppelganger”

This is not a problem with browserify since it hashes every required file and never includes a duplicate.

[ThaUnknown]

I went hard with v2's dependency pruning, and once this or this happens we'll be down to 206KB from v1's 875KB which is a 75% reduction in bundle size, which is still a lot more than v0's <100KB, but the feature difference is immense

from looking at the dependency tree there isn't much more to prune, most of the biggest modules come from webtorrent's ecosystem itself, or from it's maintainers, in filesize order:

./index.js + 26 modules
./lib/torrent.js
./node_modules/bittorrent-protocol/index.js
./node_modules/@thaunknown/simple-peer/index.js
./node_modules/streamx/index.js
./node_modules/path-browserify/index.js
./node_modules/events/events.js
./lib/server.js
./node_modules/bittorrent-tracker/lib/client/websocket-tracker.js
./node_modules/create-torrent/index.js
./lib/peer.js
./node_modules/fsa-chunk-store/index.js

<10kb:

./node_modules/bencode/index.js + 4 modules
./node_modules/mime/types/standard.js
./node_modules/bittorrent-tracker/client.js
./node_modules/parse-torrent/index.js
./node_modules/clone/clone.js
./node_modules/@thaunknown/simple-websocket/index.js
./node_modules/ut_metadata/index.js
./node_modules/debug/src/common.js
./node_modules/torrent-discovery/index.js
./node_modules/debug/src/browser.js
./node_modules/magnet-uri/index.js
./node_modules/limiter/lib/tokenBucket.js
./node_modules/uint8-util/browser.js + 2 modules
./lib/file.js
./node_modules/limiter/lib/rateLimiter.js
./lib/webconn.js
./node_modules/rc4/rc4.js
./node_modules/@thaunknown/thirty-two/lib/thirty-two/index.js
./package.json
./node_modules/speed-limiter/lib/throttle.js
./node_modules/lru/index.js
./node_modules/bencode/lib/decode.js
./node_modules/bencode/lib/encode.js
./node_modules/debug/node_modules/ms/index.js
./lib/file-iterator.js
./node_modules/range-parser/index.js
./node_modules/mime/Mime.js
./lib/rarity-map.js
./node_modules/cache-chunk-store/index.js
./node_modules/torrent-piece/index.js
./node_modules/speed-limiter/lib/throttle-group.js
./node_modules/bitfield/lib/esm/index.js
./node_modules/chunk-store-iterator/index.js
./node_modules/memory-chunk-store/index.js
./node_modules/base64-arraybuffer/dist/base64-arraybuffer.es5.js
./node_modules/bencode/lib/encoding-length.js
./node_modules/run-parallel-limit/index.js
./node_modules/lt_donthave/index.js
./node_modules/uint8-util/util.js
./node_modules/err-code/index.js
./node_modules/escape-html/index.js
./node_modules/immediate-chunk-store/index.js
./node_modules/junk/index.js
./node_modules/bittorrent-tracker/lib/common.js
./node_modules/block-iterator/index.js

<1kb:

webpack/runtime/create fake namespace object
./polyfills/process-fast.js
./node_modules/run-parallel/index.js
./node_modules/throughput/index.js
./node_modules/once/once.js
./node_modules/wrappy/wrappy.js
./node_modules/fast-fifo/index.js
./node_modules/fast-fifo/fixed-size.js
./node_modules/bencode/lib/util.js
./node_modules/inherits/inherits_browser.js
./node_modules/bep53-range/index.js
./node_modules/get-browser-rtc/index.js
./node_modules/bittorrent-tracker/lib/client/tracker.js
./node_modules/fast-readable-async-iterator/index.js
./node_modules/addr-to-ip-port/index.js
./node_modules/cross-fetch-ponyfill/browser.js
./node_modules/queue-microtask/index.js
./node_modules/random-iterate/index.js
./node_modules/limiter/lib/clock.js
webpack/runtime/define property getters
webpack/runtime/make namespace object
./node_modules/cpus/browser.js
./node_modules/join-async-iterator/index.js
./node_modules/unordered-array-remove/index.js
./node_modules/bittorrent-tracker/index.js
./node_modules/speed-limiter/index.js
./node_modules/piece-length/index.js
./node_modules/speed-limiter/lib/utils.js
./node_modules/queue-tick/queue-microtask.js
./node_modules/limiter/index.js
./node_modules/mime/lite.js

once either of the above issues are fixed, we'll have dropped stream and all its sub dependencies, all stream utility modules, buffer and it's dependencies, there isnt much more tbh