Terrible slow builds when more than 1000 files in a module

[sznowicki]

Full explanation and background in original, ViteJS issue.

Long story short, when there is a bigger bundle, in my case, a design system monorepo that contains ~2600 icons (each as a React component, referenced in an index file that Maps them as ['name', () => import('../component/name')], rollup build takes in my case 46 minutes (while ParcelJS is much faster, ~30 seconds).

I tried to debug it and it seems like rollup graphs dependencies in a way that it creates loop iteration for each file and within that iteration it iterates over every files in a bundle, which seems to me like O(n^2) performance.

I've made a repro repo to demo the problem. You can compare there ViteJS that uses Rollup and Parceljs (that uses custom bundler I believe).

Also for reference, my findings in a ViteJS discussion thread.

rollup/src/Graph.ts

Line 165 in 27c0557

private async generateModuleGraph(): Promise<void> {

I believe this is worth looking, even though it's not that common problem because not many people have this big marketing iconography in their repos ;)

[lukastaegert]

Thanks for the analysis and the reproduction. I can confirm that we have an issue, but my own debugging revealed is actually not in generateModuleGraph but in a small, innocent-looking function makeUnique https://github.com/rollup/rollup/blob/master/src/utils/renderNamePattern.ts#L33-L33

This function is called for every chunk to check if the name for a new chunk would conflict with an existing one. Some time ago, a logic was added to ignore name casing. This logic first transformed all existing names to lower case, which is the O(n^2) problem you observed as with more than 1000 chunks, you easily have half a million toLowerCase() calls. I can confirm that removing that solves the problem.

I am currently working on a proper solution to fix this, though, while still preventing clashes on case-insensitive file systems.

[lukastaegert]

Fix at #4643