Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can code splitting be done according to chunk size? #4327

Closed
sanyuan0704 opened this issue Dec 29, 2021 · 50 comments · Fixed by #4705
Closed

Can code splitting be done according to chunk size? #4327

sanyuan0704 opened this issue Dec 29, 2021 · 50 comments · Fixed by #4705

Comments

@sanyuan0704
Copy link

Similar to the maxSize configuration of webpack, can you split different chunks according to the chunk size in manualChunks? For example, the final total size is 500 KB, can it be automatically split into vendor1.js, vendor2.js, vendor3.js, and the size of each vendor does not exceed 200KB?

@sanyuan0704
Copy link
Author

I have tried to accumulate chunk size in manualChunks, and return a new name when the threshold is exceeded. But the final chunk result did not meet expectations, and most of the chunk content was severely reduced because it was processed by tree shaking.

@gajus
Copy link

gajus commented Jul 29, 2022

Similar issue. Described in https://stackoverflow.com/q/73161429/368691

I am using Rollup with Vite and vite-plugin-ssr.

The production build it produces with standard configuration includes a ton of tiny files, e.g.

dist/client/assets/chunk-adbd3755.js                                                            0.69 KiB / gzip: 0.43 KiB
dist/client/assets/chunk-adbd3755.js.map                                                        0.10 KiB
dist/client/assets/chunk-d703547b.js                                                            2.35 KiB / gzip: 1.15 KiB
dist/client/assets/chunk-d703547b.js.map                                                        7.79 KiB
dist/client/assets/chunk-860778e7.js                                                            0.95 KiB / gzip: 0.51 KiB
dist/client/assets/chunk-860778e7.js.map                                                        3.26 KiB
dist/client/assets/chunk-90693398.js                                                            2.37 KiB / gzip: 0.94 KiB
dist/client/assets/chunk-90693398.js.map                                                        10.18 KiB
dist/client/assets/chunk-a3b7c495.js                                                            2.21 KiB / gzip: 1.04 KiB
dist/client/assets/chunk-a3b7c495.js.map                                                        5.33 KiB
dist/client/assets/chunk-d321b675.js                                                            0.67 KiB / gzip: 0.46 KiB
dist/client/assets/chunk-d321b675.js.map                                                        4.34 KiB
dist/client/assets/chunk-c5fff223.js                                                            0.36 KiB / gzip: 0.23 KiB
dist/client/assets/chunk-c5fff223.js.map                                                        1.17 KiB
dist/client/assets/chunk-93f6d1e0.js                                                            2.42 KiB / gzip: 0.93 KiB
dist/client/assets/chunk-93f6d1e0.js.map                                                        6.81 KiB
dist/client/assets/chunk-84638231.js                                                            0.90 KiB / gzip: 0.52 KiB
dist/client/assets/chunk-84638231.js.map                                                        2.43 KiB
dist/client/assets/chunk-7b2a7fc5.js                                                            16.33 KiB / gzip: 3.33 KiB

I am assuming this is determined by Rollup configuration, and I would like to force rollup to target producing chunks that are at least 5KiB in size, i.e. if there are multiple small chunks, they should be put in the same file.

@lukastaegert any advice around how to improve this?

Example website https://contra.com/gajus

@gajus
Copy link

gajus commented Jul 29, 2022

CC @brillout

@brillout
Copy link

https://contra.com/gajus

On that website it's 175 chunks. The one's I inspected are all small. It seems that something is wrong with the code splitting heuristic.

@lukastaegert
Copy link
Member

lukastaegert commented Jul 30, 2022

The problem is not simple. In order to reduce chunk size, Rollup would need to run modules that are not needed yet, so we would need to extend the chunking with a logic to determine when it is "safe" to do that, i.e. running the module has not side effects. Which is entirely possible, of course. Another point is to determine the chunk size contribution of a module. To do that, we need to do something like a "trial rendering" of the module, + some fake minification, which would be costly. Still something we could do, it just amounts to a lot of work which is why I put this lower on my agenda. But I am aware of the problem.
The question is what creates the amount of chunks, is it just dynamic imports? Or is it a number of static entries?

For dynamic imports, Rollup already has an optimization for the reduction of the number of chunks, which is to take into account that one of the dynamic importers must already be in memory before the imported module is run. So any code shared between the importer and the imported module is merged into the chunk of the importer (if it is a single one, or into a chunk where the dependencies shared between all importers are placed).

If you know a static entry is already in memory when another entry is loaded, you can encode the same relationship via implicitlyLoadedAfterOneOf when emitting the entry. And of course you can manually merge stuff via manualChunks.

@gajus
Copy link

gajus commented Aug 7, 2022

The question is what creates the amount of chunks, is it just dynamic imports? Or is it a number of static entries?

Apart from using vite-plugin-ssr to create entries (there are 70 pages), nothing is special about our application.

For instance, I don't understand why Rollup is trying to put each Icon component in a separate chunk (which ends up just a few bytes), when they are imported like regular modules across the application.

Here is the full build log:

https://gist.github.com/gajus/0149233592085181331dde7076fc50b1

This is example website:

https://contra.com/p/gkOQlbLq-validating-postgre-sql-results-and-inferring-query-static-types

This is out Vite configuration:

import path from 'path';
import { default as react } from '@vitejs/plugin-react';
import { defineConfig } from 'vite';
import { default as ssr } from 'vite-plugin-ssr/plugin';

const { VITE_BASE_URL } = process.env;

export default defineConfig(({ ssrBuild }) => {
  let build;

  if (!ssrBuild) {
    build = {
      emptyOutDir: true,
      manifest: true,
      minify: true,
      polyfillModulePreload: false,
      rollupOptions: {
        output: {
          chunkFileNames: 'chunk-[name].[hash].js',
          entryFileNames: 'entry-[name].[hash].js',
          inlineDynamicImports: false,
          sourcemap: true,
        },
      },
      sourcemap: true,
    };
  }

  return {
    base: VITE_BASE_URL ?? '/static/',
    build,
    plugins: [
      react({
        babel: {
          plugins: [
            'babel-plugin-relay',
            [
              'babel-plugin-styled-components',
              {
                displayName: true,
                fileName: false,
                pure: true,
                ssr: true,
              },
            ],
          ],
        },
      }),
      ssr(),
    ],
    resolve: {
      alias: {
        '@': path.resolve(__dirname, './src/'),
      },
    },
  };
});

If you try to inspect this page using Google tools, it doesn't even load – presumably because of 200 JavaScript chunks that that page needs to load.

https://search.google.com/test/mobile-friendly/result?id=ULZhtOfQfp1Gxj2wYhyCUw

For context, I am familiar with manualChunks, and tried configuring chunks into what logically makes sense, e.g.

manualChunks: (id: string) => {
  // Some IDs are not file paths, e.g.
  // 'virtual:vite-plugin-ssr:pageFiles:client:client-routing'
  if (!id.includes(__dirname)) {
    return undefined;
  }

  // Rollup ID starts with a weird zero-width character that I was not able
  // to remove using trim() or regex.
  const pathStartIndex = id.indexOf('/');

  // Rollup IDs might include search parameters, e.g.
  // 'node_modules/react/cjs/react.production.min.js?commonjs-exports'
  const filePath = path.relative(
    __dirname,
    id.slice(pathStartIndex).split('?')[0]
  );

  if (filePath.startsWith('src/components/Icons/')) {
    return 'icons';
  }

  if (filePath.startsWith('src/hooks/')) {
    return 'hooks';
  }

  if (filePath.startsWith('node_modules/')) {
    return 'vendor';
  }

  return undefined;
},

However, attempting to modify chunks produces an error runtime:

chunk-hooks.87dd4e61.js:1 Uncaught TypeError: Cannot read properties of undefined (reading 'exports')
    at chunk-hooks.87dd4e61.js:1:1449

Even putting everything into a single chunk breaks parts of the website (because of side effects):

manualChunks: () => {
  return 'everything';
},

Really not sure how to even attempt to tackle this problem.

@lukastaegert
Copy link
Member

Neither have I because this is packed with Vite plugins and no, we are not the maintainers of Vite and I am not familiar with most of this.

@gajus
Copy link

gajus commented Aug 7, 2022

Neither have I because this is packed with Vite plugins and no, we are not the maintainers of Vite and I am not familiar with most of this.

Can you think of a configuration that would improve the odds of rollup working at least with a single chunk?

At this point, I would rather just have the entire app in a single chunk than what is happening now.

But it looks like having it all in a single chunk causes out of order execution, e.g.

2react-dom.production.min.js:189 ReferenceError: Cannot access 'ContraPaymentWorkFlowModals$1' before initialization
    at ContraPayments.tsx:10:24
    at async ContraPayments.tsx:10:18
Mi @ react-dom.production.min.js:189
react-dom.production.min.js:189 ReferenceError: Cannot access 'index$2' before initialization
    at ProfileLayout.tsx:68:34
    at async ProfileLayout.tsx:68:28
Mi @ react-dom.production.min.js:189
react-dom.production.min.js:189 ReferenceError: Cannot access 'ContraPaymentWorkFlowModals$1' before initialization
    at ContraPayments.tsx:10:24
    at async ContraPayments.tsx:10:18
Mi @ react-dom.production.min.js:189
react-dom.production.min.js:189 ReferenceError: Cannot access 'index$1' before initialization
    at projects.page.client.tsx:25:21
    at async projects.page.client.tsx:25:15

@gajus
Copy link

gajus commented Aug 7, 2022

Made a little progress. You can make single chunk strategy work if you remove all dynamic imports, i.e. Had to refactor out all instances of:

const Inquiry = lazy(async () => {
  const module = await import('persona');

  return {
    default: module.Inquiry,
  };
});

At this point, a single chunk is better than 200+ chunks that are loaded on every page.

@andreasvirkus
Copy link

Neither have I because this is packed with Vite plugins and no, we are not the maintainers of Vite and I am not familiar with most of this.

Vite claims that they use Rollup's default chunking (vitejs/vite#9565 (comment)).

Will try to replicate this issue with a vanilla rollup config early next week. We're also disturbed by this chunking strategy and using Vite. Would really love to group some chunks by defining a minimum chunk size, as atm we're seeing microbial chunks (some even down to ~60 bytes 🙄)

Would there be potential to create something similar to Webpack's splitChunks.minSize, or are there too large fundamental differences to expose this as a single variable in the config?

@brillout
Copy link

Also probably related: #4547 (in this case there are even a lot of empty chunks).

@gajus
Copy link

gajus commented Aug 12, 2022

minSize and maxSize configurations would be a game changer.

Too many small chunks kill website usability. Google literally stopped indexing our website when we had 300 small chunks loading.

A few very large chunks kill compile and evaluation time. We made a ton of optimizations to improve page loading time of https://contra.com/gajus. But a single large chunk strategy is severely penalizing our critical rendering path.

Will donate USD 1,000 to any individual if they deliver improvements that achieve controlled chunk sizes (offer stands until the end of year). Ideally as contribution to rollup, but could be done as a rollup plugin too.

@lukastaegert lukastaegert reopened this Aug 13, 2022
@lukastaegert
Copy link
Member

So some update from me:

  • I do not think there is an obvious bug in the algorithm, but the main problem is rather the use of a lot of dynamic imports.
  • The problem is that Rollup by the way it works cannot import a chunk without running all of its code. Therefore the chunking carefully needs to calculate the largest possible chunks so that for each possible order of dynamic imports, each import will only load exactly the files that have been missing at this point. If you have 70 dynamic imports that are all independent, this could theoretically mean 2^70 chunks (if we have that many source files and the imports are really specially constructed).
  • Rollup already does a lot to reduce the problem by taking all the information it has into account. E.g. if a dynamically imported module itself contains a dynamic module, we treat them as dependent and produce fewer chunks as we take into account that all modules of the first dynamic import are already in memory.
  • Still, I am very much aware of the problem and yes, there is something we can do: Usually, many modules do not have side effects when executed e.g. because they just contain some function or class definitions. For such a module, it would be safe to run it earlier than we actually need it. A solution to the problem would be to merge small chunks that only contain such modules into larger chunks.
  • An additional problem is that the chunking runs very early—long before we know what the resulting output might be. So it is kind of hard to estimate the resulting chunk size. We would need to do a trial rendering and probably some fake minification (e.g. remove all white-space and comments) to get a size estimate.

But all of this is doable. However, work on this is currently very much blocked by the 3.0 release that fundamentally changes the larger topology of how rendering is done in Rollup. Once this is released, I can dedicate effort to implement this. This could very much happen this year (I hope to pull of the release in September).

(If we pull that off and the result makes you happy, we would also be happy to receive any donations in response...)

@brillout
Copy link

That's great to hear 💚. Thanks for the answer.


the chunking runs very early

Ok that probably explains #4547 then (because the mentioned Vite app uses a Vite transformer that prunes all server-side code).

Note that with upcoming techniques like React Server Components, such transformers that heavily prune server-side code will start to become mainstream. So I think running the chunking algorithm at the end (or at least after Vite transformers) will become crucial.


If I were to completely disable code splitting (while still enabling bundling), all I would be left with x + y bundles where x is the number of input entries and y the number of dynamic imports, correct?

At that point, it's all about 1. determining common modules between bundles, and 2. determining if it's worth it to extract the common parts into chunks.

I believe the core of the problem is about 2. that tiny chunks are being considered "worth it".

E.g. gajus's app has 200+ chunks but cearly doesn't have 200+ input entries / dynamic imports.

But reading your reply, it seems that code splitting works completley differently.

@brillout
Copy link

brillout commented Aug 13, 2022

Ah, actually, I'm just realizing it's not that easy because modules are not allowed to be imported twice. This means chunking is not only an optimization but also a correctness thing. I'm starting to see why it's complex.

@lukastaegert
Copy link
Member

So I think running the chunking algorithm at the end (or at least after Vite transformers) will become crucial.

It is still run after all load and transform hooks (basically it is the first step of bundle.generate), so there is plenty of opportunity to prune code during the build phase. What you should not do is prune code in renderChunk. For obvious reasons we cannot generate chunks After renderChunk because that one requires chunks. But rendedChunk is when minifiers run, hence the problems to get accurate sizes.

@gajus
Copy link

gajus commented Oct 17, 2022

Does Rollup v3 in any way help with this issue?

@lukastaegert
Copy link
Member

I have now time to pursue this issue again, as I will.

@brillout
Copy link

@gajus FYI Lukas mentioned "minimal chunk size" in his Vite talk.

@RomanPerin
Copy link

hi @lukastaegert do you have a vision of which API/functionality will be added to address discussed issue?
do you expect to have just a possibility to have minimal hunk size or the solution might be much more general like providing to choose one of the possible algorithms like big/middle/optimal chunks...?

@lukastaegert
Copy link
Member

Not sure yet. But I will start with simple first, and I am already sure there will be no "optimal" size.

@moosemanf
Copy link

we would be also interested in the solution, hoping for the best ;)

@lukastaegert
Copy link
Member

Here is finally something to play around with: #4705
Feedback very much welcome!
I tried to describe the algorithm in the PR. It attempts to be as optimal as possible, but one weak point is that chunk size measurements are rather inaccurate.
I made the option experimental for now as I would still consider to make it a plugin interface with much more power (i.e. roll-your-own-algorithm), but I am not sure we need that.

@rollup-bot
Copy link
Collaborator

This issue has been resolved via #4705 as part of rollup@3.3.0. You can test it via npm install rollup.

@gajus
Copy link

gajus commented Nov 12, 2022

Will DM Lukas re donation first thing Monday morning.

Thank you

@gajus
Copy link

gajus commented Nov 12, 2022

@lukastaegert I realize this is experimental, but it seems to not produce the desired result.

I've set experimentalMinChunkSize: 500_000, which produces:

-rw-r--r--    1 gajus  staff   1.4M Nov 12 12:18 chunk-NewEditor.bfa24576.js
-rw-r--r--    1 gajus  staff   1.0M Nov 12 12:18 chunk-styleOverrides.673cda8d.js
-rw-r--r--    1 gajus  staff   488K Nov 12 12:18 manifest.json
-rw-r--r--    1 gajus  staff   383K Nov 12 12:18 chunk-mml-react.esm.80d700af.js
-rw-r--r--    1 gajus  staff   323K Nov 12 12:18 entry-entry-client-routing.5126989c.js
-rw-r--r--    1 gajus  staff   312K Nov 12 12:18 entry-entry-server-routing.f5408d83.js
-rw-r--r--    1 gajus  staff   304K Nov 12 12:18 chunk-NewEditor.bfa24576.js.br
-rw-r--r--    1 gajus  staff   269K Nov 12 12:18 chunk-MainLayout.46cabd77.js
-rw-r--r--    1 gajus  staff   246K Nov 12 12:18 chunk-Editor.701b2212.js
-rw-r--r--    1 gajus  staff   229K Nov 12 12:18 chunk-styleOverrides.673cda8d.js.br
-rw-r--r--    1 gajus  staff   221K Nov 12 12:18 chunk-index.fa6d413f.js
-rw-r--r--    1 gajus  staff   199K Nov 12 12:18 chunk-Main.35e7333e.js
-rw-r--r--    1 gajus  staff   188K Nov 12 12:18 chunk-ActivityLog.af8dd7dc.js
-rw-r--r--    1 gajus  staff   171K Nov 12 12:18 chunk-ContraSlateEditorWrapper.4c5c89e0.js
-rw-r--r--    1 gajus  staff   138K Nov 12 12:18 chunk-ContraPayments.f68ea1eb.js
-rw-r--r--    1 gajus  staff    90K Nov 12 12:18 chunk-ProductizedServiceFormView.5347c469.js
-rw-r--r--    1 gajus  staff    88K Nov 12 12:18 chunk-useLinkDetails.41b029bc.js
-rw-r--r--    1 gajus  staff    84K Nov 12 12:18 chunk-index.es.a882db1a.js
-rw-r--r--    1 gajus  staff    84K Nov 12 12:18 chunk-PaidProjectPaymentWrapper.df29285c.js
-rw-r--r--    1 gajus  staff    81K Nov 12 12:18 chunk-ProfileLayout.df155323.js
-rw-r--r--    1 gajus  staff    80K Nov 12 12:18 chunk-core.esm.c104686c.js
-rw-r--r--    1 gajus  staff    79K Nov 12 12:18 chunk-mml-react.esm.80d700af.js.br
-rw-r--r--    1 gajus  staff    77K Nov 12 12:18 chunk-WalletLayout.styles.47226433.js
-rw-r--r--    1 gajus  staff    74K Nov 12 12:18 chunk-EmojiPicker.be5ec844.js
-rw-r--r--    1 gajus  staff    74K Nov 12 12:18 chunk-index.c3c87db8.js
-rw-r--r--    1 gajus  staff    73K Nov 12 12:18 chunk-ProjectProposalPageBackground.8e6966af.js
-rw-r--r--    1 gajus  staff    73K Nov 12 12:18 chunk-TransactionLogDetailView.e24f61a1.js
-rw-r--r--    1 gajus  staff    72K Nov 12 12:18 chunk-ConsolidatedSearchAndSavedUserList.4393640e.js
-rw-r--r--    1 gajus  staff    54K Nov 12 12:18 chunk-ReviewProposalBackButton.1d76a36a.js
-rw-r--r--    1 gajus  staff    54K Nov 12 12:18 chunk-Editor.701b2212.js.br
-rw-r--r--    1 gajus  staff    54K Nov 12 12:18 chunk-index.fa6d413f.js.br
-rw-r--r--    1 gajus  staff    51K Nov 12 12:18 chunk-JoinChildrenWithComponent.386962ee.js
-rw-r--r--    1 gajus  staff    50K Nov 12 12:18 chunk-WrappedUnlockedFeaturesModal.5a679a8c.js
-rw-r--r--    1 gajus  staff    48K Nov 12 12:18 chunk-ContraSlateEditorWrapper.4c5c89e0.js.br
-rw-r--r--    1 gajus  staff    48K Nov 12 12:18 chunk-FullScreenProjectModal.2e43de42.js
-rw-r--r--    1 gajus  staff    47K Nov 12 12:18 chunk-NewProfileSidebar.ecd4e86a.js
-rw-r--r--    1 gajus  staff    45K Nov 12 12:18 chunk-SubmitPaidProjectTermsV2Mutation.3c2d4e77.js
-rw-r--r--    1 gajus  staff    43K Nov 12 12:18 chunk-DefaultEmojiPicker.6cfe8335.js
-rw-r--r--    1 gajus  staff    41K Nov 12 12:18 chunk-Main.35e7333e.js.br
-rw-r--r--    1 gajus  staff    40K Nov 12 12:18 chunk-SignProposalDocument.d6b9a887.js
-rw-r--r--    1 gajus  staff    39K Nov 12 12:18 chunk-PaidProjectTabs.79ae2c4f.js
-rw-r--r--    1 gajus  staff    38K Nov 12 12:18 chunk-NeedToSignUp.895502cf.js
-rw-r--r--    1 gajus  staff    38K Nov 12 12:18 chunk-CreateCard.4f42112f.js
-rw-r--r--    1 gajus  staff    36K Nov 12 12:18 chunk-SelectMenu.3326b7ab.js
-rw-r--r--    1 gajus  staff    35K Nov 12 12:18 chunk-index.modern.d62cd1ba.js
-rw-r--r--    1 gajus  staff    35K Nov 12 12:18 chunk-ActiveContraDefaultPaymentCard.60a47b7e.js
-rw-r--r--    1 gajus  staff    33K Nov 12 12:18 chunk-ContraPayments.f68ea1eb.js.br
-rw-r--r--    1 gajus  staff    33K Nov 12 12:18 chunk-prefetch.5e604e70.js
-rw-r--r--    1 gajus  staff    32K Nov 12 12:18 chunk-SetupAccountsModal.45613a19.js
-rw-r--r--    1 gajus  staff    32K Nov 12 12:18 chunk-TopIndependentBadgeWrapper.6b402b20.js
-rw-r--r--    1 gajus  staff    31K Nov 12 12:18 chunk-yup.module.fde4cc2e.js
-rw-r--r--    1 gajus  staff    27K Nov 12 12:18 chunk-MainLayout.46cabd77.js.br
-rw-r--r--    1 gajus  staff    26K Nov 12 12:18 chunk-EditableAvatar.bb06acb3.js
-rw-r--r--    1 gajus  staff    24K Nov 12 12:18 chunk-core.esm.c104686c.js.br
-rw-r--r--    1 gajus  staff    23K Nov 12 12:18 chunk-EmojiPicker.be5ec844.js.br
-rw-r--r--    1 gajus  staff    23K Nov 12 12:18 chunk-WorkOpportunityCard.b524d7f4.js
-rw-r--r--    1 gajus  staff    20K Nov 12 12:18 chunk-WorkAvailabilityActionMenu.6dc991ae.js
-rw-r--r--    1 gajus  staff    20K Nov 12 12:18 chunk-ProviderDetails.d41b744b.js
-rw-r--r--    1 gajus  staff    20K Nov 12 12:18 chunk-index.es.a882db1a.js.br
-rw-r--r--    1 gajus  staff    19K Nov 12 12:18 chunk-TagGroup.2757be9c.js
-rw-r--r--    1 gajus  staff    18K Nov 12 12:18 chunk-assertRenderHook.60eaff87.js
-rw-r--r--    1 gajus  staff    18K Nov 12 12:18 chunk-useConfetti.28b376f8.js
-rw-r--r--    1 gajus  staff    17K Nov 12 12:18 chunk-DiscoverTalentPageBackground.9919b17a.js
-rw-r--r--    1 gajus  staff    16K Nov 12 12:18 chunk-ProfileLayout.df155323.js.br
-rw-r--r--    1 gajus  staff    16K Nov 12 12:18 chunk-ProjectProposalPageBackground.8e6966af.js.br
-rw-r--r--    1 gajus  staff    16K Nov 12 12:18 chunk-ActivityLog.af8dd7dc.js.br
-rw-r--r--    1 gajus  staff    16K Nov 12 12:18 chunk-RatingCell.b8646076.js
-rw-r--r--    1 gajus  staff    16K Nov 12 12:18 chunk-ProductizedServiceFormView.5347c469.js.br
-rw-r--r--    1 gajus  staff    15K Nov 12 12:18 chunk-fuse.esm.ae3a1bf9.js
-rw-r--r--    1 gajus  staff    15K Nov 12 12:18 chunk-CardListItemAnimation.160ef20b.js
-rw-r--r--    1 gajus  staff    15K Nov 12 12:18 chunk-useSortableListView.6e557ff8.js
-rw-r--r--    1 gajus  staff    14K Nov 12 12:18 manifest.json.br
-rw-r--r--    1 gajus  staff    14K Nov 12 12:18 chunk-ProfileOpportunities.77d5f5ba.js
-rw-r--r--    1 gajus  staff    13K Nov 12 12:18 chunk-AvatarGroup.c63f1828.js
-rw-r--r--    1 gajus  staff    13K Nov 12 12:18 chunk-TransactionLogDetailView.e24f61a1.js.br
-rw-r--r--    1 gajus  staff    13K Nov 12 12:18 chunk-TextField.ea1adc9c.js
-rw-r--r--    1 gajus  staff    13K Nov 12 12:18 chunk-WalletLayout.styles.47226433.js.br
-rw-r--r--    1 gajus  staff    13K Nov 12 12:18 chunk-WorkSectionWalletLayout.styles.6bf0a207.js
-rw-r--r--    1 gajus  staff    13K Nov 12 12:18 chunk-LinkCalloutExtension.e1c69bd9.js
...
-rw-r--r--    1 gajus  staff   824B Nov 12 12:18 chunk-Checklist.6228fd51.js.br
-rw-r--r--    1 gajus  staff   818B Nov 12 12:18 chunk-EmptyState.c79a400e.js.br
-rw-r--r--    1 gajus  staff   800B Nov 12 12:18 chunk-CryptoPayoutAvatarIcon.b1249fe3.js.br
-rw-r--r--    1 gajus  staff   785B Nov 12 12:18 chunk-NewProjectButton.943dbecf.js.br
-rw-r--r--    1 gajus  staff   755B Nov 12 12:18 chunk-index.4088b726.js
-rw-r--r--    1 gajus  staff   747B Nov 12 12:18 chunk-InlineHeadlineEditor.styles.64b189bd.js.br
-rw-r--r--    1 gajus  staff   744B Nov 12 12:18 chunk-TextLink.0f7e395d.js
-rw-r--r--    1 gajus  staff   736B Nov 12 12:18 chunk-MilestonesModal.styles.b9eded0e.js.br
-rw-r--r--    1 gajus  staff   727B Nov 12 12:18 chunk-AccountRegistrationForm.styles.be1fd2f4.js.br
-rw-r--r--    1 gajus  staff   724B Nov 12 12:18 chunk-Rect.c47b9398.js
-rw-r--r--    1 gajus  staff   711B Nov 12 12:18 chunk-SidebarTipsCallout.644220bb.js.br
-rw-r--r--    1 gajus  staff   708B Nov 12 12:18 chunk-index.f11456b1.js
-rw-r--r--    1 gajus  staff   694B Nov 12 12:18 chunk-NewWorkOpportunitySkeleton.082e08d6.js.br
-rw-r--r--    1 gajus  staff   692B Nov 12 12:18 chunk-HoverCropperButton.c3bb6d69.js.br
-rw-r--r--    1 gajus  staff   691B Nov 12 12:18 chunk-InlineFormElement.377f1263.js.br
-rw-r--r--    1 gajus  staff   685B Nov 12 12:18 chunk-Checkbox.3705cf3a.js.br
-rw-r--r--    1 gajus  staff   623B Nov 12 12:18 chunk-PublishingLayout.f1e09837.js.br
-rw-r--r--    1 gajus  staff   612B Nov 12 12:18 chunk-Settings.9d8901be.js.br
-rw-r--r--    1 gajus  staff   608B Nov 12 12:18 chunk-utils.b27d3a80.js
-rw-r--r--    1 gajus  staff   595B Nov 12 12:18 chunk-SendAuthenticationTokenMutation.graphql.aa0b616e.js.br
-rw-r--r--    1 gajus  staff   567B Nov 12 12:18 chunk-getProjectCostBreakupFromBudget.27c5f4f1.js.br
-rw-r--r--    1 gajus  staff   563B Nov 12 12:18 chunk-OrSeparator.e24452bd.js
-rw-r--r--    1 gajus  staff   547B Nov 12 12:18 chunk-BlankCardMessage.d6070e9d.js
-rw-r--r--    1 gajus  staff   534B Nov 12 12:18 chunk-OverflowToolsMenu.cbc5ee46.js.br
-rw-r--r--    1 gajus  staff   529B Nov 12 12:18 chunk-actionTypes.89d863ff.js
-rw-r--r--    1 gajus  staff   517B Nov 12 12:18 chunk-AddButton.00bc2386.js.br
-rw-r--r--    1 gajus  staff   511B Nov 12 12:18 chunk-Loader.04802487.js
-rw-r--r--    1 gajus  staff   510B Nov 12 12:18 chunk-useDebouncedCallback.4772dd80.js.br
-rw-r--r--    1 gajus  staff   494B Nov 12 12:18 chunk-Table.04b6a349.js.br
-rw-r--r--    1 gajus  staff   494B Nov 12 12:18 chunk-expandedTransformKuleditorToContraSlate.0d9a40fb.js.br
-rw-r--r--    1 gajus  staff   476B Nov 12 12:18 chunk-OrganizationsList.7b1f6e70.js
-rw-r--r--    1 gajus  staff   468B Nov 12 12:18 chunk-InputFieldErrorMessage.be1829fb.js
-rw-r--r--    1 gajus  staff   457B Nov 12 12:18 chunk-ResourceDetails.styles.3460d264.js.br
-rw-r--r--    1 gajus  staff   451B Nov 12 12:18 chunk-Tabs.styles.9bb5c498.js.br
-rw-r--r--    1 gajus  staff   449B Nov 12 12:18 chunk-InlineFormElement.styles.37e2e562.js.br
-rw-r--r--    1 gajus  staff   443B Nov 12 12:18 chunk-StackableLayout.64440178.js.br
-rw-r--r--    1 gajus  staff   413B Nov 12 12:18 chunk-ContraAvatarIcon.fb135bc9.js
-rw-r--r--    1 gajus  staff   398B Nov 12 12:18 chunk-constants.78e12766.js
-rw-r--r--    1 gajus  staff   395B Nov 12 12:18 chunk-Container.61e1bc1f.js
-rw-r--r--    1 gajus  staff   370B Nov 12 12:18 chunk-LocationText.3fa3cd53.j

I would not expect any of those under 1KB chunks. The full configuration is:

{
  emptyOutDir: true,
  manifest: true,
  minify: true,
  polyfillModulePreload: false,
  rollupOptions: {
    output: {
      chunkFileNames: 'chunk-[name].[hash].js',
      entryFileNames: 'entry-[name].[hash].js',
      experimentalMinChunkSize: 500_000,
      inlineDynamicImports: false,
    },
  },
}

@lukastaegert
Copy link
Member

lukastaegert commented Nov 12, 2022

The first question is of course: Are you using the Vite 4 alpha, or a Vite branch that supports Rollup 3, or are otherwise sure you are running Rollup 3? But if you build does not give you a warning "Unknown output options: experimentalMinChunkSize", then you are probably fine.

Then the most likely cause is that Rollup is not sure that the small chunks are free of side effects. If a chunk contains a side effect, you cannot safely merge it into another chunk as the side effect may be triggered at the wrong time. It may be interesting to see what is inside those small chunks to figure out if this really is the problem. Maybe there is also some low-hanging fruit regard side effect detection.

It would also be interesting to see if the flag has any effect at all:

  • are there more chunks when you do not use the setting? If not, then we need to check again if the correct Rollup version is triggered. Otherwise, how many chunks got merged?
  • does setting it to Infinity change anything?

@gajus
Copy link

gajus commented Nov 12, 2022

The first question is of course: Are you using the Vite 4 alpha, or a Vite branch that supports Rollup 3, or are otherwise sure you are running Rollup 3? But if you build does not give you a warning "Unknown output options: experimentalMinChunkSize", then you are probably fine.

Yes, using Vite 4 alpha and forcing resolution to Rollup version 3.3.0 version.

For what it is worth, this option did merge some chunks (total number of chunks reduced from 850+ to ~600).

Here is one of the chunks (dist/client/chunk-CryptoPayoutAvatarIcon.b1249fe3.js):

import{aM as i,ap as s,aq as c}from"./chunk-NewEditor.bfa24576.js";import{a as r,j as t}from"./chunk-index.fa6d413f.js";import{S as l}from"./chunk-ContraPayments.f68ea1eb.js";const d=e=>r("svg",{width:"1em",height:"1em",viewBox:"0 0 20 20",fill:"none",xmlns:"http://www.w3.org/2000/svg",role:"img",...e,children:[t("path",{d:"m11.095 1.183 8.584 7.821a.975.975 0 0 1 .254 1.08.975.975 0 0 1-.915.626h-1.371v7.836c0 .311-.252.563-.563.563H3.785a.563.563 0 0 1-.562-.563V10.71H1.852a.975.975 0 0 1-.915-.626.975.975 0 0 1 .254-1.08l8.583-7.82a.977.977 0 0 1 1.322 0Z",fill:"url(#green-bank_svg__a)"}),t("path",{d:"M11.11 6.055 9.76 15.967M12.688 7.857h-3.38a1.577 1.577 0 1 0 0 3.154h2.253a1.577 1.577 0 0 1 0 3.154h-3.83",stroke:"#F7F7F7",strokeLinecap:"round",strokeLinejoin:"round"}),t("defs",{children:r("linearGradient",{id:"green-bank_svg__a",x1:10.435,y1:.926,x2:10.435,y2:19.109,gradientUnits:"userSpaceOnUse",children:[t("stop",{stopColor:"#C3DC2B"}),t("stop",{offset:1,stopColor:"#8CC83D"})]})})]}),I=e=>e?i({firstName:e.firstName,lastName:e.lastName},"fullName"):"Deleted user",o=Object.freeze({lg:32,sm:18}),m=Object.freeze({lg:16,sm:8}),g=s.div`
  ${({theme:e,size:a})=>c`
    background: ${e.colors.gray10};
    padding: ${m[a]}px;
    border-radius: 50%;
    display: flex;
    justify-content: center;
    align-items: center;
  `}
`,N=({size:e="sm",icon:a})=>t(g,{size:e,children:t(a||d,{height:o[e],width:o[e]})}),n=Object.freeze({lg:32,sm:18}),h=Object.freeze({lg:16,sm:8}),p=s.div`
  ${({theme:e,size:a})=>c`
    background: ${e.colors.gray10};
    padding: ${h[a]}px;
    border-radius: 50%;
    display: flex;
    justify-content: center;
    align-items: center;
  `}
`,b=({size:e="sm"})=>t(p,{size:e,children:t(l,{height:n[e],width:n[e]})});export{b as C,d as G,N as P,I as g};

and the actual code:

import { SocialUSDCSolidIcon } from '@contra/ui-kit';
import styled, { css } from 'styled-components';

type PayoutAvatarSize = 'lg' | 'sm';

type PayoutAvatarIconProps = {
  size?: PayoutAvatarSize;
};

const ICON_SIZE: Record<PayoutAvatarSize, number> = Object.freeze({
  lg: 32,
  sm: 18,
});

const ICON_PADDING: Record<PayoutAvatarSize, number> = Object.freeze({
  lg: 16,
  sm: 8,
});

const IconCircleFrame = styled.div<{ size: PayoutAvatarSize }>`
  ${({ theme, size }) => css`
    background: ${theme.colors.gray10};
    padding: ${ICON_PADDING[size]}px;
    border-radius: 50%;
    display: flex;
    justify-content: center;
    align-items: center;
  `}
`;

export const CryptoPayoutAvatarIcon = ({
  size = 'sm',
}: PayoutAvatarIconProps) => {
  return (
    <IconCircleFrame size={size}>
      <SocialUSDCSolidIcon height={ICON_SIZE[size]} width={ICON_SIZE[size]} />
    </IconCircleFrame>
  );
};

I assume styled.div is the side-effect here – is there a way to ignore these?

@lukastaegert
Copy link
Member

Yes, styled.div is the main culprit. Along with Object.freeze, as it turns out. Even though applying it to an object literal instead of a variable is arguably not a side effect, we could definitely improve here.

If styled.div was not a tagged template literal function but called as a regular function, you could "just" put /*#__PURE__*/ in front of each call. But those annotations are ignored for tagged literals for compatibility reasons. Another way would be to wrap styled.div in a creative way

const wrappedStyledDiv = (...args) => /*#__PURE__*/styled.div.apply(styled, args)

and then replace calls to styled.div with wrappedStyledDiv. For me, this had the desired effect
https://rollupjs.org/repl/?version=3.3.0&shareable=JTdCJTIybW9kdWxlcyUyMiUzQSU1QiU3QiUyMm5hbWUlMjIlM0ElMjJtYWluLmpzJTIyJTJDJTIyY29kZSUyMiUzQSUyMmltcG9ydCUyMHN0eWxlZCUyQyUyMCU3QiUyMGNzcyUyMCU3RCUyMGZyb20lMjAnc3R5bGVkLWNvbXBvbmVudHMnJTNCJTVDbiU1Q25jb25zdCUyMElDT05fU0laRSUyMCUzRCUyMCUyRiolMjNfX1BVUkVfXyolMkZPYmplY3QuZnJlZXplKCU3QiU1Q24lMjAlMjBsZyUzQSUyMDMyJTJDJTVDbiUyMCUyMHNtJTNBJTIwMTglMkMlNUNuJTdEKSUzQiU1Q24lNUNuY29uc3QlMjBJQ09OX1BBRERJTkclMjAlM0QlMjAlMkYqJTIzX19QVVJFX18qJTJGT2JqZWN0LmZyZWV6ZSglN0IlNUNuJTIwJTIwbGclM0ElMjAxNiUyQyU1Q24lMjAlMjBzbSUzQSUyMDglMkMlNUNuJTdEKSUzQiU1Q24lNUNuY29uc3QlMjBzdHlsZWREaXYlMjAlM0QlMjAoLi4uYXJncyklMjAlM0QlM0UlMjAlMkYqJTIzX19QVVJFX18qJTJGc3R5bGVkLmRpdi5hcHBseShzdHlsZWQlMkMlMjBhcmdzKSU1Q24lNUNuY29uc3QlMjBJY29uQ2lyY2xlRnJhbWUlMjAlM0QlMjBzdHlsZWREaXYlNjAlNUNuJTIwJTIwJTI0JTdCKCU3QiUyMHRoZW1lJTJDJTIwc2l6ZSUyMCU3RCklMjAlM0QlM0UlMjBjc3MlNjAlNUNuJTIwJTIwJTIwJTIwYmFja2dyb3VuZCUzQSUyMCUyNCU3QnRoZW1lLmNvbG9ycy5ncmF5MTAlN0QlM0IlNUNuJTIwJTIwJTIwJTIwcGFkZGluZyUzQSUyMCUyNCU3QklDT05fUEFERElORyU1QnNpemUlNUQlN0RweCUzQiU1Q24lMjAlMjAlMjAlMjBib3JkZXItcmFkaXVzJTNBJTIwNTAlMjUlM0IlNUNuJTIwJTIwJTIwJTIwZGlzcGxheSUzQSUyMGZsZXglM0IlNUNuJTIwJTIwJTIwJTIwanVzdGlmeS1jb250ZW50JTNBJTIwY2VudGVyJTNCJTVDbiUyMCUyMCUyMCUyMGFsaWduLWl0ZW1zJTNBJTIwY2VudGVyJTNCJTVDbiUyMCUyMCU2MCU3RCU1Q24lNjAlM0IlNUNuJTIyJTJDJTIyaXNFbnRyeSUyMiUzQXRydWUlN0QlNUQlMkMlMjJvcHRpb25zJTIyJTNBJTdCJTIyZm9ybWF0JTIyJTNBJTIyZXMlMjIlMkMlMjJuYW1lJTIyJTNBJTIybXlCdW5kbGUlMjIlMkMlMjJhbWQlMjIlM0ElN0IlMjJpZCUyMiUzQSUyMiUyMiU3RCUyQyUyMmdsb2JhbHMlMjIlM0ElN0IlN0QlN0QlMkMlMjJleGFtcGxlJTIyJTNBbnVsbCU3RA==

But of course it is not nice, and you still need to "fix" the Object.freeze calls. I will see if I cannot find an easier way, maybe a simple option to blanked mark functions as pure if their name matches an expression. And I will also see if I can improve on Object.freeze.

@gajus
Copy link

gajus commented Nov 16, 2022

Let me know what are the next steps here – esp. how I can support the effort.

@lukastaegert
Copy link
Member

Yes, I do have something for you to try: #4718

Basically, using that branch/rollup@beta, try adding

treeshake: {
  pureFunctions: ['styled.div', 'Object.freeze']
}

and see what the effect is. I also have some idea how to improve the basic algorithm even when not using this, but it will take another couple of days to implement.

@gajus
Copy link

gajus commented Nov 18, 2022

As far as I can tell, the particular chunk that we were looking at (CryptoPayoutAvatarIcon) is no longer included.

We are now down to 377 chunks, which is a huge improvement from where we started.

This is my Rollup configuration:

{
  output: {
    chunkFileNames: 'chunk-[name].[hash].js',
    entryFileNames: 'entry-[name].[hash].js',
    experimentalMinChunkSize: 500_000,
    inlineDynamicImports: false,
  },
  treeshake: {
    manualPureFunctions: ['styled', 'styled.div', 'Object.freeze'],
  },
},

(I assume that pureFunctions in your message was a typo.)

However, we still have a couple hundred small chunks:

-rw-r--r--   1 gajus  staff   1.4K Nov 18 09:21 chunk-Tabs.styles.d6fdb765.js
-rw-r--r--   1 gajus  staff   1.4K Nov 18 09:21 chunk-StackableLayout.6a0286d1.js
-rw-r--r--   1 gajus  staff   1.4K Nov 18 09:21 chunk-EditableHeadline.99d1fbe9.js
-rw-r--r--   1 gajus  staff   1.4K Nov 18 09:21 chunk-Callout.ab41f3a1.js
-rw-r--r--   1 gajus  staff   1.4K Nov 18 09:21 chunk-Rect.469a7283.js
-rw-r--r--   1 gajus  staff   1.4K Nov 18 09:21 chunk-index.33ba5db1.js
-rw-r--r--   1 gajus  staff   1.3K Nov 18 09:21 chunk-InlineFormElement.styles.ef8db2f5.js
-rw-r--r--   1 gajus  staff   1.2K Nov 18 09:21 chunk-AvatarPinnedIcon.5efaa270.js
-rw-r--r--   1 gajus  staff   1.2K Nov 18 09:21 chunk-DurationAndRate.styles.88fc40df.js
-rw-r--r--   1 gajus  staff   1.1K Nov 18 09:21 chunk-TextLink.e3e7d80d.js
-rw-r--r--   1 gajus  staff   1.0K Nov 18 09:21 chunk-actionTypes.afeffaa2.js
-rw-r--r--   1 gajus  staff   991B Nov 18 09:21 chunk-Loader.c5bc06a4.js
-rw-r--r--   1 gajus  staff   964B Nov 18 09:21 chunk-OrganizationsList.7b92b8df.js
-rw-r--r--   1 gajus  staff   903B Nov 18 09:21 chunk-OrSeparator.b658ef5a.js
-rw-r--r--   1 gajus  staff   560B Nov 18 09:21 chunk-LocationText.0b576325.js
-rw-r--r--   1 gajus  staff   429B Nov 18 09:21 chunk-InputFieldErrorMessage.44886bb4.js

Looking at the contents of these files:

import { type ReactNode } from 'react';
import styled from 'styled-components';
import { Text } from '../../Primitives/index';

const StyledErrorMessage = styled(Text)`
  color: var(--color-uiErrorRegular);
`;

export const InputFieldErrorMessage = ({
  children,
  ...props
}: {
  children: ReactNode;
}) => (
  <StyledErrorMessage as="span" textStyle="captionSmall" {...props}>
    {children}
  </StyledErrorMessage>
);

and output:

import { N as styled, aI as Text } from "./chunk-projectCover.b01513ef.js";
import { j as jsx } from "./chunk-index.f1e29f4b.js";
const StyledErrorMessage = styled(Text)`
  color: var(--color-uiErrorRegular);
`;
const InputFieldErrorMessage = ({
  children,
  ...props
}) => /* @__PURE__ */ jsx(StyledErrorMessage, {
  as: "span",
  textStyle: "captionSmall",
  ...props,
  children
});
export {
  InputFieldErrorMessage as I
};

My guess is that the outstanding problem is styled(Text).

As you can see from my config, I tried adding styled, but that didn't do the trick here.

Is this misconfiguration on my part?

@lukastaegert
Copy link
Member

I see, the difference here is that you are basically calling the return value of styled(Text) in the tagged template literal, and that was not (yet) marked as pure. I took the liberty to extend and simplify the manualPureFunctions option so that

  • anything returned by a manual pure function is itself considered to be a pure function recursively
  • properties of manual pure functions are also pure functions
  • and any combination, i.e. any property of the return value of the property of the return value of a manual pure function is considered pure

That means you can just write manualPureFunctions: ['styled', 'Object.freeze'] and it should cover all cases. Can you give it another shot and report results?

I am still not done with the topic though, as I have another optimization to the minChunkSize algorithm I want to add + improvements for Object.freeze handling.

@gajus
Copy link

gajus commented Nov 21, 2022

Updating to version v3.4.0-1 reduced chunks to 366, down from 377.

This is the current configuration:

manualPureFunctions: [
  'styled',
  'Object.freeze',
  'forwardRef',
  'css',
  'createContext',
],

Chunks:

-rw-r--r--   1 gajus  staff   3.4K Nov 21 11:41 chunk-RecentTransactions.c33dc6c0.js
-rw-r--r--   1 gajus  staff   3.4K Nov 21 11:41 chunk-expandedTransformKuleditorToContraSlate.8549466b.js
-rw-r--r--   1 gajus  staff   3.3K Nov 21 11:41 chunk-ShareButton.5269a9a8.js
-rw-r--r--   1 gajus  staff   3.1K Nov 21 11:41 chunk-EmptyState.7a6eafbc.js
-rw-r--r--   1 gajus  staff   2.8K Nov 21 11:41 chunk-DurationBudgetInputs.styles.e88026a8.js
drwxr-xr-x  86 gajus  staff   2.7K Nov 21 11:41 assets
-rw-r--r--   1 gajus  staff   2.7K Nov 21 11:41 chunk-ProjectSquareCard.b987ce3a.js
-rw-r--r--   1 gajus  staff   2.5K Nov 21 11:41 chunk-util.4d3709d5.js
-rw-r--r--   1 gajus  staff   2.5K Nov 21 11:41 chunk-InlineHeadlineEditor.styles.548ae522.js
-rw-r--r--   1 gajus  staff   2.5K Nov 21 11:41 chunk-PublishingLayout.2bb7bc5c.js
-rw-r--r--   1 gajus  staff   2.2K Nov 21 11:41 chunk-Settings.40889406.js
-rw-r--r--   1 gajus  staff   2.2K Nov 21 11:41 chunk-useDebouncedCallback.2aaf62eb.js
-rw-r--r--   1 gajus  staff   2.0K Nov 21 11:41 chunk-hooks.3a64c3ab.js
-rw-r--r--   1 gajus  staff   1.9K Nov 21 11:41 chunk-Table.b04dbe87.js
-rw-r--r--   1 gajus  staff   1.7K Nov 21 11:41 contra-logo-white-on-black-128.png
-rw-r--r--   1 gajus  staff   1.6K Nov 21 11:41 chunk-v4.a4521536.js
-rw-r--r--   1 gajus  staff   1.4K Nov 21 11:41 chunk-Tabs.styles.b07964bb.js
-rw-r--r--   1 gajus  staff   1.4K Nov 21 11:41 chunk-index.33ba5db1.js
-rw-r--r--   1 gajus  staff   1.3K Nov 21 11:41 chunk-InlineFormElement.styles.4306fd98.js
-rw-r--r--   1 gajus  staff   988B Nov 21 11:41 chunk-Loader.779b58a7.js

Example chunk input (RecentTransactions.tsx):

/* eslint-disable relay/unused-fields */

import { useFragment } from 'react-relay';
import { graphql } from 'relay-runtime';
import styled, { css } from 'styled-components';
import { type RecentTransactions_userAccount$key } from '@/__generated__/RecentTransactions_userAccount.graphql';
import { Button } from '@/components/Buttons/index';
import { Link } from '@/components/Link/index';
import { Text } from '@/components/Primitives/index';
import { RecentTransactionSummary } from '@/features/wallet/index';
import { useUserTypeSelector } from '@/hooks/index';
import { type StubRouteHelpers } from '@/utilities/routeHelpers';

const Container = styled.div`
  ${({ theme }) => css`
    background-color: ${theme.colors.white30};
    padding: 24px;
    border-radius: 20px;
    width: 100%;

    > * {
      width: 100%;
      margin: 0;
    }

    ${theme.mediaQueries.lg} {
      padding: 32px;
    }
  `}
`;

type RecentTransactionsProps = {
  nodeRef: RecentTransactions_userAccount$key;
};

export const RecentTransactions = ({ nodeRef }: RecentTransactionsProps) => {
  const data = useFragment<RecentTransactions_userAccount$key>(
    graphql`
      fragment RecentTransactions_userAccount on UserAccount {
        id
        transactionSummary: transactionHistory(first: 3) {
          edges {
            node {
              id
            }
          }
        }
        ...RecentTransactionSummaryFragment
      }
    `,
    nodeRef
  );

  const {
    routeHelpers: { selectedUserTypeRouteBase },
  } = useUserTypeSelector();

  // const renderEmptyState = true;
  const renderEmptyState =
    data.transactionSummary.edges.map((edge) => edge?.node.id).filter(Boolean)
      .length === 0;

  return (
    <Container>
      {renderEmptyState ? (
        <>
          <Text textStyle="subtitleRegular">Recent Transactions</Text>
          <Text
            color="--color-ui-black--medium-emphasis"
            pb={24}
            textStyle="captionRegular"
          >
            You currently don&apos;t have any transactions.
          </Text>
        </>
      ) : (
        <RecentTransactionSummary stripeAccountRef={data} />
      )}
      <Button
        as={Link}
        removeMinWidth
        to={(routes: StubRouteHelpers) =>
          routes.wallet(selectedUserTypeRouteBase)
        }
        variant="secondary"
      >
        View Transactions
      </Button>
    </Container>
  );
};

Output:

import { b as reactRelay, a as jsxs, F as Fragment, j as jsx } from "./chunk-index.f1e29f4b.js";
import { aT as useUserTypeSelector, aB as Text, aF as Button, aN as Link, N as styled, O as Ce } from "./chunk-constants.edbcb661.js";
import { R as RecentTransactionSummary } from "./chunk-WorkSectionWalletSidebar.640d4e81.js";
import "./chunk-ActiveContraCard.styles.2614e9c7.js";
import "./chunk-prefetch.95008803.js";
const node = function() {
  var v0 = {
    "alias": null,
    "args": null,
    "kind": "ScalarField",
    "name": "id",
    "storageKey": null
  };
  return {
    "argumentDefinitions": [],
    "kind": "Fragment",
    "metadata": null,
    "name": "RecentTransactions_userAccount",
    "selections": [
      v0,
      {
        "alias": "transactionSummary",
        "args": [
          {
            "kind": "Literal",
            "name": "first",
            "value": 3
          }
        ],
        "concreteType": "MoneyTransactionHistoryConnection",
        "kind": "LinkedField",
        "name": "transactionHistory",
        "plural": false,
        "selections": [
          {
            "alias": null,
            "args": null,
            "concreteType": "MoneyTransactionHistoryEdge",
            "kind": "LinkedField",
            "name": "edges",
            "plural": true,
            "selections": [
              {
                "alias": null,
                "args": null,
                "concreteType": null,
                "kind": "LinkedField",
                "name": "node",
                "plural": false,
                "selections": [
                  v0
                ],
                "storageKey": null
              }
            ],
            "storageKey": null
          }
        ],
        "storageKey": "transactionHistory(first:3)"
      },
      {
        "args": null,
        "kind": "FragmentSpread",
        "name": "RecentTransactionSummaryFragment"
      }
    ],
    "type": "UserAccount",
    "abstractKey": null
  };
}();
node.hash = "52b49181421748f65a239e1f9c82f260";
const Container = styled.div`
  ${({
  theme
}) => Ce`
    background-color: ${theme.colors.white30};
    padding: 24px;
    border-radius: 20px;
    width: 100%;

    > * {
      width: 100%;
      margin: 0;
    }

    ${theme.mediaQueries.lg} {
      padding: 32px;
    }
  `}
`;
const RecentTransactions = ({
  nodeRef
}) => {
  const data = reactRelay.exports.useFragment(node, nodeRef);
  const {
    routeHelpers: {
      selectedUserTypeRouteBase
    }
  } = useUserTypeSelector();
  const renderEmptyState = data.transactionSummary.edges.map((edge) => edge == null ? void 0 : edge.node.id).filter(Boolean).length === 0;
  return /* @__PURE__ */ jsxs(Container, {
    children: [renderEmptyState ? /* @__PURE__ */ jsxs(Fragment, {
      children: [/* @__PURE__ */ jsx(Text, {
        textStyle: "subtitleRegular",
        children: "Recent Transactions"
      }), /* @__PURE__ */ jsx(Text, {
        color: "--color-ui-black--medium-emphasis",
        pb: 24,
        textStyle: "captionRegular",
        children: "You currently don't have any transactions."
      })]
    }) : /* @__PURE__ */ jsx(RecentTransactionSummary, {
      stripeAccountRef: data
    }), /* @__PURE__ */ jsx(Button, {
      as: Link,
      removeMinWidth: true,
      to: (routes) => routes.wallet(selectedUserTypeRouteBase),
      variant: "secondary",
      children: "View Transactions"
    })]
  });
};
export {
  RecentTransactions as R
};

@lukastaegert
Copy link
Member

No, I checked the file locally and it is working. I think we are hitting another limit: Rollup does not find a reasonable match for that file to merge with. I have some ideas how to improve the base merging algorithm further, I will let you know for the next iteration. Maybe I can also add some logging then to give you more of an idea what is behind this.

@lukastaegert
Copy link
Member

@gajus can you give #4723 a spin and report the results? It also contains some crude logging to tell you how the algorithm is faring.

@gajus
Copy link

gajus commented Nov 25, 2022

Created 448 chunks
----- pure  side effects
small 262   177
  big 0     9

Trying to find merge targets for 177 chunks smaller than 500 kB with side effects...
177 chunks smaller than 500 kB with side effects remaining.

Trying to find merge targets for 11 pure chunks smaller than 500 kB...
1 pure chunks smaller than 500 kB remaining.

187 chunks remaining.

Are these the logs you were looking for? I cannot seem to see other logs.

Output of RecentTransactions.tsx chunk:

import { r as reactRelay } from "./chunk-index.0055ed09.js";
import { T as Text, B as Button, s as styled, C as Ce } from "./chunk-ErrorBoundary.380bfeab.js";
import { a as jsxs, F as Fragment, j as jsx, L as Link } from "./entry-src/pages/index.page.client.4d2d9d66.js";
import { R as RecentTransactionSummary } from "./chunk-RecentTransactionSummary.37584201.js";
import "./chunk-ContraCard.20e563c2.js";
import "./chunk-Tooltip.c8dd331a.js";
import "./entry-src/pages/discover-talent.page.client.673bb672.js";
import { u as useUserTypeSelector } from "./entry-src/pages/account-deleted.page.client.95433318.js";
const node = function() {
  var v0 = {
    "alias": null,
    "args": null,
    "kind": "ScalarField",
    "name": "id",
    "storageKey": null
  };
  return {
    "argumentDefinitions": [],
    "kind": "Fragment",
    "metadata": null,
    "name": "RecentTransactions_userAccount",
    "selections": [
      v0,
      {
        "alias": "transactionSummary",
        "args": [
          {
            "kind": "Literal",
            "name": "first",
            "value": 3
          }
        ],
        "concreteType": "MoneyTransactionHistoryConnection",
        "kind": "LinkedField",
        "name": "transactionHistory",
        "plural": false,
        "selections": [
          {
            "alias": null,
            "args": null,
            "concreteType": "MoneyTransactionHistoryEdge",
            "kind": "LinkedField",
            "name": "edges",
            "plural": true,
            "selections": [
              {
                "alias": null,
                "args": null,
                "concreteType": null,
                "kind": "LinkedField",
                "name": "node",
                "plural": false,
                "selections": [
                  v0
                ],
                "storageKey": null
              }
            ],
            "storageKey": null
          }
        ],
        "storageKey": "transactionHistory(first:3)"
      },
      {
        "args": null,
        "kind": "FragmentSpread",
        "name": "RecentTransactionSummaryFragment"
      }
    ],
    "type": "UserAccount",
    "abstractKey": null
  };
}();
node.hash = "52b49181421748f65a239e1f9c82f260";
const Container = styled.div`
  ${({
  theme
}) => Ce`
    background-color: ${theme.colors.white30};
    padding: 24px;
    border-radius: 20px;
    width: 100%;

    > * {
      width: 100%;
      margin: 0;
    }

    ${theme.mediaQueries.lg} {
      padding: 32px;
    }
  `}
`;
const RecentTransactions = ({
  nodeRef
}) => {
  const data = reactRelay.exports.useFragment(node, nodeRef);
  const {
    routeHelpers: {
      selectedUserTypeRouteBase
    }
  } = useUserTypeSelector();
  const renderEmptyState = data.transactionSummary.edges.map((edge) => edge == null ? void 0 : edge.node.id).filter(Boolean).length === 0;
  return /* @__PURE__ */ jsxs(Container, {
    children: [renderEmptyState ? /* @__PURE__ */ jsxs(Fragment, {
      children: [/* @__PURE__ */ jsx(Text, {
        textStyle: "subtitleRegular",
        children: "Recent Transactions"
      }), /* @__PURE__ */ jsx(Text, {
        color: "--color-ui-black--medium-emphasis",
        pb: 24,
        textStyle: "captionRegular",
        children: "You currently don't have any transactions."
      })]
    }) : /* @__PURE__ */ jsx(RecentTransactionSummary, {
      stripeAccountRef: data
    }), /* @__PURE__ */ jsx(Button, {
      as: Link,
      removeMinWidth: true,
      to: (routes) => routes.wallet(selectedUserTypeRouteBase),
      variant: "secondary",
      children: "View Transactions"
    })]
  });
};
export {
  RecentTransactions as R
};

Smallest chunks:

-rw-r--r--   1 gajus  staff   9.7K Nov 25 13:39 chunk-SelectLanguages.76e8ab0c.js
-rw-r--r--   1 gajus  staff   8.8K Nov 25 13:39 chunk-paidProjectCard.1450fdbf.js
-rw-r--r--   1 gajus  staff   7.6K Nov 25 13:39 chunk-AuthorBlock.0622fbdb.js
-rw-r--r--   1 gajus  staff   7.2K Nov 25 13:39 chunk-ExperimentalServiceCardProject.f264b3ba.js
-rw-r--r--   1 gajus  staff   7.2K Nov 25 13:39 chunk-HoverCropperButton.bcac6d38.js
-rw-r--r--   1 gajus  staff   6.4K Nov 25 13:39 chunk-NumericInputWithSymbol.c8a9c6c8.js
-rw-r--r--   1 gajus  staff   6.4K Nov 25 13:39 chunk-OnboardingTooltip.610545b5.js
-rw-r--r--   1 gajus  staff   6.3K Nov 25 13:39 chunk-SidebarSocialLink.a582a549.js
-rw-r--r--   1 gajus  staff   6.1K Nov 25 13:39 chunk-useEmailLogIn.fba195c9.js
-rw-r--r--   1 gajus  staff   6.1K Nov 25 13:39 chunk-constants.2967294b.js
-rw-r--r--   1 gajus  staff   5.8K Nov 25 13:39 chunk-recoverProjectProposalState.b63fa933.js
-rw-r--r--   1 gajus  staff   5.8K Nov 25 13:39 chunk-ProjectDurationSelect.1a5f7b51.js
-rw-r--r--   1 gajus  staff   5.7K Nov 25 13:39 chunk-ShareButton.95ff5f21.js
-rw-r--r--   1 gajus  staff   5.5K Nov 25 13:39 chunk-CalendarIcon.4b6a1ddb.js
-rw-r--r--   1 gajus  staff   5.5K Nov 25 13:39 chunk-OrganizationsList.76b28926.js
-rw-r--r--   1 gajus  staff   5.3K Nov 25 13:39 chunk-index.029f3330.js
-rw-r--r--   1 gajus  staff   5.3K Nov 25 13:39 chunk-ServiceLayout.ac55f6ea.js
-rw-r--r--   1 gajus  staff   4.8K Nov 25 13:39 chunk-OpportunityBudgetWizard.styles.6f5c81f4.js
-rw-r--r--   1 gajus  staff   4.5K Nov 25 13:39 chunk-isSlateContent.23129b0a.js
-rw-r--r--   1 gajus  staff   3.8K Nov 25 13:39 chunk-Checklist.1d49427f.js
-rw-r--r--   1 gajus  staff   3.6K Nov 25 13:39 chunk-RecentTransactions.dc603b13.js
-rw-r--r--   1 gajus  staff   3.5K Nov 25 13:39 chunk-BudgetInfo.051b1054.js
-rw-r--r--   1 gajus  staff   3.3K Nov 25 13:39 chunk-EmptyState.653cc49f.js
-rw-r--r--   1 gajus  staff   3.1K Nov 25 13:39 chunk-projectCover.f661f5f1.js

@lukastaegert
Copy link
Member

lukastaegert commented Nov 25, 2022

Are these the logs you were looking for

Exactly. And these are actually quite interesting. In the first pass, it tried to "grow" the 177 small side effect chunks by merging in suitable chunks from the 262 pure chunks.

Trying to find merge targets for 177 chunks smaller than 500 kB with side effects...
177 chunks smaller than 500 kB with side effects remaining.

Now this looks like nothing was merged, but this is far from true. It just means that after this phase, there were still 177 side effect chunks below the size limit. The next line indeed tells us how much was merged

Trying to find merge targets for 11 pure chunks smaller than 500 kB...

The 11 is actually what remained from the 262 pure chunks we originally had. So in the first phase, the 177 chunks with side effects were grown by merging in 251 small pure chunks.

1 pure chunks smaller than 500 kB remaining.

That is what I would have expected considering pure chunks can be merged arbitrarily. Basically, all remaining pure chunks were merged into one catch-all chunk.

This also tells us what needs to be done to further reduce the number of chunks: Get rid of more side effects. Because in the end, it is the 177 chunks with side effects that make up the majority of the remaining chunks, and those cannot be merged with each other.

From the example chunk you are giving me it still looks like it could only be the styled template string that is considered a side effect. Simply pasting it into the Rollup REPL agrees.

I think in the end it is actually theme.colors.white30 that is the side effect! Because it does not know what theme is here, it assumes the worst (which would be like null), in which case this would throw. Now usually this is not the kind of side effect that critical logic relies upon. Which is why you can actually turn off this part of the side effect detection via the treeshake.propertyReadSideEffects option.

I would actually add

treeshake: {
  preset: 'recommended',
  manualPureFunctions: [
    'styled',
    'forwardRef',
    'css',
    'createContext',
  ]
}

as the recommended preset includes this. You can probably even go for smallest. You can also remove Object.freeze from the manualPureFunctions list as that has now been improved in recent Rollup.

Would be interested to hear how these settings affect your outcome.

@gajus
Copy link

gajus commented Nov 28, 2022

As a quick update, I am debugging issues with our build at the moment. It seems some chunks are being loaded out of order and result in errors like:

Uncaught ReferenceError: Cannot access 'logger$2' before initialization

If I disable experimentalMinChunkSize, this does not happen.

@lukastaegert
Copy link
Member

Ah I see. It could be we are introducing circular references with some of the merging that were not present before. Will need to think about this.

@gajus
Copy link

gajus commented Nov 28, 2022

Ah I see. It could be we are introducing circular references with some of the merging that were not present before. Will need to think about this.

Just circling back to confirm that it seems most probable that this is a Rollup issue. I stripped out anything that I thought may have be causing this, but at the end, it all comes down to the presence of experimentalMinChunkSize configuration.

@gajus
Copy link

gajus commented Dec 5, 2022

@lukastaegert Do the latest releases include any related fixes?

@lukastaegert
Copy link
Member

Not yet, I took a slight detour to hopefully improve upon #4732, but I will go back to minChunkSize now.

@lukastaegert
Copy link
Member

Ok, I improved #4723 to prevent merges if the merge would create a cyclic dependency. Would be nice if you could run it again against your code base and report the results (number of chunks, chunk sizes, console output).

@orenmizr
Copy link

orenmizr commented Feb 2, 2023

@lukastaegert using rollup via vite 4.0 - can i log the rollout output so i can see what it is building ? vite info flag did not work

@lukastaegert
Copy link
Member

Write a minimal plugin like this:

({
  name: 'log-build',
  generateBundle(options, bundle) {
    console.log(bundle); // or something a little more refined
  }
});

@dhlolo
Copy link

dhlolo commented Jul 5, 2023

I guess that the questioner seems to need a max Chunk size, rather than min chunk? Is there a convenient option like 'splitChunks.maxSize' in webpack?

@sbernard31
Copy link

I have a vue + vite 5 applicaiton (rollup@4.5.0).
Some of my chunk was bigger than 500kB, so I tried to improved this using dynamic import in router to have smaller chunk.
This works but now I have too small chunk like :

../target/dist/assets/index-HFHy5uE0.js                                0.06 kB │ gzip:   0.08 kB
../target/dist/assets/VDivider-0ijIaHwP.js                             0.45 kB │ gzip:   0.28 kB
../target/dist/assets/VListItemGroup-KaxAC2-f.js                       0.56 kB │ gzip:   0.35 kB

So I tried with :

rollupOptions: {
  output: {
    experimentalMinChunkSize: Infinity
  }
}

And nothing changes, chunks have same size and I get this output :

Initially, there are
26 chunks, of which
26 are below minChunkSize.
After merging chunks, there are
26 chunks, of which
26 are below minChunkSize.

@bfricka
Copy link

bfricka commented Dec 15, 2023

Currently, this algorithm performs very, very poorly if you have multiple dynamic imports.

manualChunks is essentially useless for many types of code, because it will often add random code to the manual chunks, which then gets exported and required by other things. For example, if I try to manually add a couple of components to a certain chunk, I found that the build adds a portion of react and classNames to it, which it then exports. Thus, this chunk which should be shared by two dynamic import components, now has exports requiring that it be imported by most of the app.

Additionally, experimentalMinChunkSize is particularly bad, because, when it works at all, it doesn't create new "common" chunks (like Webpack), but instead will simply add the small chunks to seemingly random other chunks. For example, I enabled this, and it optimized out a small chunk that only exports a string, but put it in my traditional Chinese language translations file. So now the constant 32, that this file was exporting, requires users to download all Chinese translations as well.

And, as I said, when it works at all. I currently have around 80 chunks under 2kB that will never be combined and only exist as separate files because multiple dynamic imports import them. All of these would be candidates for either inlining or combining into a common chunk. None of them are dynamic, none of them have side effects. Here's an example of one such file (unminified), a pretty generic sort of function that just acts as a getter to determine whether a component is mounted. It's referenced in 88 places in this codebase and could easily be in a common chunk:

import { t as reactExports } from "./index-4761524d.js";
const useMounted = () => {
  const mountedRef = reactExports.useRef(true);
  reactExports.useEffect(() => {
    mountedRef.current = true;
    return () => {
      mountedRef.current = false;
    };
  }, []);
  return () => mountedRef.current;
};
export {
  useMounted as u
};

This file minified is 0.25 kB. It has no business sitting on its own. Hopefully it's obvious that this doesn't have any side effects.

The rest of the files are similar.

A few things should to happen for this to be viable for production code:

  1. Heuristics for combining small chunks need to improve. Something in the logic is clearly incorrect if the above file can't be combined.
  2. A commons chunk (or chunks) need to be created, rather than just adding combined small chunks to random other chunks. By defining commons chunks, we say that this is the random assortment of shared code between dynamic chunks. It might be possible to be smart about this, but it's probably best not to be, because the logic gets complicated (and computationally expensive) very quickly.
  3. Both min and max chunk size. Having both is a nice to have, and both should be treated as targets, not hard rules.
  4. More inlining. The above mentioned 32 is a primitive that could simply be inlined. Webpack also does this: There's likely to be some duplication in chunks, and this is an acceptable trade-off.

I've tried everything reasonable to fix this issue, but I think my only option at this point is to use Webpack for production builds. This is obviously very painful, because we use Vite for development, since it's so fast. Having an entirely different build path is sort of crazy, but I don't see any other way.

@SunHuawei
Copy link

SunHuawei commented Feb 1, 2024

The original requirement of this issue is about maxSize which is what I want. Is that in plan?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.