Glob support in optimizeDeps.include #5419

sxzz · 2021-10-25T19:55:52Z

Clear and concise description of the problem

In some libraries, files in different paths need to be loaded frequently. The page will be refreshed every pre-build. So it is necessary to pre-build all the files that may be used at startup.

So I think glob should be supported to easily add files that may be needed.

Alternative

No response

Additional context

Related issue #699 #1545
Related pr #1571

Validations

Follow our Code of Conduct
Read the Contributing Guidelines.
Read the docs.
Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

sxzz · 2021-10-25T19:58:56Z

Chinese: 在某些库中，需要频繁地加载不同路径下的文件。每次预编译都会刷新页面。所以需要把可能要用到的所有文件都在启动时预编译。因此我想应该支持 glob 来更便捷地添加可能需要的文件。

bluwy · 2022-03-28T05:53:05Z

I have a need of this before for packages with many sub-exports, e.g. my-pkg/some-module and many other modules that could be known ahead of time. But Vite currently only understands what you import and dynamically prebundle thereafter.

I think globbing would be tricky to work with exports since only a subset of files are allowed to be imported, I wonder if the better solution is to have Vite auto optimize all sub-exports if the exports field exists.

ghost · 2022-06-02T11:59:11Z

We have non-blocking optimization #8280, and it seems that this feature becomes unimportant.

patak-dev · 2022-06-02T12:27:31Z

Interesting. I think we could try Blu's proposal here, or maybe to avoid surprises only support something like include: ['my-pkg/*'] to include all the exports of that package. Or maybe a generic glob over all the names of dependencies in the package JSON plus their exports? Maybe it is fast enough.

sxzz · 2022-06-24T13:13:49Z

I think this feature is still needed.

Kapture.2022-06-24.at.21.01.40.mp4

Reproduction repo: https://github.com/sxzz/vite-optimize-deps (cannot work on stackblitz)

Reproduction steps:

clear all cache, rm -fr node_modules/.vite
run vite --force
Once the page is loaded you can see these messages on the terminal.

9:01:28 p.m. [vite] ✨ new dependencies optimized: element-plus/es, element-plus/es/components/card/style/css
9:01:28 p.m. [vite] ✨ optimized dependencies changed. reloading

Click About, vite will refresh the page. It means all states will be reseted, and the terminal will display the messages belows.

9:01:35 p.m. [vite] ✨ new dependencies optimized: element-plus/es/components/button/style/css
9:01:35 p.m. [vite] ✨ optimized dependencies changed. reloading

Click About again, it will work finally.

sapphi-red · 2022-06-26T12:26:05Z

@sxzz I found that * could be used for exclude actually (it is not documented in Vite so it may change in future though). https://esbuild.github.io/api/#external

Setting ['./node_modules/element-plus/es/components/*/style/css2.mjs'] to optimizeDeps.exclude reduces reload.

Also for your specific case, ['./node_modules/element-plus/es/components/base/style/css2.mjs'] will work too.

patak-dev · 2022-06-26T12:45:06Z

I think we should add support for glob imports, at least document that * works. Thanks for taking the time to check this @sapphi-red, the second one is a nice trick to for element-plus.

@sxzz let us know if this works for you, better to wait to beta.3 though to test as there are a few other PRs to improve dev server perf about to be merged

patak-dev · 2022-06-26T15:02:54Z

@sapphi-red do you have a repro of this working with that exclude? Trying it now and I still see the reloads (that could be related to something different)

sxzz · 2022-06-26T16:07:26Z

@sapphi-red Hmmmm seems that it can't work for me.

sapphi-red · 2022-06-26T16:16:50Z

It worked for me with npm and yarn(non pnp). But it didn't work with pnpm.
It seems symlink is affecting this.

sapphi-red · 2022-06-26T16:33:29Z

This worked with pnpm. So symlinks does not work well.

import { defineConfig, normalizePath } from 'vite'
import vue from '@vitejs/plugin-vue'
import fs from 'fs/promises'
import path from 'path'
import url from 'url'

const exclude = './' + normalizePath(
  path.relative(
    url.fileURLToPath(new URL('./', import.meta.url)),
    path.join(
      await fs.realpath('./node_modules/element-plus/'),
      'es/components/*/style/css2.mjs'
    )
  )
)

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [vue()],
  optimizeDeps: {
    exclude: [exclude]
  }
})

sxzz · 2022-06-26T16:40:16Z

Stills can't work for me.

sapphi-red · 2022-06-26T16:42:35Z

The first reload is expected. Does it reload two times?

patak-dev · 2022-06-26T16:48:39Z

Something odd is going on. I'll check it out. The first reload shouldnt be there either

sxzz · 2022-06-26T16:55:41Z

@sapphi-red Yes,

sapphi-red · 2022-06-26T17:11:11Z

@sxzz Your screenshot shows reloading is happening once and pre-bundle is happening twice.

sxzz · 2022-06-26T17:31:24Z

Oh, just found it can avoid to refresh page now, and I had a better experience now as the same as 2.x.

But it seems to be complicated for users to write such a config.

Actually, I wrote a config to pre-optimize deps to fix this issue before.

optimizeDeps: {
  include: await glob(['element-plus/es/components/*/style/css.mjs'], {
    cwd: $r('node_modules'),
    onlyFiles: true,
  }),
},

Suggestion: Third-party dep can provide a metafile to help Vite consider which files should be included or excluded to be optimized. Out of the box for Vite users.

By the way, I have a question: Why some times it is possible to optimize dep without refreshing the page, and sometimes it is necessary to refresh the page.

patak-dev · 2022-06-26T17:36:51Z

@sxzz see this comment #8721 (comment)

patak-dev · 2022-06-27T08:56:05Z

@sxzz as a summary of the new strategy:

By default, dynamic routes are not crawled because it looks at this point like a better default.
You can add the most common visited routes to optimizeDeps.entries and there wouldn't be any reloads. Adding this to the config works well in beta.4:

optimizeDeps: {
    entries: ['src/App.vue', 'src/components/Footer.vue'],
    ...

You could also achieve the same with optimizeDeps.include of some key deps, which leads to a faster cold start.

We need more data to see if not crawling dynamic routes is the best default here. An alternative could be to crawl by default leading to no reloads but slower cold start and let the user opt-out of certain dynamic imports:

optimizeDeps: {
    scanDynamicImports: string | Regex | (string | Regex)[] | boolean

By default it would be scanDynamicImports: true

sxzz · 2022-06-27T22:38:49Z

@patak-dev I see. Thanks for your answer.

patak-dev · 2022-07-01T05:53:20Z

@sxzz the idea above is now implemented in:

feat: improved cold start using deps scanner #8869

I think that 'dynamic-scan' should be the best for vue-router based projects, at the expense of keeping the complexity of the esbuild scanner around.

This for out-of-the-box cold start, the best possible perf is still 'lazy' + including some deps in optimizeDeps.include (I'm going to work on DX around hinting wich ones should be included)

Let me know if you test the PR.

sxzz · 2022-07-01T14:15:47Z

Repo: https://github.com/sxzz/vite3-optimize-deps

pre-scan / dynamic-scan is great, but the users have to add some esbuild plugins to optimizeDeps.esbuildOptions.plugins. If some plugins is not supported on esbuild, then some files cannot be found and optimized.

  optimizeDeps: {
    devStrategy: 'dynamic-scan',
    esbuildOptions: {
      plugins: [
        Transform({
          plugins: [
            VueEsbuild(),
            ComponentsEsbuild({
              resolvers: [ElementPlusResolver()],
            }),
          ],
        }),
      ],
    },
  },

dynamic-scan without esbuild plugins / lazy / refreshing 2 times.

  vite:deps new dependencies found: vue +0ms
  vite:deps Crawling dependencies using entries:
  vite:deps   /Users/kevin/Developer/reproduction/vite3-optimize-deps/src/App.vue +0ms
  vite:deps Scan completed in 6.83ms: {
  vue: '/Users/kevin/Developer/reproduction/vite3-optimize-deps/node_modules/.pnpm/vue@3.2.37/node_modules/vue/dist/vue.runtime.esm-bundler.js'
} +7ms
  vite:deps deps bundled in 30.24ms +39ms
  vite:deps ✨ optimized dependencies unchanged +1ms
  vite:deps new dependencies found: element-plus/es
  vite:deps   element-plus/es/components/button/style/css +212ms
  vite:deps deps bundled in 367.61ms +369ms
10:18:51 p.m. [vite] ✨ optimized dependencies changed. reloading
  vite:deps new dependencies found: element-plus/es/components/card/style/css +463ms
  vite:deps deps bundled in 336.22ms +338ms

dynamic-scan with esbuild plugins / no refreshing.

  vite:deps new dependencies found: vue +0ms
  vite:deps Crawling dependencies using entries:
  vite:deps   /Users/kevin/Developer/reproduction/vite3-optimize-deps/src/App.vue +0ms
  vite:deps Scan completed in 68.39ms: {
  'element-plus/es': '/Users/kevin/Developer/reproduction/vite3-optimize-deps/node_modules/.pnpm/element-plus@2.2.6_vue@3.2.37/node_modules/element-plus/es/index.mjs',
  'element-plus/es/components/button/style/css': '/Users/kevin/Developer/reproduction/vite3-optimize-deps/node_modules/.pnpm/element-plus@2.2.6_vue@3.2.37/node_modules/element-plus/es/components/button/style/css.mjs',
  vue: '/Users/kevin/Developer/reproduction/vite3-optimize-deps/node_modules/.pnpm/vue@3.2.37/node_modules/vue/dist/vue.runtime.esm-bundler.js',
  'element-plus/es/components/card/style/css': '/Users/kevin/Developer/reproduction/vite3-optimize-deps/node_modules/.pnpm/element-plus@2.2.6_vue@3.2.37/node_modules/element-plus/es/components/card/style/css.mjs'
} +68ms
  vite:deps deps bundled in 698.01ms +771ms
  vite:deps ✨ optimized dependencies unchanged +0ms

eager is the best. Without any optimizeDeps.esbuildOptions.plugins. The page can be opened directly without any refreshes. All deps will be optimized for the first time.

  vite:deps new dependencies found: vue
  vite:deps   element-plus/es
  vite:deps   element-plus/es/components/button/style/css
  vite:deps   element-plus/es/components/card/style/css +4s

sxzz added the enhancement: pending triage label Oct 25, 2021

bluwy added the p2-nice-to-have Not breaking anything but nice to have (priority) label Mar 28, 2022

bluwy mentioned this issue Apr 30, 2022

[Feature request] Allow import.meta.glob from node_modules #5728

Closed

4 tasks

bluwy added enhancement New feature or request and removed enhancement: pending triage labels Oct 21, 2022

bluwy mentioned this issue Oct 21, 2022

feat: bring prebundleSvelteLibraries out of experimental sveltejs/vite-plugin-svelte#476

Merged

bluwy self-assigned this Dec 30, 2022

bluwy mentioned this issue Mar 8, 2023

optimizeDeps ignores deep imports #12326

Closed

7 tasks

bluwy mentioned this issue Mar 14, 2023

feat(optimizer): support glob includes #12414

Merged

9 tasks

bluwy mentioned this issue May 14, 2023

vite切换路由出现reloading #13144

Closed

7 tasks

patak-dev closed this as completed in #12414 Jun 7, 2023

github-actions bot locked and limited conversation to collaborators Jun 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Glob support in optimizeDeps.include #5419

Glob support in optimizeDeps.include #5419

sxzz commented Oct 25, 2021

sxzz commented Oct 25, 2021

bluwy commented Mar 28, 2022

ghost commented Jun 2, 2022

patak-dev commented Jun 2, 2022

sxzz commented Jun 24, 2022 •

edited

sapphi-red commented Jun 26, 2022 •

edited

patak-dev commented Jun 26, 2022

patak-dev commented Jun 26, 2022

sxzz commented Jun 26, 2022

sapphi-red commented Jun 26, 2022 •

edited

sapphi-red commented Jun 26, 2022

sxzz commented Jun 26, 2022

sapphi-red commented Jun 26, 2022

patak-dev commented Jun 26, 2022

sxzz commented Jun 26, 2022

sapphi-red commented Jun 26, 2022

sxzz commented Jun 26, 2022 •

edited

patak-dev commented Jun 26, 2022

patak-dev commented Jun 27, 2022

sxzz commented Jun 27, 2022

patak-dev commented Jul 1, 2022

sxzz commented Jul 1, 2022 •

edited

Glob support in optimizeDeps.include #5419

Glob support in optimizeDeps.include #5419

Comments

sxzz commented Oct 25, 2021

Clear and concise description of the problem

Suggested solution

Alternative

Additional context

Validations

sxzz commented Oct 25, 2021

bluwy commented Mar 28, 2022

ghost commented Jun 2, 2022

patak-dev commented Jun 2, 2022

sxzz commented Jun 24, 2022 • edited

sapphi-red commented Jun 26, 2022 • edited

patak-dev commented Jun 26, 2022

patak-dev commented Jun 26, 2022

sxzz commented Jun 26, 2022

sapphi-red commented Jun 26, 2022 • edited

sapphi-red commented Jun 26, 2022

sxzz commented Jun 26, 2022

sapphi-red commented Jun 26, 2022

patak-dev commented Jun 26, 2022

sxzz commented Jun 26, 2022

sapphi-red commented Jun 26, 2022

sxzz commented Jun 26, 2022 • edited

patak-dev commented Jun 26, 2022

patak-dev commented Jun 27, 2022

sxzz commented Jun 27, 2022

patak-dev commented Jul 1, 2022

sxzz commented Jul 1, 2022 • edited

sxzz commented Jun 24, 2022 •

edited

sapphi-red commented Jun 26, 2022 •

edited

sapphi-red commented Jun 26, 2022 •

edited

sxzz commented Jun 26, 2022 •

edited

sxzz commented Jul 1, 2022 •

edited