Switch away from webpack for bundling

[davidbrochart]

Problem

JupyterLab uses webpack for bundling, which is slow.

Proposed Solution

I'm wondering if it could be possible to switch to esbuild.

Additional context

esbuild is used in anywidget and the developer experience is so much better. Maybe @manzt has ideas.

[jtpio]

Thanks @davidbrochart for starting this discussion.

esbuild is great. It could be interesting to start experimenting with it for building JupyterLab and see if it speeds up build and improve the developer experience. We might also consider Vite.

One main blocker for fully adopting it would be support for federated extensions. It might already be possible as there seems to be an example for keeping compatibility with the webpack module federation: https://github.com/jacob-ebey/esbuild-federation-example.

Linking to #13456 as related.

[jtpio]

If speed is the main motivation, there is also rspack which aims to be a drop-in replacement for webpack: https://github.com/web-infra-dev/rspack

They have plans for supporting module federation: web-infra-dev/rspack#1930

[manzt]

Yes, I was just going to mention support for module federation (and it being a historically webpack-specific thing) as the main blocker last time I looked into it. This is also the reason that I still need to bundle anywidget for JupyterLab with jupyter labextension build still (I wasn't able to reverse engineer the output into something that I could generate with esbuild).

There was some discussion on standardizing module federation across build systems, but I haven't seen much on the topic since this blog post.

Just to note, IMO the main "dev experience" improvement with these new bundlers (i.e., esbuild or Vite) comes from their use of native ESM. It's easy for tools like Vite (and even anywidget) to "hot-swap" invalidated parts of the module graph without needing to rebundle the entire application. To that end, I'm interested in the "native" module federation (mentioned in the blog post) which seems to be based on browser-standard ESM and importmaps.

[krassowski]

Another selling point for a switch would be having a bundler which runs in the browser environment (in a webworker) so that we can rebuild without depending on node. If it just worked in the dev more, it would greatly simplify the stack for newcomers who are making minimal changes to the frontend.

[terlan4]

If speed is the main motivation, there is also rspack which aims to be a drop-in replacement for webpack: https://github.com/web-infra-dev/rspack

They have plans for supporting module federation: web-infra-dev/rspack#1930

@jtpio , is this only thing that blocks us from using rspack? Also I am wondering if we can already use rspack in small projects or module federation is essential for how jupyterlab works in our project

[krassowski]

Apparently rspack v0.5 now supports module federation v1.5: https://www.rspack.dev/blog/module-federation-added-to-rspack.html

Some highlights from the second link:

Webpack: 500-3000ms per build - production
Rspack: 130-350ms per build - production

Progressive migration to rspack can be achieved via federation. If you have webpack locked plugins or cannot perform a full cut over to rspack, via module federation you can allow rspack and webpack to share dependencies and code, meaning more code could be built via rspack while the webpack host does less work but still gets the same result. example: Webpack Rspack interop

Migrate one at a time. Since the interfaces between webpack (@module-federation/enhanced) and rspack are shared, users can switch over any existing federation build or remote to rspack. We recommend any remaining webpack builds using @module-federation/enhanced which leverages our new design and exports ModuleFederationPlugin. You can, however, still use the stock plugin that ships in webpack core. Rspack should slot in seamlessly with existing federated applications.

[jtpio]

Maybe it could be interesting to open a draft PR that switches to rspack. Since it aims to be a drop-in replacement, and we don't use that many webpack plugins, maybe it will "just work".

If that's the case, we could then consider switching to it in the next 4.x minor version, if it ends up improving extension development and if it doesn't introduce breaking changes of course. Or consider providing this option as part of a separate package (as proposed in #13456).

[krassowski]

Sounds like a good idea. While I do not expect a huge change for the interactive experience when running in watch mode, this could have a big impact on Binder build times and help with testing PRs. Adding to 4.2 for consideration.

[gt-tm-3310]

@krassowski could you please share little bit why rspack wouldn't make difference in watch mode? Are there still things that rspack should support so that we could gain speed in watch mode?

I was interested in rspack mainly for this reason. By the way, I would be happy to hear if anyone knows different ways to have faster watch mode. Thanks

[jtpio]

FYI I quickly looked into building an extension (bootstrapped from https://github.com/jupyterlab/extension-template), and compile it with rspack.

So far it seems to be working fine, at least for a basic extension, which is good news! And the iteration time is even faster when using the builtin:swc-loader loader for transpiling TypeScript.

Below is a screencast of the Rspack-based builder running in watch mode (with builtin:swc-loader), and editing the code within JupyterLab (with jupyterlab-lsp and the typescript-language-server):

jupyterlab-rspack-extension-development.webm

For now the code for that "builder" package is almost like @jupyterlab/builder, but with some parts replaced by the @rspack/core equivalents.

I'll see if this could be published as a package, so it could already be tested by third-party extensions as an alternative to jupyter labextension build for now.

It also means that if we want to undertake the transition to Rspack, we could also just focused on the extension author facing part as a first step (via @jupyterlab/builder), and still have JupyterLab rely on webpack (if the transition takes longer than expected for 4.2.0).

[jtpio]

I'll see if this could be published as a package, so it could already be tested by third-party extensions as an alternative to jupyter labextension build for now.

Pushed some work-in-progress to https://github.com/jtpio/jupyterlab-rspack. There are still a few things missing (logged in issues), but it's looking promising and can at least be tested. This repo will likely be archived once we have done the switch here in the JupyterLab repo.

[ellisonbg]

Thanks for investigating this everyone, sounds promising!

[jtpio]

Also opened #16005 to get a better idea of how this would affect the JupyterLab code base, and check if some things are working as expected yet.