Webpack 5 support

[mlassbo]

I'm submitting a feature request
Add support for Webpack 5.

We (at Hogia Group) need to start preparing our Aurelia apps for Webpack 5. It is in beta but Webpack reccommends to use it for production. Also, Webpack 5 comes with Module Federation that might become an important part of building micro frontends using Aurelia and we need aurelia-webpack-plugin to support webpack 5 to be able to evaluate this properly. I've been in contact with webpack core member Zack Jackson @ScriptedAlchemy, the creator of Module Federation, about this and he has offered his help with the upgrade if needed.

• Library Version:
  aurelia-webpack-plugin 4.0.0

Please tell us about your environment:

• Operating System:
  Windows 10

• Node Version:
  12.16.0

• NPM Version:
  6.13.4

• JSPM OR Webpack AND Version
  webpack 5.0.0-beta.17

• Browser:
  all

• Language:
  all

Current behavior:
When upgrading to webpack 5 beta 17 build fails with the following error:

Error: Cannot find module 'webpack/lib/BasicEvaluatedExpression'
Require stack:

• C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\AureliaDependenciesPlugin.js
• C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\AureliaPlugin.js
• C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\index.js
• C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\webpack.config.js
• C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\webpack-cli\bin\utils\convert-argv.js
• C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\webpack-cli\bin\cli.js
• C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\webpack\bin\webpack.js
  at Function.Module._resolveFilename (internal/modules/cjs/loader.js:981:15)
  at Function.Module._load (internal/modules/cjs/loader.js:863:27)
  at Module.require (internal/modules/cjs/loader.js:1043:19)
  at require (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:161:20)
  at Object. (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\AureliaDependenciesPlugin.js:4:34)
  at Module._compile (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:192:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
  at Module.load (internal/modules/cjs/loader.js:1001:32)
  at Function.Module._load (internal/modules/cjs/loader.js:900:14)
  at Module.require (internal/modules/cjs/loader.js:1043:19)
  at require (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:161:20)
  at Object. (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\AureliaPlugin.js:4:37)
  at Module._compile (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:192:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
  at Module.load (internal/modules/cjs/loader.js:1001:32)
  at Function.Module._load (internal/modules/cjs/loader.js:900:14)
  at Module.require (internal/modules/cjs/loader.js:1043:19)
  at require (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:161:20)
  at Object. (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\index.js:6:10)
  at Module._compile (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:192:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
  at Module.load (internal/modules/cjs/loader.js:1001:32)
  at Function.Module._load (internal/modules/cjs/loader.js:900:14)
  at Module.require (internal/modules/cjs/loader.js:1043:19)
  at require (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:161:20)
  at Object. (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\webpack.config.js:2:27)
  at Module._compile (C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\v8-compile-cache\v8-compile-cache.js:192:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)
  at Module.load (internal/modules/cjs/loader.js:1001:32)
  at Function.Module._load (internal/modules/cjs/loader.js:900:14) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
  'C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\AureliaDependenciesPlugin.js',
  'C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\AureliaPlugin.js',
  'C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\aurelia-webpack-plugin\dist\index.js',
  'C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\webpack.config.js',
  'C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\webpack-cli\bin\utils\convert-argv.js',
  'C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\webpack-cli\bin\cli.js',
  'C:\repos\TEMPLATE_WebComponent_InProgress\Api\Web\App\node_modules\webpack\bin\webpack.js'
  ]
  }

Expected/desired behavior:
Build runs without errors

• What is the motivation / use case for changing the behavior?
  We (at Hogia Group) need to start preparing our Aurelia apps for Webpack 5. It is in beta but Webpack reccommends to use it for production. Also, Webpack 5 comes with Module Federation that might become an important part of building micro frontends using Aurelia and we need aurelia-webpack-plugin to support webpack 5 to be able to evaluate this properly. I've been in contact with webpack core member Zack Jackson @ScriptedAlchemy, the creator of Module Federation, about this and he has offered his help with the upgrade if needed.

[jods4]

I wrote the Webpack 3 and 4 versions of the plugin but unfortunately I don't have time to upgrade it to 5 at the moment.

I might do it later on when I'm less stressed, although if someone wants to submit a PR that fixes APIs that have changed in Webpack 4 you're welcome.

A few things that come to my mind right now:

1. We need a strategy regarding compatibility with Webpack 4.
   My thinking is: Aurelia 2 won't need this plugin as it is designed from scratch to be modern-tooling-friendly. This plugin is unlikely to receive big changes at this point.
   So I'd just give up compatibility with a major release. Webpack 4 users can continue to use aurelia-webpack-plugin@4.0.0 without loosing anything and Webpack 5 users take a dependency on aurelia-webpack-plugin@5.0.0

2. The module federation thing might be a different beast, just like concatenated modules required extra changes. Or maybe not, but it's really not a given. aurelia-loader has some hard requirements when it comes to module names and this plugin is all about making sure every module ends up with the right name. Guaranteeing this in separate builds might require some tweaks. I agree it's a very interesting feature that I would like to see supported as well!

[fkleuver]

My thinking is: Aurelia 2 won't need this plugin as it is designed from scratch to be modern-tooling-friendly.

That's mostly correct. Conventions do rely on a thin layer of tooling, but without conventions you don't have to do anything and Aurelia 2 will indeed work OOTB with any bundler including webpack 5.

However, since conventions in au2 is just a simple pre-processing step that adds decorators, I don't imagine much (if any) difficulty at all in supporting that.

On the note of au2, we are getting close to alpha. Is upgrading to au2 something you'd be interested in on the short to mid term as an alternative, or are there strong reasons to stick to au1 for some time to come?

[jods4]

However, since conventions in au2 is just a simple pre-processing step that adds decorators, I don't imagine much (if any) difficulty at all in supporting that.

Yeah this could be a simple webpack loader.

Au1 needed complex stuff to inform webpack of the dependency graph + ensuring everything would be properly available to aurelia-loader, which is purely based on module names.

Is upgrading to au2 something you'd be interested in on the short to mid term as an alternative, or are there strong reasons to stick to au1 for some time to come?

I'm assuming this question was for OP.

[mlassbo]

Thanks for your responses!

For our Micro Frontends we could (and should) definitively go for au2 right away if its production ready. When it comes to our monolith au1 apps I cant really say. Some of our teams might upgrade right away and others move more slowly. Perhaps we can find a way to make Module Federation work nicely between Micro Frontends even if they are hosted in an au1 app bundled with webpack 4.

Got any examples of an au2 app bundled with webpack?

[ScriptedAlchemy]

As one of the core maintainers of webpack and the creator of module federation. I'm willing to work with project authors if you need me

[fkleuver]

Happy to hear that @ScriptedAlchemy, and great job on the Module Federation. We're very excited about it!

The compatibility issue in a nutshell: Aurelia version 1 is built on top of a requirejs-based loader architecture that stems from before ES Modules were standardized.
We use a special marker function PLATFORM.moduleName('./path-to-module') that our webpack plugin finds in the AST and turn them into a shape that webpack understands (I didn't write the plugin but I believe it transforms them to native imports, @jods4 correct me if I'm wrong).

Unless you're aware of any specific concerns off the top of your head, I think we're mostly looking at simply try to upgrade to webpack 5 and see what breaking changes we need to address and take it from there. If @jods4 is too busy I can have a crack at it later this week.

Got any examples of an au2 app bundled with webpack?

Scattered around at the moment. I would recommend to wait for alpha announcement (within a few weeks), which will come with significantly more docs and examples than we currently have.

[jods4]

that our webpack plugin finds in the AST and turn them into a shape that webpack understands (I didn't write the plugin but I believe it transforms them to native imports, @jods4 correct me if I'm wrong).

It doesn't work at the syntax level.
It turns them into native, custom webpack dependencies using the "internal" webpack apis. Those dependencies have several special characteristics such as not being eligible for merging with another module (except if they are root of the resulting module), always preserving their name, etc.

moduleName is not the only type of Aurelia dependency with need to detect. Examples would include the implicit association between Resource.js and Resource.html, <require> used in an html template, etc.

[fkleuver]

It turns them into native, custom webpack dependencies using the "internal" webpack apis (...)

I see, that sounds like it might be a bit more work then.

Examples would include the implicit association between Resource.js and Resource.html, used in an html template, etc.

But those are AST transforms right? Or at least that's what they are in the au2 conventions plugin, so if they currently aren't, maybe we can simplify things a bit by doing so, if that makes sense?

[jods4]

But those are AST transforms right? Or at least that's what they are in the au2 conventions plugin, so if they currently aren't, maybe we can simplify things a bit by doing so, if that makes sense?

Not really. Because the "things" are not loaded directly in code with an import, rather they are loaded dynamically at runtime by aurelia-loader, which works in a peculiar way with Webpack.

This was one of the main driver for design changes in Au2, I'm afraid we did the best we could to retrofit Au1 design into the webpack era.

[fkleuver]

I'm afraid we did the best we could to retrofit Au1 design into the webpack era.

What about dynamic import era? AFAIK this plugin was implemented way before that was available, but now it so happens to be that import('./some-resources') precisely implements the LoaderPlugin interface

interface LoaderPlugin {
  fetch(address: string): Promise<any>;
}

Dynamic imports is something webpack understands OOTB. Has that by any chance been looked into?

[jods4]

What about dynamic import era?

If you do everything with dynamic imports you probably won't need this plugin and can go with vanilla Webpack (it was the goal at least).

We can't expect all our users to migrate their codebase to that new syntax. I know I won't.
So supporting "old-school" Aurelia is pretty much a requirement.

[fkleuver]

We can't expect all our users to migrate their codebase to that new syntax

I think I wasn't clear in what I meant, sorry. I mean that we use the new syntax on our side in some fashion. For example, by doing an AST transform from PLATFORM.moduleName() to import(). If I'm not mistaken, webpack is smart enough to figure out things that the browser can't, like adding the file extension and the /index convention. Does that make sense?

[jods4]

I see.
This would be crazy hard to do, impossible in some cases.

Let's say I have this module in my project:

export const route = { name: 'home', module: moduleName('./home.ts') };

export class HomeViewModel { }

You'd need to figure out in what Aurelia API the route.module export is gonna end up if you want a chance to make a meaningful transformation.
It can mean changing the API on the other end of the pipe.
In general cases it's impossible.

webpack-plugin has cases that are not even linked to actual source code.
In my project I use GlobDependenciesPlugin to ensure everything in my ViewModule folders is considered an Aurelia module and included -- I do not use moduleName in my code.

[fkleuver]

You're right. The transform suggestion must have been a brainfart moment on my part, for some reason I thought of it as a string-to-promise function but of course it's just string-to-string and has nothing to do with imports. My bad.

My original thought was an implementation of aurelia-loader where the load method simply calls import(), but that won't help us either since webpack can't statically analyze dynamic expressions.

So to come back to the complexity problem of doing the transformation that you thought I meant just now - we might be able to do what with v2's AOT once it's done, because that's built as a more or less fully fledged ESNext runtime that evaluates everything that's reachable (it bails only at things like fetch calls and DOM input bindings). That's at least several months away though.

In the mean time, the quickest way for us to achieve this webpack interop is probably to add import-friendly API's in places where it's still missing. @bigopon already added some static API's for templating and routing so we might already be there.
That does mean users will have to migrate their apps to using dynamic import syntax if they want to use module federation - I wonder if that's a type of investment they're willing to make as an addition. @mlassbo ?

If that's acceptable then we don't need to touch the v1 webpack plugin beyond ensuring general v5 api compatibility. Of course we can try to look at module federation specifically, but I'm worried we might be opening a can of worms of something that seems fine in the beginning but issues keep popping up. Going native-compatible really feels like the safer bet.

Eventually we can pull v1 along with the more advanced AOT tooling of v2, but again, that's at least several months away. It's all about what we can offer in the mean time and make the right trade-offs.

[mlassbo]

For our sake there's no rush to make this work with v1. If we can be rest assured there will be support for webpack 5 and module federation for v2 we can continue evaluating the technology to see if it's a good fit for our micro frontend challanges.

[jods4]

I'm a little busy right now but I want to bring the plugin to Webpack 5.
I'd like to support module federation and I think it should be possible (maybe even easy) but there's no guarantee until we try and find out.

@fkleuver A key goal of v2 should be to get rid of aurelia-loader and instead rely on import or stuff that can easily be transformed into import by a basic loader. This means full compatibility with every JS bundler out there without plugins or crazy hacks.

[fkleuver]

@jods4 Yes, v2 has been aurelia-loader-free from the get-go. We only have a thin (optional) transform layer to apply conventions, and all that does is add the import statement and @customElement decorator to the component if there is a same-named html file and the decorator isn't there already. It also transforms <import> tags to import statements.

The AOT I'm talking about is primarily for build-time optimizations and a few minor quality-of-life improvements, such as making autoinject work with interfaces as well instead of just classes. No crazy hacks there in order to make v2 work in the first place, just optional stuff to further improve things.

[ScriptedAlchemy]

Implementation of MF has been pretty easy with other OSS upgrades so far. So I’d imagine leveraging it will be fine as long as you can run WP5 :)

[luboid]

Hi there,
It will be nice if plugin has support for externals in WebPack 5:

https://webpack.js.org/configuration/externals/

@jods4

[jods4]

@luboid I don't think externals require support from this plugin.

• If your external is unrelated to Aurelia (e.g. jQuery) then it should just work.
• If your external contains Aurelia resources, don't even try it's not gonna work. It would require much more than a tweak to this plugin, but also runtime support from aurelia-loader.

[luboid]

@jods4

It will be something needed in the Micro Frontend world. When you have a shell (Aurelia app) that loads different modules (Micro Frontends) as Web Components (Aurelia), here can be used also IFrame to load the standalone app (for isolation and unload code). This is one of the possible ways to achieve the Micro Frontend pattern. And you want to share resources between them with browser cache. Shareable resources (like Aurelia runtime, etc) can be placed JS Libs, by this you can easily fix bugs in vendor's things and vise versa. The application can be left only with the module (App) domain logic code.

We now have around 26 modules on DurandalJS, every one of them comprises one bundle for Durandal itself, one bundle for the module (App) and other major libs are loaded on demand by RequireJS, when a module needs them. And that is wrapped around with Shell.

[bigopon]

@luboid I think what you want can be achieved by a combination of Webpack exposing Aurelia core modules & provide plugin to alias to those globals in the micro apps

[luboid]

@bigopon yes, I think that is the point of WebPack Externals to alias global libs (umd, etc), more interesting is that they have some experimental feature to load them async, however. "Webpack exposing Aurelia" I think this needs to be a most likely builder which to pack Aurelia core modules or to inspect some projects and to build intersect between them, I don't know.

[jods4]

@luboid Bundling "other major libs" as externals should work without problems.
Bundling anything that contains a resource loaded by aurelia-loader (which basically includes all Aurelia resources such as custom elements, custom attributes, value converters, etc.) is not gonna work without some runtime support from aurelia-loader.

Some alternative APIs that are more bundler-friendly were introduced at some point, which may help getting this to work if you tweak your app to use the new APIs. I am not sure if this was properly documented, @bigopon do you have any pointers?

Given the current focus on Aurelia 2, which is built from the scratch to be bundler-friendly, I think it's unlikely that the runtime capabilities of Aurelia 1 change significantly at this point.

[luboid]

@jods4 Then I will be waiting to see what will come with Aurelia 2

I'm not sure that I'm clear, because of that I will post one link, go and see Step 6: Sharing Libraries Between Micro Frontends

[deleonio]

When should Aurelia v2 come?

[deleonio]

PR: #169

[Devvox93]

@bigopon Not to rush you, but do you have enough info to make a rough estimation of when this could be ready?
Also, please let me know if there's a way I can help (by testing maybe?) :)

[bigopon]

Cant really give any update yet atm. Im atively looking into it. Maybe in a few days ill give an update of what the changes could be. Also will publish something that can be tested as soon as I can

Update 1: I'm reading the webpack v5 source code & debugging the webpack v4 with our plugin to see what changes are needed. Have been able to understand more but still not enough to refactor the plugin. I'm positive still 🤞
Update 2: After some more investigation, it seems the majority of the current plugin is still applicable. There'll still be some internal poking & hacking, but it's not too painful

[jboysen]

@bigopon, again, we really appreciate this. Could you give any insights into your findings, and if we can help in any way? Cheers

[bigopon]

I'm swapping pieces of our impl with the equivalence in v5. Will come to module.reasons soon, and push it up somewhere. Will update more.

[bigopon]

Here is my current draft for the needed changes: #170. I've listed work needed for all the plugins in 3 simple parts for now: typings & code & comments. Overtime, more details will be filled in. Will update my progress there as I continue the refactoring.

[bigopon]

An update:

With fdae3aa, it now can build correctly with normal usages.
Test app configuration can be seen at

• https://github.com/aurelia/webpack-plugin/tree/fdae3aab2238bbf61a6269853540a820eb10f115/tests/app-simple
  Webpack configuration for the test app can be seen at
• https://github.com/aurelia/webpack-plugin/blob/fdae3aab2238bbf61a6269853540a820eb10f115/tests/app-simple/webpack.config.js

What left to be tested:

• Glob dependencies plugin (includeAll)
• Sub folder plugin (will test with aurelia-charts)
• better documentation of what's been changed
• comments to aid reviews

cc @jods4

[bigopon]

The work can be tested by installing the plugin from the branch in that PR, the dist files have been built, can be used directly.

[mlassbo]

Nice work! I ran a quick test with fdae3aa and webpack 5 on one of our smaller apps (routing but no chunks) and got everything working fine. Will give it a go with a more complex app soon.

[bigopon]

@mlassbo thanks. With the latest commit. It should be able to cover all range of usages now, at least I hope so. Will merge and do a beta release soon.

[elitastic]

Thanks for your great work, @bigopon!

Looks like the aurelia-bootstrapper package is not yet compatible with webpack 5:

ERROR in ./node_modules/.pnpm/aurelia-bootstrapper@2.3.3/node_modules/aurelia-bootstrapper/dist/native-modules/aurelia-bootstrapper.js
Module not found: Error: Can't resolve 'aurelia-pal-nodejs' in 'C:\my-app\node_modules\.pnpm\aurelia-bootstrapper@2.3.3\node_modules\aurelia-bootstrapper\dist\native-modules'
 @ ./src/main.ts 15:29-60

If I analyzed correctly, "process.browser" is no longer available in webpack 5, that's why aurelia-pal-nodejs is tried to be loaded instead of aurelia-pal-browser. :(

[bigopon]

@elitastic did you get this issue on a plain new project, or an existing one?
If it's an existing one, then it sounds like some thing else, rather than an issue in the plugin entirely. If everything is as expected, it probably shouldn't reach to the process.browser check.

[MaximBalaganskiy]

I’ve had the same error when tried to remove aurelia plugin completely. Webpack recognises “.require()” string suddenly and tries to bundle nodejs package

[elitastic]

It's an existing project. Two further, perhaps relevant disclosures:

• This is a build error, not a runtime error
• There's a manual bootstrap in place for my aurelia app, like:

new AureliaPlugin({ aureliaApp: undefined }),

import { bootstrap } from 'aurelia-bootstrapper';

bootstrap(async (aurelia: Aurelia) =>
{
    aurelia.use
        .standardConfiguration()
        ....
}

Could this be a problem?

[MaximBalaganskiy]

@elitastic the reason is webpack trying to bundle nodejs package. Olds aurelia plug-in does something to suppress that

[elitastic]

Thank you for your quick and good help! I could solve it by explicitly providing "target: 'web'" in webpack config. (I call webpack over gulp, perhaps that leads into some confusion between node and web...)

It builds successfully now, unfortunately I get the following runtime error now:

Template markup must be wrapped in a <template> element e.g. <template> <!-- markup here --> </template>

This is for my root component, which is wrapped withing a template tag (was working with previous aurelia webpack plugin).

Do you have an idea? Sorry for bothering you! :/

[bigopon]

@elitastic no problem at all. Thanks for the issue, it helps fix the error in advance, before others hit it 👍
For your current build issue, is the follow steps enough to reproduce:

• import bootstrap from aurelia-bootstrapper in main
• manual bootstrap
• what else ...?
  ?

[bigopon]

Also, I think we should open a new issue, so that we can avoid spamming others, as the support for v5 is already in. This is more like a bug.

[josundt]

Are we far away from a 5.0.0-beta1 or 5.0.0-rc1 release here?

[Expaso]

@bigopon Thank you very much for your hard work! This enables us to upgrade our dependencies again! I hope Aurelia will be around for a very long time in the future!

[thomas-darling]

@bigopon Great to see the progress here - it's much appreciated!
I just tried to install the plugin from the master branch, but I'm running into this error:

[bigopon]

@thomas-darling it's been fixed.
I'll close this issue for now, so that we don't accidentally spam many people. Feel free to open a new issue to help with bug fixing if you run into one. Thanks everyone for the journey.

[PodgeHeavin]

Is there a RC or Beta version?

[bigopon]

@EisenbergEffect will do a release soon. Please use master for now if possible

[Expaso]

@bigopon You're the best! Thanks a million!

[josundt]

When can we expect a release (alpha, beta, rc)?