feat: Expose a way to set publicPath in remoteEntry

[ScriptedAlchemy]

What kind of change does this PR introduce?

This change exposes a way to set publicPath (see https://webpack.js.org/guides/public-path/#on-the-fly) on a remote so that the remote can still be deployed separately and hosted from /, but when it's hosted from say app/${appName} (with requests still being routed to the root of the remote host, i.e. remoteHost/ - that's where all the assets live, like JS, CSS etc.), it could be configured to make asset requests to /app/${appName}.

Did you add tests for your changes?

Not yet, opening this draft PR to facilitate a discussion.

Does this PR introduce a breaking change?

No

What needs to be documented once your changes are merged?

How to make use of this feature, what it enables.

Alternative Design

An alternative design would be to expose a mutable property called __webpack_public_path__ instead of setPublicPath, in order to match the semantics of webpack core more (see https://webpack.js.org/guides/public-path/#on-the-fly). This has the benefit of it being a getter as well, without having to define a matching getPublicPath().

Open Questions

Where to invoke

Once this feature is exposed, what would be the best place to invoke this? In mine and @sebinsua's experiments, we're dynamically loading remoteEntry.js in the host app code so we have a way of knowing when it's loaded, and can then check window[remoteIdentifier] !== undefined and call window[remoteIdentifier].setPublicPath(`/app/${appName}`).

@ScriptedAlchemy has suggested this kind of config:

remotes: {
  'some-app': {
    publicPath: '/app/some-app/'
  }
}

CommonJS

How would this work in CommonJS environments, would we just ignore it (likely would)?

[webpack-bot]

For maintainers only:

• This needs to be documented (issue in webpack/webpack.js.org will be filed when merged)
• This needs to be backported to webpack 4 (issue will be created when merged)

[webpack-bot]

Thank you for your pull request! The most important CI builds succeeded, we’ll review the pull request soon.

[ScriptedAlchemy]

@sokra we all good to merge this?

[NMinhNguyen]

@evilebottnawi @sokra Could you please review this PR? 🙂

[sokra]

@NMinhNguyen Could you explain why you need this feature? I'm a bit on the fence with this feature and don't think we need it.

There are a bunch of approaches how publicPath can be handled:

Static publicPath

The most common approach is probably to provide a constant publicPath via output.publicPath. This might be server-relative or even absolute if module federation should work across servers.

publicPath inferred from script

One could use document.currentScript.src to automatically infer the publicPath from the script tag. This would allow the bundle to used anywhere. The __webpack_public_path__ module variable can be used to set the publicPath at runtime.

A configuration for this could be:

entry: {
  remote: "./setup-public-path"
},
plugins: [
  new ModuleFederationPlugin({
    name: "remote", // the same as the entry
    // ...
  })
]

// setup-public-path.js
__webpack_public_path__ = document.currentScript.src + "/../";

offer an host API to set the publicPath

You can expose a module to allow the host to set the public path. Note that chunk loading also uses the publicPath so it's important that the module is placed in the entry chunk.

A configuration for this could be:

entry: {
  remote: "./public-path"
},
plugins: [
  new ModuleFederationPlugin({
    name: "remote", // the same as the entry
    exposes: ["./public-path"]
    // ...
  })
]

// public-path.js
export function set(value) { __webpack_public_path__ = value; }

// in the host
const publicPath = await import("remote/public-path");
publicPath.set("/whatever");

// bootstrap app

Note that no other remote module must be loaded before publicPath is set.

[NMinhNguyen]

@sokra thanks very much for your reply! I think your last solution is the one that would benefit us the most - I hadn't considered exposing another remote module just for this - I tried a named export from the same remote module, but yeah it was kind of too late.

Could you explain why you need this feature?

So we have a host application that mounts child applications at /app/app-name, e.g. https://some-host.com/app/foo. Child applications themselves are accessible standalone and are hosted at https://foo-app.com, with their assets living in the root. To avoid CORS issues, we thought that it'd be simplest to host child apps from the host domain, which means that https://foo-app.com would actually be accessible via https://some-host.com/app/foo where https://some-host.com/app/foo/* requests are proxied to https://foo-app.com/*, e.g. for all assets such as JS, CSS, and other static files. Now in order for this approach to work, the host is currently doing

const remoteEntry = window[moduleIdentifier]; // e.g. window['foo']
remoteEntry.setOptions({ publicPath: '/app/foo' });

// Based on https://twitter.com/ScriptedAlchemy/status/1248732087600832512
// because these apps aren't known at build time but are read from a database
const App = React.lazy(() => remoteEntry.get('App').then(factory => 
  factory()
));

// elsewhere
<App />

Hope this was clear. But yeah, I think your last option would help us here.

[yordis]

Same from my side, imagine a tool like Grafana where we install things dynamically and we need to route the publicPath based on the runtime since a service decides the slug for particular subsystems (another webpack build).

[sebinsua]

I've been working with @NMinhNguyen, too.

Could you explain why you need this feature? I'm a bit on the fence with this feature and don't think we need it.

We're in an enterprise environment in which many applications are on different domains and aren't able to control their CORS headers. They'd otherwise be inaccessible but we can route them through a proxy to gain access. So for example we can make http://child-app.other-domain.com available at http://host-app.main-domain.com/app/child-app/.

This fails because when the script tag is loaded, as by default the publicPath is ./ and this points towards the assets of the host application.

We could hard-code this as something else (relative or absolute URL), but then the applications' could no longer be loaded standalone via their deployed location. Therefore, we want a way of dynamically altering the publicPath.

I know that CORS is not necessary for loading <script> tags or assets, however we are using fetch to load data from the same domain, and also we have some code to ensure that if a team changes the variable name within the remoteEntry.js script it still works (we fetch the text and then parse out the global var from the remoteEntry.js script in order to be able to stitch it into whatever we were expecting for a particular application or to error if this is not recoverable - to ensure that we are robust to mistakes by different teams).

There was also another person within the thread that requested a dynamic way of getting the URL, but they had a different reason for wanting this than we do.

(1.) publicPath inferred from script

@sokra At first this seemed to have promise because it would mean we were hands-off at the point of importing*, however I have a feeling this won't work because the first script to evaluate when loading a remoteEntry.js contains an IIFE which internally has a r.p = './' (minified). Therefore this will get executed first and set it up incorrectly, and then later on when the first real entry chunk loads to setup __webpack_public_path__ it will be too late.

(2.) You can expose a module to allow the host to set the public path. Note that chunk loading also uses the publicPath so it's important that the module is placed in the entry chunk.

This might work.

* I guess one issue would be that it wouldn't support modules without being able to use import.meta.url for these cases.

Edit: As an extra complication, the document.currentScript option might also need @soda/get-current-script to support Firefox and IE 9-11.

[sokra]

(1.) publicPath inferred from script

@sokra At first this seemed to have promise because it would mean we were hands-off at the point of importing*, however I have a feeling this won't work because the first script to evaluate when loading a remoteEntry.js contains an IIFE which internally has a r.p = './' (minified). Therefore this will get executed first and set it up incorrectly, and then later on when the first real entry chunk loads to setup __webpack_public_path__ it will be too late.

I'm pretty sure it will work. __webpack_public_path__ is transpiled by webpack to r.p and will override the value provided by config. All entries in entry will execute before the container entry.

[NMinhNguyen]

Actually, me and @sebinsua spoke about this more and I'm not sure how it could work at all. To expand on his comment about __webpack_public_path__ = '/' in remoteEntry.js. This publicPath is what is used by get() to construct a URL to load a remote module, so this value needs to be set correctly, before we can even begin to load a remote module (chicken and egg kind of problem?).

[sokra]

Let me try it and create a test case to validate if this works correctly.

[ScriptedAlchemy]

Might be useful for us to add a webpack hook somewhere in this file, the ability to extend ContainerPlugins interface is very powerful. By adding another property to the interface, I was able to stream chunks over s3 when running in serverless runtime environments.

Going to open a PR with some basic hooks, likely might need some refinements

[NMinhNguyen]

Would the hooks be to facilitate extensions via plugins instead of this particular option landing in core? Or is the outcome still dependent on whether or not it is possible to set the public path from the outside?

[ScriptedAlchemy]

Well, I think that this pattern is likely needed as official scope and support grows. That said, if we have hooks in some key locations, you’d technically be able to add additional code like this to remote entry. I’ve had to copy core files over to extend classes for various things -it’s a hassle and would increase flexibility. Of which - one could set their public path or anything else.

This PR is not exclusive on implementing hooks in the core. I thought of this PR as I worked on exposing the file system api - both needed to add properties to the interface

[NMinhNguyen]

Cool, just trying to get more clarity :) there’s probably some benefit in not exposing functionality and increasing bundle sizes unnecessarily.

[sebinsua]

@sokra

FYI, setting the ModuleFederationPlugin options.name to be the same as an entry name as you did within the examples you gave above, produced the following error message (Conflicting entry option filename = remoteEntry.js vs undefined):

Here's the branch where I tried this.

I'm not sure whether the errror above is expected and unsupported, or whether it means there is a bug in the implementation (@ScriptedAlchemy?).

I tested it because I thought that maybe the config you supplied was a method of forcing the module to be in the same entry chunk so perhaps would avoid the chicken-and-egg problem.

If this isn't going to be fixed, I'd also be happy if either we landed the code in the OP, or for extension hooks which could add this feature to be created within ModuleFederationPlugin.

I tried simpler setups within our codebase, but it turns out that even without a proxy, we can get 404s unless we're able to control the publicPath since we can't set it to / or ./ without it being broken (these paths don't make any sense at all if you're loading something as a remote module). Also, I want the build to be able to be served from multiple domains so don't want to define the publicPath statically. I think this is going to be quite a common ask.

[ScriptedAlchemy]

Yeah that’ll cause problems. We have in the backlog to write stronger validation and error handling

[NMinhNguyen]

@sokra sorry to be pinging you again, but I just wanted to see what the future of this PR is. Would you suggest waiting until extension points are added to the plugin (e.g. via Tapable), and then a userland plugin could be written instead? Or do you think there’s still some value in this functionality in core?

[adamhaeger]

@ScriptedAlchemy Hi Zack! Thanks for the excellent work on module federation, truly a game changer. Im working on implementing module federation in a micro frontend project, and need be able to load code in different environments. It seems to me that without the ability to dynamically set the publicPath like this PR will implement, I will have to create a separate bundle for each environment with the publicPath hardcoded for each environment.

Im therefore wondering what is happening with this PR, or if there is any other way to avoid having to create separate builds for each environment?

[codepunkt]

@sokra

There are a bunch of approaches how publicPath can be handled:

Static publicPath

Can't be switched at runtime.

publicPath inferred from script

offer an host API to set the publicPath

Neither of these work with webpack-dev-server (see webpack/webpack-dev-server#2692)

[chiel]

Is there movement of any sort on this? We're looking into module federation and since we want to deploy the same bundles to our staging environment as to production, this complicates things somewhat. Of course we could build two bundles during CI, but that feels a bit excessive when all we want to do is change a url.

[chiel]

Another thing btw that is not clear to me - since we use the same bundle to deploy both to staging and production, is it possible to dynamically set the urls for the remotes section in the ModuleFederationPlugin?

  plugins: [
    new ModuleFederationPlugin({
      name: 'host',
      remotes: {
        exports: 'exports@http://localhost:3002/v1/entry.js',
      },
      shared: {
        react: { singleton: true },
        'react-dom': { singleton: true },
      },
    }),
  ],

[de-perotti]

@chiel yup, just use environment variables aka process.env.NODE_ENV or anything youd like

[chiel]

Sure, but that's at build time - I meant at run time.

[codepunkt]

It's not possible without hacks at the moment.

This is also a major problem for us because we deploy the module federation stuff on premise, where we don't know the domains/ips/ports beforehand.

/cc @sokra

[mihaisavezi]

It's not possible without hacks at the moment.

@codepunkt What hacks?

Has anyone found any workarounds until this feature gets the green light?

[chiel]

Haven't really seen any movement on this for a while now. Is there a recommended approach to using federated modules with a dynamic public path at runtime?

[ScriptedAlchemy]

Can't set dynamically. but PublicPath:"auto" circumvents the hard coded public paths we have right now. But you can programmatically change it without startup code.

[chiel]

@ScriptedAlchemy but if you have nested federated modules, each of those would need its publicPath set, no? In our case, the federated modules are being loaded from a cdn - wouldn't setting publicPath: 'auto' attempt to load it from the same host?

[chiel]

But you can programmatically change it without startup code.

More specifically, how would I achieve this? By creating a function in each remote entry which allows setting the public path, or?

[mpritchin]

It's not possible without hacks at the moment.

@codepunkt What hacks?

Has anyone found any workarounds until this feature gets the green light?

Hi @mihaisavezi,
It looks pretty weird, but should work:)
You can use https://www.npmjs.com/package/replace-in-file-webpack-plugin to replace static string path with some code, which dynamic build it.
For example:

webpack.config.example:

...
plugins: [ // plugins section in webpack config file
    new ModuleFederationPlugin({
      name: "app1",
      remotes: {
        app2: "app2@__cdn_url__/app2-module/remoteEntry.js",
      },
      shared: { react: { singleton: true }, "react-dom": { singleton: true } },
    }),
    new ReplaceInFileWebpackPlugin([{
      dir: 'dist',
      test: /\.js$/,
      rules: [{
        search: '"__cdn_url__',
        replace: 'window.cdnUrl + "'
      }]
    }]),
    new HtmlWebpackPlugin({
      template: "./public/index.html",
    }),
  ],
...

After that instead "__cdn_url__/app2-module/remoteEntry.js" in bundle we will have window.cdnUrl + "/app2-module/remoteEntry.js"
Now you can set variable window.cdnUrl let's say in index.html

<html>
<head>
    <script>
      window.cdnUrl = 'http://your-cdn.com'
    </script>
</head>
<body>
<div id="root"></div>
<script src="/static/js/main.e41e93ac.js"></script>
</body>
</html>

[goloveychuk]

What is the status of this?
My usecase is

1. remote entry is loaded with requirejs, factory is called async, so currentScript is not correct, public path is wrong.
2. when adding new entry to exposes (set-public-path), it renders webpack.e and trying to load it as async chunk, which cannot be done because of incorrect publicPath.

[sokra]

@goloveychuk see #10703 (comment)

[yordis]

@sokra I would like to know how to tackle the following situation.

Context

Imagine a web app similar to Grafana/GCP/AWS/Azure, a Hub of many subproducts.

If I would like to decentralize the development of the web app, I will need:

• A dynamic system loading
• Be able to deploy the subsystems independently.
• The subsystem bundle doesn't know where its being deploy, either (I don't know the public URL until it is mounted)

You could see a video related to this: Straw Hat Admin and Wepback Module Federation

For example:

Given the following routing:

https://console.sokra.cloud/product/{product_name}/*

I will take the product_name and load some entry points for that particular subsystem/subproduct, then mount it on the page.

The subpath allocated to a particular product is dynamic, so you can't assume that product_name always map to the same service.

And you know where the subsystem bundle deploys until much later.

Problem

How can I bundle the subsystem independently without knowing where they will be deployed?

My shell could load from CDN: https://cdn.sokra.cloud/shell while my subsystems may load from CDN: `https://subsystems.cdn.sokra.cloud/* and so on (the topology changes).

For subsystem A, the public path may be different from where it is being mounted. So in order to load its files, I need to be able to control its public path.

---

Does that make sense to you?

Are you familiar with Grafana?

[NMinhNguyen]

@goloveychuk see #10703 (comment)

Let me try it and create a test case to validate if this works correctly.

@sokra were you ever able to validate this? Per #10703 (comment) I don't think the workarounds outlined in your post would help?

[alexander-akait]

#10703 (comment) keep publicPath(s) outside of bundle (on window or something else) and set them in runtime

[yordis]

I am not sure what you mean @alexander-akait would be nice if you share some article, code snippet, or something. Going back to read again the docs, maybe I missed something.

[NMinhNguyen]

#10703 (comment) keep publicPath(s) outside of bundle (on window or something else) and set them in runtime

I'm still not sure how that is applicable to #10703 (comment) because publicPath defines where bundles are loaded from?

[goloveychuk]

This example works, thanks sokra.
It's important to have same entry and name in ModuleFederationPlugin. It somehow merges remote chunk and entry.

entry: {
    // index: "./src/index",
    remoteasd: "./src/setPublicPath"
},
new ModuleFederationPlugin({
    name: "remoteasd", //should be the same as entry name
    filename: "remote.js",
    library: { type: 'umd', },
    exposes: {'./setPublicPath': "./src/setPublicPath"}, //
    ...
}

Output is

return /******/ (() => { // webpackBootstrap
/******/ 	"use strict";
/******/ 	var __webpack_modules__ = ({

/***/ "./src/setPublicPath.js":
/*!******************************!*\
  !*** ./src/setPublicPath.js ***!
  \******************************/
/***/ ((__unused_webpack_module, __webpack_exports__, __webpack_require__) => {

__webpack_require__.r(__webpack_exports__);
/* harmony export */ __webpack_require__.d(__webpack_exports__, {
/* harmony export */   "setPublicPatASD": () => /* binding */ setPublicPatASD
/* harmony export */ });
const setPublicPatASD = newPublicPath => {
  __webpack_require__.p = newPublicPath;
};

/***/ }),

/***/ "webpack/container/entry/remoteasd":
/*!***********************!*\
  !*** container entry ***!
  \***********************/
/***/ ((__unused_webpack_module, exports, __webpack_require__) => {

var moduleMap = {
	"./setPublicPath": () => { // usually it's __webpack_require__.e here!
		return Promise.resolve().then(() => () => (__webpack_require__(/*! ./src/setPublicPath */ "./src/setPublicPath.js")));
	}
};
var get = (module, getScope) => {
	__webpack_require__.R = getScope;
	getScope = (
		__webpack_require__.o(moduleMap, module)
			? moduleMap[module]()
			: Promise.resolve().then(() => {
				throw new Error('Module "' + module + '" does not exist in container.');
			})
	);
	__webpack_require__.R = undefined;
	return getScope;
};

[ScriptedAlchemy]

for those still reading this. Most cases dynamic public path can be publicPath: auto