Support meteor.nodeModules.recompile package.json configuration option.

[benjamn]

In the spirit of meteor.mainModule and meteor.testModule (#9690), this PR introduces a new package.json-based configuration for selecting npm packages to recompile using the Meteor compiler plugins system. This configuration is intended to replace the unconditional compilation introduced by #10585.

Example:

{
  "name": "your-application",
  ...
  "meteor": {
    "mainModule": ...,
    "testModule": ...,
    "nodeModules": {
      "recompile": {
        "very-modern-package": ["client", "server"],
        "alternate-notation-for-client-and-server": true,
        "somewhat-modern-package": "legacy",
        "another-package": ["legacy", "server"]
      }
    }
  }
}

The keys of the meteor.nodeModules.recompile configuration object are npm package names, and the values specify for which bundles those packages should be recompiled using the Meteor compiler plugins system, as if the packages were part of the Meteor application.

For example, if an npm package uses const/let syntax or arrow functions, that's fine for modern and server code, but you would probably want to recompile the package when building the legacy bundle. To accomplish this, specify ”legacy" or ["legacy"] as the value of the package's property, similar to somewhat-modern-package above. These strings and arrays of strings have the same meaning as the second argument to api.addFiles(files, where) in a package.js file.

This configuration serves pretty much the same purpose as symlinking an application directory into node_modules, but without any symlinking: https://forums.meteor.com/t/litelement-import-litelement-html/45042/8?u=benjamn

The meteor.nodeModules.recompile configuration currently applies to the application node_modules directory only (not to Npm.depends dependencies in Meteor packages). Recompiled packages must be direct dependencies of the application.

[CaptainN]

Would it be possible to add an additional prop which would override the source entry point defined in "main" or "module".

I'm thinking about "material-ui" in particular. It ships ES6+ modules, in addition to pre-transpiled esm files, which is what it links in it's module field. But there's no obvious way to access that. With an override, we could do that ourselves, and possible gain some benefit in the modern bundle.

  "main": "./index.js",
  "module": "./esm/index.js",
  "name": "@material-ui/core",

That's what shows in "@material-ui/core"'s package.json file. If we could redefine that to use "module": "./es/index.js" that could be very useful. I'm not sure how many packages deploy assets such as this.

{
  "meteor": {
    "nodeModules": {
      "recompile": {
        "@material-ui/core": true
      },
      "alternaiveModule": { // or maybe "moduleEntries"
        "@material-ui/core": "./es/index.js"
      }
    }
  }
}

[trusktr]

I feel like compile may be a better term than recompile, because not all node_modules code is necessarily compiled in the first place. Some people publish hand-written JS for common denominators of environments, or just modern code and then use their own build tools to consume the packages.

[trusktr]

Would it be possible to add an additional prop which would override the source entry point defined in "main" or "module".

That would be a great feature!

Sometimes module might point to code with everything compiled except for import/export statements, but we may want to actually import relative to the src folder which some packages do publish. This would give us more control over the output by letting us compile original sources (when those are present).

It could get tricky though, as tsconfig files can vary from project to project (I publish tsconfig files, and in some cases I compile TypeScript from node_modules because it's just easy to publish that way and consume them in some build setup designed to work with those files).

[HemalR]

@benjamn Thanks for this! I'm in the process of trying to decipher which package is causing my syntax error on IE:

Does the pattern you mentioned allow the names of meteor packages as well as npm packages?

I.e. Could I use this in my package.json:

"meteor": {
    "nodeModules": {
      "recompile": {
        "aldeed:collection2": [
          "client",
          "server"
        ]
      }
    }
  },

At least, the collection2 package is what IE's terrible debugger seems to be pointing to (though I can't see what/why):

Ps - If anybody has any tips/tricks on trying to narrow down the package in question that IE doesn't like, then I'd love to hear it.

Thanks in advance

[CaptainN]

I don't think recompile is meant for atmosphere packages, but you do need a setting for simpl-schema. Try:

"meteor": {
    "nodeModules": {
      "recompile": {
        "simpl-schema": "legacy"
      }
    }
  },

[HemalR]

Ok, after many hours yesterday, I decided to try create a new project from scratch to see if I can find it that way. Repo here: https://github.com/HemalR/meteor-ie-breaker/branches

Two branches - the master doesn't work and the 'works' branch does.

The only difference between the two is the presence of the aldeed:collection2 package. I tried adding simpl-schema to the recompile list under package.json to see if that makes a difference, but no luck there.

So that does seem to be the culprit. As I type this, I'm wondering whether the collection2 package is even necessary or whether it has been replaced (don't know why that thought is going through my head). I'll explore and update here if that is the case...

[CaptainN]

There is a bug with the array syntax #10791 try it with the string "legacy" instead.

[HemalR]

@CaptainN YES! That does the trick (on the tester package at least - now to try on production)

[HemalR]

Alright, confirmed that this added to package.json fixes things:

 "meteor": {
    "mainModule": {
      "client": "./client/main.js",
      "server": "./server/main.js"
    },
    "nodeModules": {
      "recompile": {
        "simpl-schema": "legacy",
        "query-string": "legacy"       
      }
    }
  },

@CaptainN - Thank you SO MUCH - I'm well into double digits of hours spent on this. B****y IE!

[CaptainN]

@HemalR You can also just set it to true instead of legacy. That will recompile it for the modern and server bundles too (in case a particular package needs that - Svelte packages generally need that).

[HemalR]

I'm a tad confused by that - the server handles ES6 doesn't it? (And so does Svelte?).

If so, then what is the reasoning for the server option (clearly I'm missing something obvious).

[CaptainN]

The server handles modern syntax, until new syntax is added (maybe like optional chaining or something like that). If a module has that, it won't work without being transpiled. Svelte always needs to be transpiled, since it's a compiler based framework.

[HemalR]

Ah got it, for the Svelte compiler 👍

Svelte is defo on my list of to-try. I imagine it would actually eliminate (or drastically reduce) IE (or any browser specific) issues with any/all packages.

[jamesgibson14]

@benjamn I have a package setup of for recompile and the recompiling is working just fine, however the package imports a few other npm package and the are completely blank with no code, even thought node module is installed. Is there something wrong I am doing?

[crapthings]

@CaptainN @filipenevola

will recompile works with babel plugin?

i'm writing a babel plugin for our project, but it looks it doesn't work with plugin

[filipenevola]

Hi @crapthings checking the code and also this comment from Ben (f61ba60)

The keys of the meteor.nodeModules.recompile configuration object are npm
package names, and the values specify for which bundles those packages
should be recompiled using the Meteor compiler plugins system, as if the
packages were part of the Meteor application.

It seems that it's going to recompile using Meteor configurations so if your plugin is applied to your Meteor code it should also be applied in the package code.

[CaptainN]

@crapthings I don't know - never tried it. I do know that the meteor build system will not respond to changes you make to your local build plugin files until you restart the build system. So, change file, restart meteor, etc.