Download cmake binary

[Julusian]

Most of the time, we already rely on access to the internet in order to download the headers needed to compile against

We could take this further, and say that if cmake was not found on the path, or if the version found is too old, then we could attempt to download it before failing the build.
That would reduce the complexity for users of libraries built with this, as there would be less chance of their users having to install cmake themselves.

[alejandroclaro]

I think downloading something during "npm install" execution, as well as during cmake execution, is a terrible idea.

Not only is this obscure and buggy, but it also makes it difficult for caching and offline mode in the package manager to work properly.

In particular, I have a project where the audit process is very important, and the internet connection during the build is prohibited for a good reason (the version and hashes of everything used during the build process must be exactly what was audited). Dealing with all these poorly designed npm packages that download its own dependencies with custom mechanics instead of relying on standard package manager mechanisms has been particularly an auditing processes nightmare.

[Julusian]

That is an case I hadn't thought of.
But as I said above if cmake was not found on the path, or if the version found is too old, so by simply making sure to install cmake first, it will not try to download anything. I am also open to adding support for an environment variable, to force it to operate in offline mode and fail instead of trying to download anything. Perhaps we can try to convince prebuild-install and other packages doing downloading to listen to a common variable to do the same?

cmake-js is already downloading (and caching) the nodejs headers that are needed to be able to compile the addons, and this is really an extension of that.
The aim here is to make a more seamless experience for using cmake-js. As right now you have to install cmake yourself, whereas using node-gyp doesnt require installing anything. We can't change what is installed by default, but we can do more behind the scenes for those users who want things to 'just work'.
The requirement for installing cmake manually was making me really consider switching to node-gyp for a project

The other option would be to bundle cmake inside the npm package, but that will be vastly unpopular with many other users, as the size of the npm package would probably hit 100mb easily once you include all the common platforms. And as cmake-js is supposed to be installed as a production dependency, that will affect anyone pulling in a library which uses cmake-js.

[alejandroclaro]

That is an case I hadn't thought of. But as I said above if cmake was not found on the path, or if the version found is too old, so by simply making sure to install cmake first, it will not try to download anything. I am also open to adding support for an environment variable, to force it to operate in offline mode and fail instead of trying to download anything. Perhaps we can try to convince prebuild-install and other packages doing downloading to listen to a common variable to do the same?

cmake-js is already downloading (and caching) the nodejs headers that are needed to be able to compile the addons, and this is really an extension of that. The aim here is to make a more seamless experience for using cmake-js. As right now you have to install cmake yourself, whereas using node-gyp doesnt require installing anything. We can't change what is installed by default, but we can do more behind the scenes for those users who want things to 'just work'. The requirement for installing cmake manually was making me really consider switching to node-gyp for a project

The other option would be to bundle cmake inside the npm package, but that will be vastly unpopular with many other users, as the size of the npm package would probably hit 100mb easily once you include all the common platforms. And as cmake-js is supposed to be installed as a production dependency, that will affect anyone pulling in a library which uses cmake-js.

I get what you say, but this has to stop somewhere. What about the compilers. Do you expect this to download GNU GCC, or CLang etc to make the experience seamless?

Downloading NodeJS/Electron headers is also a terrible idea. Just to exemplify, recently the process was broken due to changing the download path of the headers for Electron. Which does not make sense to me that to be able to re-generate a system, you are forced to wait for a new version and then increase the version of the dependencies.

This is messy, obscure, fragile and insecure. There are no integrity and authenticity validation over the download source and this is happening in many NPM packages. You as a client have little or no control over this, and it is becoming extremely complicated to have control of the construction process of a simple application. This is extremely worrying.

I understand your frustration. It's real. node-gyp is just extremely basic and terrible in practice for more than very basic native modules. cmake-js, NodeJS, Electron and the NPM ecosystem in general is far from one could be expecting too in terms of making things seamless in a standardized way. There are just too many hacks around everything.

[Julusian]

I get what you say, but this has to stop somewhere. What about the compilers. Do you expect this to download GNU GCC, or CLang etc to make the experience seamless?

Well no, I am only aiming for more seamless than today. Ideally I would like for it to be possible to use cmake-js anywhere node-gyp works without needing to install anything more.
I think it is reasonable to expect that a compiler is installed, installing nodejs on windows even helps you with that. But if we want to lure libraries away from node-gyp I think it has to be just as easy for their users to get it compiling. As a library author, node-gyp still has an appeal for some things as it is much more likely to work everywhere without a constant stream of support requests from users who dont know what cmake is or how to install it.

Honestly I am unsure if I will do this task, I am still unsure if it is worth the complexity. If libraries were to provide a sufficient range of prebuilds, then in theory it wouldnt be necessary to compile locally and would negate the need for this.
Perhaps it should only be done for windows and macos? With linux it is really simple to install cmake the same way as gcc/clang, and will typically have more knowledgeable users, so maybe serves no purpose?

Downloading NodeJS/Electron headers is also a terrible idea. Just to exemplify, recently the process was broken due to changing the download path of the headers for Electron. Which does not make sense to me that to be able to re-generate a system, you are forced to wait for a new version and then increase the version of the dependencies.

You will have to complain to nodejs about that one. They are the ones who don't ship the headers with the windows binaries (but do for every other os). I havent checked what electron do, but I assume it is the same.
Perhaps on other platforms we could try and use the installed version before downloading a copy? I suspect finding it will be the challenge, especially to make it work on every distro and install method.

Yes the electron download path changing is annoying and a good example of the flaws of this, but I dont see a practical solution here to avoid downloading the headers. Asking the user to manually download the headers to the right place is going to confuse so many first-time/casual contributors to open source projects using these libraries. And putting them in an npm package will make it way too large.
Between this mirror changing, and the occasional issue from electron changing things in beta versions, if it werent for node-api now being a thing I would be really tempted to mirror the headers somewhere to make the url and contents be under the control of this library.

That said, in 7.0 this will be a lot better for projects using node-api. Thanks to https://github.com/nodejs/node-api-headers, there is a npm package containing everything we need (except msvc, but we can generate the missing piece). We dont need such a specific version of them due to their stability promise. So in 7.0, targetting node-api will not download anything on any platform. The exception is if you need to use extra headers which break the node-api stability guarantee, but that isnt supported in 7.0 yet.

This is messy, obscure, fragile and insecure. There are no integrity and authenticity validation over the download source and this is happening in many NPM packages. You as a client have little or no control over this, and it is becoming extremely complicated to have control of the construction process of a simple application. This is extremely worrying.

Yeah, that is a good point. But it will be very possible to verify the checksum/signature of the binary before allowing it to be executed. And again this cmake downloading will be entirely avoidable. I havent looked in too much depth at the header downloading, but I think it does verify the checksum, I dont know if it is possible to check a signature. Equally, that signature checking is only going to work until the key expires, and cause the pain of the mirror path changing again.

---

If you are using cmake-js in an application rather than a library, then I think it is then also reasonable to expect contributors to have cmake installed.

But if your application is using a library which uses cmake-js, then I think it shouldnt be necessary. Requiring cmake can likely be avoided 99% of the time by providing a complete set of prebuilds. But if the library you are using is missing a prebuild then that can be really tedious for your application.
Dealing with casual contributors not reading or understanding the output of yarn install or npm i is really tedious. It often takes some prompting to be able to see any meaningful log output, if they even realise that the yarn install is what went wrong. Often they assume it was successful without reading, and complain that it doesnt run properly.

Alternatively, I could make the getting started guide for the application say to install cmake, but I firstly need to know that a dependency is using cmake-js (if it has some prebuilds, I may be unaware). And do I really want to make every contributor install cmake just in case there isnt a prebuild for them? What platforms should be made to install it?
But by cmake-js downloading cmake, that shold remove any need to think about mentioning cmake for at least windows and macos. Maybe linux does need to, but users there are more likely to figure it out themselves.

Again, this is all about easing the pain of using libraries which use cmake-js, and trying to make it as easy for users as projects based on node-gyp.