Node 14 compatibility

[jaxxstorm]

We need to determine if node 14 works with Pulumi. There are related issues indicated that there may be some breaking grpc changes that might affect us:

nodejs/node#32463
grpc/grpc-node#1407

[danfhernandez]

Ran into a node 14 issue this morning that may be related. I downgraded and things started working properly. See error log below:

node-pre-gyp WARN Using needle for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(404): https://node-precompiled-binaries.grpc.io/grpc/v1.24.2/node-v83-darwin-x64-unknown.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for grpc@1.24.2 and node@14.2.0 (node-v83 ABI, unknown) (falling back to source compile with node-gyp) 
No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.

gyp: No Xcode or CLT version detected!
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
gyp ERR! System Darwin 19.4.0
gyp ERR! command "/usr/local/Cellar/node/14.2.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--library=static_library" "--module=/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown/grpc_node.node" "--module_name=grpc_node" "--module_path=/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown" "--napi_version=6" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v83"
gyp ERR! cwd /Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc
gyp ERR! node -v v14.2.0
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/Cellar/node/14.2.0/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --library=static_library --module=/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown/grpc_node.node --module_name=grpc_node --module_path=/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1051:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
node-pre-gyp ERR! System Darwin 19.4.0
node-pre-gyp ERR! command "/usr/local/Cellar/node/14.2.0/bin/node" "/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build" "--library=static_library"
node-pre-gyp ERR! cwd /Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc
node-pre-gyp ERR! node -v v14.2.0
node-pre-gyp ERR! node-pre-gyp -v v0.14.0
node-pre-gyp ERR! not ok 
Failed to execute '/usr/local/Cellar/node/14.2.0/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --library=static_library --module=/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown/grpc_node.node --module_name=grpc_node --module_path=/Users/dan/Documents/playground/support/s3-zendesk-import/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
npm WARN s3-zendesk-import@ No description
npm WARN s3-zendesk-import@ No repository field.
npm WARN s3-zendesk-import@ No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! grpc@1.24.2 install: `node-pre-gyp install --fallback-to-build --library=static_library`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the grpc@1.24.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/dan/.npm/_logs/2020-05-08T15_00_50_613Z-debug.log

[lukehoban]

grpc/grpc-node#1407

In 2.0 we no longer use grpc we use @grpc/grpc-js.

npm ERR! Failed at the grpc@1.24.2 install script.

@danfhernandez Were you using Pulumi 1.x (or old packages for any dependencies?)

[lukehoban]

Hmm - yeah - this repros on pulumi new aws-typescript with Node 14. Sounds like we must still have some transitive dependency on grpc?

[leezen]

It looks like it's b/c awsx->@pulumi/docker->@pulumi/pulumi (1.14)->grpc

[danfhernandez]

package.json:

{
    "name": "node-14-error",
    "devDependencies": {
        "@types/node": "^10.0.0"
    },
    "dependencies": {
        "@pulumi/pulumi": "^2.0.0",
        "@pulumi/aws": "^2.0.0",
        "@pulumi/awsx": "^0.18.10"
    }
}

Error log:

/Users/dan/Documents/playground/support/node-14-error % pulumi version 
v2.1.0
/Users/dan/Documents/playground/support/node-14-error % node -v
v14.0.0
/Users/dan/Documents/playground/support/node-14-error % pulumi new aws-typescript
This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (node-14-error) 
project description: (A minimal AWS TypeScript Pulumi program) 
Created project 'node-14-error'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
stack name: (dev) 
Created stack 'dev'

aws:region: The AWS region to deploy into: (us-east-1) 
Saved config

Installing dependencies...


> deasync@0.1.20 install /Users/dan/Documents/playground/support/node-14-error/node_modules/deasync
> node ./build.js

`darwin-x64-node-14` exists; testing
Binary is fine; exiting

> grpc@1.24.2 install /Users/dan/Documents/playground/support/node-14-error/node_modules/grpc
> node-pre-gyp install --fallback-to-build --library=static_library

No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.

gyp: No Xcode or CLT version detected!
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
gyp ERR! System Darwin 19.4.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--library=static_library" "--module=/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown/grpc_node.node" "--module_name=grpc_node" "--module_path=/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown" "--napi_version=6" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v83"
gyp ERR! cwd /Users/dan/Documents/playground/support/node-14-error/node_modules/grpc
gyp ERR! node -v v14.0.0
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --library=static_library --module=/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown/grpc_node.node --module_name=grpc_node --module_path=/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1051:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
node-pre-gyp ERR! System Darwin 19.4.0
node-pre-gyp ERR! command "/usr/local/bin/node" "/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build" "--library=static_library"
node-pre-gyp ERR! cwd /Users/dan/Documents/playground/support/node-14-error/node_modules/grpc
node-pre-gyp ERR! node -v v14.0.0
node-pre-gyp ERR! node-pre-gyp -v v0.14.0
node-pre-gyp ERR! not ok 
Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --library=static_library --module=/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown/grpc_node.node --module_name=grpc_node --module_path=/Users/dan/Documents/playground/support/node-14-error/node_modules/grpc/src/node/extension_binary/node-v83-darwin-x64-unknown --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! grpc@1.24.2 install: `node-pre-gyp install --fallback-to-build --library=static_library`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the grpc@1.24.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/dan/.npm/_logs/2020-05-08T16_11_06_081Z-debug.log
error: npm install failed; rerun manually to try again, then run 'pulumi up' to perform an initial deployment: exit status 1
/Users/dan/Documents/playground/support/node-14-error % 

[lukehoban]

Appears this is actually just a template issue - we still depend on v0.18.0 of awsx in the default aws templates.

[leezen]

In case others come across this issue, if you have awsx that's not ^0.20.0, you'll need to bump the dependency so that you won't transitively pick up @pulumi/docker 1.x which will bring in a version of @pulumi/pulumi that depends on grpc

[leezen]

I'm going to leave the issue open to discuss if there are other areas of compatibility we need to confirm.

[lukehoban]

@leezen I believe folks have been using Pulumi with Node 14 now for several months - anything else we need to track before closing this out? Notably we don't appear to be testing against Node 14 in CI - even though it is the "Current" release right now (https://nodejs.org/en/about/releases/) - so we likely should at least fix that.

[leezen]

Agreed we need to have this in CI before closing this out. Will queue that up for the next iteration.