Skip to content

Node 14 ArrayBuffer changes causes grpc to crash. #1407

Closed
@strigona-worksight

Description

@strigona-worksight

Problem description

Node 14 has some breaking changes regarding ArrayBuffers as noted here nodejs/node#30782 and https://nodejs.org/en/blog/release/v14.0.0/

Multiple ArrayBuffers pointing to the same base address are no longer allowed by V8. This may impact native addons.

Reproduction steps

I don't have solid/standalone reproduction steps as of yet, but there is a relevant issue here with the same stacktrace that we received: pulumi/pulumi#4258 (comment)

Environment

  • OS name, version and architecture: Ubuntu 19.10
  • Node version: 14.0.0
  • Node installation method: nvm
  • If applicable, compiler version:
  • Package name and version: gRPC@1.24.2

Additional context

nodejs/node#32463

Activity

murgatroid99

murgatroid99 commented on May 6, 2020

@murgatroid99
Member

Thank you for the information. I will try to fix this; any additional information will help.

I suggest switching to @grpc/grpc-js. We will be concentrating most of our development efforts on that library going forward.

strigona-worksight

strigona-worksight commented on May 6, 2020

@strigona-worksight
Author

Thanks for your reply! We'll look into switching over to the grpc-js.

strigona-worksight

strigona-worksight commented on Jun 9, 2020

@strigona-worksight
Author

We had to hold off on switching to grpc-js as the healthcheck is still using the native grpc.

We tried out grpc 1.24.3 with node v14.4.0 and haven't had any issues yet. The release says Node 14 support was added, though I couldn't find any commits tied to that claim. Not sure if this is safe to close or not.

hugebdu

hugebdu commented on Sep 6, 2020

@hugebdu
Contributor

Same here

#
# Fatal error in , line 0
# Check failed: result.second.
#
#
#
#FailureMessage Object: 0x7ffeefbfe100
 1: 0x100112612 node::NodePlatform::GetStackTracePrinter()::$_3::__invoke() [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 2: 0x10100c7b2 V8_Fatal(char const*, ...) [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 3: 0x1004d37b4 v8::internal::GlobalBackingStoreRegistry::Register(std::__1::shared_ptr<v8::internal::BackingStore>) [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 4: 0x1001f9436 v8::ArrayBuffer::GetBackingStore() [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 5: 0x10006cfa5 node::ArrayBufferViewContents<char, 64ul>::Read(v8::Local<v8::ArrayBufferView>) [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 6: 0x10008684c void node::Buffer::(anonymous namespace)::StringSlice<(node::encoding)1>(v8::FunctionCallbackInfo<v8::Value> const&) [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 7: 0x1002582e8 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 8: 0x10025787c v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
 9: 0x100256fa2 v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
10: 0x100a780b9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]
11: 0x100a11382 Builtins_InterpreterEntryTrampoline [/Users/daniels/.nvm/versions/node/v14.9.0/bin/node]

Process finished with exit code 132 (interrupted by signal 4: SIGILL)

cannot switch to @grpc/grpc-js due to missing deadlines/cancellations support

:(

strigona-worksight

strigona-worksight commented on Sep 8, 2020

@strigona-worksight
Author

@hugebdu What version of grpc and node are you using? We haven't had any crashes with the latest versions.

hugebdu

hugebdu commented on Sep 9, 2020

@hugebdu
Contributor

@strigona-worksight

grpc: 1.24.3
node: 14.9.0

happens only in gRPC server. client is fine.

#
# Fatal error in , line 0
# Check failed: result.second.
#
#
#
#FailureMessage Object: 0x7ffc3bbf0ed0
 1: 0xa6f5f1  [node]
 2: 0x19cb8c4 V8_Fatal(char const*, ...) [node]
 3: 0xe58379 v8::internal::GlobalBackingStoreRegistry::Register(std::shared_ptr<v8::internal::BackingStore>) [node]
 4: 0xba38a8 v8::ArrayBuffer::GetBackingStore() [node]
 5: 0x9dd394  [node]
 6: 0xbe254b  [node]
 7: 0xbe3af6  [node]
 8: 0xbe4176 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [node]
 9: 0x13fe0b9  [node]
sh: line 1: 19292 Illegal instruction     (core dumped)
hugebdu

hugebdu commented on Oct 27, 2020

@hugebdu
Contributor

minimal reproduction code I've managed to distill.
related to grpc.Metadata

https://github.com/hugebdu/node14-grpc-core-crashes

murgatroid99

murgatroid99 commented on Oct 27, 2020

@murgatroid99
Member

OK, knowing that the bug triggers from receiving metadata narrowed it down enough. I think the issue is with the use of this function. I think I can publish a patch soon to fix that.

hugebdu

hugebdu commented on Oct 27, 2020

@hugebdu
Contributor

that would be awesome!
unfortunately we cannot switch to js implementation and obviously want to upgrade to node 14 once LTS.

thanks a lot

murgatroid99

murgatroid99 commented on Nov 12, 2020

@murgatroid99
Member

I have published version 1.24.4 with a change to address this bug. Please try that out and tell us if you still have these problems.

hugebdu

hugebdu commented on Nov 16, 2020

@hugebdu
Contributor

@murgatroid99
works like a charm!
confirmed.

thanks a lot.

murgatroid99

murgatroid99 commented on Nov 16, 2020

@murgatroid99
Member

OK, I'm going to consider this fixed and close the issue. If anyone experiences this kind of error with version 1.24.4 or later, please comment and I will reopen the issue.

3 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @murgatroid99@hugebdu@strigona-worksight@bilihuang

        Issue actions

          Node 14 ArrayBuffer changes causes grpc to crash. · Issue #1407 · grpc/grpc-node