SendMsg Method Not-Concurrency Safe

[steve-gray]

What version of gRPC are you using?

We are using go 1.10, with a GRPC 3 service using the latest versions of grpc-go and the protoc code generator.

What version of Go are you using (go version)?

1.10

What operating system (Linux, Windows, …) and version?

OSX, Linux

What did you do?

We have a service where clients connect with a subscription profile, and we respond each time an item arrives on the corresponding response stream. We recently changed the sending send to use multiple goroutines to fetch data from multiple sources, and send them all to the same client.

What did you expect to see?

Concurrent sends should serialize onto the connection internally for transmission to the remote side. We were not able to achieve this until we wrapped the Send(item) call with a sync.Mutex.

What did you see instead?

It appears that the SendMsg method is not thread/goroutine safe - and we observed that during a race, one of the requests would succeed, with all others hanging indefinately - as if there's a mechanism that's trying to mitigate this, but it's not working as-intended.

[electronjoe]

FWIW I ran across your report while troubleshooting another proximal issue - the documentation for
SendMsg indicates it is NOT safe to simultaneously use from two goroutines:

// It's safe to have a goroutine calling SendMsg and another goroutine calling
// recvMsg on the same stream at the same time.
// But it is not safe to call SendMsg on the same stream in different goroutines.

https://godoc.org/google.golang.org/grpc#Stream

[steve-gray]

For the sake of a single mutex at the stage where it interacts with the stream, that can be resolved - doing this now myself in code around Recv and Send calls - but that means stuff that doesn’t need to be serialised (ie: marshalling/unmarshal) is also inside the block, whereas if this was done on the GRPC package the scope could be minimised Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Scott Moeller <notifications@github.com> Sent: Monday, May 21, 2018 1:19:29 AM To: grpc/grpc-go Cc: Steve Gray; Author Subject: Re: [grpc/grpc-go] SendMsg Method Not-Concurrency Safe (#2094) FWIW I ran across your report while troubleshooting another proximal issue - the documentation for SendMsg indicates it is NOT safe to simultaneously use from two goroutines: // It's safe to have a goroutine calling SendMsg and another goroutine calling // recvMsg on the same stream at the same time. // But it is not safe to call SendMsg on the same stream in different goroutines. https://godoc.org/google.golang.org/grpc#Stream — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#2094 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AN45vx0Zap2iHb8C4ItWVOUhjmucyu4mks5t0YmBgaJpZM4UFpA0>.

[lyuxuan]

This has been discussed in #1879, please refer to it for more info.