Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[binder] Handle outbound flow control #27243

Merged
merged 1 commit into from
Sep 11, 2021

Conversation

TaWeiTu
Copy link
Member

@TaWeiTu TaWeiTu commented Sep 3, 2021

Tested locally with Java servers.

Depends on #27228.

@TaWeiTu TaWeiTu added the release notes: no Indicates if PR should not be in release notes label Sep 3, 2021
@TaWeiTu TaWeiTu force-pushed the flow-control-outbound branch 5 times, most recently from 97e51fa to 877fb1a Compare September 6, 2021 13:25
@TaWeiTu
Copy link
Member Author

TaWeiTu commented Sep 8, 2021

Ping.

while (ptr < data.size()) {
while (num_outgoing_bytes_ >=
num_acknowledged_bytes_ + kFlowControlWindowSize) {
cv_.Wait(&mu_);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably shouldn't block here, it would be pretty hard to debug if we don't receive ACK from the other end for some reason (since the program will just hang).

We probably want a helper class or something to handle chunking so that flow control logic (which can be orthogonal to other logic) don't make WireWriterImpl become too complex

If that's too much to implement in the same PR, probably add a 10ms deadline to cv_.Wait() (and log ERROR if deadline exceeded) and add a TODO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about having a new class like FlowControl to take care of the flow control mechanism. The transport now holds a FlowControl (which internally holds a WieWtier to deal with the wire format) and the transport sends RPC call through FlowControl instead of WireWriter.

What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's probably too much abstraction? It should be possible to implement the flow control logic within WireWriterImpl and make the code readable? Probably introduce some kind of queue as member variable and some member functions

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@sifmelcara
Copy link
Member

Seems like some inbound code are also in the outbound commit? Will review after inbound commit is merged

@TaWeiTu
Copy link
Member Author

TaWeiTu commented Sep 9, 2021

Seems like some inbound code are also in the outbound commit? Will review after inbound commit is merged

Sure, the inbound flow-control PR has been merged.

@TaWeiTu TaWeiTu force-pushed the flow-control-outbound branch 6 times, most recently from c3ae8a0 to 034e539 Compare September 10, 2021 09:25
src/core/ext/transport/binder/wire_format/wire_writer.cc Outdated Show resolved Hide resolved
src/core/ext/transport/binder/wire_format/wire_writer.cc Outdated Show resolved Hide resolved
grpc_core::MutexLock lock(&mu_);
RETURN_IF_ERROR(binder_->PrepareTransaction());
WritableParcel* parcel = binder_->GetWritableParcel();
RETURN_IF_ERROR(parcel->WriteInt64(num_bytes));
return binder_->Transact(BinderTransportTxCode::ACKNOWLEDGE_BYTES);
}

void WireWriterImpl::RecvAck(int64_t num_bytes) {
grpc_core::MutexLock lock(&mu_);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no deadlock?

Is it because cv_.WaitWithDeadline allows us to obtain mu_ here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting on a conditional variable will release the mutex IIRC.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. (I know it is true in boost, just want to confirm if that's also the case in abseil)

src/core/ext/transport/binder/wire_format/wire_writer.h Outdated Show resolved Hide resolved
@TaWeiTu TaWeiTu force-pushed the flow-control-outbound branch 3 times, most recently from 67228b0 to 840c703 Compare September 11, 2021 01:57
@TaWeiTu TaWeiTu enabled auto-merge (squash) September 11, 2021 07:04
@TaWeiTu TaWeiTu merged commit 52e5b64 into grpc:master Sep 11, 2021
lidizheng pushed a commit to lidizheng/grpc that referenced this pull request Sep 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lang/core release notes: no Indicates if PR should not be in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants