Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(spanner): fix retry unit tests with RESOURCE_EXHAUSTED errors #10117

Merged
merged 2 commits into from
May 6, 2024

Conversation

rahul2393
Copy link
Contributor

Fixes: #10070, #10069

@rahul2393 rahul2393 requested review from a team as code owners May 6, 2024 06:27
@product-auto-label product-auto-label bot added the api: spanner Issues related to the Spanner API. label May 6, 2024
@rahul2393 rahul2393 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 6, 2024
@@ -731,7 +731,7 @@ func (s *inMemSpannerServer) BatchCreateSessions(ctx context.Context, req *spann
s.mu.Lock()
defer s.mu.Unlock()
if s.maxSessionsReturnedByServerInTotal > int32(0) && int32(len(s.sessions)) >= s.maxSessionsReturnedByServerInTotal {
return nil, gstatus.Error(codes.ResourceExhausted, "No more sessions available")
return nil, gstatus.Error(codes.OutOfRange, "No more sessions available")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to change this to a different error code? I didn't understand this change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the mock response, it has nothing to do with actual cloud spanner backend response code, but unfortunately since ResourceExhausted is treated as retryable now we are changing to some other error code to make it fail and not retry the error which was assumption when writing the test case/this mock code before.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Can we document this assumption as a comment on top of the mock response and the test? This way developers can understand the reason behind sending this irrelevant error code?

Also how about marking it as INTERNAL error? OutOfRange error doesn't seem to be relevant to the problem? I will leave it up to you since this mock error code is not serving any purpose.

Copy link
Contributor Author

@rahul2393 rahul2393 May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually OutOfRange suits more here https://developers.google.com/actions-center/reference/grpc-api/status_codes, since in mock code we want to make sure session count never exceeds maxSessionsReturnedByServerInTotal, while Internal Errors should be returned when

Internal errors. This means that some invariants expected by the underlying system have been broken. This error code is reserved for serious errors.

Copy link
Contributor

@harshachinta harshachinta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for fixing this Rahul.
I would prefer documenting the behavior on top of the assertion and the place where the mock error is returned. Otherwise this becomes confusing!

@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 6, 2024
@rahul2393 rahul2393 merged commit b91f658 into main May 6, 2024
12 checks passed
@rahul2393 rahul2393 deleted the fix_resource_exhausted_errors branch May 6, 2024 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the Spanner API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

spanner: fix TestBatchCreateSessions_ServerExhausted
3 participants