Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pubsub): report publisher outstanding metrics #6187

Merged

Conversation

schallert
Copy link
Contributor

Fixes #6180.

@schallert schallert requested review from a team as code owners June 15, 2022 23:33
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: pubsub Issues related to the Pub/Sub API. labels Jun 15, 2022
@schallert
Copy link
Contributor Author

Marking as draft for a bit. Based on internal testing, there's one more path where we need to upsert the topic tag on the context.

@schallert schallert marked this pull request as ready for review June 16, 2022 17:10
@schallert
Copy link
Contributor Author

Updated with the tag fix. With the most recent commit, the behavior we see in our testing is:

  1. While metrics are buffered but before the first publish call:
opencensus_cloud_google_com_go_pubsub_publisher_outstanding_messages{topic="<INTERNAL_TOPIC>"} 8
  1. After the publish call, outstanding goes back to 0 and published_messages increments to the total:
opencensus_cloud_google_com_go_pubsub_published_messages{error="",status="OK",topic="<INTERNAL_TOPIC>"} 8
opencensus_cloud_google_com_go_pubsub_publisher_outstanding_messages{topic="<INTERNAL_TOPIC>"} 0

@hongalex hongalex added the kokoro:run Add this label to force Kokoro to re-run the tests. label Jun 16, 2022
@kokoro-team kokoro-team removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Jun 16, 2022
func (f *flowController) recordOutstandingBytes(ctx context.Context, n int64) {
if f.purpose == flowControllerPurposeTopic {
recordStat(ctx, PublisherOutstandingBytes, n)
} else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're not going to check the unknown value here, we should either remove that value above or explicitly check f.purpose == flowControllerPurposeSubscription here.

case flowControllerPurposeTopic:
recordStat(ctx, PublisherOutstandingMessages, n)
case flowControllerPurposeSubscription:
fallthrough
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made this fall through to maintain backwards compatibility with the old behavior. However, as written there shouldn't be any cases where a "real" client has an unknown purpose, so I'm happy to make default a noop. Or I can remove the unknown value entirely and default to flowControllerPurposeSubscription.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should remove the unknown value since flowcontroller is only used for publisher/subscribers here. Thanks!

@hongalex hongalex added the kokoro:run Add this label to force Kokoro to re-run the tests. label Jun 23, 2022
@kokoro-team kokoro-team removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Jun 23, 2022
@hongalex hongalex added the kokoro:run Add this label to force Kokoro to re-run the tests. label Jun 23, 2022
@kokoro-team kokoro-team removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Jun 23, 2022
@hongalex hongalex merged commit cc1528b into googleapis:main Jun 23, 2022
@schallert schallert deleted the schallert/publisher_outstanding_metrics branch June 23, 2022 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the Pub/Sub API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pubsub: Emit outstanding messages metrics for publishers
3 participants