refs(metrics) Count/measure the number of issues on the issue page. #35422

antonpirker · 2022-06-07T07:43:56Z

Request for comments and discussion:

This is about dogfooding the measurements API:
getsentry/sentry-python#1359

The idea is to count/measure the number of issues displayed on the "Issues" page. (One metric for "unresolved", "for review" and "ignored" respectively)

This has multiple benefits:

we test the measurements API and storing/processing of the measurements
we know how many issues the users see on their issues page
when we will improve grouping we will see the impact the improvement has on the number of issues on the issues page.
all leads to workflow 2.0!

antonpirker · 2022-06-07T07:45:19Z

One thing I notices slapping together this PR: we probably should have a "static API" for set_measurement so the users does not need to create a new transaction, but just adds the measurements to the current transaction.

sl0thentr0py

you need to do several things right now to use the 'experimental' set_measurement api on the SDK.

require at least 1.5.12 here
add _experiments={"custom_measurements": True}) to the sdk init
don't start a new transaction but use the current one on the scope

transaction = Hub.current.scope.transaction
if transaction:
    transaction.set_measurement('foobar', 123)

eventually we'll add top-level api helpers, but since this is experimental this is not exposed all the way yet.

antonpirker · 2022-06-07T14:27:00Z

Incorporated feedback of @sl0thentr0py. Now it is a good base for discussion.

sl0thentr0py

looks good from sdk side, someone from product should still review the other stuff

We started exporting SENTRY_DSN back in #32524 in order to fix errors in lib.sh not being reported. Having this variable exported can cause a lot of errors from a developers' development to show up as dev env errors. Exporting inside of lib.sh does not persist in the developers' environment.

smeubank · 2022-06-20T16:50:15Z

hi @edwardgou-sentry would you mind checking this PR?

smeubank · 2022-06-20T17:13:02Z

@antonpirker was just discussing with @jas-kas and they have some examples for testing locally using transaction.duration, measurements.longtaskcount, and total.db.calls. Could we instrument in Sentry to start collecting those measurements in the wild on the sentry org?

Ideally a python dev workin on product would do it, and they could make sure the SDK impl makes sense for them. @k-fish would someone on your team be able to support here? With this PR as an example of setting custom measurements. There is this notion page with example of what kinds of dashboards should be possible after these start being collected

https://www.notion.so/sentry/Demo-Video-Brainstorm-for-Internal-Stakeholder-Review-662e0a51301b42f380529fc705535fe0

smeubank · 2022-06-20T17:37:59Z

@antonpirker you also mention some measurement which we are currently doing with another tool, can you remind me where those are? Maybe they are some other use cases we can surface, document/promote/dog food

edwardgou-sentry · 2022-06-20T17:51:03Z

Hey @smeubank, by this: Could we instrument in Sentry to start collecting those measurements in the wild on the sentry org?, do you mean ingesting these custom measurements in prod?

I think we can start adding instrumentation to those measurements as well, but those measurements won't actually get "collected" yet (ie extracted to metrics). This is also true for the measurements in this pr.

There is an allowlist in relay preventing these custom measurements from being extracted at the moment. The ingest team is working on removing this allowlist and preparing to properly ingest these custom measurements, so we won't be able to collect on these measurements until then.

Sorry if this is information you already knew. 😄

smeubank · 2022-06-20T18:01:15Z

There is an allowlist in relay preventing these custom measurements from being extracted at the moment.

now that you write it, it sounds familiar, but can't swear that I know anything ever 😄

but then relay just throws them away right, it doesn't cuase any issues? If that is the case then I would merge any set_measurements changes already. So once relay changes are in place then good to go already

edwardgou-sentry · 2022-06-20T18:02:33Z

but then relay just throws them away right, it doesn't cuase any issues? If that is the case then I would merge any set_measurements changes already. So once relay changes are in place then good to go already

Yup, i believe this is correct based on my understanding too!

edwardgou-sentry · 2022-06-20T18:07:16Z

src/sentry/api/endpoints/organization_group_index.py

@@ -345,6 +346,10 @@ def get(self, request: Request, organization) -> Response:
        self.add_cursor_headers(request, response, cursor_result)

        # TODO(jess): add metrics that are similar to project endpoint here
+        transaction = Hub.current.scope.transaction
+        if transaction:
+            transaction.set_measurement("issues.unresolved", len(cursor_result))


I could be wrong here since I'm not in issues code very much, but if a user visits the For Review tab directly via url (ie /organizations/sentry/issues/?query=is%3Aunresolved+is%3Afor_review+assigned_or_suggested%3A%5Bme%2C+none%5D&sort=inbox), wouldn't this measurement become inaccurate? It's reporting for issues.unresolved but the actual measurement should be for issues.for_review.

untitaker · 2022-06-21T11:17:25Z

src/sentry/api/endpoints/organization_group_index.py

@@ -345,6 +346,10 @@ def get(self, request: Request, organization) -> Response:
        self.add_cursor_headers(request, response, cursor_result)

        # TODO(jess): add metrics that are similar to project endpoint here
+        transaction = Hub.current.scope.transaction
+        if transaction:
+            transaction.set_measurement("issues.unresolved", len(cursor_result))


why don't we have a global sentry_sdk.set_measurement for this?

commented above

eventually we'll add top-level api helpers, but since this is experimental this is not exposed all the way yet.

github-actions · 2022-07-20T00:04:35Z

This pull request has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

antonpirker · 2022-08-29T09:35:12Z

just silently closing this. It is not that important anymore. Thanks everyone for the reviews.

Cound the number of issues on the issue page.

93375bc

sl0thentr0py requested changes Jun 7, 2022

View reviewed changes

Update current transaction and correctly initialice sentry sdk

142cdc1

vercel bot deployed to Preview – sentry June 7, 2022 14:27 View deployment

vercel bot deployed to Preview – storybook June 7, 2022 14:28 View deployment

sl0thentr0py approved these changes Jun 7, 2022

View reviewed changes

antonpirker and others added 3 commits June 8, 2022 14:31

Merge branch 'master' into antonpirker-measurments-dog-fooding

68b9d62

Merge branch 'master' into antonpirker-measurments-dog-fooding

56f3302

antonpirker marked this pull request as ready for review June 8, 2022 12:37

antonpirker requested a review from a team as a code owner June 8, 2022 12:37

antonpirker requested a review from a team June 8, 2022 12:37

antonpirker requested review from a team as code owners June 8, 2022 12:37

vercel bot deployed to Preview – sentry June 8, 2022 12:41 View deployment

vercel bot deployed to Preview – storybook June 8, 2022 12:42 View deployment

AniketDas-Tekky removed the request for review from a team June 8, 2022 16:43

smeubank requested a review from edwardgou-sentry June 20, 2022 16:47

edwardgou-sentry reviewed Jun 20, 2022

View reviewed changes

untitaker approved these changes Jun 21, 2022

View reviewed changes

github-actions bot added the Status: Stale label Jul 20, 2022

sl0thentr0py added Status: Backlog and removed Status: Stale labels Jul 20, 2022

antonpirker closed this Aug 29, 2022

github-actions bot locked and limited conversation to collaborators Sep 13, 2022

asottile-sentry deleted the antonpirker-measurments-dog-fooding branch December 1, 2023 16:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refs(metrics) Count/measure the number of issues on the issue page. #35422

refs(metrics) Count/measure the number of issues on the issue page. #35422

antonpirker commented Jun 7, 2022 •

edited

antonpirker commented Jun 7, 2022

sl0thentr0py left a comment •

edited

antonpirker commented Jun 7, 2022

sl0thentr0py left a comment

smeubank commented Jun 20, 2022

smeubank commented Jun 20, 2022

smeubank commented Jun 20, 2022

edwardgou-sentry commented Jun 20, 2022

smeubank commented Jun 20, 2022 •

edited

edwardgou-sentry commented Jun 20, 2022

edwardgou-sentry Jun 20, 2022

untitaker Jun 21, 2022

sl0thentr0py Jun 28, 2022

github-actions bot commented Jul 20, 2022

antonpirker commented Aug 29, 2022

refs(metrics) Count/measure the number of issues on the issue page. #35422

refs(metrics) Count/measure the number of issues on the issue page. #35422

Conversation

antonpirker commented Jun 7, 2022 • edited

antonpirker commented Jun 7, 2022

sl0thentr0py left a comment • edited

Choose a reason for hiding this comment

antonpirker commented Jun 7, 2022

sl0thentr0py left a comment

Choose a reason for hiding this comment

smeubank commented Jun 20, 2022

smeubank commented Jun 20, 2022

smeubank commented Jun 20, 2022

edwardgou-sentry commented Jun 20, 2022

smeubank commented Jun 20, 2022 • edited

edwardgou-sentry commented Jun 20, 2022

edwardgou-sentry Jun 20, 2022

Choose a reason for hiding this comment

untitaker Jun 21, 2022

Choose a reason for hiding this comment

sl0thentr0py Jun 28, 2022

Choose a reason for hiding this comment

github-actions bot commented Jul 20, 2022

antonpirker commented Aug 29, 2022

antonpirker commented Jun 7, 2022 •

edited

sl0thentr0py left a comment •

edited

smeubank commented Jun 20, 2022 •

edited