Stats from database/sql

[jared-mackey]

Hi, first thanks for all the hard work here. This is a great package.

Maybe it exists and I just missed it, but could we periodically report stats from database/sql as metrics automatically for instrumented pools?

[felixge]

Thanks for the suggestion! I like the idea.

@DataDog/tracing-go What do y'all think? Do we already do metrics for other contrib packages? Or only the runtime metrics stuff?

[katiehockman]

Thanks for reaching out. We're currently looking into this, in particular we're curious if any of the other language tracers are sending metrics for their contribs. We send runtime metrics about tracers, but don't have any prior art for doing this in dd-trace-go. We're also looking into whether other products already provide this kind of data. We'll get back in touch!

[jared-mackey]

Hey @mtoffl01 👋🏻

I seen you’ve made some great progress on this recently. Very excited! Do you have an ETA when this will be available in one of the releases? Thanks again!

[mtoffl01]

Hey @jared-mackey, the feature is slated to be included in the next release (v1.63.0)! We would greatly appreciate your feedback once you have the chance to upgrade and test the feature out.

[jared-mackey]

Hi @mtoffl01, I seen it just released (🎉), but I am not quite sure I know how to use it. I have runtime metrics enabled already from previous usages and have added the WithDBStats() func opt to my sql.Open() call. Yet, I am not getting any metrics reported on any interval. Is there documentation I should reference? Thanks!

[mtoffl01]

@jared-mackey There is documentation for use, here: https://pkg.go.dev/gopkg.in/DataDog/dd-trace-go.v1@v1.63.0/contrib/database/sql#WithDBStats
Note: I just realized the example code snippet in the docs is inaccurate, I'll update it now. But all you have to do is pass WithDBStats as a config option to the sqltrace Register or Open functions:
sqltrace.Register(driverName, connStr, sqltrace.WithDBStats())

Have you looked for metrics prefixed with datadog.tracer.sql? We're working on a Datadog dashboard that should display all of the metrics for you so you don't have to go looking for them. If you enable debug mode on the tracer (tracer.Start(tracer.WithDebugMode(true))), you can look out for:

1. logs that imply the feature is working: "Traced DB connection found: DB stats will be gathered and sent every %v." and "Reporting DB.Stats metrics..."
2. logs that imply the feature failed: "No traced DB connection found; cannot pull DB stats."

[jared-mackey]

Hi @mtoffl01, thanks for those details. This is working for me on my local machine but not in my deployed environments (which is where I was testing yesterday) and I cannot figure out why. Does it require a specific agent version or anything of the likes? I see "Reporting DB.Stats..." when running with debug enabled but I don't see them come through. I am receiving other metrics just fine (including golang runtime ones). I am using the unix socket both locally and in the deployed environment.

[jared-mackey]

Ah, I figured it out. My local had the agent listening on localhost:8125 along with the unix socket. The tracer seems to be using that instead of the unix socket despite having export DD_DOGSTATSD_URL=unix:///var/run/datadog/dsd.socket set. Which is weird because it seems like the runtime metrics properly respect that variable. Closing that port on the agent showed it not functioning just like in my deployed environments.

So it seems like there may be 2 bugs.

1. It does not read or use DD_DOGSTATSD_URL like I would expect it to.
2. Setting it via tracer.WithDogstatsdAddress(os.Getenv("DD_DOGSTATSD_URL")) still does not work (I triple checked the variable being set correctly). I suspect it's just not handling unix sockets appropriately somewhere but that's just a guess.

[darccio]

Closing as the feature request is complete. Any bug or issue, please fill a new one.