ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec #2623

darccio · 2024-03-20T16:48:26Z

What does this PR do?

Defaults dynamic config origin to default when created.

It also sets one specific case to env_var according to the telemetry API spec (more detail in AIT-8580).

Motivation

Fix failing telemetry system-tests in DataDog/system-tests#2063.

Reviewer's Checklist

Changed code has unit tests for its functionality at or near 100% coverage.
System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
There is a benchmark for any new code, or changes to existing code.
If this interacts with the agent in a new way, a system test has been added.
Add an appropriate team label so this PR gets put in the right place for the release notes.
Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

For Datadog employees:

If this PR touches code that handles credentials of any kind, such as Datadog API keys, I've requested a review from @DataDog/security-design-and-guidance.
This PR doesn't touch any of that.

Unsure? Have a question? Request a review!

…ctive Tracing telemetry spec (re: AIT-8580)

ddtrace/tracer/remote_config_test.go

mtoffl01 · 2024-03-22T21:07:56Z

ddtrace/tracer/option.go

@@ -334,7 +334,9 @@ func newConfig(opts ...StartOption) *config {
 	}
 	c.headerAsTags = newDynamicConfig("trace_header_tags", nil, setHeaderTags, equalSlice[string])
 	if v := os.Getenv("DD_TRACE_HEADER_TAGS"); v != "" {
-		WithHeaderTags(strings.Split(v, ","))(c)
+		c.headerAsTags.update(strings.Split(v, ","), originEnvVar)


This just made me wonder whether the other settings that are available through RC should also get origin: originEnvVar on their dynamicConfig when enabled via env var, e.g, DD_TRACE_ENABLED
Maybe newDynamicConfig() should accept a value for origin, or maybe these we should also call dynamicConfig.update(val, originEnvVar) for these other options

@mtoffl01 You are right. I left this one as example and reminder to add more in a separate PR. Thanks!

I agree with this. Ideally the constructor should set the origin.

If for some reason we can't use the constructor to do so, the generic dynamicConfig type should provide a method to update the startup and current values and their origins instead of manipulating the fields directly in ddtrace/tracer/option.go.

mtoffl01

I did leave one comment about potential changes, but those changes could be applied to a separate PR (since the title of this one explicitly states "support default origin" and my comment is for env_var origin)

darccio · 2024-03-25T09:28:30Z

@avivenzio-dd Before merging this one, what do you think it's the best course to ensure it's implementing right the spec?

avivenzio-dd · 2024-03-25T13:05:25Z

@avivenzio-dd Before merging this one, what do you think it's the best course to ensure it's implementing right the spec?

@darccio good call

Can we start up a small service and send some traces using the changes in this branch to a staging org? or is there a way to run the system tests based on this branch? I'm assuming sending some traces is probably easier.

We can verify the telemetry events are correct using their debug logs.

ahmed-mez

Looks good!
For future reference, I believe the dynamicConfig structure should be encapsulated in a separate package.
I saw we already bypass its method and access some field (e.g 25d3eb5) which is not the intended behavior (not thread-safe, error-prone and leaks the implementation details)

ahmed-mez · 2024-04-03T08:36:54Z

ddtrace/tracer/option.go

@@ -334,7 +334,9 @@ func newConfig(opts ...StartOption) *config {
 	}
 	c.headerAsTags = newDynamicConfig("trace_header_tags", nil, setHeaderTags, equalSlice[string])
 	if v := os.Getenv("DD_TRACE_HEADER_TAGS"); v != "" {
-		WithHeaderTags(strings.Split(v, ","))(c)
+		c.headerAsTags.update(strings.Split(v, ","), originEnvVar)


I agree with this. Ideally the constructor should set the origin.

If for some reason we can't use the constructor to do so, the generic dynamicConfig type should provide a method to update the startup and current values and their origins instead of manipulating the fields directly in ddtrace/tracer/option.go.

pr-commenter · 2024-04-03T15:33:29Z

Benchmarks

Benchmark execution time: 2024-05-03 11:00:34

Comparing candidate commit a08e194 in PR branch dario.castane/AIT-9990-default-origin with baseline commit 3a426ca in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 0 unstable metrics.

darccio · 2024-04-04T12:19:37Z

@avivenzio-dd The best way I found is to temporarily modify the GitHub CI config to run the parametric tests from DataDog/system-tests#2063. This week is heavy on ER thus reducing our capacity to do anything more sophisticated.

A first run shows that it's correctly running from the linked PR in system tests, combined with this PR. It failed anyway because we are returning default instead of env_var in a test, so ~~I'm going to fix that~~ I fixed it.

Edit: a second run seems to fail too but no longer due to the origin field's value. Please, can you check it? I think it's because you are not expecting the runtime ID as tag.

github-actions · 2024-04-25T01:44:50Z

This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.

avivenzio-dd · 2024-04-25T12:09:02Z

@darccio is there something pending from my side on this? I think the tests should be all updated now.

darccio · 2024-04-25T13:14:15Z

@avivenzio-dd No, it's on my roof. Let me try to give it a try tomorrow.

darccio · 2024-05-03T13:21:25Z

@avivenzio-dd I did another try but it fails. I'm having availability issues as I've been reassigned to a R&D project for one or two months. How do you want to proceed?

ddtrace/tracer: support default origin on dynamic config to support A…

c65f534

…ctive Tracing telemetry spec (re: AIT-8580)

darccio added the team:apm-go label Mar 20, 2024

darccio requested a review from a team as a code owner March 20, 2024 16:48

darccio changed the title ~~ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec (re: AIT-8580)~~ ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec Mar 20, 2024

darccio requested a review from ahmed-mez March 20, 2024 16:50

avivenzio-dd reviewed Mar 20, 2024

View reviewed changes

ddtrace/tracer/remote_config_test.go Show resolved Hide resolved

darccio mentioned this pull request Mar 21, 2024

contrib/google.golang.org/grpc.v12: replace missing {From,New}Context with equivalent functions #2624

Merged

8 tasks

mtoffl01 reviewed Mar 22, 2024

View reviewed changes

mtoffl01 approved these changes Mar 22, 2024

View reviewed changes

ahmed-mez approved these changes Apr 3, 2024

View reviewed changes

Merge branch 'main' into dario.castane/AIT-9990-default-origin

40be10c

.github/workflows: use specific branch to test new parametric tests

430d876

darccio requested a review from a team as a code owner April 4, 2024 12:06

ddtrace/tracer: set trace_sample_rate origin to env_var if present

46bb463

github-actions bot added the stale Stuck for more than 1 month label Apr 25, 2024

darccio removed the stale Stuck for more than 1 month label Apr 29, 2024

Merge branch 'main' into dario.castane/AIT-9990-default-origin

a08e194

darccio requested a review from a team as a code owner May 3, 2024 10:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec #2623

ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec #2623

darccio commented Mar 20, 2024

mtoffl01 Mar 22, 2024 •

edited

darccio Mar 25, 2024

ahmed-mez Apr 3, 2024

mtoffl01 left a comment

darccio commented Mar 25, 2024

avivenzio-dd commented Mar 25, 2024

ahmed-mez left a comment

ahmed-mez Apr 3, 2024

pr-commenter bot commented Apr 3, 2024 •

edited

darccio commented Apr 4, 2024 •

edited

github-actions bot commented Apr 25, 2024

avivenzio-dd commented Apr 25, 2024

darccio commented Apr 25, 2024

darccio commented May 3, 2024

ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec #2623

Are you sure you want to change the base?

ddtrace/tracer: support default origin on dynamic config to support Active Tracing telemetry spec #2623

Conversation

darccio commented Mar 20, 2024

What does this PR do?

Motivation

Reviewer's Checklist

mtoffl01 Mar 22, 2024 • edited

Choose a reason for hiding this comment

darccio Mar 25, 2024

Choose a reason for hiding this comment

ahmed-mez Apr 3, 2024

Choose a reason for hiding this comment

mtoffl01 left a comment

Choose a reason for hiding this comment

darccio commented Mar 25, 2024

avivenzio-dd commented Mar 25, 2024

ahmed-mez left a comment

Choose a reason for hiding this comment

ahmed-mez Apr 3, 2024

Choose a reason for hiding this comment

pr-commenter bot commented Apr 3, 2024 • edited

Benchmarks

darccio commented Apr 4, 2024 • edited

github-actions bot commented Apr 25, 2024

avivenzio-dd commented Apr 25, 2024

darccio commented Apr 25, 2024

darccio commented May 3, 2024

mtoffl01 Mar 22, 2024 •

edited

pr-commenter bot commented Apr 3, 2024 •

edited

darccio commented Apr 4, 2024 •

edited