[BUG] Hostname providers spamming traces

[adrianlungu]

The providerCatalog has a set of providers that seem to be running regardless of any configuration given to the tracer; this sometimes leads to a behavior where traces are being spammed, as follows, with a call to egress 169.254.169.254.

Ideally this could be configured somehow, or at least have a way to address what to do in case a provider cannot be reached so we can silence it if we don't use some of the specified providers; or maybe make it customizable which provider should be contacted and which not ?

Version of dd-trace-go

v1.60.3

Describe what happened:

The internal hostname providers are being called and leads to a spam of traces with egress 169.254.169.254, url being http://169.254.169.254/computeMetadata/v1/instance/hostname.

Describe what you expected:

Not to have these traces sent to datadog.

Steps to reproduce the issue:
N/A - it can be reproduced just by enabling the tracer.

Additional environment details (Version of Go, Operating System, etc.):

• Go 1.22 (same behavior with 1.21.x)
• MacOS 14.3

[darccio]

@adrianlungu The provider that triggers the HTTP request is only activated if the hostname has one of the following prefixes:

 	defaultPrefixes = []string{"ip-", "domu", "ec2amaz-"}

How does your hostname look like? Also, how is your service instrumented?

[adrianlungu]

@darccio from what I can tell, defaultPrefixes are only being checked for ec2 hostnames, while http://169.254.169.254/computeMetadata/v1/instance/hostname is a gce metadataURL.

I'm not sure how that applies here. In /internal/hostname/providers.go we can see that fromGce, fromAzure and maybe others, return a hostname without checking anything up-front, thus resulting in these calls that end up being logged as 401 as we don't use ec2, gce and/or azure at the same time from a single service.

[darccio]

Thanks @adrianlungu, you are right. I'll check it again.

[darccio]

@adrianlungu I saw my error, gce and aws use the same internal IP, but different APIs (as one could expect 😆). Sorry for the confusion and misleading comment.

As you pointed, we are running these checks unconditionally through ddtrace/tracer.Start -> ddtrace/tracer#tracer.hostname -> ddtrace/internal/hostname.Get (which spawns a goroutine to asynchronously update the hostname).

The first check in providerCatalog is fromConfig. This check tries to resolve the hostname from environment variable DD_HOSTNAME. Setting this variable will stop further checks down the list.

Does this solve your issue?