OkHttpMetricsEventListener can lead to tag cardinality explosion with default instrumentation

[kelunik]

The path how to prevent this is unclear, see #4561 (comment):

I don't especially like adding flags for each tag. However, cardinality explosion with the default instrumentation is also undesirable. HTTP client instrumentation can be difficult to do in such a way it necessarily avoids cardinality issues due to difficulty in tagging uri in a low cardinality way out-of-the-box and the cardinality of host/port depends on usage/environment. For that reason, in other instrumentations we have offered a flag for route tags (see MicrometerHttpRequestExecutor for Apache HttpClient 5). Alternatively we've had the concept of a tags provider (see JettyClientTagsProvider) where we can have a default that users can override. Basically, we haven't been consistent and I'm not sure what the best path forward to unify on is, but maybe there is another approach we could take to allow disabling this for some clients where it is known ahead of time route tags are not wanted.

The mechanism how to prevent this should be unified.

[shakuzen]

There are two parts here, I think. One is documenting better what users can do about it today without changes to the instrumentation:

At least it is a good idea to cross reference the MeterFilter documentation from the documentation specifically for OkHttp (and other HTTP clients) and mention this potential cardinality problem.

The other is providing a way to customize the instrumentation where it is known ahead of time that this will be a problem for a given client in an application. This was originally about the OkHttp implementation, but we should in addition or instead look at all the HTTP client instrumentations to solve this in a consistent way since they are generally all susceptible to this problem.