OTLP: Use optimized OTel metric translator, converting directly to Mimir format

[aknuds1]

What this PR does

Use optimized OTel metric translator in OTLP endpoint, converting directly to Mimir format.

The translation code is generated by downloading Go sources from mimir-prometheus, and transforming them to use Mimir protobuf types and enums via the gopatch tool. The generated files (_generated.go suffix) are checked into Git, and CI verifies that they are up to date with the mimir-prometheus version depended on.

The workflow should be that when we update the mimir-prometheus dependency, we should also execute make generate-otlp, to make sure we have the latest translation code. It's a minor hassle, but I think the performance gains are worth it.

TODO

• Modify CI check to generate code and check difference (suggested by @pracucci)

Benchmark results

✗ benchstat main-otlphandler.txt direct-otlphandler.txt 
goos: darwin
goarch: arm64
pkg: github.com/grafana/mimir/pkg/distributor
                        │ main-otlphandler-6.txt │     direct-otlphandler-6.txt      │
                        │         sec/op         │   sec/op     vs base              │
OTLPHandler/protobuf-12              254.3µ ± 1%   231.3µ ± 4%  -9.04% (p=0.002 n=6)
OTLPHandler/JSON-12                  379.4µ ± 1%   354.5µ ± 3%  -6.55% (p=0.002 n=6)
geomean                              310.6µ        286.3µ       -7.80%

                        │ main-otlphandler-6.txt │      direct-otlphandler-6.txt       │
                        │          B/op          │     B/op      vs base               │
OTLPHandler/protobuf-12             370.0Ki ± 0%   251.6Ki ± 0%  -31.99% (p=0.002 n=6)
OTLPHandler/JSON-12                 416.9Ki ± 0%   298.7Ki ± 0%  -28.36% (p=0.002 n=6)
geomean                             392.7Ki        274.1Ki       -30.20%

                        │ main-otlphandler-6.txt │     direct-otlphandler-6.txt      │
                        │       allocs/op        │  allocs/op   vs base              │
OTLPHandler/protobuf-12              5.127k ± 0%   5.120k ± 0%  -0.14% (p=0.002 n=6)
OTLPHandler/JSON-12                  8.160k ± 0%   8.153k ± 0%  -0.09% (p=0.002 n=6)
geomean                              6.468k        6.461k       -0.11%

Which issue(s) this PR fixes or relates to

Checklist

• Tests updated.
• Documentation added.
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
• about-versioning.md updated with experimental features.

[krajorama]

From the weekly: we have a general pattern to pick up changes: first delete the generated files *_generated.go and then re-generate and check if there's any change picked up by git. See the use of ./tools/find-diff-or-untracked.sh in the Makefile. This handles files missing/added and if the generator itself fails.

[krajorama]

I'm not sure I understand why we need to download the sources? If we put a reference to the sources anywhere (make a test/dummy command), they would be downloaded by go mod vendor wouldn't they?

[aknuds1]

That's a good idea @krajorama - my thinking was that these sources wouldn't be vendored. I guess it's preferable to rely on vendoring.