Fluentbit OpenTelemetry Plugins drops some fields from metrics

[thchan]

Bug Report

Describe the bug
When using the Fluentbit OpenTelemetry plugins to receive and send OTLP/HTTP metrics, some metrics fields are dropped that are supported in the OpenTelemetry specifications.
Some fields that are dropped include “units” that are dropped from all metric types, “min”, “max” and “aggregation_temporality” in histograms.

To Reproduce
Send OTLP/HTTP formatted metric data with all fields defined through fluentbit with the OpenTelemetry input and output plugins. Have fluent-bit send the data to a Opentelemetetry compatible endpoint.

Metrics from fluent-bit received by OpenTelemetry endpoint:

  "metrics": [
	{
	  "name": "cpu.utilization",
	  "description": "-",
	  "gauge": {
		"dataPoints": [
		  {
			"timeUnixNano": "1714743093300000000",
			"asDouble": 0
		  }
		]
	  }
	},
	{
	  "name": "testPeriodicHistogram",
	  "description": "-",
	  "histogram": {
		"dataPoints": [
		  {
			"timeUnixNano": "1714743093300000000",
			"count": "2",
			"sum": 11,
			"bucketCounts": [
			  "0",
			  "0",
			  "2",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0"
			],
			"explicitBounds": [
			  0,
			  5,
			  10,
			  25,
			  50,
			  75,
			  100,
			  250,
			  500,
			  750,
			  1000,
			  2500,
			  5000,
			  7500,
			  10000
			]
		  }
		]
	  }
	}
  ]

Expected behavior
Valid OTLP metric fields should not be dropped when going through fluentbit.

Input and expected output:

"metrics": [
	{
	  "name": "cpu.utilization",
	  "unit": "percent",
	  "gauge": {
		"dataPoints": [
		  {
			"startTimeUnixNano": "1714743034087000000",
			"timeUnixNano": "1714743093300000000",
			"asDouble": 0
		  }
		]
	  }
	},
	{
	  "name": "testPeriodicHistogram",
	  "unit": "count",
	  "histogram": {
		"dataPoints": [
		  {
			"startTimeUnixNano": "1714743034087000000",
			"timeUnixNano": "1714743093300000000",
			"count": "2",
			"sum": 11,
			"bucketCounts": [
			  "0",
			  "0",
			  "2",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0",
			  "0"
			],
			"explicitBounds": [
			  0,
			  5,
			  10,
			  25,
			  50,
			  75,
			  100,
			  250,
			  500,
			  750,
			  1000,
			  2500,
			  5000,
			  7500,
			  10000
			],
			"min": 5.5,
			"max": 5.5
		  }
		],
		"aggregationTemporality": 2
	  }
	}
]

Your Environment

• Version used: 3.0.3
• Configuration:

[SERVICE]
    Flush     5
    storage.type memory
    Daemon    off
    Log_Level debug

[INPUT]
    Name  opentelemetry
    Listen 127.0.0.1
    Port 3001

[OUTPUT]
    Name  stdout
    Match *

[OUTPUT]
    Name  opentelemetry
    Host  127.0.0.1
    Port  3003
    Metrics_uri /v1/metrics
    Logs_uri /v1/logs
    Traces_uri /v1/traces
    Tls Off
    Match *

• Operating System: Cross-Compiled for Android

Additional context
The purpose is to use fluent-bit to forward OpenTelemetry formatted data instead of using the collector provided by the OpenTelemetry Project.

OpenTelemetry Metrics protobuf file