Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add auto-configure support for logging-otlp #4879

Merged
merged 7 commits into from Nov 28, 2022
Merged

Add auto-configure support for logging-otlp #4879

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
918faf7
Add autoconfigure support for logging-otlp
ben-roling Oct 21, 2022
601e82a
Add NotOnClasspath tests
ben-roling Nov 3, 2022
0fa4d94
Fix formatting
ben-roling Nov 4, 2022
0cd2f4f
Declare logging-otlp as experimental, add unit test
jack-berg Nov 21, 2022
8b5a2dc
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
jack-berg Nov 22, 2022
318f2d6
Spotless
jack-berg Nov 26, 2022
8b3c3e6
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
jack-berg Nov 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 13 additions & 0 deletions sdk-extensions/autoconfigure/README.md
View file
Open in desktop
Expand Up @@ -159,6 +159,19 @@ The logging exporter prints the name of the span along with its attributes to st
| otel.metrics.exporter=logging | OTEL_METRICS_EXPORTER=logging | Select the logging exporter for metrics |
| otel.logs.exporter=logging | OTEL_LOGS_EXPORTER=logging | Select the logging exporter for logs |

### Logging OTLP JSON exporter

The logging-otlp exporter writes the telemetry data to the JUL logger in OLTP JSON form. It's a more verbose output mainly used for testing and debugging.

| System property | Environment variable | Description |
|------------------------------------|------------------------------------|----------------------------------------------------|
| otel.traces.exporter=logging-otlp | OTEL_TRACES_EXPORTER=logging-otlp | Select the logging OTLP JSON exporter for tracing |
| otel.metrics.exporter=logging-otlp | OTEL_METRICS_EXPORTER=logging-otlp | Select the logging OTLP JSON exporter for metrics |
| otel.logs.exporter=logging-otlp | OTEL_LOGS_EXPORTER=logging-otlp | Select the logging OTLP JSON exporter for logs |
jack-berg marked this conversation as resolved.
Show resolved Hide resolved

**NOTE:** While the `OtlpJsonLogging{Signal}Exporters` are stable, specifying their use
via `otel.{signal}.exporter=logging-otlp` is experimental and subject to change or removal.

## Propagator

The propagators determine which distributed tracing header formats are used, and which baggage propagation header formats are used.
Expand Down
7 changes: 7 additions & 0 deletions sdk-extensions/autoconfigure/build.gradle.kts
View file
Open in desktop
Expand Up @@ -17,6 +17,7 @@ dependencies {

compileOnly(project(":exporters:jaeger"))
compileOnly(project(":exporters:logging"))
compileOnly(project(":exporters:logging-otlp"))
compileOnly(project(":exporters:otlp:all"))
compileOnly(project(":exporters:otlp:logs"))
compileOnly(project(":exporters:otlp:common"))
Expand Down Expand Up @@ -144,6 +145,12 @@ testing {
}
}
}
val testLoggingOtlp by registering(JvmTestSuite::class) {
dependencies {
implementation(project(":exporters:logging-otlp"))
implementation("com.google.guava:guava")
}
}
val testOtlp by registering(JvmTestSuite::class) {
dependencies {
implementation(project(":exporters:otlp:all"))
Expand Down
7 changes: 7 additions & 0 deletions ...gure/src/main/java/io/opentelemetry/sdk/autoconfigure/LogRecordExporterConfiguration.java
View file
Open in desktop
Expand Up @@ -12,6 +12,7 @@
import io.opentelemetry.api.metrics.MeterProvider;
import io.opentelemetry.exporter.internal.retry.RetryUtil;
import io.opentelemetry.exporter.logging.SystemOutLogRecordExporter;
import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingLogRecordExporter;
import io.opentelemetry.exporter.otlp.http.logs.OtlpHttpLogRecordExporter;
import io.opentelemetry.exporter.otlp.http.logs.OtlpHttpLogRecordExporterBuilder;
import io.opentelemetry.exporter.otlp.logs.OtlpGrpcLogRecordExporter;
Expand Down Expand Up @@ -91,6 +92,12 @@ static LogRecordExporter configureExporter(
"Logging Log Exporter",
"opentelemetry-exporter-logging");
return SystemOutLogRecordExporter.create();
case "logging-otlp":
ClasspathUtil.checkClassExists(
"io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingLogRecordExporter",
"OTLP JSON Logging Log Exporter",
"opentelemetry-exporter-logging-otlp");
return OtlpJsonLoggingLogRecordExporter.create();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about providing a ConfigurableLogRecordExporterProvider and letting the autoconfigure discover the exporter automatically?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could give a shot if it's desired. Initially I was just following the code pattern used for the "logging" exporter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking this may be a good idea to do for all the exporters from a separation of concerns perspective. If each exporter artifact contains a Configurable{Signal}ExporterProvider, then the code to interpret ConfigProperties becomes distributed into each artifacts, reducing the complexity of autoconfigure, especially around testing which has become cumbersome.

What do you think @jkwatson?

BTW, for the purposes of this PR I do think we should continue the current pattern. If we agree on implementing providers, we can open a separate issue to track that work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then the code to interpret ConfigProperties becomes distributed into each artifacts, reducing the complexity of autoconfigure, especially around testing which has become cumbersome.

this sounds great, and could help guide users into good practices when they are trying to extend autoconfigure with their own items (e.g. exporters)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, for the purposes of this PR I do think we should continue the current pattern. If we agree on implementing providers, we can open a separate issue to track that work.

👍

default:
LogRecordExporter spiExporter = spiExportersManager.getByName(name);
if (spiExporter == null) {
Expand Down
12 changes: 12 additions & 0 deletions ...nfigure/src/main/java/io/opentelemetry/sdk/autoconfigure/MetricExporterConfiguration.java
View file
Open in desktop
Expand Up @@ -11,6 +11,7 @@

import io.opentelemetry.exporter.internal.retry.RetryUtil;
import io.opentelemetry.exporter.logging.LoggingMetricExporter;
import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingMetricExporter;
import io.opentelemetry.exporter.otlp.http.metrics.OtlpHttpMetricExporter;
import io.opentelemetry.exporter.otlp.http.metrics.OtlpHttpMetricExporterBuilder;
import io.opentelemetry.exporter.otlp.metrics.OtlpGrpcMetricExporter;
Expand Down Expand Up @@ -49,6 +50,9 @@ static MetricReader configureExporter(
case "logging":
metricExporter = configureLoggingExporter();
break;
case "logging-otlp":
metricExporter = configureLoggingOtlpExporter();
break;
default:
MetricExporter spiExporter = configureSpiExporter(name, config, serviceClassLoader);
if (spiExporter == null) {
Expand All @@ -69,6 +73,14 @@ private static MetricExporter configureLoggingExporter() {
return LoggingMetricExporter.create();
}

private static MetricExporter configureLoggingOtlpExporter() {
ClasspathUtil.checkClassExists(
"io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingMetricExporter",
"OTLP JSON Logging Metrics Exporter",
"opentelemetry-exporter-logging-otlp");
return OtlpJsonLoggingMetricExporter.create();
}

// Visible for testing.
@Nullable
static MetricExporter configureSpiExporter(
Expand Down
7 changes: 7 additions & 0 deletions ...configure/src/main/java/io/opentelemetry/sdk/autoconfigure/SpanExporterConfiguration.java
View file
Open in desktop
Expand Up @@ -15,6 +15,7 @@
import io.opentelemetry.exporter.jaeger.JaegerGrpcSpanExporter;
import io.opentelemetry.exporter.jaeger.JaegerGrpcSpanExporterBuilder;
import io.opentelemetry.exporter.logging.LoggingSpanExporter;
import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingSpanExporter;
import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporter;
import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporterBuilder;
import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter;
Expand Down Expand Up @@ -98,6 +99,12 @@ static SpanExporter configureExporter(
"Logging Trace Exporter",
"opentelemetry-exporter-logging");
return LoggingSpanExporter.create();
case "logging-otlp":
ClasspathUtil.checkClassExists(
"io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingSpanExporter",
"OTLP JSON Logging Trace Exporter",
"opentelemetry-exporter-logging-otlp");
return OtlpJsonLoggingSpanExporter.create();
default:
SpanExporter spiExporter = spiExportersManager.getByName(name);
if (spiExporter == null) {
Expand Down
39 changes: 39 additions & 0 deletions ...ns/autoconfigure/src/test/java/io/opentelemetry/sdk/autoconfigure/NotOnClasspathTest.java
View file
Open in desktop
Expand Up @@ -81,6 +81,18 @@ void loggingSpans() {
+ "classpath");
}

@Test
void loggingSpansOtlp() {
assertThatThrownBy(
() ->
SpanExporterConfiguration.configureExporter(
"logging-otlp", EMPTY, NamedSpiManager.createEmpty(), MeterProvider.noop()))
.isInstanceOf(ConfigurationException.class)
.hasMessageContaining(
"OTLP JSON Logging Trace Exporter enabled but opentelemetry-exporter-logging-otlp not found on "
+ "classpath");
}

@Test
void loggingMetrics() {
assertThatThrownBy(
Expand All @@ -96,6 +108,21 @@ void loggingMetrics() {
+ "classpath");
}

@Test
void loggingMetricsOtlp() {
assertThatThrownBy(
() ->
MetricExporterConfiguration.configureExporter(
"logging-otlp",
EMPTY,
MetricExporterConfiguration.class.getClassLoader(),
(a, unused) -> a))
.isInstanceOf(ConfigurationException.class)
.hasMessageContaining(
"OTLP JSON Logging Metrics Exporter enabled but opentelemetry-exporter-logging-otlp not found on "
+ "classpath");
}

@Test
void loggingLogs() {
assertThatThrownBy(
Expand All @@ -108,6 +135,18 @@ void loggingLogs() {
+ "classpath");
}

@Test
void loggingLogsOtlp() {
assertThatThrownBy(
() ->
LogRecordExporterConfiguration.configureExporter(
"logging-otlp", EMPTY, NamedSpiManager.createEmpty(), MeterProvider.noop()))
.isInstanceOf(ConfigurationException.class)
.hasMessageContaining(
"OTLP JSON Logging Log Exporter enabled but opentelemetry-exporter-logging-otlp not found on "
+ "classpath");
}

@Test
void otlpGrpcMetrics() {
assertThatCode(
Expand Down
61 changes: 61 additions & 0 deletions ...onfigure/src/testLoggingOtlp/java/io/opentelemetry/sdk/autoconfigure/LoggingOtlpTest.java
View file
Open in desktop
@@ -0,0 +1,61 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.sdk.autoconfigure;

import static org.assertj.core.api.Assertions.assertThat;
import static org.awaitility.Awaitility.await;

import com.google.common.collect.ImmutableMap;
import io.github.netmikey.logunit.api.LogCapturer;
import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingLogRecordExporter;
import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingMetricExporter;
import io.opentelemetry.exporter.logging.otlp.OtlpJsonLoggingSpanExporter;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import java.util.concurrent.TimeUnit;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

class LoggingOtlpTest {

@RegisterExtension
LogCapturer spansCapturer =
LogCapturer.create().captureForType(OtlpJsonLoggingSpanExporter.class);

@RegisterExtension
LogCapturer metricsCapturer =
LogCapturer.create().captureForType(OtlpJsonLoggingMetricExporter.class);

@RegisterExtension
LogCapturer logsCapturer =
LogCapturer.create().captureForType(OtlpJsonLoggingLogRecordExporter.class);

@Test
void configures() {
OpenTelemetrySdk sdk =
AutoConfiguredOpenTelemetrySdk.builder()
.setConfig(
DefaultConfigProperties.createForTest(
ImmutableMap.of(
"otel.traces.exporter", "logging-otlp",
"otel.metrics.exporter", "logging-otlp",
"otel.logs.exporter", "logging-otlp")))
.setResultAsGlobal(false)
.build()
.getOpenTelemetrySdk();

sdk.getTracerProvider().get("tracer").spanBuilder("test").startSpan().end();
sdk.getMeterProvider().get("meter").counterBuilder("counter").build().add(10);
sdk.getSdkLoggerProvider().get("logger").logRecordBuilder().setBody("message").emit();

sdk.getSdkLoggerProvider().forceFlush().join(10, TimeUnit.SECONDS);
sdk.getSdkMeterProvider().forceFlush().join(10, TimeUnit.SECONDS);
sdk.getSdkLoggerProvider().forceFlush().join(10, TimeUnit.SECONDS);

await().untilAsserted(() -> assertThat(spansCapturer.getEvents().size()).isGreaterThanOrEqualTo(1));
await().untilAsserted(() -> assertThat(metricsCapturer.getEvents().size()).isGreaterThanOrEqualTo(1));
await().untilAsserted(() -> assertThat(logsCapturer.getEvents().size()).isGreaterThanOrEqualTo(1));
}
}