[Logs] Support dependency injection in logging build-up

[CodeBlanch]

Changes

Enabled dependency injection scenarios in logging provider build-up similar to what we have for Tracing & Metrics except no dependency on the hosting project is required.

Public API

namespace OpenTelemetry
{
   public static class Sdk
   {
+     public static OpenTelemetryLoggerOptions CreateLoggerProviderBuilder() {}
   }
}

namespace OpenTelemetry.Logs
{
   public class OpenTelemetryLoggerOptions
   {
+      public OpenTelemetryLoggerOptions SetIncludeFormattedMessage(bool enabled) {}
+      public OpenTelemetryLoggerOptions SetIncludeScopes(bool enabled) {}
+      public OpenTelemetryLoggerOptions SetParseStateValues(bool enabled) {}
+      public OpenTelemetryLoggerOptions AddProcessor<T>() where T : BaseProcessor<LogRecord> {}
+      public OpenTelemetryLoggerOptions ConfigureServices(Action<IServiceCollection> configure) {}
+      public OpenTelemetryLoggerOptions ConfigureProvider(Action<IServiceProvider, OpenTelemetryLoggerProvider> configure) {}
   }

+  public static class OpenTelemetryLoggerOptionsExtensions
   {
+      public static OpenTelemetryLoggerProvider Build(this OpenTelemetryLoggerOptions options) {}
   }

   public class OpenTelemetryLoggerProvider
   {
+      public OpenTelemetryLoggerProvider AddProcessor(BaseProcessor<LogRecord> processor) {}
-      public OpenTelemetryLoggerProvider(Action<OpenTelemetry.Logs.OpenTelemetryLoggerOptions!> configure) {} // This was never released. Replaced by Sdk.CreateLoggerProviderBuilder
   }

+  public static class OpenTelemetryEventSourceLoggerOptionsExtensions
   {
+      public static OpenTelemetryLoggerOptions AddEventSourceLogEmitter(this OpenTelemetryLoggerOptions options, Func<string, EventLevel?> shouldListenToFunc) {}
   }
}

Scenarios enabled

These are all things that were not possible before.

Register a processor registered through services

services.AddLogging(configure =>
{
    configure.AddOpenTelemetry(options => options.AddProcessor<CustomProcessor>());
});

Register a processor through services which is automatically configured

services.AddSingleton<BaseProcessor<LogRecord>>(sp => new CustomProcessor());

services.AddLogging(configure => configure.AddOpenTelemetry());

Create extension methods which register services and configure provider (for library authors)

   public static OpenTelemetryLoggerOptions AddMyLibraryFeature(this OpenTelemetryLoggerOptions options)
   {
       return options
          .ConfigureServices(services => services
             .AddSingleton<MyLibraryDependency1>()
             .AddSingleton<MyLibraryDependency2>())
          .AddProcessor<MyLibraryProcessor>();
   }

Configure the provider after it has been created and the service provider is available

   services.AddLogging(configure =>
   {
       configure.AddOpenTelemetry(options =>
       {
           options.ConfigureProvider((sp, provider) =>
           {
               var someDependency = sp.GetRequiredService<SomeDependency>();
   
               provider.AddProcessor(new MyCustomProcessor(someDependency));
           });
       });
   });

Detached configuration method

If users or library authors wanted to expose some logging registration off of IServiceCollection instead of ILoggingBuilder that is possible like this:

public static IServiceCollection AddOpenTelemetryLoggingFeature(this IServiceCollection services)
{
   // This line is optional but ensures that the provider will be registered if the host is not directly calling AddOpenTelemetry itself
   services.AddLogging(configure => configure.AddOpenTelemetry());

   services.AddSingleton<MyLibraryDependency>();
   services.Configure<OpenTelemetryLoggerOptions>(options => { ...do configuration stuff...});
   return services;
}

Build things that need access to the provider

This is the main reason I worked on this. See: #3454 (comment). OpenTelemetryEventSourceLogEmitter needs to access the provider in order to create a LogEmitter. There was previously no way to get access to it when using the ILoggingBuilder API. Now it is possible to build services/extensions like what I am adding on this PR with AddEventSourceLogEmitter to smooth out the experience:

IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureLogging(builder =>
    {
        builder.ClearProviders();

        // Step 1: Configure OpenTelemetry logging...
        builder.AddOpenTelemetry(options =>
        {
            options
                .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("MyService"))
                .AddConsoleExporter()
                // Step 2: Register OpenTelemetryEventSourceLogEmitter to listen to events...
                .AddEventSourceLogEmitter(
                    (name) => name == MyEventSource.Name ? EventLevel.Informational : null);
        });
    })
    .Build();

This is done using the ConfigureProvider scenario above.

It is also possible to do this by registering a detached configuration action directly with services...

services.AddSingleton<Action<IServiceProvider, OpenTelemetryLoggerProvider>>(MyProviderConfigurationMethod);

TODOs

• Appropriate CHANGELOG.md updated for non-trivial changes
• Changes in public API reviewed
• Unit tests

[codecov]

Codecov Report

Merging #3504 (7057343) into main (72f4e07) will increase coverage by 0.11%.
The diff coverage is 97.31%.

@@            Coverage Diff             @@
##             main    #3504      +/-   ##
==========================================
+ Coverage   87.23%   87.35%   +0.11%     
==========================================
  Files         275      278       +3     
  Lines        9959    10083     +124     
==========================================
+ Hits         8688     8808     +120     
- Misses       1271     1275       +4     

Impacted Files	Coverage Δ
...gs/Options/OpenTelemetryLoggerOptionsExtensions.cs	75.00% <75.00%> (ø)
.../OpenTelemetry/Logs/OpenTelemetryLoggerProvider.cs	94.62% <92.30%> (-1.43%)	⬇️
...enTelemetry/Logs/OpenTelemetryLoggingExtensions.cs	97.22% <95.83%> (-2.78%)	⬇️
...OpenTelemetryEventSourceLoggerOptionsExtensions.cs	100.00% <100.00%> (ø)
...lemetry/Logs/Options/OpenTelemetryLoggerOptions.cs	100.00% <100.00%> (ø)
...etry/Logs/Options/OpenTelemetryLoggerOptionsSdk.cs	100.00% <100.00%> (ø)
src/OpenTelemetry/Sdk.cs	100.00% <100.00%> (ø)
src/OpenTelemetry/ProviderExtensions.cs	81.81% <0.00%> (-9.10%)	⬇️
...Telemetry/Metrics/PeriodicExportingMetricReader.cs	72.72% <0.00%> (-5.46%)	⬇️
...ter.ZPages/Implementation/ZPagesActivityTracker.cs	97.14% <0.00%> (-2.86%)	⬇️
... and 4 more

[alanwest]

Would a static method on Sdk make sense? Like Sdk.CreateMeterProviderBuilder and Sdk.CreateTracerProviderBuilder

[CodeBlanch]

Good idea I like it! At the moment the mechanics are slightly different in that there is no Build method for OpenTelemetryLoggerProvider. I could introduce a small class LoggerProviderBuilder so it matches more closely what we have to metrics + traces. Thoughts on that?

[alanwest]

If folks aren't thrilled by that idea, then I think Sdk.CreateLoggerProvider (i.e., w/o Builder) wouldn't be terrible.

But, consistency would have a nice feel to it, so my vote is a simple class enabling Sdk.CreateLoggerProviderBuilder(...).Build()

[CodeBlanch]

@alanwest OK this now has more of a builder pattern. Updated the description. LMK what you think!

[alanwest]

Lot of good stuff here! Reviewing these DI/options PRs are always tough for me - always feel like I have to page a bunch of stuff in my head to keep the Rube Goldberg machine straight 😆.

Your write up of scenarios is hugely helpful. Also it was helpful for me to take a look at what's been done for other providers to enable these scenarios. I reviewed the ConsoleLoggerExtensions to better grok what you've done here.

Regarding your last - maybe most important scenario - I like it! Makes the code for wiring up the EventSource extension clean.

[alanwest]

Feels a little off to have some things configured via the options like IncludeFormattedMessage and other things configured via methods off the builder.

For metrics we have some similar options like max streams that we set using builder methods:

Sdk.CreateMeterProviderBuilder()
   .SetMaxMetricStreams(100);

This PR is growing, so maybe a follow up? Though a follow up would mean the potential for some breaking changes.

[CodeBlanch]

Ya this was something I went back and forth on. The bools suck because you can't chain them which kind of breaks the builder pattern. That's why I added the delegate. We could add "Set" methods to preserve the chaining but some downsides to that: More public API and maybe confusing to have properties with get/set and also "Set" methods?

@utpilla @cijothomas @reyang Thoughts?

[reyang]

I guess it all started from SetSampler.

I personally would find it slightly more user-friendly with builder.SetSomething if the number of things to be set is small enough, and the things to be set are fairly complex (e.g. very hard to be configuration-based). I would be less happy if there are too many simple options that I have to chain the builder calls:

// it is just more difficult if I want to implement a configuration file

Sdk.CreateMeterProviderBuilder()
   .SetMaxMetricStreams(100),
   .SetX(true),
   .SetY("abc"),
   .SetZ(false);

[reyang]

Btw, now that we started talking about configuration, do you think ConfigureResource should be SetResource? 😆

[CodeBlanch]

@reyang You are going to love this, we actually have SetResourceBuilder & ConfigureResource both today 🤣 ConfigureResource was a recent addition.

Subtle differences between the two. Set clobbers anything that was configured. Configure is additive. For users configuring their host, I would expect them to call Set (but they could also Configure). For libraries/extension authors, they should call Configure.

We should probably obsolete the Set version because in Configure you can call Clear on the builder if you really wanted to reset it. (Pretty sure, didn't double-check a Clear exists.)

[reyang]

I would vote for eventually getting rid of ConfigureXyz (or at least use it carefully) because "configure" is a very general term that can have several different meanings (e.g. it could be additive and commutative; it could be additive but not commutative; it could be idempotent...).

If we're trying to be pedantic, "Append" is probably good for Processor and Exporter (coz it is not commutative due to the sequential processing behavior), "Set" is probably good for Sampler, "Configure" is probably good for anything that takes a complex "XyzOptions" object.

[CodeBlanch]

I removed the delegate and replaced it with 3 new "Set" methods. Don't disagree with any of this, it just seems like a bigger conversation we need to have beyond this PR.

[reyang]

+1, I would suggest that we scope this out for now. Maybe revisit it after this PR.

[alanwest]

I suspect we have other places this question applies, but for public APIs should we notate possible exceptions in the comments?

/// <exception cref="NullReferenceException">
/// Thrown when the <c>textMapPropagator</c> is null.
/// </exception>

[TimothyMothra]

I think this would be nice to have, but we would need some tooling to identify.

• would need to discover them all
• would need to keep it up to date when exceptions are added or removed

[CodeBlanch]

Totally agree with this. We should have these comments but without tooling support I worry it will often be wrong, missing, or outright lies 😄

[alanwest]

Are you saying you're trying to replace good jobs with machines? This PR got political all of the sudden 😆

Jokes aside, yes I completely agree and it's a discussion that's beyond the scope of this PR.

[reyang]

Are you saying you're trying to replace good jobs with machines? This PR got political all of the sudden 😆

Machines probably don't need such <exception> comment, I assume they will just disassemble the assembly to figure out what exception can be thrown from the callee. 🤣

[alanwest]

I think these are great improvements. I think this conversation is a good one for us to follow up on separately #3504 (comment). I agree that settling on using Set* vs. Configure* vs. a delegate should be a broader conversation than the goals of this PR.

[CodeBlanch]

Merged. There are still some rough edges but now that we're in an alpha/beta stage getting ready for net7 I think it is better to get it in and continue to improve it.