Project Tracking: Sampling

[jmacd]

Project-tracking for the Sampling SIG:

Probability sampling threshold w/ OTEP 235 for traces signal

Status: Merged.

OTEP 235 specifies how to encode information about the selection probability of a sampled span. This makes use of Trace Context Level 2.

https://github.com/open-telemetry/oteps/blob/main/text/trace/0235-sampling-threshold-in-trace-state.md

Compositional logic for samplers

Status: In Review

OTEP 250 adds definitions for EachOf, AnyOf, Conjunction, and RuleBased samplers for combining arbitrary samplers. It also defines Consistent forms of composition which can be implemented in a much more direct and efficient way, with changes in the Sampler API.

OTEP 240 proposes a Yaml syntax for expressing these compositional forms.

These two OTEPs, implemented together inside a Trace SDK, should make it possible to develop an OTel-specified remote sampler configuration similar to Jaeger's.

• Multi-policy sampling #2113
• Remote controlled sampling rules using attributes #2085
• Leaky-bucket rate limiting sampler #1769

Probability sampling semantic conventions

Status: In Review

open-telemetry/semantic-conventions#793 extends the semantic conventions with the conventions documented in OTEP 235 for traces. Additionally, this extends the semantic conventions to log records with the use of attributes instead of TraceState fields.

Intermediate Span Processor for OTel Collector

Status: In Review

open-telemetry/opentelemetry-collector-contrib#31918 describes an upgrade to the OTel Collector's probabilisticsamplerprocessor to emit OTEP 235 tracestate information, allowing it to convey sampling probability. A new package was added to the collector-contrib with common logic for this purpose, which will be useful in other components, see https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/sampling.

Specification update for Trace SDK built-in Samplers

Status: Draft

The TraceIDRatioBased sampler originally specified in the OTel 1.0 traces specification contains a lingering TODO related to OTEP 235. Here is a draft of the specification change #3910 we are looking at.

• Review approach & specify algorithm for TraceIdRatioBasedSampler (ProbabilitySampler) #1413
• Complete the TraceIdRatio specification #1826

Version-2 Sampler API prototype

Status: Requirements gathering

A number of issues have come up related to limitations in the original (Traces v1.0) Sampler API that deserve to be fixed by a future Sampling API. This will be necessary for the compositional samplers proposed in OTEP 250. Here are a list of other issues we hope to address in the V2 sampler API:

• Instrumentation library info and Resources are not available to Samplers #1588
• Support for tail-based sampling #1720
• Sampling when span has links #1877
• Remove unreasonable restriction on Sampler's description to be immutable #2095
• SpanProcessor without sampling #2543
• Best way to to support traceresponse and load balancer deferred sampling? #2914
• Let SDKs export all the spans regardless of their sampled flag #2986
• Introduce means of producing metrics from all Spans regardless of sampling decision. #3145
• Sampling: Each exporter should have isolated control over its sampler decision and custom processing #3284
• Clarification on how processors handle sampling decisions. #3779
• Give samplers the span ID #3991

[jpkrohling]

@jmacd, does this need to be tracked as an official project? I could be the sponsor for this, and we could be formally tracking this.

https://github.com/open-telemetry/community/blob/main/project-management.md#project-proposal