Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does span execute span. end #5429

Closed
panRui935074243 opened this issue Apr 25, 2024 · 2 comments
Closed

Does span execute span. end #5429

panRui935074243 opened this issue Apr 25, 2024 · 2 comments
Labels
bug Something isn't working instrumentation: otelhttp question Further information is requested response needed Waiting on user input before progress can be made

Comments

@panRui935074243
Copy link

panRui935074243 commented Apr 25, 2024

Description

Transferring from service A to service B, service B received the spanid created in the transport, which was not executed on span. end, resulting in nodes A and B being at the same level

Environment

  • go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp version: [0.49.0]

Steps To Reproduce

ctx, span := tracer.Start(r.Context(), t.spanNameFormatter("", r), opts...)

if t.clientTrace != nil {
	ctx = httptrace.WithClientTrace(ctx, t.clientTrace(ctx))
}

labeler := &Labeler{}
ctx = injectLabeler(ctx, labeler)

r = r.Clone(ctx) // According to RoundTripper spec, we shouldn't modify the origin request.

// use a body wrapper to determine the request size
var bw bodyWrapper
// if request body is nil or NoBody, we don't want to mutate the body as it
// will affect the identity of it in an unforeseeable way because we assert
// ReadCloser fulfills a certain interface and it is indeed nil or NoBody.
if r.Body != nil && r.Body != http.NoBody {
	bw.ReadCloser = r.Body
	// noop to prevent nil panic. not using this record fun yet.
	bw.record = func(int64) {}
	r.Body = &bw
}

span.SetAttributes(semconvutil.HTTPClientRequest(r)...)
t.propagators.Inject(ctx, propagation.HeaderCarrier(r.Header))

res, err := t.rt.RoundTrip(r)
if err != nil {
	span.RecordError(err)
	span.SetStatus(codes.Error, err.Error())
	span.End()
	return res, err
}

// metrics
metricAttrs := append(labeler.Get(), semconvutil.HTTPClientRequestMetrics(r)...)
if res.StatusCode > 0 {
	metricAttrs = append(metricAttrs, semconv.HTTPStatusCode(res.StatusCode))
}
o := metric.WithAttributes(metricAttrs...)
t.requestBytesCounter.Add(ctx, bw.read.Load(), o)
// For handling response bytes we leverage a callback when the client reads the http response
readRecordFunc := func(n int64) {
	t.responseBytesCounter.Add(ctx, n, o)
}

// traces
span.SetAttributes(semconvutil.HTTPClientResponse(res)...)
span.SetStatus(semconvutil.HTTPClientStatus(res.StatusCode))

res.Body = newWrappedBody(span, readRecordFunc, res.Body)

// Use floating point division here for higher precision (instead of Millisecond method).
elapsedTime := float64(time.Since(requestStartTime)) / float64(time.Millisecond)

t.latencyMeasure.Record(ctx, elapsedTime, o)

return res, err

Expected behavior

If err is nil, execute span. end

I tested it above and I'm not sure if it's a configuration issue with me

@panRui935074243 panRui935074243 added the bug Something isn't working label Apr 25, 2024
@dmathieu
Copy link
Member

Your code sample is taken from transport.go.
And I suppose your question is: in the RoundTrip method, span.End() is only called if err != nil, here:

res, err := t.rt.RoundTrip(r)
if err != nil {
span.RecordError(err)
span.SetStatus(codes.Error, err.Error())
span.End()
return res, err
}

So yes, even if there is no error, span.End() will be called.
You can see span being assigned to a body wrapper here:

res.Body = newWrappedBody(span, readRecordFunc, res.Body)

This body wrapper can close the span in two cases:

When the body is being read, and we reach the end of the content:
https://github.com/open-telemetry/opentelemetry-go-contrib/blob/8c7c4deeeb9b4cc91672f44f74745b52e3eba1c5/instrumentation/net/http/otelhttp/transport.go#L246-248

When the body is being closed:

func (wb *wrappedBody) Close() error {
wb.recordBytesRead()
wb.span.End()
if wb.body != nil {
return wb.body.Close()
}
return nil
}

We need to do this so streaming requests are counted entirely. Otherwise, we could be closing the span before the entire body is received, and therefore before the action is actually finished.

This also means the body must be closed for the span to be ended. If you don't close the body, your span will be finished.
This aligns with the requirements from the net/http package though: https://pkg.go.dev/net/http#pkg-overview

The caller must close the response body when finished with it

Hopefully, that answers your question.

@MrAlias MrAlias added question Further information is requested response needed Waiting on user input before progress can be made labels Apr 26, 2024
@panRui935074243
Copy link
Author

I didn't look carefully, thank you

@dmathieu dmathieu closed this as not planned Won't fix, can't repro, duplicate, stale Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working instrumentation: otelhttp question Further information is requested response needed Waiting on user input before progress can be made
Projects
None yet
Development

No branches or pull requests

3 participants