Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Semconv migration Part2 Server - duplicate support #5400

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
195 changes: 195 additions & 0 deletions instrumentation/net/http/otelhttp/internal/semconv/dup.go
@@ -0,0 +1,195 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
MadVikingGod marked this conversation as resolved.
Show resolved Hide resolved

package semconv // import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp/internal/semconv"

import (
"io"
"net/http"
"slices"
"strings"

"go.opentelemetry.io/otel/attribute"
semconvOld "go.opentelemetry.io/otel/semconv/v1.20.0"
semconvNew "go.opentelemetry.io/otel/semconv/v1.24.0"
)

type dupHTTPServer struct{}

var _ HTTPServer = dupHTTPServer{}

// RequestTraceAttrs returns trace attributes for an HTTP request received by a
// server.
//
// The server must be the primary server name if it is known. For example this
// would be the ServerName directive
// (https://httpd.apache.org/docs/2.4/mod/core.html#servername) for an Apache
// server, and the server_name directive
// (http://nginx.org/en/docs/http/ngx_http_core_module.html#server_name) for an
// nginx server. More generically, the primary server name would be the host
// header value that matches the default virtual host of an HTTP server. It
// should include the host identifier and if a port is used to route to the
// server that port identifier should be included as an appropriate port
// suffix.
//
// If the primary server name is not known, server should be an empty string.
// The req Host will be used to determine the server instead.
func (d dupHTTPServer) RequestTraceAttrs(server string, req *http.Request) []attribute.KeyValue {
// old http.target http.scheme net.host.name net.host.port http.scheme net.host.name net.host.port http.method net.sock.peer.addr net.sock.peer.port user_agent.original http.method http.status_code net.protocol.version
// new http.request.header server.address server.port network.local.address network.local.port client.address client.port url.path url.query url.scheme user_agent.original server.address server.port url.scheme http.request.method http.response.status_code error.type network.protocol.name network.protocol.version http.request.method_original http.response.header http.request.method network.peer.address network.peer.port network.transport http.request.method http.response.status_code error.type network.protocol.name network.protocol.version

const MaxAttributes = 24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const MaxAttributes = 24
const maxAttributes = 24

attrs := make([]attribute.KeyValue, MaxAttributes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to allocate more space than needed when not all attributes are included. Can we have an issue track setting this to the exact size from the start?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it will, this is because optimizing for allocated space is over optimizing. When you measure at the http.Handler the extra capacity is reused. I have built a benchmark to demonstrate this, but they exist outside of this code so I didn't include in the PR

Instead I've optimized for time spent in the function, by changing the workflow from Check if Attributes Exist to count ->Create the Slice -> Check if Attributes exist to Append to Make a Slice -> Check if Attributes exists to Append.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not

Suggested change
attrs := make([]attribute.KeyValue, MaxAttributes)
attrs := make([]attribute.KeyValue, 0, MaxAttributes)

and append?

var host string
var p int
if server == "" {
host, p = splitHostPort(req.Host)
} else {
// Prioritize the primary server name.
host, p = splitHostPort(server)
if p < 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

p < 0 doesn't invalidate the host. Shouldn't this check the host value as well first?

Suggested change
if p < 0 {
if host == "" && p < 0 {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a copy of the logic from the current implementation.

If there is no server, then it will take what it can from the request.Host. If there is a server (server !="") it will use the host from that, and if the port is present it will use that.
Notice that last splitHostPort only will set port.

_, p = splitHostPort(req.Host)

Check warning on line 62 in instrumentation/net/http/otelhttp/internal/semconv/dup.go

View check run for this annotation

Codecov / codecov/patch

instrumentation/net/http/otelhttp/internal/semconv/dup.go#L60-L62

Added lines #L60 - L62 were not covered by tests
}
}

attrs[0] = semconvOld.NetHostName(host)
attrs[1] = semconvNew.ServerAddress(host)
i := 2
if hostPort := requiredHTTPPort(req.TLS != nil, p); hostPort > 0 {
attrs[i] = semconvOld.NetHostPort(hostPort)
attrs[i+1] = semconvNew.ServerPort(hostPort)
i += 2
}
i += d.method(req.Method, attrs[i:]) // Max 3
i += d.scheme(req.TLS != nil, attrs[i:]) // Max 2

if peer, peerPort := splitHostPort(req.RemoteAddr); peer != "" {
// The Go HTTP server sets RemoteAddr to "IP:port", this will not be a
// file-path that would be interpreted with a sock family.
attrs[i] = semconvOld.NetSockPeerAddr(peer)
attrs[i+1] = semconvNew.NetworkPeerAddress(peer)
i += 2
if peerPort > 0 {
attrs[i] = semconvOld.NetSockPeerPort(peerPort)
attrs[i+1] = semconvNew.NetworkPeerPort(peerPort)
i += 2
}
}

if useragent := req.UserAgent(); useragent != "" {
// This is the same between v1.20, and v1.24
attrs[i] = semconvNew.UserAgentOriginal(useragent)
i++
}

if clientIP := serverClientIP(req.Header.Get("X-Forwarded-For")); clientIP != "" {
attrs[i] = semconvOld.HTTPClientIP(clientIP)
attrs[i+1] = semconvNew.ClientAddress(clientIP)
i += 2
}

if req.URL != nil && req.URL.Path != "" {
attrs[i] = semconvOld.HTTPTarget(req.URL.Path)
attrs[i+1] = semconvNew.URLPath(req.URL.Path)
i += 2
}

protoName, protoVersion := netProtocol(req.Proto)
if protoName != "" && protoName != "http" {
attrs[i] = semconvOld.NetProtocolName(protoName)
attrs[i+1] = semconvNew.NetworkProtocolName(protoName)
i += 2

Check warning on line 112 in instrumentation/net/http/otelhttp/internal/semconv/dup.go

View check run for this annotation

Codecov / codecov/patch

instrumentation/net/http/otelhttp/internal/semconv/dup.go#L110-L112

Added lines #L110 - L112 were not covered by tests
}
if protoVersion != "" {
attrs[i] = semconvOld.NetProtocolVersion(protoVersion)
attrs[i+1] = semconvNew.NetworkProtocolVersion(protoVersion)
i += 2
}

return slices.Clip(attrs[:i])
}

func (d dupHTTPServer) method(method string, attrs []attribute.KeyValue) int {
if method == "" {
attrs[0] = semconvOld.HTTPMethod(http.MethodGet)
attrs[1] = semconvNew.HTTPRequestMethodGet
return 2

Check warning on line 127 in instrumentation/net/http/otelhttp/internal/semconv/dup.go

View check run for this annotation

Codecov / codecov/patch

instrumentation/net/http/otelhttp/internal/semconv/dup.go#L125-L127

Added lines #L125 - L127 were not covered by tests
}
attrs[0] = semconvOld.HTTPMethod(method)
if attr, ok := methodLookup[method]; ok {
attrs[1] = attr
return 2
}

if attr, ok := methodLookup[strings.ToUpper(method)]; ok {
attrs[1] = attr
} else {
// If the Original method is not a standard HTTP method fallback to GET
attrs[1] = semconvNew.HTTPRequestMethodGet
}
attrs[2] = semconvNew.HTTPRequestMethodOriginal(method)
return 3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The index state is not tracked in a variable. This method is going to be very inflexible to changes. Can we track the index with a variable instead?

Or just use attrs[:0] and append to it? That way len(attrs) can always be returned?

}

func (d dupHTTPServer) scheme(https bool, attrs []attribute.KeyValue) int { // nolint:revive
if https {
attrs[0] = semconvOld.HTTPSchemeHTTPS
attrs[1] = semconvNew.URLScheme("https")
return 2

Check warning on line 149 in instrumentation/net/http/otelhttp/internal/semconv/dup.go

View check run for this annotation

Codecov / codecov/patch

instrumentation/net/http/otelhttp/internal/semconv/dup.go#L147-L149

Added lines #L147 - L149 were not covered by tests
}
attrs[0] = semconvOld.HTTPSchemeHTTP
attrs[1] = semconvNew.URLScheme("http")
return 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar here regarding state tracking.

}

// ResponseTraceAttrs returns trace attributes for telemetry from an HTTP response.
//
// If any of the fields in the ResponseTelemetry are not set the attribute will be omitted.
func (d dupHTTPServer) ResponseTraceAttrs(resp ResponseTelemetry) []attribute.KeyValue {
attributes := []attribute.KeyValue{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to rely on the compiler strategy to allocate space for the attributes. Likely over-allocating. Can we similar to above pre-allocate capacity? Or can we change above to also follow this?


if resp.ReadBytes > 0 {
attributes = append(attributes,
semconvOld.HTTPRequestContentLength(int(resp.ReadBytes)),
semconvNew.HTTPRequestBodySize(int(resp.ReadBytes)),
)
}
if resp.ReadError != nil && resp.ReadError != io.EOF {
// This is not in the semantic conventions, but is historically provided
attributes = append(attributes, attribute.String("http.read_error", resp.ReadError.Error()))
}
if resp.WriteBytes > 0 {
attributes = append(attributes,
semconvOld.HTTPResponseContentLength(int(resp.WriteBytes)),
semconvNew.HTTPResponseBodySize(int(resp.WriteBytes)),
)
}
if resp.WriteError != nil && resp.WriteError != io.EOF {
// This is not in the semantic conventions, but is historically provided
attributes = append(attributes, attribute.String("http.write_error", resp.WriteError.Error()))
}
if resp.StatusCode > 0 {
attributes = append(attributes,
semconvOld.HTTPStatusCode(resp.StatusCode),
semconvNew.HTTPResponseStatusCode(resp.StatusCode),
)
}

return attributes
}

// Route returns the attribute for the route.
func (d dupHTTPServer) Route(route string) attribute.KeyValue {
return semconvNew.HTTPRoute(route)

Check warning on line 194 in instrumentation/net/http/otelhttp/internal/semconv/dup.go

View check run for this annotation

Codecov / codecov/patch

instrumentation/net/http/otelhttp/internal/semconv/dup.go#L193-L194

Added lines #L193 - L194 were not covered by tests
}
156 changes: 156 additions & 0 deletions instrumentation/net/http/otelhttp/internal/semconv/dup_test.go
@@ -0,0 +1,156 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package semconv

import (
"fmt"
"net/http"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

"go.opentelemetry.io/otel/attribute"
)

func TestDupTraceRequest(t *testing.T) {
t.Setenv("OTEL_HTTP_CLIENT_COMPATIBILITY_MODE", "http/dup")
serv := NewHTTPServer()
want := func(req testServerReq) []attribute.KeyValue {
return []attribute.KeyValue{
attribute.String("http.method", "GET"),
attribute.String("http.request.method", "GET"),
attribute.String("http.scheme", "http"),
attribute.String("url.scheme", "http"),
attribute.String("net.host.name", req.hostname),
attribute.String("server.address", req.hostname),
attribute.Int("net.host.port", req.serverPort),
attribute.Int("server.port", req.serverPort),
attribute.String("net.sock.peer.addr", req.peerAddr),
attribute.String("network.peer.address", req.peerAddr),
attribute.Int("net.sock.peer.port", req.peerPort),
attribute.Int("network.peer.port", req.peerPort),
attribute.String("user_agent.original", "Go-http-client/1.1"),
attribute.String("http.client_ip", req.clientIP),
attribute.String("client.address", req.clientIP),
attribute.String("net.protocol.version", "1.1"),
attribute.String("network.protocol.version", "1.1"),
attribute.String("http.target", "/"),
attribute.String("url.path", "/"),
}
}
testTraceRequest(t, serv, want)
}

func TestDupMethod(t *testing.T) {
testCases := []struct {
method string
n int
want []attribute.KeyValue
}{
{
method: http.MethodPost,
n: 2,
want: []attribute.KeyValue{
attribute.String("http.method", "POST"),
attribute.String("http.request.method", "POST"),
},
},
{
method: "Put",
n: 3,
want: []attribute.KeyValue{
attribute.String("http.method", "Put"),
attribute.String("http.request.method", "PUT"),
attribute.String("http.request.method_original", "Put"),
},
},
{
method: "Unknown",
n: 3,
want: []attribute.KeyValue{
attribute.String("http.method", "Unknown"),
attribute.String("http.request.method", "GET"),
attribute.String("http.request.method_original", "Unknown"),
},
},
}

for _, tt := range testCases {
t.Run(tt.method, func(t *testing.T) {
attrs := make([]attribute.KeyValue, 5)
n := dupHTTPServer{}.method(tt.method, attrs[1:])
require.Equal(t, tt.n, n, "Length doesn't match")
require.ElementsMatch(t, tt.want, attrs[1:n+1])
})
}
}

func TestDupTraceResponse(t *testing.T) {
testCases := []struct {
name string
resp ResponseTelemetry
want []attribute.KeyValue
}{
{
name: "empty",
resp: ResponseTelemetry{},
want: nil,
},
{
name: "no errors",
resp: ResponseTelemetry{
StatusCode: 200,
ReadBytes: 701,
WriteBytes: 802,
},
want: []attribute.KeyValue{
attribute.Int("http.request_content_length", 701),
attribute.Int("http.request.body.size", 701),
attribute.Int("http.response_content_length", 802),
attribute.Int("http.response.body.size", 802),
attribute.Int("http.status_code", 200),
attribute.Int("http.response.status_code", 200),
},
},
{
name: "with errors",
resp: ResponseTelemetry{
StatusCode: 200,
ReadBytes: 701,
ReadError: fmt.Errorf("read error"),
WriteBytes: 802,
WriteError: fmt.Errorf("write error"),
},
want: []attribute.KeyValue{
attribute.Int("http.request_content_length", 701),
attribute.Int("http.request.body.size", 701),
attribute.String("http.read_error", "read error"),
attribute.Int("http.response_content_length", 802),
attribute.Int("http.response.body.size", 802),
attribute.String("http.write_error", "write error"),
attribute.Int("http.status_code", 200),
attribute.Int("http.response.status_code", 200),
},
},
}

for _, tt := range testCases {
t.Run(tt.name, func(t *testing.T) {
got := dupHTTPServer{}.ResponseTraceAttrs(tt.resp)
assert.ElementsMatch(t, tt.want, got)
})
}
}
10 changes: 9 additions & 1 deletion instrumentation/net/http/otelhttp/internal/semconv/env.go
Expand Up @@ -6,6 +6,8 @@ package semconv // import "go.opentelemetry.io/contrib/instrumentation/net/http/
import (
"fmt"
"net/http"
"os"
"strings"

"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
Expand Down Expand Up @@ -52,7 +54,13 @@ type HTTPServer interface {
func NewHTTPServer() HTTPServer {
// TODO (#5331): Detect version based on environment variable OTEL_HTTP_CLIENT_COMPATIBILITY_MODE.
// TODO (#5331): Add warning of use of a deprecated version of Semantic Versions.
return oldHTTPServer{}
env := strings.ToLower(os.Getenv("OTEL_HTTP_CLIENT_COMPATIBILITY_MODE"))
switch env {
case "http/dup":
return dupHTTPServer{}
default:
return oldHTTPServer{}
}
}

// ServerStatus returns a span status code and message for an HTTP status code
Expand Down