Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tracing-subscriber: please log that there was an error serializing an event #1965

Closed
jaskij opened this issue Mar 1, 2022 · 1 comment
Closed
Assignees
Labels
crate/subscriber Related to the `tracing-subscriber` crate good first issue Good for newcomers kind/feature New feature or request

Comments

@jaskij
Copy link

jaskij commented Mar 1, 2022

Feature Request

tokio-rs/valuable#88 came because I realized that tracing-subscriber, specifically the json formatter, missed some of the logged events.

Crates

tracing-subscriber

Motivation

Logs, and traces, carry information. It is good to know that some of it was lost. In my case what failed to serialize was the error which caused abnormal termination.

Proposal

If an event fails to serialize (whether in json or a different formatter), I'd like to see a new event generated, informing me about it. Best would be if this was optional, as I'm sure not everyone would want it.

Alternatives

The other alternatives are to either silently ignore the failed event (what happens now), or panic. Panic seems unreasonably harsh, especially since tracing is used by long running services.

@hawkw hawkw added kind/feature New feature or request good first issue Good for newcomers crate/subscriber Related to the `tracing-subscriber` crate labels Apr 22, 2022
bryangarza added a commit to bryangarza/tracing that referenced this issue Apr 29, 2022
This patch adds a single `eprintln` line to let the user know that we
were unable to format a specific event. We are not emitting an actual
tracing Event, to avoid the risk of a cycle (the new Event could
trigger the same formatting error again).

Resolves tokio-rs#1965.
bryangarza added a commit to bryangarza/tracing that referenced this issue Apr 29, 2022
This patch adds a single `eprintln` line to let the user know that we
were unable to format a specific event. We are not emitting an actual
tracing Event, to avoid the risk of a cycle (the new Event could
trigger the same formatting error again).

Resolves tokio-rs#1965.
bryangarza added a commit to bryangarza/tracing that referenced this issue Apr 29, 2022
When `Format_event::format_event(...)` returns an errror, we are
currently silently dropping that
Span/Event. tokio-rs/valuable#88 explains
one such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever a Span/Event is dropped.

This patch adds a single `eprintln` line to let the user know that we
were unable to format a specific event. We are not emitting an actual
tracing Event, to avoid the risk of a cycle (the new Event could
trigger the same formatting error again).

Resolves tokio-rs#1965.
bryangarza added a commit to bryangarza/tracing that referenced this issue Apr 29, 2022
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

This patch adds a single `eprintln` line to let the user know that we
were unable to format a specific event. We are not emitting an actual
tracing Event, to avoid the risk of a cycle (the new Event could
trigger the same formatting error again).

Resolves tokio-rs#1965.
bryangarza added a commit that referenced this issue Jul 25, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
@bryangarza bryangarza self-assigned this Aug 11, 2022
@bryangarza
Copy link
Member

Resolved by #2102

davidbarsky added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
davidbarsky added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
davidbarsky added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
davidbarsky added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
davidbarsky added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
davidbarsky added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
davidbarsky added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
hawkw added a commit that referenced this issue Aug 24, 2022
…ter (#2102)

Motivation:
When `Format_event::format_event(...)` returns an error, we are
currently silently dropping that
Event. tokio-rs/valuable#88 explains one
such case in which this was encountered (due to a bug in
valuable-serde). We want to be made aware whenever an Event is dropped.

Solution:
Write to the Writer with an error message to let the user know that
we were unable to format a specific event. If writing to the Writer fails,
we fall back to writing to stderr. We are not emitting an actual tracing
Event, to avoid the risk of a cycle (the new Event could trigger the
same formatting error again).

Resolves #1965.

Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: David Barsky <me@davidbarsky.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crate/subscriber Related to the `tracing-subscriber` crate good first issue Good for newcomers kind/feature New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants