Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tracing: less noisy, more useful log records for spans #613

Merged
merged 6 commits into from
Feb 28, 2020

Commits on Feb 28, 2020

  1. tracing: log span lifecycle events under a separate target

    ## Motivation
    
    Currently, the `tracing` crate's "log" feature always logs span
    lifecycle (creation/enter/exit/close) events, even when the span has no
    fields. This can be quite noisy. Since the enter, exit, and close logs
    don't include the span's fields, it doesn't add that much additional
    context to the surrounding log records.
    
    ## Solution
    
    In order to continue recording this data while improving the
    signal-to-noise ratio of `tracing`-generated logs, this commit changes
    the log integration to log span lifecycle events under a separate
    target, "tracing::span". This way, log users can opt in to this data
    when it is desired, but logs are generally much less noisy. Span
    creation is still logged to the default target when the span has fields,
    as this often provides useful information, but is sent to the
    "tracing-span" target when there are no fields.
    
    Fixes #607
    hawkw committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    fccdf68 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4e6182b View commit details
    Browse the repository at this point in the history
  3. tracing: include span IDs in logs when available

    When using `tracing` with the `log-always`, the IDs of enabled spans are
    available, but not included in the span lifecycle `log` records. These
    could be used to enable tracking individual span's lifecycles via these
    log records, but this is not currently possible.
    
    This commit updates the `log` integration so that when a span is enabled
    by a subscriber, and `log` integration is enabled, that span's `log`
    records will include its ID. Now, when multiple spans with the same name
    are in the logs, we have a rudimentary ability to correlate those span's
    lifecycle events, without introducing so much complexity that just using
    a `tracing` subscriber would be a better solution.
    
    For example, when a subscriber is assigning IDs to spans, the logs look
    like this:
    
    ```
    ERROR log_with_trace foo=5
    ERROR log_with_trace foo=42
    ERROR log_with_trace foo=39
    WARN  log_with_trace hello world;
    INFO  log_with_trace hello world; thingy=42 other_thingy=666
    TRACE tracing::span ++ foo; span=1
    TRACE tracing::span -> foo; span=1
    TRACE log_with_trace hello san francisco; foo=3 bar=4
    TRACE tracing::span <- foo; span=1
    TRACE tracing::span -- foo; span=1
    TRACE log_with_trace ++ foo; bar=3 baz=false; span=2
    TRACE tracing::span -- foo; span=2
    TRACE log_with_trace hello world foo=1 bar=2
    ```
    
    Signed-off-by: Eliza Weisman <eliza@buoyant.io>
    hawkw committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    e17b8cc View commit details
    Browse the repository at this point in the history
  4. much nicer semicolon placement

    Signed-off-by: Eliza Weisman <eliza@buoyant.io>
    hawkw committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    ba76df3 View commit details
    Browse the repository at this point in the history
  5. remove accidentally added file

    Signed-off-by: Eliza Weisman <eliza@buoyant.io>
    hawkw committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    559cef0 View commit details
    Browse the repository at this point in the history
  6. Update tracing/src/lib.rs

    Co-Authored-By: David Barsky <me@davidbarsky.com>
    hawkw and davidbarsky committed Feb 28, 2020
    Configuration menu
    Copy the full SHA
    8b10e76 View commit details
    Browse the repository at this point in the history