Skip to content

Commit

Permalink
Merge branch 'main' into patch-5
Browse files Browse the repository at this point in the history
  • Loading branch information
pellared authored Aug 1, 2023
2 parents 6887614 + 19ca392 commit 58baadc
Show file tree
Hide file tree
Showing 4 changed files with 147 additions and 95 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ release.

- Add experimental metric attributes advice API.
([#3546](https://github.com/open-telemetry/opentelemetry-specification/pull/3546))
- Revise the exemplar default reservoirs.
([#3627](https://github.com/open-telemetry/opentelemetry-specification/pull/3627))
- Do not require `MetricReader` to have `ForceFlush`.
([#3609](https://github.com/open-telemetry/opentelemetry-specification/pull/3609))

Expand Down Expand Up @@ -67,6 +69,8 @@ release.
([#3559](https://github.com/open-telemetry/opentelemetry-specification/pull/3559))
- Make SDK Logger Creation more normative.
([#3529](https://github.com/open-telemetry/opentelemetry-specification/pull/3529))
- Clarify how log appender use Scope name and attributes.
([#3583](https://github.com/open-telemetry/opentelemetry-specification/pull/3583))

### Resource

Expand Down
90 changes: 6 additions & 84 deletions specification/logs/bridge-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,17 @@
- [Optional and required parameters](#optional-and-required-parameters)
- [Concurrency requirements](#concurrency-requirements)
- [Artifact Naming](#artifact-naming)
- [Usage](#usage)
* [How to Create a Log4J Log Appender](#how-to-create-a-log4j-log-appender)
* [Implicit Context Injection](#implicit-context-injection)
* [Explicit Context Injection](#explicit-context-injection)

<!-- tocstop -->

</details>

<b>Note: this document defines a log *backend* API. The API is not intended to be called
by application developers directly. It is provided for logging library authors
to build [log appenders](#how-to-create-a-log4j-log-appender), which use
this API to bridge between existing logging libraries and the OpenTelemetry log
data model.</b>
<b>Note: this document defines a log *backend* API. The API is not intended
to be called by application developers directly. It is provided for logging
library authors to build
[log appenders](./supplementary-guidelines.md#how-to-create-a-log4j-log-appender),
which use this API to bridge between existing logging libraries and the
OpenTelemetry log data model.</b>

The Logs Bridge API consist of these main classes:

Expand Down Expand Up @@ -156,78 +153,3 @@ in the event OpenTelemetry were to add a user facing API, the Logs Bridge API wo
be a natural starting point. Therefore, Log Bridge API artifact, package, and class
names MUST NOT include the terms "bridge", "appender", or any other qualifier
that would prevent evolution into a user facing API.

## Usage

### How to Create a Log4J Log Appender

A [log appender](../glossary.md#log-appender--bridge) implementation can be used
to bridge logs into the [Log SDK](./sdk.md)
OpenTelemetry [LogRecordExporters](sdk.md#logrecordexporter). This approach is
typically used for applications which are fine with changing the log transport
and is [one of the supported](README.md#direct-to-collector) log collection
approaches.

The log appender implementation will typically acquire a [Logger](#logger) from the
global [LoggerProvider](#loggerprovider) at startup time, then
call [Emit LogRecord](#emit-a-logrecord) for `LogRecord`s received from the
application.

[Implicit Context Injection](#implicit-context-injection)
and [Explicit Context Injection](#explicit-context-injection) describe how an
Appender injects `TraceContext` into `LogRecord`s.

![Appender](img/appender.png)

This same approach can be also used for example for:

- Python logging library by creating a Handler.
- Go zap logging library by implementing the Core interface. Note that since
there is no implicit Context in Go it is not possible to get and use the
active Span.

Log appenders can be created in OpenTelemetry language libraries by OpenTelemetry
maintainers, or by 3rd parties for any logging library that supports a similar
extension mechanism. This specification recommends each OpenTelemetry language
library to include out-of-the-box Appender implementation for at least one
popular logging library.

### Implicit Context Injection

When Context is implicitly available (e.g. in Java) the Appender can rely on
automatic context propagation by NOT explicitly setting `Context` when
calling [emit a LogRecord](#emit-a-logrecord).

Some log libraries have mechanisms specifically tailored for injecting
contextual information info logs, such as MDC in Log4j. When available, it may
be preferable to use these mechanisms to set the Context. A log appender can
then fetch the Context and explicitly set it when
calling [emit a LogRecord](#emit-a-logrecord). This allows the correct Context
to be included even when log records are emitted asynchronously, which can
otherwise lead the Context to be incorrect.

TODO: clarify how works or doesn't work when the log statement call site and the
log appender are executed on different threads.

### Explicit Context Injection

In order for `TraceContext` to be recorded in `LogRecord`s in languages where
the Context must be provided explicitly (e.g. Go), the end user must capture the
Context and explicitly pass it to the logging subsystem. The log appender must
take this Context and explicitly set it when
calling [emit a LogRecord](#emit-a-logrecord).

Support for OpenTelemetry for logging libraries in these languages typically can
be implemented in the form of logger wrappers that can capture the context once,
when the span is created and then use the wrapped logger to execute log
statements in a normal way. The wrapper will be responsible for injecting the
captured context in the logs.

This specification does not define how exactly it is achieved since the actual
mechanism depends on the language and the particular logging library used. In
any case the wrappers are expected to make use of the Trace Context API to get
the current active span.

See
[an example](https://docs.google.com/document/d/15vR7D1x2tKd7u3zaTF0yH1WaHkUr2T4hhr7OyiZgmBg/edit#heading=h.4xuru5ljcups)
of how it can be done for zap logging library for Go.
118 changes: 118 additions & 0 deletions specification/logs/supplementary-guidelines.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Supplementary Guidelines

Note: this document is NOT a spec, it is provided to support the Logs
[API](./bridge-api.md) and [SDK](./sdk.md) specifications, it does NOT add any
extra requirements to the existing specifications.

<details>
<summary>Table of Contents</summary>

<!-- Re-generate TOC with `markdown-toc --no-first-h1 -i` -->

<!-- toc -->

- [Usage](#usage)
* [How to Create a Log4J Log Appender](#how-to-create-a-log4j-log-appender)
* [Logger Name](#logger-name)
* [Context](#context)
+ [Implicit Context Injection](#implicit-context-injection)
+ [Explicit Context Injection](#explicit-context-injection)

<!-- tocstop -->

</details>

## Usage

### How to Create a Log4J Log Appender

A [log appender](../glossary.md#log-appender--bridge) implementation can be used
to bridge logs into the [Log SDK](./sdk.md)
OpenTelemetry [LogRecordExporters](sdk.md#logrecordexporter). This approach is
typically used for applications which are fine with changing the log transport
and is [one of the supported](README.md#direct-to-collector) log collection
approaches.

The log appender implementation will typically acquire a
[Logger](./bridge-api.md#logger), then
call [Emit LogRecord](./bridge-api.md#emit-a-logrecord) for `LogRecord`s
received from the application.

[Implicit Context Injection](#implicit-context-injection)
and [Explicit Context Injection](#explicit-context-injection) describe how an
Appender injects `TraceContext` into `LogRecord`s.

![Appender](img/appender.png)

This same approach can be also used for example for:

- Python logging library by creating a Handler.
- Go zap logging library by implementing the Core interface. Note that since
there is no implicit Context in Go it is not possible to get and use the
active Span.

Log appenders can be created in OpenTelemetry language libraries by OpenTelemetry
maintainers, or by 3rd parties for any logging library that supports a similar
extension mechanism. This specification recommends each OpenTelemetry language
library to include out-of-the-box Appender implementation for at least one
popular logging library.

### Logger Name

If the logging library has a concept that is similar to OpenTelemetry's
definition of the [Instrumentation Scope's](../glossary.md#instrumentation-scope)
name, then the appender's implementation should use that value as the
name parameter when [obtaining the Logger](./bridge-api.md#get-a-logger).

This is for example applicable to:

- Logger name in [Log4J](https://javadoc.io/doc/org.apache.logging.log4j/log4j-api/latest/org.apache.logging.log4j/org/apache/logging/log4j/Logger.html)
- Channel name in [Monolog](https://github.com/Seldaek/monolog/blob/3.4.0/doc/01-usage.md#leveraging-channels)

Appenders should avoid setting any attributes of the Instrumentation Scope.
Doing so may result in different appenders setting different attributes on the
same Instrumentation Scope, obtained with the same identity of the
[Logger](./bridge-api.md#get-a-logger), which, according to the specification,
is an error.

### Context

#### Implicit Context Injection

When Context is implicitly available (e.g. in Java) the Appender can rely on
automatic context propagation by NOT explicitly setting `Context` when
calling [emit a LogRecord](./bridge-api.md#emit-a-logrecord).

Some log libraries have mechanisms specifically tailored for injecting
contextual information info logs, such as MDC in Log4j. When available, it may
be preferable to use these mechanisms to set the Context. A log appender can
then fetch the Context and explicitly set it when
calling [emit a LogRecord](./bridge-api.md#emit-a-logrecord). This allows the correct Context
to be included even when log records are emitted asynchronously, which can
otherwise lead the Context to be incorrect.

TODO: clarify how works or doesn't work when the log statement call site and the
log appender are executed on different threads.

#### Explicit Context Injection

In order for `TraceContext` to be recorded in `LogRecord`s in languages where
the Context must be provided explicitly (e.g. Go), the end user must capture the
Context and explicitly pass it to the logging subsystem. The log appender must
take this Context and explicitly set it when
calling [emit a LogRecord](./bridge-api.md#emit-a-logrecord).

Support for OpenTelemetry for logging libraries in these languages typically can
be implemented in the form of logger wrappers that can capture the context once,
when the span is created and then use the wrapped logger to execute log
statements in a normal way. The wrapper will be responsible for injecting the
captured context in the logs.

This specification does not define how exactly it is achieved since the actual
mechanism depends on the language and the particular logging library used. In
any case the wrappers are expected to make use of the Trace Context API to get
the current active span.

See
[an example](https://docs.google.com/document/d/15vR7D1x2tKd7u3zaTF0yH1WaHkUr2T4hhr7OyiZgmBg/edit#heading=h.4xuru5ljcups)
of how it can be done for zap logging library for Go.
30 changes: 19 additions & 11 deletions specification/metrics/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ linkTitle: SDK
+ [TraceBased](#tracebased)
* [ExemplarReservoir](#exemplarreservoir)
* [Exemplar defaults](#exemplar-defaults)
+ [SimpleFixedSizeExemplarReservoir](#simplefixedsizeexemplarreservoir)
+ [AlignedHistogramBucketExemplarReservoir](#alignedhistogrambucketexemplarreservoir)
- [MetricReader](#metricreader)
* [MetricReader operations](#metricreader-operations)
+ [RegisterProducer(metricProducer)](#registerproducermetricproducer)
Expand Down Expand Up @@ -969,20 +971,21 @@ The `ExemplarReservoir` SHOULD avoid allocations when sampling exemplars.

### Exemplar defaults

The SDK will come with two types of built-in exemplar reservoirs:
The SDK SHOULD include two types of built-in exemplar reservoirs:

1. SimpleFixedSizeExemplarReservoir
2. AlignedHistogramBucketExemplarReservoir
1. `SimpleFixedSizeExemplarReservoir`
2. `AlignedHistogramBucketExemplarReservoir`

By default, explicit bucket histogram aggregation with more than 1 bucket will
use `AlignedHistogramBucketExemplarReservoir`. All other aggregations will use
`SimpleFixedSizeExemplarReservoir`.

_SimpleExemplarReservoir_
This Exemplar reservoir MAY take a configuration parameter for the size of the
reservoir pool. The reservoir will accept measurements using an equivalent of
the [naive reservoir sampling
algorithm](https://en.wikipedia.org/wiki/Reservoir_sampling)
#### SimpleFixedSizeExemplarReservoir

This reservoir MUST use an uniformly-weighted sampling algorithm based on the
number of samples the reservoir has seen so far to determine if the offered
measurements should be sampled. For example, the [simple reservoir sampling
algorithm](https://en.wikipedia.org/wiki/Reservoir_sampling) can be used:

```
bucket = random_integer(0, num_measurements_seen)
Expand All @@ -991,10 +994,15 @@ algorithm](https://en.wikipedia.org/wiki/Reservoir_sampling)
end
```

Additionally, the `num_measurements_seen` count SHOULD be reset at every
collection cycle.
Any stateful portion of sampling computation SHOULD be reset every collection
cycle. For the above example, that would mean that the `num_measurements_seen`
count is reset every time the reservoir is collected.

This Exemplar reservoir MAY take a configuration parameter for the size of the
reservoir pool.

#### AlignedHistogramBucketExemplarReservoir

_AlignedHistogramBucketExemplarReservoir_
This Exemplar reservoir MUST take a configuration parameter that is the
configuration of a Histogram. This implementation MUST keep the last seen
measurement that falls within a histogram bucket. The reservoir will accept
Expand Down

0 comments on commit 58baadc

Please sign in to comment.