Add an option to limit length of values of attributes and metric valu…

…es (open-telemetry#1130)
kenfinnigan · Aug 3, 2021 · 7cef355 · 7cef355
1 parent c35c212
commit 7cef355
Show file tree

Hide file tree

Showing 6 changed files with 93 additions and 17 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -11,6 +11,10 @@ release.
 
 ### Traces
 
+- Add generalized attribute count and attribute value length limits and relevant
+  environment variables.
+  ([#1130](https://github.com/open-telemetry/opentelemetry-specification/pull/1130))
+- Adding environment variables for event and link attribute limits. ([#1751](https://github.com/open-telemetry/opentelemetry-specification/pull/1751))
 - Adding SDK configuration for Jaeger remote sampler ([#1791](https://github.com/open-telemetry/opentelemetry-specification/pull/1791))
 
 ### Metrics

diff --git a/spec-compliance-matrix.md b/spec-compliance-matrix.md
@@ -79,6 +79,7 @@ formats is required. Implementing more than one format is optional.
 | [IdGenerators](specification/trace/sdk.md#id-generators)                                         |          | +  | +    |    | +      | +    |        |     | +    | +   |      | +     |
 | [SpanLimits](specification/trace/sdk.md#span-limits)                                             | X        | +  | +    |    | +      | +    |        |     |      | -   |      | +     |
 | [Built-in `SpanProcessor`s implement `ForceFlush` spec](specification/trace/sdk.md#forceflush-1) |          |    |      |    | +      | +    |        |     | +    | +   |      |       |
+| [Attribute Limits](specification/common/common.md#attribute-limits)                              | X        |    |      |    |        |      |        |     |      |     |      |       |
 
 ## Baggage
 
@@ -141,12 +142,15 @@ Note: Support for environment variables is optional.
 |OTEL_TRACES_EXPORTER                          | - | +  |   | +    | +  | +    |   | -  | - |    |     |
 |OTEL_METRICS_EXPORTER                         | - | +  |   | +    | -  | -    |   | -  | - | -  | -   |
 |OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT               | - | +  |   | +    | +  | -    |   | +  | - |    |     |
+|OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT        |   |    |   |      |    |      |   |    |   |    |     |
 |OTEL_SPAN_EVENT_COUNT_LIMIT                   | - | +  |   | +    | +  | -    |   | +  | - |    |     |
 |OTEL_SPAN_LINK_COUNT_LIMIT                    | - | +  |   | +    | +  | -    |   | +  | - |    |     |
 |OTEL_EVENT_ATTRIBUTE_COUNT_LIMIT              |   |    |   |      |    |      |   |    |   |    |     |
 |OTEL_LINK_ATTRIBUTE_COUNT_LIMIT               |   |    |   |      |    |      |   |    |   |    |     |
 |OTEL_TRACES_SAMPLER                           | - | +  |   | +    | +  | +    |   | -  | - |    |     |
 |OTEL_TRACES_SAMPLER_ARG                       | - | +  |   | +    | +  | +    |   | -  | - |    |     |
+|OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT             |   |    |   |      |    |      |   |    |   |    |     |
+|OTEL_ATTRIBUTE_COUNT_LIMIT                    |   |    |   |      |    |      |   |    |   |    |     |
 
 ## Exporters
 

diff --git a/specification/common/common.md b/specification/common/common.md
@@ -8,6 +8,8 @@ Table of Contents
 </summary>
 
 - [Attributes](#attributes)
+  - [Attribute Limits](#attribute-limits)
+    - [Exempt Entities](#exempt-entities)
 
 </details>
 
@@ -41,3 +43,52 @@ both containing an array of strings to represent a mapping
 `header_keys[i] -> header_values[i]`).
 
 See [Attribute Naming](attribute-naming.md) for naming guidelines.
+
+### Attribute Limits
+
+Execution of erroneous code can result in unintended attributes. If there are no
+limits placed on attributes, they can quickly exhaust available memory, resulting
+in crashes that are difficult to recover from safely.
+
+By default an SDK SHOULD apply truncation as per the list of
+[configurable parameters](#attribute-limits-configuration) below.
+
+If an SDK provides a way to:
+
+- set an attribute value length limit such that for each
+  attribute value:
+  - if it is a string, if it exceeds that limit (counting any character in it as
+    1), SDKs MUST truncate that value, so that its length is at most equal
+    to the limit,
+  - if it is an array of strings, then apply the above rule to each of the
+    values separately,
+  - otherwise a value MUST NOT be truncated;
+- set a limit of unique attribute keys such that:
+  - for each unique attributes key, addition of which would result in exceeding
+    the limit, SDK MUST discard that key/value pair.
+
+There MAY be a log emitted to indicate to the user that an attribute was
+truncated or discarded. To prevent excessive logging, the log MUST NOT be
+emitted more than once per record on which an attribute is set.
+
+If the SDK implements the limits above, it MUST provide a way to change these
+limits programmatically. Names of the configuration options SHOULD be the same as
+in the list below.
+
+An SDK MAY implement model-specific limits, for example
+`SpanAttributeCountLimit`. If both a general and a model-specific limit are
+implemented, then the SDK MUST first attempt to use the model-specific limit, if
+it isn't set and doesn't have a default, then the SDK MUST attempt to use the
+general limit.
+
+<a name="attribute-limits-configuration"></a>
+**Configurable parameters:**
+
+* `AttributeCountLimit` (Default=128) - Maximum allowed attribute count per record;
+* `AttributeValueLengthLimit` (Default=Infinity) - Maximum allowed attribute value length;
+
+#### Exempt Entities
+
+Attributes, which belong to Metrics, are exempt from the limits described above
+at this time, as discussed in
+[Metrics Attribute Limits](../metrics/sdk.md#attribute-limits).
diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md
@@ -23,6 +23,7 @@ Table of Contents
 </summary>
 
 * [MeterProvider](#meterprovider)
+* [Attribute Limits](#attribute-limits)
 * [MeasurementProcessor](#measurementprocessor)
 * [MetricProcessor](#metricprocessor)
 * [MetricExporter](#metricexporter)
@@ -207,6 +208,13 @@ meter_provider
     .set_exporter(ConsoleExporter())
 ```
 
+## Attribute Limits
+
+Attributes which belong to Metrics are exempt from the
+[common rules of attribute limits](../common/common.md#attribute-limits) at this
+time. Attribute truncation or deletion could affect identitity of metric time
+series and it requires further analysis.
+
 ## MeasurementProcessor
 
 `MeasurementProcessor` is an interface which allows hooks when a

diff --git a/specification/sdk-environment-variables.md b/specification/sdk-environment-variables.md
@@ -78,19 +78,32 @@ Depending on the value of `OTEL_TRACES_SAMPLER`, `OTEL_TRACES_SAMPLER_ARG` may b
 | OTEL_BSP_MAX_QUEUE_SIZE        | Maximum queue size                             | 2048    |                                                       |
 | OTEL_BSP_MAX_EXPORT_BATCH_SIZE | Maximum batch size                             | 512     | Must be less than or equal to OTEL_BSP_MAX_QUEUE_SIZE |
 
-## Span Collection Limits
+## Attribute Limits
+
+SDKs SHOULD only offer environment variables for the types of attributes, for
+which that SDK implements truncation mechanism.
+
+See the SDK [Attribute Limits](common/common.md#attribute-limits) section for the definition of the limits.
+
+| Name                              | Description                          | Default | Notes |
+| --------------------------------- | ------------------------------------ | ------- | ----- |
+| OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT | Maximum allowed attribute value size |         | Empty value is treated as infinity. Non-integer and negative values are invalid. |
+| OTEL_ATTRIBUTE_COUNT_LIMIT        | Maximum allowed span attribute count | 128     |       |
+
+## Span Limits <a name="span-collection-limits"></a>
 
 **Status**: [Stable](document-status.md)
 
 See the SDK [Span Limits](trace/sdk.md#span-limits) section for the definition of the limits.
 
-| Name                             | Description                                    | Default | Notes |
-| -------------------------------- | ---------------------------------------------- | ------- | ----- |
-| OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT  | Maximum allowed span attribute count           | 128     |       |
-| OTEL_SPAN_EVENT_COUNT_LIMIT      | Maximum allowed span event count               | 128     |       |
-| OTEL_SPAN_LINK_COUNT_LIMIT       | Maximum allowed span link count                | 128     |       |
-| OTEL_EVENT_ATTRIBUTE_COUNT_LIMIT | Maximum allowed attribute per span event count | 128     |       |
-| OTEL_LINK_ATTRIBUTE_COUNT_LIMIT  | Maximum allowed attribute per span link count  | 128     |       |
+| Name                                   | Description                                    | Default | Notes |
+| -------------------------------------- | ---------------------------------------------- | ------- | ----- |
+| OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT | Maximum allowed attribute value size           |         | Empty value is treated as infinity. Non-integer and negative values are invalid. |
+| OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT        | Maximum allowed span attribute count           | 128     |       |
+| OTEL_SPAN_EVENT_COUNT_LIMIT            | Maximum allowed span event count               | 128     |       |
+| OTEL_SPAN_LINK_COUNT_LIMIT             | Maximum allowed span link count                | 128     |       |
+| OTEL_EVENT_ATTRIBUTE_COUNT_LIMIT       | Maximum allowed attribute per span event count | 128     |       |
+| OTEL_LINK_ATTRIBUTE_COUNT_LIMIT        | Maximum allowed attribute per span link count  | 128     |       |
 
 ## OTLP Exporter
 

diff --git a/specification/trace/sdk.md b/specification/trace/sdk.md
@@ -319,20 +319,16 @@ Optional parameters:
 
 ## Span Limits
 
-Erroneous code can add unintended attributes, events, and links to a span. If
-these collections are unbounded, they can quickly exhaust available memory,
-resulting in crashes that are difficult to recover from safely.
+Span attributes MUST adhere to the [common rules of attribute limits](../common/common.md#attribute-limits).
 
-To protect against such errors, SDK Spans MAY discard attributes, links, and
-events that would increase the number of elements of each collection beyond
-the configured limit.
+SDK Spans MAY also discard links and events that would increase the number of
+elements of each collection beyond the configured limit.
 
 If the SDK implements the limits above it MUST provide a way to change these
 limits, via a configuration to the TracerProvider, by allowing users to
 configure individual limits like in the Java example bellow.
 
-The name of the configuration options SHOULD be `AttributeCountLimit`,
-`EventCountLimit` and `LinkCountLimit`. The options MAY be bundled in a class,
+The name of the configuration options SHOULD be `EventCountLimit` and `LinkCountLimit`. The options MAY be bundled in a class,
 which then SHOULD be called `SpanLimits`. Implementations MAY provide additional
 configuration such as `AttributePerEventCountLimit` and `AttributePerLinkCountLimit`.
 
@@ -354,7 +350,7 @@ public final class SpanLimits {
 
 **Configurable parameters:**
 
-* `AttributeCountLimit` (Default=128) - Maximum allowed span attribute count;
+* [all common options applicable to attributes](../common/common.md#attribute-limits-configuration)
 * `EventCountLimit` (Default=128) - Maximum allowed span event count;
 * `LinkCountLimit` (Default=128) - Maximum allowed span link count;
 * `AttributePerEventCountLimit` (Default=128) - Maximum allowed attribute per span event count;