First cut at AI semconv

docs/ai/README.md

@@ -0,0 +1,30 @@
+<!--- Hugo front matter used to generate the website version of this page:
+linkTitle: AI
+path_base_for_github_subdir:
+  from: content/en/docs/specs/semconv/ai/_index.md
+  to: database/README.md
+--->
+
+# Semantic Conventions for AI systems
+
+**Status**: [Experimental][DocumentStatus]
+
+This document defines semantic conventions for the following kind of AI systems:
+
+* LLMs
+* Vector Embeddings (todo)
+* Vector Databases (todo)
+
+Semantic conventions for LLM operations are defined for the following signals:
+
+* [LLM Spans](llm-spans.md): Semantic Conventions for LLM requests.
+* [LLM chains](llm-chains.md): Semantic Conventions for LLM chains.
+
+Technology specific semantic conventions are defined for the following databases:
+
+* [OpenAI](openai.md): Semantic Conventions for *OpenAI*.
+* [Anthropic](anthropic.md): Semantic Conventions for *Anthropic*.
+* [Cohere](cohere.md): Semantic Conventions for *Cohere*.
+* [Replicate](replicate.md): Semantic Conventions for *Replicate*.
+
+[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.22.0/specification/document-status.md

docs/ai/llm-spans.md

@@ -0,0 +1,65 @@
+<!--- Hugo front matter used to generate the website version of this page:
+linkTitle: LLM Calls
+--->
+
+# Semantic Conventions for LLM requests
+
+**Status**: [Experimental][DocumentStatus]
+
+<!-- Re-generate TOC with `markdown-toc --no-first-h1 -i` -->
+
+<!-- toc -->
+
+- [LLM Request attributes](#llm-request-attributes)
+- [Semantic Conventions for specific LLM technologies](#semantic-conventions-for-specific-llm-technologies)
+
+<!-- tocstop -->
+
+**Span kind:** MUST always be `CLIENT`.
+
+The **span name** SHOULD be set to a low cardinality value representing the request made to an LLM.
+It MAY be a name of the API endpoint for the LLM being called.
+
+## LLM Request attributes
+
+These attributes track input data and metadata for a request to an LLM. Each attribute represents a concept that is common to most LLMs.
+
+<!-- semconv ai(tag=llm-request) -->
+| Attribute  | Type | Description  | Examples  | Requirement Level |
+|---|---|---|---|---|
+| `llm.model` | string | The name of the LLM a request is being made to. If the LLM is supplied by a vendor, then the value must be the exact name of the model used. If the LLM is a fine-tuned custom model, the value SHOULD have a more specific name than the base model that's been fine-tuned. | `gpt-4` | Required |
+| `llm.prompt` | string | The full prompt string sent to an LLM in a request. If the LLM accepts a more complex input like a JSON object made up of several pieces (such as OpenAI's different message types), this field is that entire JSON object encoded as a string. | `\n\nHuman:You are an AI assistant that tells jokes. Can you tell me a joke about OpenTelemetry?\n\nAssistant:` | Required |
+| `llm.max_tokens` | int | The maximum number of tokens the LLM generates for a request. | `100` | Recommended |
+| `llm.temperature` | float | The temperature setting for the LLM request. | `0.0` | Recommended |
+| `llm.top_p` | float | The top-p setting for the LLM request. | `1.0` | Recommended |
+| `llm.stream` | bool | Whether the LLM responds with a stream. | `false` | Recommended |
+| `llm.stop_sequences` | array | Array of strings the LLM uses as a stop sequence. | `["stop1"]` | Recommended |
+
+`llm.model` has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
+
+| Value  | Description |
+|---|---|
+| `gpt-4` | GPT-4 |
+| `gpt-4-32k` | GPT-4 with 32k context window |
+| `gpt-3.5-turbo` | GPT-3.5-turbo |
+| `gpt-3.5-turbo-16k` | GPT-3.5-turbo with 16k context window|
+| `claude-instant-1` | Claude Instant (latest version) |
+| `claude-2` | Claude 2 (latest version) |
+<!-- endsemconv -->
+
+## LLM Response attributes
+
+These attributes track output data and metadata for a response from an LLM. Each attribute represents a concept that is common to most LLMs.
+
+<!-- semconv ai(tag=llm-response) -->
+| Attribute  | Type | Description  | Examples  | Requirement Level |
+|---|---|---|---|---|
+| `llm.completion` | string | The full response string from an LLM. If the LLM responds with a more complex output like a JSON object made up of several pieces (such as OpenAI's message choices), this field is the content of the response. If the LLM produces multiple responses, then this field is left blank, and each response is instead captured in an attribute determined by the specific LLM technology semantic convention for responses.| `Why did the developer stop using OpenTelemetry? Because they couldn't trace their steps!` | Required |
+
+## Semantic Conventions for specific LLM technologies
+
+More specific Semantic Conventions are defined for the following database technologies:
+
+* [OpenAI](openai.md): Semantic Conventions for *OpenAI*.
+
+[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.22.0/specification/document-status.md

docs/ai/openai.md

@@ -0,0 +1,34 @@
+<!--- Hugo front matter used to generate the website version of this page:
+linkTitle: OpenAI
+--->
+
+# Semantic Conventions for OpenAI
+
+**Status**: [Experimental][DocumentStatus]
+
+The Semantic Conventions for [OpenAI](https://www.microsoft.com/sql-server) extend the [LLM Semantic Conventions](llm-spans.md)
+that describe common LLM request attributes in addition to the Semantic Conventions
+described on this page.
+
+## OpenAI LLM attributes
+
+<!-- semconv llm.openai(tag=llm-request-tech-specific) -->
+| Attribute  | Type | Description  | Examples  | Requirement Level |
+|---|---|---|---|---|
+| `llm.openai.messages.<index>.role` | string | The assigned role for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `system` | Required |
+| `llm.openai.messages.<index>.message` | string | The message for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `You are an AI system that tells jokes about OpenTelemetry.` | Required |
+| `llm.openai.messages.<index>.name` | string | If present, the message for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `You are an AI system that tells jokes about OpenTelemetry.` | Required |
+| `llm.openai.messages.<index>.function_call.name` | string | If present, name of an OpenAI function for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `get_weather_forecast` | Required |
+| `llm.openai.messages.<index>.function_call.arguments` | string | If present, the arguments to call a function call with for an OpenAI function for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `{"type": "object", "properties": {}}` | Required |
+| `llm.openai.functions.<index>.name` | string | If present, name of an OpenAI function for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `get_weather_forecast` | Required |
+| `llm.openai.functions.<index>.parameters` | string | If present, JSON-encoded string of the parameter object of an OpenAI function for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `{"type": "object", "properties": {}}` | Required |
+| `llm.openai.functions.<index>.description` | string | If present, description of an OpenAI function for a given OpenAI request, denoted by `<index>`. The value for `<index>` starts with 0, where 0 is the first message. | `Gets the weather forecast.` | Required |
+| `llm.openai.n` | int | If present, the number of messages an OpenAI request responds with. | `2` | Recommended |
+| `llm.openai.presence_penalty` | float | If present, the `presence_penalty` used in an OpenAI request. Value is between -2.0 and 2.0. | `-0.5` | Recommended |
+| `llm.openai.frequency_penalty` | float | If present, the `frequency_penalty` used in an OpenAI request. Value is between -2.0 and 2.0. | `-0.5` | Recommended |
+| `llm.openai.logit_bias` | string | If present, the JSON-encoded string of a `logit_bias` used in an OpenAI request. | `{2435:-100, 640:-100}` | Recommended |
+| `llm.openai.user` | string | If present, the `user` used in an OpenAI request. | `bob` | Recommended |
+
+
+
+[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.22.0/specification/document-status.md