Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more ECS url fields #181

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/url/url.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,26 @@ This document defines semantic conventions that describe URL and its components.
| `url.path` | string | The [URI path](https://www.rfc-editor.org/rfc/rfc3986#section-3.3) component [2] | `/search` | Recommended |
| `url.query` | string | The [URI query](https://www.rfc-editor.org/rfc/rfc3986#section-3.4) component [3] | `q=OpenTelemetry` | Recommended |
| `url.fragment` | string | The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component | `SemConv` | Recommended |
| `url.registered_domain` | string | The highest registered url domain, stripped of the subdomain.
For example, the registered domain for "foo.example.com" is "example.com".
This value can be determined precisely with a list like the public suffix list (`http://publicsuffix.org`). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". | `example.com` | Opt-In |
| `url.subdomain` | string | The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.
For example the subdomain portion of `www.east.mydomain.co.uk` is "east". If the domain has multiple levels of subdomain, such as `sub2.sub1.example.com`, the subdomain field should contain "sub2.sub1", with no trailing period. | `east` | Opt-In |
| `url.top_level_domain` | string | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".
This value can be determined precisely with a list like the public suffix list (`http://publicsuffix.org`). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as `co.uk`. | `co.uk` | Opt-In |
| `url.username` | string | Username of the request. | `user42` | Opt-In |
| `url.password` | string | Password of the request. | `changeme` | Opt-In |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should definitely add some caveat here about sensitive information :)

I forget how we decided to call this out or if that's still TBD, but I'd love a not on these fields about sensitivity.

| `url.extension` | string | The field contains the file extension from the original request url, excluding the leading dot.
The file extension is only set if it exists, as not every url has a file extension.
The leading period must not be included. For example, the value must be "png", not ".png".
Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz"). | `png` | Opt-In |
| `url.domain` | string | Domain of the url, such as `www.opentelemetry.io`.
In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the domain field.
If the URL contains a literal IPv6 address enclosed by [ and ] (IETF RFC 2732), the [ and ] characters should also be captured in the domain field. | `www.opentelemetry.io` | Opt-In |
| `url.port` | int | Port of the request | `9090` | Opt-In |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant with server.port.

| `url.original` | string | Unmodified original URL as seen in the event source.
Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path.
This field is meant to represent the URL as it was observed, complete or not. | `https://www.opentelemetry.io/search/?q=container` | Opt-In |

**[1]:** For network calls, URL usually has `scheme://host[:port][path][?query][#fragment]` format, where the fragment is not transmitted over HTTP, but if it is known, it should be included nevertheless.
`url.full` MUST NOT contain credentials passed via URL in form of `https://username:password@www.example.com/`. In such case username and password should be redacted and attribute's value should be `https://REDACTED:REDACTED@www.example.com/`.
Expand Down
91 changes: 91 additions & 0 deletions model/url.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,94 @@ groups:
type: string
brief: 'The [URI fragment](https://www.rfc-editor.org/rfc/rfc3986#section-3.5) component'
examples: ["SemConv"]
- id: registered_domain
requirement_level: opt_in
type: string
brief: >
Copy link
Member

@joaopgrassi joaopgrassi Aug 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to use brief: | here to be able to achieve multiple lines I think. Right now, the table is broken into multiple lines

image

It's also the same in the other attributes

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even with that the table was kinda broken. I moved some content into the note section and only kept the basic definitions in the brief.

The highest registered url domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix
list (`http://publicsuffix.org`). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".
examples: [ "example.com" ]
- id: subdomain
requirement_level: opt_in
type: string
brief: >
The subdomain portion of a fully qualified domain name includes all of
the names except the host name under the registered_domain. In a partially
qualified domain, or if the the qualification level of the full name cannot
be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of `www.east.mydomain.co.uk` is "east".
If the domain has multiple levels of subdomain, such as `sub2.sub1.example.com`,
the subdomain field should contain "sub2.sub1", with no trailing period.
examples: [ "east" ]
- id: top_level_domain
requirement_level: opt_in
type: string
brief: >
The effective top level domain (eTLD), also known as the domain suffix,
is the last part of the domain name. For example, the top level domain
for example.com is "com".

This value can be determined precisely with a list like the public suffix list
(`http://publicsuffix.org`). Trying to approximate this by simply taking the last
label will not work well for effective TLDs such as `co.uk`.
examples: [ "co.uk" ]
- id: username
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add username and password? @trask linked to them being deprecated? https://www.ietf.org/rfc/rfc3986.txt

Copy link
Member Author

@ChrsMark ChrsMark Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I was not aware of those being deprecated.
@AlexanderWert @trisch-me do you you know more about this and if those would be deprecated from ECS?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instrumentations also cover older http clients which would still contain username and password.
So, I think username and password could still be useful as opt-in attributes.

requirement_level: opt_in
type: string
brief: Username of the request.
examples: [ "user42" ]
- id: password
requirement_level: opt_in
type: string
brief: Password of the request.
examples: [ "changeme" ]
- id: extension
requirement_level: opt_in
type: string
brief: >
The field contains the file extension from the original request url,
excluding the leading dot.

The file extension is only set if it exists, as not every url has
a file extension.

The leading period must not be included. For example, the value must
be "png", not ".png".

Note that when the file name has multiple extensions (example.tar.gz),
only the last one should be captured ("gz", not "tar.gz").
examples: [ "png" ]
- id: domain
requirement_level: opt_in
type: string
brief: >
Domain of the url, such as `www.opentelemetry.io`.

In some cases a URL may refer to an IP and/or port directly,
without a domain name. In this case, the IP address would go to the domain field.

If the URL contains a literal IPv6 address enclosed by [ and ] (IETF RFC 2732),
the [ and ] characters should also be captured in the domain field.
examples: [ "www.opentelemetry.io" ]
- id: port
requirement_level: opt_in
type: int
brief: Port of the request
examples: [ 9090 ]
- id: original
requirement_level: opt_in
type: string
brief: >
Unmodified original URL as seen in the event source.

Note that in network monitoring, the observed URL may be
a full URL, whereas in access logs, the URL is often just represented as a path.

This field is meant to represent the URL as it was observed, complete or not.
examples: [ "https://www.opentelemetry.io/search/?q=container" ]