Introduce semantic conventions for CloudEvents

[joaopgrassi]

Resolves #1949

This PR adds semantic conventions for CloudEvents. I centralized the context of why these attributes are needed and where does the values come from in the issue above.

[tigrannajaryan]

Are these expected to be recorded as Span attributes or as attributes of an Event of a Span?

[joaopgrassi]

They are expected to be recorded as attributes on the span. The Go SDK for CloudEvents offers an interface which is called by the client when sending and receiving events. This interface (which can be implemented by users of the SDK) is responsible for starting the spans and there is where the attributes above are used.

Here is an example: https://github.com/dynatrace-oss-contrib/sdk-go/blob/1f612ea3705bdba7343ac7d428c078bdc398ff5d/observability/opentelemetry/v2/client/otel_observability_service.go#L78

[tigrannajaryan]

How is this different from service.name? If it is different we need to make it clear in the spec.

[joaopgrassi]

I also had the same question, but I'm not sure it refers to the same thing. The CloudEvents spec says:

Source: Identifies the context in which an event happened. Often this will include information such as the type of the event source, the organization publishing the event or the process that produced the event. The exact syntax and semantics behind the data encoded in the URI is defined by the event producer.

If it were only for the portion: process that produced the event then service.name would suffice I think. But because of the other details mentioned, I think it' doesn't fit.

Also, in the examples section for source they list:

- Internet-wide unique URI with a DNS authority.
    @cloudevents
    mailto:cncf-wg-serverless@lists.cncf.io
- Universally-unique URN with a UUID:
    urn:uuid:6e8bc430-9c3a-11d9-9669-0800200c9a66
- Application-specific identifiers
    /cloudevents/spec/pull/123
    /sensors/tn-1234567/alerts
    1-555-123-4567

Particularly /cloudevents/spec/pull/123 strikes me of an example of "context in which an event happened". That is clearly an endpoint of sorts that produced an event.

I was going to add such things in here, but I thought that duplicating the CloudEvents spec text here is not great, that's why I only left the link to it. But I can add something to make it clear that it is different than service.name. What do you think?

[Oberon00]

Is this the correct way to represent cloud events? With this PR we have "meta-tracing", i.e. we trace how we trace using CloudEvents. Wouldn't it make more sense to describe how to map cloud events to existing semantic conventions?

EDIT: My above question is based on a misunderstand I think. Cloud events are actually often used to trigger workflows, not (only) to log events. So it might make sense to see them as messaging system. But the key thing would be to find a sensible destination. I don't think the current one is right.

[kenfinnigan]

It might be my naive understanding, but are CloudEvents a "messaging system" or a "method of packaging" an event/message to be sent to a messaging system?

I believe CloudEvents can be sent/received by AMQP, Kafka, likely others. What would happen in those situations?

[joaopgrassi]

It might be my naive understanding, but are CloudEvents a "messaging system" or a "method of packaging" an event/message to be sent to a messaging system?

At least from my limited understanding of it it's somewhat both. For example, with the SDK for Go, you can also send/receive events using the "client" the CloudEvents SDK provides: https://github.com/cloudevents/sdk-go/blob/main/v2/client/client.go, using the protocols they offer. These attributes here then gets added to the spans created from this client.

[pyohannes]

CloudEvents is not a messaging system, but it's a specification for describing event data. You can use CloudEvents with Kafka, but I'd assume one still wants messaging.system set to Kafka in that case.

@joaopgrassi Can you make it to one of our messaging semantic conventions WGs Thursday 8am PST? This would be an appropriate forum to discuss the relation between messaging semantic conventions and CloudEvents.

[joaopgrassi]

Hi @pyohannes, sure I will join the one today.

Yes, you are right on the statement above. My limited contact with cloudevents was with the Go SDK and it seems it differs greatly from the other languages. From their SDK spec:

An SDK is not required to implement a wrapper around the transport, the focus should be around allowing programming models to work with the high level Event object, and providing tools to take the Event and turn it into something that can be used with the implementation transport selected.

I took a look at the other SDKs (.NET, Java, Python) and they seem to "only" provide ways of "converting" a CloudEvent to the appropriate type to be sent via the chosen transport. In that sense, I agree that these semantic conventions maybe don't make much sense.

But: the Go SDK is different, in the sense that it does offer a wrapper around the transports. Consider this sample for Kafka for example.

The client.go is used to send the events, and if we look internally, it offers a point to add "observability". That's where the attributes in this PR are used.

But thinking more: Even if sending events with Kafka directly and not via a wrapper provided by the CloudEvents SDK (like in the example you mentioned), I can see it could be beneficial to have things like the event_type or spec_version in the span that was either generated manually by the user OR by the instrumentation library (though that might not be possible to find out if it's a cloudevent or not..).

[Oberon00]

I wonder if we should just specify these attributes completely independent of messaging. For example, you could also send a CloudEvent via HTTP or RPC or insert it into/read from a DB. I think decoupling would make things simpler.

For sending the cloud event in particular, the attributes could sensibly be applied both to (a parent span of) the sending itself and a span for the actual operation the event is about. E.g. if I have an event about a document being changed, I could add the cloud event attributes to the span that describes the operation of changing the document. I I then HTTP POST the span, I could do that as a plain HTTP child span of that operation.

[joaopgrassi]

E.g. if I have an event about a document being changed, I could add the cloud event attributes to the span that describes the operation of changing the document.

Agreed. The more I think the more I tend to agree that maybe decoupling these attributes from "messaging" makes more sense.

It's a very general thing and multiple ways to "send an event". I think I was naive to believe the other SDKs would behave like the one in Go and didn't have the big picture in mind. Having said that, I wonder where would these attributes fit? 🤔

[Oberon00]

Having said that, I wonder where would these attributes fit?

You mean in the spec? I would suggest a completely new top-level document under https://github.com/open-telemetry/opentelemetry-specification/tree/main/semantic_conventions/trace. I don't think they fit https://github.com/open-telemetry/opentelemetry-specification/blob/main/semantic_conventions/trace/general.yaml, since CloudEvent spans could stand on their own.

[pyohannes]

As discussed in today's messaging WG, please submit another PR that adds CloudEvents attributes as separate semantic conventions, independent of messaging. This will allow users to use CloudEvents attributes as add-on for both messaging and HTTP scenarios.

[pyohannes]

@joaopgrassi I think you can close this PR, as #1978 replaces it.