Needs clarification: interoperability of extensions

[alanconway]

This is not a spec bug,downgrading to a doc issue. See the explanation at #508 (comment)

Original text:

The current definition of extension attributes makes interoperability essentially impossible:

"This specification places no restriction on the type or semantics of the extension attributes. Each definition of an extensions SHOULD fully define all aspects of the attribute - e.g. its name, semantic meaning and possible values or even to indicate that it places no restrictions on its values."

There is no way for a general purpose cloudevents library to represent such an extension unless it has been customized to be aware of it. There is no way for a transport binding implementation to decode or encode such an extensions unless it has been so customized. There is no way for a generic event bridge to even pass-through such extension values unless it is aware of them. I'm trying to fix cloudevents/sdk-go and the cloudevents AMQP transport to deal with extensions and AFAIK it's impossible except for int, bool and string valued extensions.

I suggest that extension attributes should be constrained to use the same type system as normal context attributes.

Even that is problematic for the JSON binding, as there's no way to distinguish time or binary values from strings. For JSON I suggest those extensions be encoded as a JSON object with a type tag:
{
"anInt": 1,
"aString": "foo",
"aTime": { "time": "1969-03-21T0:00:00Z"},
"aBinary": { "binary": "base64encodedstuff" }
}

I think that would be sufficient to round-trip extension value safely thru any binding that supports the cloudevents type system.

There is also the issue of URI and URI-reference, I think it's OK to encode those as strings in JSON since they are so closely allied to strings, and any endpoint that knows what the extension means will be capable of using a string. It's not 100% faithful to the CE type system but probably workable.
.

[alanconway]

A simpler alternative would be to restrict extensions to int, bool and string. In that case I would be done already :)

[alanconway]

Another alternative would be to restrict extensions to always be binary, and leave the encoding and decoding to the endpoints who have knowledge of the extension. Extension authors could use a well-known encoding like JSON, protobuf or whatever they fancy. Such extensions would not be mapped to the underlying type-system of a binding like AMQP, or the language type system of a library like sdk-go, but that's OK since the binding/library will never need to interpret them.

[clemensv]

Extensions are meant to live within the type system but without any further restrictions. Extensions are just attributes, and attributes must be passed onwards by intermediaries verbatim if they don't understand them. In some cases, that may require some level of type transcoding going from a rich type system like AMQP to a lesser type system like HTTP headers. However, that transcoding is not lossy because we have a set of rules for how the canonical string representation looks like. If you understand an extension at an intermediary or endpoint (which is the only case that matters) and you expect an integer but you are getting a string, the rules provide for that string to contain a properly encoded integer.

If you believe that we should clarify the fact that the type system applies, that would seem like an easy PR to add.

[duglin]

@alanconway see https://github.com/cloudevents/spec/pull/505/files#diff-958e7270f96f5407d7d980f500805b1bR409
does that help?

[duglin]

To your points though.... IMO while PR 505 restricts things to the type system, as a receiver of an unknown extension I would just capture it as binary and pass it along to the app in that way. Then even if we did allow for things to be outside of our type system it would still work. I don't see why an unknown extension would be treated any differently than an http body with an unknown content-type - it's just passed along as binary. I think this falls into the category of "doing less for the user is more".

[alanconway]

PR #505 helps but does not solve my core practical problem:

Given a JSON event with an unknown extension, which is a JSON string, do I map that to AMQP string, binary or time extension? To preserve the CE type system faithfully I should really use AMQP described strings for URI and URI-ref - i.e. encode them as strings but with a tag indicating they should be treated as URI/URI-ref.

I guess the core problem is that the preferred event encoding (JSON) does not actually implement the CE type system faithfully for extensions. I can't "capture it as binary" as there's no common binary representation. I need type hints for extensions in the JSON format. E.g.:

 { "myTimeExt": { "time": "rfctime"}, "myBin":{"binary": "base64"}, "myURI": {"uri": "..."}} etc.

Then it would be straightforward to translate to/from JSON and other formats that faithfully represent the type system in whatever way they choose. E.g. AMQP can has native support for time and binary, and has the "described type" mechanism for string-like types such as URI.

I'm trying to make the existing JSON-focussed sdk-go work for multi-protocol, multi-format applications. The current sdk relies heavily on JSON - it basically assumes json.RawMessage is acceptable as a "universal binary format" which does not work for pretty much anything except JSON.

[clemensv]

If you get an unknown attribute and that is a string, you forward it as a string. No special processing.

A consumer who is aware of the extension and expects a time expression but finds a string, can reasonably expect for the string to contain a time expression per our canonicalization rules. In the worst case, all values degrade from a native and efficient representation into canonical strings, but we have rules for what those strings ought to contain, and the receiver has a stance on what the type ought to be because they know the extension.

[alanconway]

@clemensv OK, I can live with that. I'll leave this issue open till I have a chance to think about whether to clarify the text, but it's now a doc issue not a bug.