The OpenAPI specification for describing REST interfaces in a standardized way has already established itself in many project and application areas. While OpenAPI has become established for synchronous interfaces, asynchronous communication places new demands on the interface definition. For this reason, AsyncAPI provides an extension to the OpenAPI definition to meet these new requirements. This blog article highlights the differences between OpenAPI and AsyncAPI in more detail.
A common scenario in larger projects involves multiple teams working and developing together. Nevertheless, there should be no dependencies between the teams. A central factor for this is that the documentation of the interfaces for the microserverices and components is available to all teams in an uncomplicated manner. For this reason, it seems to be a reasonable use case to store API definitions in a registry as a central point of access. Apicurio Registry is intended to enable this for various specifications, e.g. OpenAPI, AsyncAPI, GraphQL, Apache Avro, Protobuf, JSON Schema and others.
This showcase will look at how AsyncAPI and Apicurio can be used. Nevertheless, the development of AsyncAPI and Apicurio is progressing rapidly and new features are added regularly. For this reason, this showcase is only a snapshot that may become outdated after a short period of time.
In a previous showcase, the interoperability of AMQP 0.9.1 and AMQP 1.0 was investigated. Since this can only be guaranteed to a limited extent, this project should additionally investigate whether it is possible to produce and consume messages in Avro format through a RabbitMQ broker.
Since this project was originally developed in the direction of an event mesh, the use of different brokers can be considered realistic. One possible scenario would involve a Kafka broker already being present in the system and existing applications obtaining their Avro schemas from a Confluent Schema Registry. Later, a RabbitMQ broker could be added to the system. In order to provide the AsyncAPI definitions centrally, Apicurio can be used at this point.
As already partially visible in the diagram above, the Producer and Consumer applications should fulfill the following:
-
An AsyncAPI definition is provided which references an Avro schema. The definition used in this project can be found at src/main/resources/asyncapi.json.
-
The AsyncAPI definition is registered at Apicurio. The Avro schema is registered at Confluent Schema Registry.
-
Clients can pull the specifications from Apicurio and resolve the reference to the Avro schema.
-
Generation of messages and/or validation against the schemas is possible.
Avro schemas can be registered at Schema Registry using the Schema Registry library by Confluent. A topic/subject name needs to be provided for each new schema. Alternatively, it is also possible to use the SpecificAvroSerializer for schema registration. The schema will be registered automatically when an object is serialized and the class/schema it belongs to is not already present.
In contrast to the Confluent Schema Registry, Apicurio is not limited to message schemas, but also supports API specifications as already mentioned. The Confluent Schema Registry could therefore also be completely replaced by Apicurio.
However, this is probably only true if the architecture is not mainly based on a Confluent ecosystem.
Depending on the use case, Apicurio can therefore be a useful addition to the Schema Registry due to the additionally supported formats.
Apicurio also provides a library to access and manage the artifacts located at the registry. Similar to Schema Registry, a unique ID is required for each artifact.
Apicurio puts an emphasis on OpenAPI support, which can be seen from the fact that Apicurio Studio was developed for OpenAPI. Apicurio Registry supports several formats, including AsyncAPI. Nevertheless, not all specification details of AsyncAPI are supported yet, which is also due to the fast development of AsyncAPI.
A central element of AsyncAPI are the external references, which can be used in almost every AsyncAPI element. This makes it possible to move the definition of components to another place in the document or outside the document. External remote references are also explicitly supported.
Thus, theoretically, external references to an Avro schema can also be defined, which can be used to format the payload of a message. Currently, it is possible to define external references when using Apicurio, but these are not automatically resolved by Apicurio Registry. As a result, only the link to the external schema is displayed in the registry, which can leave important information about the message payload unknown to the user.
In contrast, the AsyncAPI Playground, for example, can resolve external references and display them (depending on the format of the reference). A similar representation would be desirable for Apicurio Registry, as is also planned for the future.
The data models library by Apicurio adds functionalities for the usage of OpenAPI and AsynAPI
specifications in Java clients. It is possible to create a new specification and to read from existing documents.
If the API specification contains a schema definition (e.g., Avro), validation against this schema is not available. Schema validation then has to be handled by custom client implementations.
As mentioned above, resolving external references is not fully supported at the moment. However, the data models library offers the possibility to implement the interface IReferenceResolver in order to be able to integrate external schemas.
The default implementation can in principle already resolve external references (as documented here and here), but not at such a deep level as allowed by the AsyncAPI specification. The AsyncAPI specification allows that the payload object in the message object also represents an external reference.
However, the data models library only checks objects for external references if the implement the IReferenceNode interface. While the message object implements this interface, the payload object in the data models library is only of type "Object". The payload object is therefore not checked for external references, so they remain unresolved.
The scenario shown in the diagram in section 2 can in principle be implemented using Apicurio, Schema Registry and RabbitMQ. However, some custom implementations are required to resolve the externally referenced Avro schema. This makes the implementation more complicated than necessary.
However, this is only true if the external reference is supposed to be inserted into the AsyncAPI definition. If the API definition does not contain external references and has a simpler structure or resolution is not necessary, this problem does not even occur.
Additional information, such as the broker URL and port may be relevant for application development. These could be obtained via the AsyncAPI definition, similar to the definition of environment variables. The Data Models Library could be used for this purpose. However, the library did not seem to be suitable for this, as the AsyncAPI document must be traversed to access the required properties.
Modelina
At the time of implementing this showcase, the Modelina tool was not yet actively promoted by AsyncAPI. In the meantime, information about it can also be found on AsyncAPI’s website.
With Modelina, data models can be generated from AsyncAPI definitions, as it is already possible for Avro models. The tool could therefore have been used in this showcase. A further evaluation and tests still have to be done.
Avro Schema Parser
The Avro Schema Parser Tool allows to resolve Avro schemas in the payload of messages (as would be desirable for Apicurio and explained in Section 4.4). Unfortunately, it looks like this tool is only available for JavaScript and no other programming language.
Apicurio can currently be used for AsyncAPI definitions with a few special restrictions. Nevertheless, AsyncAPI is still in a dynamic development and is getting more and more popularity. It is expected that with widespread use of AsyncAPI, some features will be added for use with Apicurio.
For example, a rendered representation of the AsyncAPI definitions would be desirable. In this Github issue it was already discussed to support AsyncAPI in Apicurio Studio.
The Apicurio Data Models Library has not proven to be the best solution for using AsyncAPI definitions in Java clients. However, the Modelina library of AsyncAPI seems to be another possibility that should be evaluated.
At the same time, the AsyncAPI team is currently developing AsyncAPI Studio. According to the documentation, Studio will be similar to Apicurio Studio, which focuses on the use with OpenAPI.
The team of AsyncAPI has some interesting ideas on the roadmap. Some of them are:
-
Glee: An AsyncAPI framework.
-
Event Gateway: An event gateway to validate, manipulate, aggregate and filter messages.
-
CLI Tool: To validate AsyncAPI definitions. New features are planned.
-
Simulator: To simulate application scenarios. In an early stage of development.
-
Cupid: Analyzes relationships between AsyncAPI documents and outputs a map of the system architecture.