-
Notifications
You must be signed in to change notification settings - Fork 3
Asynchronous Web API
Asynchronous WebAPI will allow to execute WebAPI calls asynchronously. Integrations don't have to wait for requests to get fully processed by Magento when making asynchronous calls.
In the product update context, this means that product APIs can be invoked and get an immediate response that the request was accepted. Separate status reporting API is offered to check on the progress of requests by ID. This functionality will be available in both Magento Commerce and Magento Open Source.
The role of asynchronous endpoint is to intercept the message to the Web API and write it to the Message Queue together with identifying information, such as UUID of the operation.
Asynchronous endpoints will be one per operation. Prefix async indicates that the endpoint is called in an asynchronous manner.
POST /async/V1/poducts
PUT /async/V1/poducts
DELETE /async/V1/products
The endpoint will allow to accept an array of entities. If one entity need to be passed, it will still be an array with the single entity.
Asynchronous Web API for the single entity converts all existing routes declared to in the webapi.xml to the asynchronous endpoints by adding the async suffix to them. They should still support route parameters and the rest of the properties and features of the Web API framework.
Persisting of multiple entities through the async single endpoints would require performing of multiple requests. Each request will be written to the message queue as a separate message.
Asynchronous Web API for the single entity is intended to simplify the configuration and usage of the async framework by the extension developers.
The request intercepted by the Async Web API framework should be written to the Message Queue. The way how Magento publisher interface works is it publishes messages to Rabbit MQ using topic name and the data. Topic name defines the schema of the data, so there are must be different topic for different messages. Messages are routed to the queues based on topic names.
Right now there is no correlation between Web API rote names and the topics and they should be introduced as a part of this work.
Here is an example of the route configuration from the Catalog webapi.xml:
<!-- Product Service -->
<route url="/V1/products" method="POST">
<service class="Magento\Catalog\Api\ProductRepositoryInterface" method="save"/>
<resources>
<resource ref="Magento_Catalog::products" />
</resources>
</route>
The unique identifiers of the route is combination of the url and method. When message is written to the RabbitMQ, topic names should be constructed on the fly using both the url and the method from the route declaration. In this example that would be v1.products.post
Consumers in Magento are executed one per queue. The handler attached to the queue is configured per topic. For every topic there should be separate handler processing it, because topic defines the content of the message. That's why or the topic generated from the webapi.xml we should attach the handlers which would execute Service Contracts configured for the Web API route.
v1.products.post should be mapped back to Magento\Catalog\Api\ProductRepositoryInterface::save when consumer for the queue is running. That can be done by converting topic back to the route, matching corresponding route in webapi.xml and reading the service node from there
The response of the service is constructed in the following way:
HTTP/1.1 202 Accepted
Location: /bulk/status/12345
All operations are executed in the context of the particular store.
re-use Asynchronous Operations interfaces
\Magento\AsynchronousOperations\Model\BulkManagement::scheduleBulk
Writes bulk data to the database and to the queue. Generates the UI to track the status of the Bulk operation. Interfaces should be improved if certain use cases are not possible to implement.
Swagger schema generator should be updated in order to reflect the asynchronous response of the async endpoint
Should be implemented in separate epic status reporting API
Request
GET /bulk/status/12345
Response
"operation": "PUT /async/V1/products"
"id": "12314",
"entities":
[
{
"id": "1213",
"status": "accepted"
},
{
"id": "1213",
"status": "in_progress"
},
{
"id": "1213",
"status": "processed"
}
]
Q: Should async bulk request be POST only, or PUT and DELETE also needed? A: PUT, DELETE should be supported. GET is not supported because should not be used to trigger lo-running backend processes.
Q: Should we support different operations as a part of one API request? A: one API request should be tight to one entity (i.e. Product) and one HTTP method (PUT, POST, etc). It will allow to support strict Sagger schema.
Q: Can we have one endpoint for all requests? A: No, based on the answer above
Q: Should the request support Swagger schema? A: yes
Q: Should more than one entity come in the request? A: Ideally we should support multiple entities sent as a part of API request. There might be concerns that with many Product entities sent to create request the payload size would be huge. However, for the patch update requests the payload can be reasonably small, while minimize the amount of requests to the system.
Q: Can we use AMQP instead? A: We can and it works reasonable well. However, the Async Web API will allow to support Web API Authorization, instead of custom RabbitMQ authorizer; Swagger schema; Clients would be built on the same protocol as the other REST apis.
http://restcookbook.com/Resources/asynchroneous-operations/
- User can call product bulk update web API asynchronously
- Code Generation for Async responses should be possible via typed Swagger schema
- Bulk API should be performant under load, better mechanism for the Bulk API tracking might be introouced