CosmosDB: Expose an API to retrieve a particular version of a given document

[liliankasem]

As part of the work we're doing to bring SDK-type bindings to out-of-proc Functions workers, we would like to handle CosmosDB trigger payloads on the worker side, rather than passing the entire payload to the worker.

Currently, Cosmos DB triggering will still require documents to be resolved on the host due to how the change feed works. When a document is added/updated, the change feed will get a snapshot of the document. Cosmos doesn't expose a way for you to get a particular version or snapshot of a document, so unless we handle the exact payload sent by the feed, we could end up retrieving a different version if we used the document ID alone to get the payload. This means that we cannot use the document ID to retrieve the triggering document on the worker side, and need to handle the payload on the host side and pass it via gRPC as we do now.

A potential solution for this would be for Cosmos to pass a snapshot ID and expose an API to retrieve a given snapshot of a document.

[ealsur]

This would require this supported as a Service feature, currently this is impossible from any client. The ASK should be first on the Service, once the Service decides to implement such a thing (if it's technically possible), then can clients expose it.

Based on current service design, it would not be possible. The database contains the latest most recent state of a document, not the intermediate changes. You can receive via Change Feed intermediate states, but the final state is what is stored. If there are any other operations between the event being received and your attempt to read the document, then you will not see the version you want, the state of the database is the final state.

sequenceDiagram
    participant D as gRPC worker
    participant A as Cosmos Trigger
    participant B as Database
    participant C as Application writer    
    C->>B: Creates Document version 1
    A->>B: Reads event with version 1
    A->>D: Notifies about document (passes id+PK)
    C->>B: Updates Document version 2
    A->>B: Reads event with version 2
    D->>B: Tries to read document for version 1, will see version 2

Loading

[liliankasem]

This would require this supported as a Service feature, currently this is impossible from any client. The ASK should be first on the Service, once the Service decides to implement such a thing (if it's technically possible), then can clients expose it.

Based on current service design, it would not be possible. The database contains the latest most recent state of a document, not the intermediate changes. You can receive via Change Feed intermediate states, but the final state is what is stored. If there are any other operations between the event being received and your attempt to read the document, then you will not see the version you want, the state of the database is the final state.

Yup totally, this was intended as a service feature request and we wanted to have somewhere related to Functions to track this rather than opening directly with CosmosDB (for now at least).

We were just trying to brainstorm ideas for if there is any way at all to make this scenario possible. However, it sounds like because of CosmosDB's service design, it's not possible to even expose such an API to for these intermediate states?

FYI: @fabiocav @mattchenderson