-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cosmos: Harden and merge tip-isa-Batch
branch
#109
Comments
Removed milestone to reflect that, while this is unlikely to ever make it to a 2.x release (targeting Microsoft.Azure.DocumentDb V2), it's also not guaranteed to make a 3.x release as the Archival/Trimming facilities in #226 will likely yield a better overall solution. The other reason it's not a no-brainer to add back in is that the write amplification effects of having to read, write, and then have the CFPs read the redundant events are such that it requires a very specific combination of things (e.g. small total JSON length of events per stream) in order for this to be the best solution |
More reflections: Learnings and realizations from implementing Archival+Pruning actually make the case stronger for implementing this:
|
There's a
tip-isa-batch
branch, which generalizes the storage scheme to provide even greater efficiency in terms of read and write costs.This work was shelved for prioritisation purposes in order to avoid having to make projectors deal cleanly with seeing an event multiple times.
Given a projector that does some practical de-duplication of events, its pretty feasible to re-introduce this storage optimization; the performance and RU cost reductions can be inspected by using
Equinox.Tool run
and/ordotnet new eqxtestbed
against said branch.As noted in #108, this storage generalization also provides benefits wrt competing writer scenarios.
The text was updated successfully, but these errors were encountered: