Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for skipping subtree revisions to increase read performance and reduce disk usage #3201

Merged
merged 5 commits into from
Dec 12, 2023

Commits on Dec 11, 2023

  1. A really dirty rip-out of the subtree revisions

    This confirms that removing this will actually work in practice.
    This commit is a breaking change and can't be merged as-is. The
    lesson to be learned is that this is the minimum change that would
    be required to make this work if there were no existing clients
    using Trillian.
    mhutchinson committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    8ca7117 View commit details
    Browse the repository at this point in the history
  2. Support both revisioned and unrevisioned subtrees

    The same schema is used for both revisioned and unrevisioned subtrees. The difference is that we always write a revision of 0 in the unrevisioned case, which still means that there will only be a single entry per subtree.
    
    The old PublicKey column has been used to store a settings object for newly created trees.
    If this settings object is found in this column, then it will be parsed and checked for a property indicating that this tree is revisionless.
    If the property is successfully confirmed to be revisionless then all writes to the subtree table will have a revision of 0, and all reads will skip the nested inner query that was causing slow queries.
    If the property cannot be confirmed to be revisionless (no settings persisted, or settings persisted but explictly say to use revisioned), then the functionality will continue in the old way.
    This preserves backwards compatibility, but makes it so that new trees will gain these features.
    
    For users with legacy trees that wish to take advantage of the smaller storage costs and faster queries of the new revisionless storage, the proposed migration mechanism is to use migrillian to clone the old tree to a new tree. If anyone is interested in doing this then I recommend speaking to us on Slack (https://join.slack.com/t/transparency-dev/shared_invite/zt-27pkqo21d-okUFhur7YZ0rFoJVIOPznQ).
    mhutchinson committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    13d5d31 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    8027a3d View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Updated CHANGELOG

    It's a big one
    mhutchinson committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    16ac323 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1ca893f View commit details
    Browse the repository at this point in the history