-
-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Round of log-resetting changes #18
Conversation
025de56
to
49a1c52
Compare
For witness ecosystem compatibility.
49a1c52
to
7cb64d2
Compare
4143875
to
5df3bfe
Compare
Based on a design by @jellevandenhooff at jellevandenhooff#1. Fixes #11 Co-authored-by: Jelle van den Hooff <jelle@vandenhooff.name>
5df3bfe
to
1ed5201
Compare
@@ -376,6 +387,7 @@ type UploadOptions struct { | |||
|
|||
var optsHashTile = &UploadOptions{Immutable: true} | |||
var optsDataTile = &UploadOptions{Compress: true, Immutable: true} | |||
var optsStaging = &UploadOptions{Compress: true, Immutable: true} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that in a very rare case it would be possible to get a staging bundle with the same hash and N but with different contents: if at height N certificates A and B get staged and the commit fails, then A gets staged and committed, and then B, the same final hash for N+2 would have two different bundles, one with A and B and one with just B.
Using a nonce for the staging bundle file name would prevent that from happening, but there is no place to store a nonce in the checkpoint.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh that's a good observation. One solution is to use the hash of the whole checkpoint that includes a timestamp, but I actually think we can show that this is fine regardless: we are worried about two staging bundles for N->M and N+k->M conflicting; the only case in which this can happen is if both N and N+k are successfully fully sequenced, because we never make progress without a full sequencing. The good news is that I think both the tiles in N->M and the tiles in N+k->M are valid to make a M tree whole from N+k, so it doesn't matter which one "wins".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that it should not matter which write wins, and I wish the correctness argument was simpler but that doesn’t matter much.
Should the file type be marked immutable? If immutable means the file can safely be cached, yes, if it means the file will always be bit-for-bit identical (and could fail writes if not), no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried writing a test for this in 4df346d but I actually think it's impossible: even if N -> N+2 and N -> N+1 manage to have the same timestamp (which is part of the leaf and so part of the tree hash and so part of the staging path) then N+1 -> N+2 can't have the same timestamp because time is required to move forward at each sequencing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to mark staging uploads not immutable though, because there are differences in the serialized values that I have actually not yet figured out.
Using sqlite to save checkpoints failed at runtime because: * Internal Mutex was a nil pointer * The help messages explaining how to create the sqlite3 database had the wrong CREATE statement * The implementation of the `LockBackend` interface had the `Fetch` function return a wrong object (a `dynamoCheckpoint` instead of a `sqliteCheckpoint`) Closes #19 Signed-off-by: Flavio Castelli <fcastelli@suse.com>
b48d04d
to
b1edbf5
Compare
Based on feedback in #18.
b1edbf5
to
e8ab779
Compare
No description provided.