Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure scanner database is accessed using the correct types #8112

Merged
merged 10 commits into from
Dec 18, 2023

Conversation

teor2345
Copy link
Collaborator

@teor2345 teor2345 commented Dec 15, 2023

Motivation

During the scanner MVP, it was easy to change the column family's types, but forget to update the code everywhere. There were at least two bugs like this, from two different developers.

Instead, we can make each column family strongly typed, so it can only be accessed using the correct types.

Preparation for #7833, let's document the easier way this PR implements.

PR Author Checklist

Check before marking the PR as ready for review:

  • Will the PR name make sense to users?
  • Does the PR have a priority label?
  • Have you added or updated tests?
  • Is the documentation up to date?

If a checkbox isn't relevant to the PR, mark it as done.

API & Rust Reference

https://docs.rs/rocksdb/latest/rocksdb/struct.WriteBatchWithTransaction.html

https://doc.rust-lang.org/book/ch10-03-lifetime-syntax.html
https://doc.rust-lang.org/std/marker/struct.PhantomData.html

Complex Code or Requirements

Solution

Production:

  • Define types for reading and writing sapling transaction ID results
  • Simplify implementations using typed column families
  • Make sure each method writes its own batch
  • Implement common methods as a trivial trait

Docs & debugging:

  • impl Debug and PartialEq on some database types to improve debugging
  • Update docs & parameter names
  • Deprecate some traits that we shouldn't use for new column families

Unrelated cleanups:

  • impl TryFrom<zcash_primitives::BlockHeight> for Height
  • Simplify logging macros in the scanner

We can't make the column family name string constant part of the type, because Rust doesn't support String const generics yet.

Testing

This PR is well covered by existing tests, which pass.

Review

This is a routine cleanup/internal enhancement.

Reviewer Checklist

Check before approving the PR:

  • Does the PR scope match the ticket?
  • Are there enough tests to make sure it works? Do the tests cover the PR motivation?
  • Are all the PR blockers dealt with?
    PR blockers can be dealt with in new tickets or PRs.

And check the PR Author checklist is complete.

Follow Up Work

Use the new TypedColumnFamily and WriteTypedBatch for existing state column families. We might discover some bugs this way.

Document how to set up a new column family using these types - #7833

@teor2345 teor2345 added C-cleanup Category: This is a cleanup P-Medium ⚡ A-state Area: State / database changes A-blockchain-scanner Area: Blockchain scanner of shielded transactions labels Dec 15, 2023
@teor2345 teor2345 self-assigned this Dec 15, 2023
@teor2345 teor2345 requested a review from a team as a code owner December 15, 2023 06:15
@teor2345 teor2345 requested review from upbqdn and removed request for a team December 15, 2023 06:15
@github-actions github-actions bot added the C-trivial Category: A trivial change that is not worth mentioning in the CHANGELOG label Dec 15, 2023
@teor2345 teor2345 removed the C-trivial Category: A trivial change that is not worth mentioning in the CHANGELOG label Dec 16, 2023
@github-actions github-actions bot added the C-trivial Category: A trivial change that is not worth mentioning in the CHANGELOG label Dec 17, 2023
Copy link
Member

@upbqdn upbqdn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK.

Looks great!

@mergify mergify bot merged commit 4b5838c into main Dec 18, 2023
102 checks passed
@mergify mergify bot deleted the scan-cleanups branch December 18, 2023 16:33
@oxarbitrage oxarbitrage mentioned this pull request Jan 17, 2024
41 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-blockchain-scanner Area: Blockchain scanner of shielded transactions A-state Area: State / database changes C-cleanup Category: This is a cleanup C-trivial Category: A trivial change that is not worth mentioning in the CHANGELOG
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants