feat(trie): Implement parity trie crate

[dimartiro]

Changes

Implements parity trie crate used in substrate

This is based on substrate and trie-db crate

Goals

• Get trie values using new lookup methods
• Lazy loading trie from DB (Get nodes from DB and decode it only if necessary)

New structures

• TrieDB
• Lookup

Changes

• MemoryDB
• Partial decoding (decode only required nodes and not the full trie)

Tests

go test ./lib/trie/... ./internal/trie/...

Issues

This is part but not complete solution for issue #2418

Primary Reviewer

@timwu20

[timwu20]

shouldn't we do a length check here?

[timwu20]

Suggested change

	func NewMemoryDBFromProof(encodedNodes [][]byte) (*InMemoryDB, error) {
	func NewInMemoryDBFromProof(encodedNodes [][]byte) (*InMemoryDB, error) {

[timwu20]

let's try to avoid using anything from common. Have you considered translating what they do in the rust trie lib using go generics?

/// Trait describing an object that can hash a slice of bytes. Used to abstract
/// other types over the hashing algorithm. Defines a single `hash` method and an
/// `Out` associated type with the necessary bounds.
pub trait Hasher: Sync + Send {
	/// The output type of the `Hasher`
	type Out: AsRef<[u8]>
		+ AsMut<[u8]>
		+ Default
		+ MaybeDebug
		+ core::cmp::Ord
		+ PartialEq
		+ Eq
		+ hash::Hash
		+ Send
		+ Sync
		+ Clone
		+ Copy;
	/// What to use to build `HashMap`s with this `Hasher`.
	type StdHasher: Sync + Send + Default + hash::Hasher;
	/// The length in bytes of the `Hasher` output.
	const LENGTH: usize;

	/// Compute the hash of the provided slice of bytes returning the `Out` type of the `Hasher`.
	fn hash(x: &[u8]) -> Self::Out;
}

[timwu20]

why do we need reference counting?

[timwu20]

So the data is never updated if there are multiple callers to MemoryDB.Insert()?

[timwu20]

Would be it better to actually create the different types for all these nodes, rather than create one Node type that will store them all?

/// Type of node in the trie and essential information thereof.
#[derive(Eq, PartialEq, Clone)]
#[cfg_attr(feature = "std", derive(Debug))]
pub enum Node<'a> {
	/// Null trie node; could be an empty root or an empty branch entry.
	Empty,
	/// Leaf node; has key slice and value. Value may not be empty.
	Leaf(NibbleSlice<'a>, Value<'a>),
	/// Extension node; has key slice and node data. Data may not be null.
	Extension(NibbleSlice<'a>, NodeHandle<'a>),
	/// Branch node; has slice of child nodes (each possibly null)
	/// and an optional immediate node data.
	Branch([Option<NodeHandle<'a>>; nibble_ops::NIBBLE_LENGTH], Option<Value<'a>>),
	/// Branch node with support for a nibble (when extension nodes are not used).
	NibbledBranch(
		NibbleSlice<'a>,
		[Option<NodeHandle<'a>>; nibble_ops::NIBBLE_LENGTH],
		Option<Value<'a>>,
	),
}

[timwu20]

can you mark as draft if you're not gonna work on this @dimartiro?