Skip to content

Commit

Permalink
Reduce checkpoint file size by 5.8+GB
Browse files Browse the repository at this point in the history
Further reduction of 4.4+GB is planned for a total
reduction of 10.2+GB.  (see TODOs at bottom).

Leaf node is encoded as:
- node type (1 byte) (new in v4)
- height (2 bytes)
- max depth (2 bytes)
- reg count (8 bytes)
- hash (2 bytes + 32 bytes)
- path (2 bytes + 32 bytes)
- payload (4 bytes + n bytes)

Encoded payload size also reduced by removing prefix (version 2 bytes +
type 1 byte).

Interim node is encoded as:
- node type (1 byte) (new in v4)
- height (2 bytes)
- max depth (2 bytes)
- reg count (8 bytes)
- lchild index (8 bytes)
- rchild index (8 bytes)
- hash (2 bytes + 32 bytes)

Trie is encoded as:
- root node index (8 bytes)
- hash (2 bytes + 32 bytes)

Removed v3 leaf node fields:
- version (2 bytes)
- left child index (8 bytes)
- right child index (8 bytes)
- payload version and type (3 bytes)

Removed v3 interim node fields:
- version (2 bytes)
- path (2 bytes)
- payload (4 bytes)

Removed v3 trie field:
- version (2 bytes)

Leaf node data is reduced by 20 bytes (2+8+8+3-1).
Interim node data is reduced by 7 bytes (2+2+4-1).
Trie is reduced by 2 bytes.

TODO: remove max depth and reg count fields from both leaf node and
interim node types.
TODO: reduce hash length from 2 bytes to 1 byte for both leaf node and
interim node types.
  • Loading branch information
fxamacker committed Feb 3, 2022
1 parent 800cca1 commit 422b75b
Show file tree
Hide file tree
Showing 5 changed files with 424 additions and 159 deletions.
12 changes: 12 additions & 0 deletions ledger/common/encoding/encoding.go
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,12 @@ func EncodePayload(p *ledger.Payload) []byte {
return buffer
}

// EncodePayloadWithoutPrefix encodes a ledger payload
// without prefix (version and type).
func EncodePayloadWithoutPrefix(p *ledger.Payload) []byte {
return encodePayload(p)
}

func encodePayload(p *ledger.Payload) []byte {
buffer := make([]byte, 0)

Expand Down Expand Up @@ -370,6 +376,12 @@ func DecodePayload(encodedPayload []byte) (*ledger.Payload, error) {
return decodePayload(rest)
}

// DecodePayloadWithoutPrefix construct a payload from an encoded byte slice
// without prefix (version and type).
func DecodePayloadWithoutPrefix(encodedPayload []byte) (*ledger.Payload, error) {
return decodePayload(encodedPayload)
}

func decodePayload(inp []byte) (*ledger.Payload, error) {

// read encoded key size
Expand Down
Loading

0 comments on commit 422b75b

Please sign in to comment.