Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test: re-enable "full blob" test #1410

Closed
roynalnaruto opened this issue Aug 28, 2024 · 1 comment
Closed

Test: re-enable "full blob" test #1410

roynalnaruto opened this issue Aug 28, 2024 · 1 comment
Assignees

Comments

@roynalnaruto
Copy link

Describe the feature you would like

Background

As per the changes made in #1394 the structure of the "blob" has been modified such that the first byte in the blob indicates whether or not the following bytes represent the batch in encoded or raw form. Essentially, if blob[0] == 1 then blob[1:] are the zstd-encoded form of the batch bytes, while if blob[0] == 0 then blob[1:] are the raw batch bytes.

This naturally means that batch274 that previously used to produce an exactly full blob post-encoding will now be a single byte more than a full blob.

As a result, the test for the "full blob", i.e. len(blob_bytes) == 4096*31 was disabled. Refer:

  • // TODO: enable this once we have another deterministic case of batch -> blob (fully packed).
    // Full blob test case
    // batch274 contains batch bytes that will produce a full blob
    // let full_blob = hex::decode(
    // fs::read_to_string("./data/test_batches/batch274.hex")
    // .expect("file path exists")
    // .trim(),
    // )
    // .expect("should load full blob batch bytes");
    // batch274 contains metadata
    // let segmented_full_blob_src = BatchData::<MAX_AGG_SNARKS>::segment_with_metadata(full_blob);
  • // segmented_full_blob_src,
  • // TODO: enable this once we have another deterministic case of batch -> blob (fully
    // packed).
    // First blob is purposely constructed to take full blob space
    // if idx == 0 {
    // let blob_data_bytes_len = batch_data.get_blob_data_bytes().len();
    // assert_eq!(
    // blob_data_bytes_len, N_BLOB_BYTES,
    // "should be full blob: expected={N_BLOB_BYTES}, got={blob_data_bytes_len}",
    // );
    // }

What to do

We need to use a batch that basically encodes to just 1 byte less than "full blob" so that after prepending the first byte (enable_encoding = 1) we arrive at a "full blob".

We could modify batch274 slightly such that in its encoded form, we have 4096*31 - 1 bytes. It is not trivial because as soon as we change any single byte, the encoder may perform much better or much poorer, resulting in either less or more bytes than required.

After we have found a batch that satisfies "full blob" criteria, we should enable the commented out code.

Additional context

No response

@DreamWuGit
Copy link
Member

closed as PR privacy-scaling-explorations#1432 merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants