Refactoring buf reader (#593)

* WIP: Add optimsitic sync * WIP: implement chain state module * WIP: Add attester incentives * Can compile * Does compile * Start working on attester incentives * Implement attestation processing * Finish unbonding * Improve events * Rename sov-attester-incentives to optimistic-workflow * Refactor * Start working on apply_sync_data_blob * Refactoring apply_tx_blob * Refactoring files * Rewriting unbonding * Refactor attestation processing * Refactor process challenge * Refactoring * Fixing compile issues * Does compile * Everything compiles * Add 2 phase unbonding back * Updating tests * Changing bonding proof * Changing bonding proof * Fixing get_with_proof * Fixing chain updates * Refactoring buf reader * Change STF interface to work on slot level * Removing Misbehavour hint from parameters * Fixing prover * Update docs * Adding chain state * Starting chain state * Fix lint * Fixing warnings * Integrating slot hooks * Populating todos * Adding data generation for different modules * Implementing automatic bank data generation * Fixing stf * Compiles * Fixing chain-state test, adding EncodeCall trait * Fixing lints * Chain state testing * Simple chain state module tests * All tests pass * Fixing lints * Fixing comments PR * Fixing comments PR * nit fix * Fixing PR comments * Deleting borsh compat * Adding back github and config files * Specifying commit for jmt * Update module-system/sov-state/src/storage.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Fixing comments PR * Applying PR comments * Update adapters/risc0/src/host.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Update adapters/risc0/src/host.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Fixing error handling * Fixing lints * Fixing lints * Fix lints * Refactor buf reader * Update rollup-interface/src/state_machine/mocks.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Addressing PR comments * Including PR comments * Update Cargo.toml Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Update adapters/risc0/src/guest.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Update adapters/risc0/src/host.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Update adapters/risc0/src/host.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Adding back genesis and call * Fix lints * Lint * Restoring default context * Fixing imports * Fixing prover compiles * Fix prover compiles * Fixing end slot hook * Reorganizing integration tests * Reorganizing integration tests * Adding unit tests for chain-state * Update chain state tests * Update apply slot signature * Fix demo-prover * Update adapters/risc0/src/guest.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Update adapters/risc0/src/host.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Fix lints * Format demo-prover * Fix feature data generators * Merge with chain-state and fix lints * Update examples/demo-stf/Cargo.toml Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Update adapters/risc0/src/guest.rs Co-authored-by: Nikolai Golub <nikolai@sovlabs.io> * Fixing pr comments * Fix lint * Fix lints --------- Co-authored-by: Preston Evans <preston.b.evans@gmail.com> Co-authored-by: Nikolai Golub <nikolai@sovlabs.io>
Sovereign-Labs · Sep 14, 2023 · a8f5d6c · a8f5d6c
1 parent 0487050
commit a8f5d6c
Show file tree

Hide file tree

Showing 3 changed files with 46 additions and 9 deletions.
diff --git a/adapters/celestia/src/types.rs b/adapters/celestia/src/types.rs
@@ -132,6 +132,7 @@ pub enum ValidationError {
     MissingTx,
     InvalidRowProof,
     InvalidSigner,
+    IncompleteData,
 }
 
 impl CelestiaHeader {

diff --git a/adapters/celestia/src/verifier/mod.rs b/adapters/celestia/src/verifier/mod.rs
@@ -248,9 +248,17 @@ impl da::DaVerifier for CelestiaVerifier {
                 let mut blob_iter = blob_ref.data();
                 let mut blob_data = vec![0; blob_iter.remaining()];
                 blob_iter.copy_to_slice(blob_data.as_mut_slice());
-                let tx_data = tx.data().acc();
-
-                assert_eq!(blob_data, *tx_data);
+                let tx_data = tx.data().accumulator();
+
+                match tx_data {
+                    da::Accumulator::Completed(tx_data) => {
+                        assert_eq!(blob_data, *tx_data);
+                    }
+                    // For now we bail and return, maybe want to change that behaviour in the future
+                    da::Accumulator::InProgress(_) => {
+                        return Err(ValidationError::IncompleteData);
+                    }
+                }
 
                 // Link blob commitment to e-tx commitment
                 let expected_commitment =

diff --git a/rollup-interface/src/state_machine/da.rs b/rollup-interface/src/state_machine/da.rs
@@ -68,6 +68,17 @@ pub trait DaVerifier {
     ) -> Result<<Self::Spec as DaSpec>::ValidityCondition, Self::Error>;
 }
 
+/// [`AccumulatorStatus`] is a wrapper around an accumulator vector that specifies
+/// whether a [`CountedBufReader`] has finished reading the underlying buffer.
+#[derive(BorshDeserialize, BorshSerialize, Serialize, Deserialize, Debug, Clone, PartialEq)]
+pub enum Accumulator {
+    /// The underlying buffer has been completely read and [`Vec<u8>`] contains the result
+    Completed(Vec<u8>),
+    /// The underlying buffer still contains elements to be read. [`Vec<u8>`] contains the
+    /// accumulated elements.
+    InProgress(Vec<u8>),
+}
+
 #[derive(Debug, Clone, Serialize, Deserialize, BorshDeserialize, BorshSerialize, PartialEq)]
 /// Simple structure that implements the Read trait for a buffer and  counts the number of bytes read from the beginning.
 /// Useful for the partial blob reading optimization: we know for each blob how many bytes have been read from the beginning.
@@ -83,7 +94,7 @@ pub struct CountedBufReader<B: Buf> {
 
     /// An accumulator that stores the data read from the blob buffer into a vector.
     /// Allows easy access to the data that has already been read
-    reading_acc: Vec<u8>,
+    accumulator: Accumulator,
 }
 
 impl<B: Buf> CountedBufReader<B> {
@@ -93,7 +104,7 @@ impl<B: Buf> CountedBufReader<B> {
         CountedBufReader {
             inner,
             counter: 0,
-            reading_acc: Vec::with_capacity(buf_size),
+            accumulator: Accumulator::InProgress(Vec::with_capacity(buf_size)),
         }
     }
 
@@ -103,9 +114,8 @@ impl<B: Buf> CountedBufReader<B> {
     }
 
     /// Getter: returns a reference to an accumulator of the blob data read by the rollup
-    /// TODO: Refactor <https://github.com/Sovereign-Labs/sovereign-sdk/issues/462>
-    pub fn acc(&self) -> &Vec<u8> {
-        &self.reading_acc
+    pub fn accumulator(&self) -> &Accumulator {
+        &self.accumulator
     }
 
     /// Contains the total length of the data (length already read + length remaining)
@@ -124,7 +134,25 @@ impl<B: Buf> Read for CountedBufReader<B> {
 
         let num_read = len_before_reading - self.inner.remaining();
 
-        self.reading_acc.extend_from_slice(&buf[..buf_end]);
+        let inner_acc_vec = match &mut self.accumulator {
+            Accumulator::Completed(_) => {
+                // The accumulator is completed, we return 0 as no data was read into self
+                return Ok(0);
+            }
+
+            Accumulator::InProgress(inner_vec) => inner_vec,
+        };
+
+        inner_acc_vec.extend_from_slice(&buf[..buf_end]);
+
+        match self.inner.remaining() {
+            0 => {
+                self.accumulator = Accumulator::Completed(inner_acc_vec.to_vec());
+            }
+            _ => {
+                self.accumulator = Accumulator::InProgress(inner_acc_vec.to_vec());
+            }
+        }
 
         self.counter += num_read;