Skip to content
This repository has been archived by the owner on Aug 21, 2024. It is now read-only.

refactor(native_blockifier): flat fields in bouncerinfo #1394

Conversation

ayeletstarkware
Copy link
Contributor

@ayeletstarkware ayeletstarkware commented Jan 29, 2024

This change is Reviewable

@codecov-commenter
Copy link

codecov-commenter commented Jan 29, 2024

Codecov Report

Attention: 53 lines in your changes are missing coverage. Please review.

Comparison is base (29d2d5f) 70.37% compared to head (26cfc50) 70.38%.

Files Patch % Lines
...ve_blockifier/src/py_transaction_execution_info.rs 0.00% 45 Missing ⚠️
...ates/native_blockifier/src/transaction_executor.rs 0.00% 8 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##           main-v0.13.1    #1394   +/-   ##
=============================================
  Coverage         70.37%   70.38%           
=============================================
  Files                60       60           
  Lines              7773     7772    -1     
  Branches           7773     7772    -1     
=============================================
  Hits               5470     5470           
+ Misses             1874     1873    -1     
  Partials            429      429           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ArniStarkware
Copy link
Contributor

Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 4 files at r1.
Reviewable status: 3 of 4 files reviewed, 4 unresolved discussions (waiting on @ayeletstarkware)


crates/blockifier/src/utils.rs line 38 at r1 (raw file):

    }
    result
}

This is the only time this is used. Cool.
Will we not want to use this util ever again?

Code quote:

pub fn merge_hashmaps<K, V>(x: &HashMap<K, V>, y: &HashMap<K, V>) -> HashMap<K, V>
where
    K: Hash + Eq + Clone,
    V: Add<Output = V> + AddAssign + Default + Clone,
{
    let mut result = x.clone();
    for (key, value) in y {
        result
            .entry(key.clone())
            .and_modify(|v| v.add_assign(value.clone()))
            .or_insert(value.clone());
    }
    result
}

crates/native_blockifier/src/py_transaction_execution_info.rs line 219 at r1 (raw file):

}

type ResourcesMap = HashMap<String, usize>;

Is this type only used in this context (like this)?
Probably not. Use this type throughout the code, (like other instances of actual_resources). Add a todo to do it another time (please link here the number of the corresponding PR) - or delete this type.

Code quote:

type ResourcesMap = HashMap<String, usize>

crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r1 (raw file):

impl PyBouncerInfo {
    pub fn calculate(
  1. Is this function never being called? Why does it exist?

  2. The pyclasses should be very simple - they should not implement any logic. On the other hand - this is just a constructor - which is exactly the kind of logic that do makes sense.

  3. Is the name calculate correct here? are there any computations happening other than some additions? (Maybe additions are enough of an operator to be considered calculations...)

Code quote:

 pub fn calculate(

Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 4 files reviewed, 3 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r1 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…
  1. Is this function never being called? Why does it exist?

  2. The pyclasses should be very simple - they should not implement any logic. On the other hand - this is just a constructor - which is exactly the kind of logic that do makes sense.

  3. Is the name calculate correct here? are there any computations happening other than some additions? (Maybe additions are enough of an operator to be considered calculations...)

Discussed F2F. Done.

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from c5f4376 to 72e8c07 Compare January 29, 2024 13:07
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 4 files reviewed, 3 unresolved discussions (waiting on @ArniStarkware)


crates/blockifier/src/utils.rs line 38 at r1 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

This is the only time this is used. Cool.
Will we not want to use this util ever again?

I added this function in the last PR (for calculate_tx_weights).


crates/native_blockifier/src/py_transaction_execution_info.rs line 219 at r1 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

Is this type only used in this context (like this)?
Probably not. Use this type throughout the code, (like other instances of actual_resources). Add a todo to do it another time (please link here the number of the corresponding PR) - or delete this type.

Done.

@ArniStarkware
Copy link
Contributor

crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r2 (raw file):

impl PyBouncerInfo {
    pub fn calculate(

The pyclasses should be very simple - they should not implement any logic. On the other hand - this is just a constructor - which is precisely the kind of logic that do makes sense.
On the other other hand - there is some computation here - So maybe it should be a util.

@giladchase WDYT?

Code quote:

impl PyBouncerInfo {
    pub fn calculate(

@ArniStarkware
Copy link
Contributor

crates/native_blockifier/src/transaction_executor.rs line 116 at r2 (raw file):

                let actual_resources = blockifier::transaction::objects::ResourcesMapping(
                    tx_execution_info.actual_resources.0.clone(),
                );

Can you use the type?

Suggestion:

                // In imports section.
                use blockifier::transaction::objects::ResourcesMapping;
                // Here
                let ResourcesMapping(
                    tx_execution_info.actual_resources.0.clone(),
                );

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 72e8c07 to 174cdca Compare January 29, 2024 13:58
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 2 of 4 files reviewed, 3 unresolved discussions (waiting on @ArniStarkware)


crates/native_blockifier/src/transaction_executor.rs line 116 at r2 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

Can you use the type?

Done.

Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r3.
Reviewable status: 3 of 4 files reviewed, 2 unresolved discussions (waiting on @ayeletstarkware)

Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: 3 of 4 files reviewed, 2 unresolved discussions (waiting on @ayeletstarkware)

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: 3 of 4 files reviewed, 3 unresolved discussions (waiting on @ArniStarkware and @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 224 at r3 (raw file):

#[derive(Clone, Default)]
// TODO(Ayelet, 24/01/2024): Consider remove message_segment_length, state_diff_size.
pub struct PyBouncerInfo {

See my comment in the Python side, I think we should work with VmExecutionResources.

@ArniStarkware
Copy link
Contributor

crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r2 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

The pyclasses should be very simple - they should not implement any logic. On the other hand - this is just a constructor - which is precisely the kind of logic that do makes sense.
On the other other hand - there is some computation here - So maybe it should be a util.

@giladchase WDYT?

@elintul ?

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 174cdca to 60d221d Compare January 30, 2024 10:23
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 4 files reviewed, 3 unresolved discussions (waiting on @ArniStarkware and @elintul)


crates/native_blockifier/src/py_transaction_execution_info.rs line 224 at r3 (raw file):

Previously, elintul (Elin) wrote…

See my comment in the Python side, I think we should work with VmExecutionResources.

Done.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r4, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @ayeletstarkware and @giladchase)


crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r2 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

@elintul ?

Did you try to read what's the best practice?


crates/native_blockifier/src/py_transaction_execution_info.rs line 229 at r4 (raw file):

    pub gas_weight: usize,
    #[pyo3(get)]
    pub range_check_builtin: usize,

Should be removed, exists in PyVmExecutionResources.

Code quote:

pub range_check_builtin: usize,

crates/native_blockifier/src/py_transaction_execution_info.rs line 243 at r4 (raw file):

        state_diff_size: usize,
    ) -> TransactionExecutionResult<Self> {
        let gas_weight = actual_resources.0.get("l1_gas_usage").ok_or_else(|| {

Can you build a VmExecutionResources out of ResourcesMapping and add them?

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @ayeletstarkware and @giladchase)


crates/native_blockifier/src/py_transaction_execution_info.rs line 243 at r4 (raw file):

Previously, elintul (Elin) wrote…

Can you build a VmExecutionResources out of ResourcesMapping and add them?

Maybe we can do it outside, like we do today (when building additional_os_resources).

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r3.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @ayeletstarkware and @giladchase)

Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r2 (raw file):

Previously, elintul (Elin) wrote…

Did you try to read what's the best practice?

a. No. Doing it right now.
b. If you didn't have a strong feeling towards that - it means we are not against having this as a logic here. Unblocking this discussion.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r2 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

a. No. Doing it right now.
b. If you didn't have a strong feeling towards that - it means we are not against having this as a logic here. Unblocking this discussion.

I'll have an opinion once I understand best practices. :)
We can also move this offline and consult with Gilad.

@ArniStarkware
Copy link
Contributor

crates/native_blockifier/src/py_transaction_execution_info.rs line 242 at r2 (raw file):

Previously, elintul (Elin) wrote…

I'll have an opinion once I understand best practices. :)
We can also move this offline and consult with Gilad.

It is fine to create impls for Rust-only methods.

Straight from documentation:
https://pyo3.rs/v0.18.1/class.html?highlight=pycell#:~:text=Since%20Rust%20allows%20any%20number%20of%20impl%20blocks%2C%20you%20can%20easily%20split%20methods%20between%20those%20accessible%20to%20Python%20(and%20Rust)%20and%20those%20accessible%20only%20to%20Rust.

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 60d221d to 96ba551 Compare January 30, 2024 14:54
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 5 files reviewed, 3 unresolved discussions (waiting on @ArniStarkware and @elintul)


crates/native_blockifier/src/py_transaction_execution_info.rs line 229 at r4 (raw file):

Previously, elintul (Elin) wrote…

Should be removed, exists in PyVmExecutionResources.

Done.


crates/native_blockifier/src/py_transaction_execution_info.rs line 243 at r4 (raw file):

Previously, elintul (Elin) wrote…

Maybe we can do it outside, like we do today (when building additional_os_resources).

Done.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r5, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 267 at r5 (raw file):

            builtin_instance_counter,
            n_memory_holes: merged_resources.n_memory_holes,
        };

Let's pass actual_resources inside, pop the L1 usage and create VmExecutionResources from the rest. Then + between the two resources (the one we created and OS ones) and use PyVmExecutionResources::from.

Code quote:

        let execution_resources = PyVmExecutionResources {
            n_steps,
            builtin_instance_counter,
            n_memory_holes: merged_resources.n_memory_holes,
        };

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 96ba551 to 8768e54 Compare January 31, 2024 09:25
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 5 files reviewed, 2 unresolved discussions (waiting on @elintul)


crates/native_blockifier/src/py_transaction_execution_info.rs line 267 at r5 (raw file):

Previously, elintul (Elin) wrote…

Let's pass actual_resources inside, pop the L1 usage and create VmExecutionResources from the rest. Then + between the two resources (the one we created and OS ones) and use PyVmExecutionResources::from.

Done.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r6, all commit messages.
Reviewable status: all files reviewed, 7 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 239 at r6 (raw file):

    pub fn calculate(
        additional_os_resources: VmExecutionResources,
        actual_resources: ResourcesMapping,

Suggestion:

actual_resources: &ResourcesMapping,

crates/native_blockifier/src/py_transaction_execution_info.rs line 247 at r6 (raw file):

                field: "l1_gas_usage".to_string(),
            }
        })?;

expect + informative error message, also lose the variable.

Code quote:

        let l1_gas_amount = actual_resources.0.get("l1_gas_usage").ok_or_else(|| {
            TransactionExecutionError::InvalidTransactionExecutionInfo {
                field: "l1_gas_usage".to_string(),
            }
        })?;

crates/native_blockifier/src/py_transaction_execution_info.rs line 249 at r6 (raw file):

        })?;

        let actual_resources = execution_resources_from_hashmap(&actual_resources.0);

Suggestion:

        let actual_resources = VmExecutionResources {
            n_steps: actual_resources.0.get(constants::N_STEPS_RESOURCE).unwrap_or_default(),
            builtin_instance_counter: HashMap::new(
                (name, actual_resources.get(name).unwrap_of_default()) for name in BUILTIN_NAMES
            ),
            n_memory_holes: actual_resources.get("n_memory_holes").unwrap_or_default(),
        };

crates/native_blockifier/src/py_transaction_execution_info.rs line 251 at r6 (raw file):

        let actual_resources = execution_resources_from_hashmap(&actual_resources.0);

        let mut merged_resources = additional_os_resources.add(&actual_resources);

I think adding them is enough.

Suggestion:

+

crates/native_blockifier/src/py_transaction_execution_info.rs line 253 at r6 (raw file):

        let mut merged_resources = additional_os_resources.add(&actual_resources);

        merged_resources.n_steps += merged_resources.n_memory_holes;

Zero it after usage.

Code quote:

merged_resources.n_memory_holes;

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 8768e54 to 9fec1f9 Compare February 1, 2024 10:01
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 2 of 6 files reviewed, 5 unresolved discussions (waiting on @ArniStarkware and @elintul)


crates/native_blockifier/src/py_transaction_execution_info.rs line 239 at r6 (raw file):

    pub fn calculate(
        additional_os_resources: VmExecutionResources,
        actual_resources: ResourcesMapping,

Done.


crates/native_blockifier/src/py_transaction_execution_info.rs line 247 at r6 (raw file):

Previously, elintul (Elin) wrote…

expect + informative error message, also lose the variable.

Done.


crates/native_blockifier/src/py_transaction_execution_info.rs line 249 at r6 (raw file):

        })?;

        let actual_resources = execution_resources_from_hashmap(&actual_resources.0);

Done.
unwrap_or_default didn't work beacuse "get" returns an Option<&usize>, and unwrap_or_default expects usize in order to create the default value.


crates/native_blockifier/src/py_transaction_execution_info.rs line 251 at r6 (raw file):

Previously, elintul (Elin) wrote…

I think adding them is enough.

didn't work


crates/native_blockifier/src/py_transaction_execution_info.rs line 253 at r6 (raw file):

Previously, elintul (Elin) wrote…

Zero it after usage.

Done.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r7, all commit messages.
Reviewable status: all files reviewed, 10 unresolved discussions (waiting on @ayeletstarkware)

a discussion (no related file):
In a Python PR: please add an assertion that the new bouncer info we get from Rust is equivalent to the one we currently calculate in Python. Only after this CI passes, can we merge this feature.



crates/native_blockifier/src/py_transaction_execution_info.rs line 249 at r6 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

Done.
unwrap_or_default didn't work beacuse "get" returns an Option<&usize>, and unwrap_or_default expects usize in order to create the default value.

I didn't understand the last part of your sentence.


crates/native_blockifier/src/py_transaction_execution_info.rs line 251 at r6 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

didn't work

Please explain what you tried and the errors you got.


crates/native_blockifier/src/py_transaction_execution_info.rs line 239 at r7 (raw file):

    pub fn calculate(
        additional_os_resources: VmExecutionResources,
        actual_resources: &ResourcesMapping,

Swap order, by importance.

Suggestion:

        tx_actual_resources: &ResourcesMapping,
        tx_additional_os_resources: VmExecutionResources,

crates/native_blockifier/src/py_transaction_execution_info.rs line 252 at r7 (raw file):

            BuiltinName::keccak,
            BuiltinName::poseidon,
        ];

Consider defining a constant list and use in other such locations throughout the code.

Code quote:

        let builtin_ordered_list = vec![
            BuiltinName::output,
            BuiltinName::pedersen,
            BuiltinName::range_check,
            BuiltinName::ecdsa,
            BuiltinName::bitwise,
            BuiltinName::ec_op,
            BuiltinName::keccak,
            BuiltinName::poseidon,
        ];

crates/native_blockifier/src/py_transaction_execution_info.rs line 260 at r7 (raw file):

            .collect();

        let actual_resources_execution_resources = VmExecutionResources {

Can the name be shadowed?

Code quote:

actual_resources_execution_resources

crates/native_blockifier/src/py_transaction_execution_info.rs line 269 at r7 (raw file):

            additional_os_resources.add(&actual_resources_execution_resources);
        merged_resources.n_steps += merged_resources.n_memory_holes;
        merged_resources.n_memory_holes = 0;

Suggestion:

        
        // Memory holes are counted as steps.
        merged_resources.n_steps += merged_resources.n_memory_holes;
        merged_resources.n_memory_holes = 0;

crates/native_blockifier/src/py_transaction_execution_info.rs line 278 at r7 (raw file):

                .0
                .get("l1_gas_usage")
                .expect("Invalid Transaction Execution Info. Field l1_gas_usage was not found."),

It's longer than I expected, let's introduce back the variable. Sorry for the back and forth.

Code quote:

            l1_gas_amount: *actual_resources
                .0
                .get("l1_gas_usage")
                .expect("Invalid Transaction Execution Info. Field l1_gas_usage was not found."),

crates/native_blockifier/src/transaction_executor.rs line 153 at r7 (raw file):

                );

                Ok((raw_tx_execution_info, py_bouncer_info))

Suggestion:

                let state_diff_size = 0;
                
                // Finalize counting logic.
                let typed_tx_execution_info = TypedTransactionExecutionInfo {
                    info: tx_execution_info,
                    tx_type: tx_type.to_string(),
                };
                let actual_resources = &typed_tx_execution_info.info.actual_resources;
                let py_bouncer_info = PyBouncerInfo::calculate(
                    additional_os_resources,
                    actual_resources,
                    message_segment_length,
                    state_diff_size,
                )?;
                
                self.staged_for_commit_state = Some(
                    transactional_state.stage(tx_executed_class_hashes, tx_visited_storage_entries),
                );
                let raw_tx_execution_info = serde_json::to_vec(&typed_tx_execution_info)?;
                Ok((raw_tx_execution_info, py_bouncer_info))

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 9fec1f9 to 1c60ce9 Compare February 4, 2024 13:34
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r8, all commit messages.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 239 at r8 (raw file):

    pub fn calculate(
        tx_actual_resources: &ResourcesMapping,
        tx_additional_os_resources: VmExecutionResources, // os

Remove?

Code quote:

// os

Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 7 unresolved discussions (waiting on @elintul)

a discussion (no related file):

Previously, elintul (Elin) wrote…

In a Python PR: please add an assertion that the new bouncer info we get from Rust is equivalent to the one we currently calculate in Python. Only after this CI passes, can we merge this feature.

I deleted Python's "_calculate_tx_weights" on the previous PR. I couldn't add it back because this function used to sum "tx_receipt.execution_info.actual_resources" and "tx_receipt.additionial_os_resources" (as we see in the code snippet below).
"tx_receipt" doesn't have "additionial_os_resources" anymore. It has but "bouncer_info", which contains the calculation already.

I believe the rust's calculation does the same thing:
1 - sums up additional_os_resources + actual_resources
2 - pops "l1_gas_usage" from actual_resources and add it to the dictionary

Also, I discovered a new field of 'actual_resources', which was added recently - "l1_blob_gas_usage".
Do we need to add it to the bouncer?

Code snippet:

def _calculate_tx_weights(self, tx_receipt: StateConsumptionReceipt) -> CountingBouncerWeights:
        gas_weight, cairo_usage = extract_l1_gas_and_cairo_usage(
            resources=tx_receipt.execution_info.actual_resources
        )
        os_cairo_usage = tx_receipt.additional_os_resources.to_dict()
        cairo_usage = add_counters(cairo_usage, os_cairo_usage)

        return {
            **cairo_usage,
            "gas_weight": gas_weight,
            "message_segment_length": tx_receipt.bouncer_info.message_segment_length,
            STATE_DIFF_SIZE_WEIGHT_NAME: tx_receipt.bouncer_info.state_diff_size,
        }
        
def extract_l1_gas_and_cairo_usage(resources: ResourcesMapping) -> Tuple[int, ResourcesMapping]:
    cairo_resource_usage = dict(resources)
    return cairo_resource_usage.pop("l1_gas_usage"), cairo_resource_usage


crates/native_blockifier/src/py_transaction_execution_info.rs line 249 at r6 (raw file):

Previously, elintul (Elin) wrote…

I didn't understand the last part of your sentence.

get returns Option<&V>:
pub fn get(&self, k: &Q) -> Option<&V>

unwrap_or_default expect usize:
the trait bound &usize: std::default::Default is not satisfied
the trait std::default::Default is implemented for usize

It works this way if you prefer it:

Code snippet:

match tx_actual_resources.0.get(constants::N_STEPS_RESOURCE) {
                Some(&value) => value,
                None => Default::default(),
            }

crates/native_blockifier/src/py_transaction_execution_info.rs line 251 at r6 (raw file):

Previously, elintul (Elin) wrote…

Please explain what you tried and the errors you got.

cannot add cairo_vm::vm::runners::cairo_runner::ExecutionResources to cairo_vm::vm::runners::cairo_runner::ExecutionResources

core::ops::arith::Add
pub fn add(self, rhs: Rhs) -> Self::Output
Performs the + operation.


crates/native_blockifier/src/py_transaction_execution_info.rs line 239 at r7 (raw file):

Previously, elintul (Elin) wrote…

Swap order, by importance.

Done


crates/native_blockifier/src/py_transaction_execution_info.rs line 252 at r7 (raw file):

Previously, elintul (Elin) wrote…

Consider defining a constant list and use in other such locations throughout the code.

Will do so in a different PR.


crates/native_blockifier/src/py_transaction_execution_info.rs line 260 at r7 (raw file):

Previously, elintul (Elin) wrote…

Can the name be shadowed?

WDYM?


crates/native_blockifier/src/py_transaction_execution_info.rs line 269 at r7 (raw file):

            additional_os_resources.add(&actual_resources_execution_resources);
        merged_resources.n_steps += merged_resources.n_memory_holes;
        merged_resources.n_memory_holes = 0;

Done


crates/native_blockifier/src/py_transaction_execution_info.rs line 278 at r7 (raw file):

Previously, elintul (Elin) wrote…

It's longer than I expected, let's introduce back the variable. Sorry for the back and forth.

Done.


crates/native_blockifier/src/transaction_executor.rs line 153 at r7 (raw file):

                );

                Ok((raw_tx_execution_info, py_bouncer_info))

Done

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 1c60ce9 to 3d6e3ec Compare February 4, 2024 14:03
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r9, all commit messages.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 249 at r6 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

get returns Option<&V>:
pub fn get(&self, k: &Q) -> Option<&V>

unwrap_or_default expect usize:
the trait bound &usize: std::default::Default is not satisfied
the trait std::default::Default is implemented for usize

It works this way if you prefer it:

Sorry, I don't follow. Let's take this offline please.


crates/native_blockifier/src/py_transaction_execution_info.rs line 251 at r6 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

cannot add cairo_vm::vm::runners::cairo_runner::ExecutionResources to cairo_vm::vm::runners::cairo_runner::ExecutionResources

core::ops::arith::Add
pub fn add(self, rhs: Rhs) -> Self::Output
Performs the + operation.

I think you should work with references.


crates/native_blockifier/src/py_transaction_execution_info.rs line 260 at r7 (raw file):

Previously, ayeletstarkware (Ayelet Zilber) wrote…

WDYM?

Use actual_resources.

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 3d6e3ec to 26cfc50 Compare February 4, 2024 16:22
Copy link
Contributor Author

@ayeletstarkware ayeletstarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 6 files reviewed, 4 unresolved discussions (waiting on @elintul)


crates/native_blockifier/src/py_transaction_execution_info.rs line 251 at r6 (raw file):

Previously, elintul (Elin) wrote…

I think you should work with references.

Done.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r10, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @ayeletstarkware)


crates/native_blockifier/src/py_transaction_execution_info.rs line 265 at r10 (raw file):

            .collect();

        let tx_actual_resources = VmExecutionResources {

Suggestion:

            .collect();
        let tx_actual_resources = VmExecutionResources {

crates/native_blockifier/src/py_transaction_execution_info.rs line 273 at r10 (raw file):

        let mut merged_resources = &tx_additional_os_resources + &tx_actual_resources;

        // Memory holes are counted as steps.

Suggestion:

        let mut merged_resources = &tx_additional_os_resources + &tx_actual_resources;
        // Memory holes are counted as steps.

crates/native_blockifier/src/py_transaction_execution_info.rs line 277 at r10 (raw file):

        merged_resources.n_memory_holes = 0;

        let execution_resources = PyVmExecutionResources::from(merged_resources);

I think we can lose this variable.

Code quote:

execution_resources

elintul
elintul previously approved these changes Feb 4, 2024
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @ayeletstarkware)

ArniStarkware
ArniStarkware previously approved these changes Feb 5, 2024
Copy link
Contributor

@ArniStarkware ArniStarkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @ayeletstarkware)

a discussion (no related file):

Previously, ArniStarkware (Arnon Hod) wrote…

Python side PR: https://github.com/starkware-industries/starkware/pull/33661

Please, some else block.


@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 26cfc50 to 5fd0bd4 Compare February 5, 2024 09:25
elintul
elintul previously approved these changes Feb 5, 2024
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 1 files at r11, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ayeletstarkware)

@ayeletstarkware ayeletstarkware force-pushed the ayelet/starknet/remove-unnecessary-fields-in-bouncerinfo branch from 5fd0bd4 to 8132dff Compare February 5, 2024 10:10
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r12, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ayeletstarkware)

@ayeletstarkware ayeletstarkware merged commit 076d46b into main-v0.13.1 Feb 5, 2024
6 of 8 checks passed
rodrigo-pino added a commit to NethermindEth/blockifier that referenced this pull request Mar 7, 2024
* Run cargo update

Signed-off-by: Dori Medini <dori@starkware.co>

* Detach native_blockifier

* Bump cairo-vm and cairo compiler

New cairo-vm version includes performance optimizations.
New cairo version only includes the new vm version.

* Release 0.4.1-rc.0

* Re-attach native_blockifier, bump papyrus_storage

* native_blockifier: Allow querying for `Blockifier` version. (starkware-libs#1249)

This allows one to do:
```
import native_blockifier
native_blockifier.blockifier_version()
```
And get the version the native was compiled with.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* Add updated version of estimate_casm_hash_computation_resources(). (starkware-libs#1314)

* Resolve conflicts.

* Reconnect nnative-blockifier

Signed-off-by: Dori Medini <dori@starkware.co>

* Add the field kzg_resources to PyBouncerInfo. (starkware-libs#1321)

* Add use_kzg_da flag to PyBlockInfo. (starkware-libs#1319)

* Merge `main-v0.13.0` into `main` (resolve conflicts).

* Remove gas price bounds from general config (starkware-libs#1329)

Signed-off-by: Dori Medini <dori@starkware.co>

* Bump cairo-vm version (starkware-libs#1330)

Signed-off-by: Dori Medini <dori@starkware.co>

* Update cairo compiler version and fix related TODOs. (starkware-libs#1332)

* Small fix in into_block_context. (starkware-libs#1337)

* Data gas prices in block context (starkware-libs#1336)

Signed-off-by: Dori Medini <dori@starkware.co>

* Use try_from instead of as (starkware-libs#1322)

* Use into instead of as (starkware-libs#1333)

* Add MAX_L1_GAS_AMOUNT_U128 (starkware-libs#1335)

* Refactor calculate_tx_gas_usage (starkware-libs#1320)

* Change additional as to try_from and try_to (starkware-libs#1342)

* Remove PyKzgResources. Reveal the state diff size. (starkware-libs#1341)

* Make `BlockContext` a newtype around `BlockInfo` (starkware-libs#1323)

The crux of this commit is the change in `block_context.rs`, which
converts `BlockContext` into a wrapper around `BlockInfo` (which used to be
`BlockContext`).

Everything else is just delegating the accessors to the internal block_info, made
by search-and-replace (no other changes).

Motivation:
- This change the groundwork for future refactor and consolidation of
contexts.
- Subsequent commits will extract non block-specific constants from
`BlockInfo` into separate `ChainInfo` and `VersionedConstants` structs,
which will be siblings of `block_info` inside `BlockContext`.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* Extract from `BlockInfo` into `ChainInfo` (starkware-libs#1344)

`BlockInfo` should only contain block-level info, whereas chain-level
information should be held at the `BlockContext` level, encapsulated in
a dedicated `ChainInfo` struct.

Changes:
Only extract from `BlockInfo` and store in `BlockContext.chain_info:
ChainInfo`. All other changes are search-replace + moving the
fee_token_address getter into `ChainInfo`.

TODO: extract more stuff from BlockInfo: version-dependant constants should
be extracted into a dedicated VersionedConstants, which will be a member of BlockContext

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* Revert changes of 776d7a5 (starkware-libs#1352)

* fix(native_blockifier): Remove `PyGasPrices` (starkware-libs#1353)

`PyX` types are intended to represent (a subset) of the real
structure of object in Python.
Currently Python is broken do to PyBlockInfo's FromPyObject expecting
there to be a `PyGasPrices`, which doesn't exist in Ptyhon

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* Rename `{into,to}_actual_cost_builder` (starkware-libs#1355)

`into` implies `owned -> owned` cast, whereas `to` can imply `borrowed
-> owned`.

See this reference for naming convention: https://rust-lang.github.io/api-guidelines/naming.html#ad-hoc-conversions-follow-as_-to_-into_-conventions-c-conv

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* add missing internal panic in cairo1 (starkware-libs#1340)

* add missing internal panic in cairo1

* update `test_stack_trace` and make it work with Cairo1 as well as Cairo0

* Align block info fields (starkware-libs#1358)

Signed-off-by: Dori Medini <dori@starkware.co>

* ci: add commitlint in CI (starkware-libs#1313)

Signed-off-by: Dori Medini <dori@starkware.co>

* Create the function calculate_tx_blob_gas_usage. (starkware-libs#1346)

* Add direct cast from PyFelt into ContractAddress + Deref (starkware-libs#1345)

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* fix: revert "Add direct cast from PyFelt into ContractAddress + Deref (starkware-libs#1345)" (starkware-libs#1364)

This reverts commit 6b524c9.

Python is fussy about derive_more, will TAL at it again later

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* refactor(fee): split tx l1 gas computation to da and non-da sum, add fee test for empty transaction (starkware-libs#1339)

* test: add stack trace regressions tests in Cairo1 for various call chains (starkware-libs#1367)

* feat(fee): preparation-add the calldata length to the resource calcaultion (starkware-libs#1361)

* build(fee): added new struct and field in preparation for using blob_gas (starkware-libs#1356)

* feat(fee): modified calculate_tx_l1_gas_usage(s) added calculate_tx_gas_and_blob_gas_usage (starkware-libs#1357)

* style: remove some unnecessary variables (starkware-libs#1366)

* feat(fee): modified calculate_tx_l1_gas_usage(s), added calculate_tx_gas_and_blob_gas_usage (starkware-libs#1370)

* refactor(state): replace GlobalContractCache default with GlobalContractCache new (starkware-libs#1368)

* feat(fee): check gas amount including discounted data gas (starkware-libs#1374)

Signed-off-by: Dori Medini <dori@starkware.co>

* test: add cairo0 variants to the new stack trace regression tests (starkware-libs#1371)

* feat(fee): calculates messages size field (starkware-libs#1290)

* fix(native_blockifier): change failure_flag type to bool (starkware-libs#1334)

* fix(execution): convert all local errors to stack trace (starkware-libs#1378)

Signed-off-by: Dori Medini <dori@starkware.co>

* fix(execution): change additional as to from (starkware-libs#1373)

* Merge `main-v0.13.1` into `main` (resolve conflicts).

* fix(native_blockifier): use PyCallType and PyEntryPointType instead of as (starkware-libs#1383)

* ci: prevent PR title check on merges

Signed-off-by: Dori Medini <dori@starkware.co>

* ci: ignore merge PRs when linting PR titles (starkware-libs#1388)

Signed-off-by: Dori Medini <dori@starkware.co>

* refactor(native_blockifier): pybouncerinfo tx_weights field (starkware-libs#1347)

* style(fee): fix error handle unpack (starkware-libs#1390)

* feat(fee): add linear combination for gas and blob_gas (starkware-libs#1360)

* refactor: change CustomHint error usage into VirtualMachineError::Other(anyhow::Error) (starkware-libs#1397)

* refactor(state): use get_nonce_updates (starkware-libs#1398)

* refactor: make the BlockContext constructor private as preparation for pre_process_block changes (starkware-libs#1385)

* refactor: change variable and function names for gas_vector (starkware-libs#1400)

* fix(fee): modified estimate_minimal_gas_vector, extracted compute_discounted_gas_from_gas_vector (starkware-libs#1401)

* refactor: modified extraction in extract_l1_blob_gas_usage (from unwrap_or(0) to expect) (starkware-libs#1402)

* test: update test_call_contract to new infra (starkware-libs#1404)

Signed-off-by: Dori Medini <dori@starkware.co>

* feat: add VersionedConstants

- Currently only covers the constants previously contained in `BlockInfo`.
TODO: Add check if this covers all chain ids, which might have different
contstants
- Remove `BlockContextArgs`: no longer needed now that BlockContext can
  be initialized by 3 args of public types.

* test: update test_replace_class to new infra (starkware-libs#1405)

Signed-off-by: Dori Medini <dori@starkware.co>

* refactor: restructure old_block_number_and_hash as a struct (starkware-libs#1403)

* chore(fee): gas computation refactor (starkware-libs#1408)

Changes:
1. Implemented get_da_gas_cost, which returns a GasVector (depending on
   use_kzg_da flag).
2. Deleted calculate_tx_gas_usage_vector, because
   calculate_tx_l1_gas_usage does the same thing.
3. Derived derive_more::Add for GasVector to easily sum up gas costs.

Signed-off-by: Dori Medini <dori@starkware.co>

* chore(fee): use GasVector as a return type for gas computations (starkware-libs#1409)

Signed-off-by: Dori Medini <dori@starkware.co>

* refactor: rename context modules

- Rename block_context.rs -> context.rs. This will hold all context
  types.
- Rename block_execution.rs -> block.rs and move `BlockInfo` and
  `GasPrices` there (`GasPrices` is only used inside `BlockInfo`).

No other changes.

* chore(fee): integer VM gas usage (starkware-libs#1410)

* chore(fee): use GasVector as a return type for gas computations

Signed-off-by: Dori Medini <dori@starkware.co>

* chore(fee): integer VM gas usage

Signed-off-by: Dori Medini <dori@starkware.co>

* perf(native_blockifier): transaction execution info add serialize (starkware-libs#1315)

* feat(fee): adding a slope paramters to the os resources (starkware-libs#1362)

* feat(fee): set the resources slope values (starkware-libs#1363)

* refactor: make `AccountTransactionContext` hold BlockContext

- Rename `AccountTransactionContext` into `TransactionInfo`: i want to
  use `Context` for something else, and `Account` is a misnomer, since
  L1Transactions also generate this instance.
- Create a new `AccountTransactionContext`, which holds `BlockContext`
  and `TransactionInfo`: This mirrors `BlockContext`, which holds
  `BlockInfo` as well as higher level contexts.
- Remove all unnecessary `get_account_tx` calls.
- Replace all functions that take both `block_context` and `tx_info` with
a single `TransactionContext`.
- Make `EntryPointExecutionContext` hold an `Arc<TransactionContext>`
  instead of both a block_context and `tx_info`: previously every entry
  point cloned the blockContext and generated a new tx_info, now they
  all share the same (identical) one.

* feat: add more constants to`VersionedConstants`

- Add gateway constants, these are only for transparency, and aren't
  used directly by the Blockifier whatsoever.
- Pass `validate_max_n_steps` into the blockifier via native_blockifier,
  to override versioned constants defaults.
- Sort json

* fix: remove dead code `block_context.rs` (starkware-libs#1423)

The module has already taken out of lib.rs in
`8ba2662f93999b526ef7fd0a7fc49d1314657184` (making this module
dead-code), but the file wasn't actually deleted there.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* fix: delete dead code block_execution.rs (starkware-libs#1425)

Was already removed from lib.rs.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* chore(execution): enforce nonzero gas prices (starkware-libs#1391)

Signed-off-by: Dori Medini <dori@starkware.co>

* refactor(execution)!: pre_process_block refactor for full_nodes simulate transaction (starkware-libs#1387)

* refactor(native_blockifier): flat fields in bouncerinfo (starkware-libs#1394)

* feat: add Starknet OS constants to `VersionedConstants` - unused

1. Currently not using the new consts, just adding
the deserialization logic and tests.
In upcoming commits this will replace the gas costs in `constants.rs`.

The idea is that values inside `cairo_constants` that are json objects,
that is, have the form:
```json
"step_gas_cost": {
  foo_cost: x,
  bar_cost: y,
  ...
}
```
should be parsed as:
```
step_gas_cost = foo_cost*x + bar_cost*y + ...,
```
where {x,y} must be unsigned integers (otherwise an error is thrown),
and where {foo_cost, bar_cost,...} must correspond to keys that
have already* been parsed in the JSON.

Note: JSON standard doesn't enforce order on the keys, but our
implementation does; we should consider using a format that ensures
order, rather than relying on serde's implementation and `IndexMap`.

2. validate the os costs on creation (except for in tests): under
this assumption `get_gas_cost` will _panic_ if given a non-whitelisted
gas cost name.

3. hide the two hashmaps inside `VersionedConstants`, they have
invariants, set accessors accordingly.

* chore(execution): saturate step bound on overflow (starkware-libs#1429)

Signed-off-by: Dori Medini <dori@starkware.co>

* feat(fee): add signature length to ActualCostBuilder (starkware-libs#1431)

* feat(execution): use gas costs only from `VersionedConstants`

- Remove from constants module and replace all usages with
  `VersionedConstants#gas_cost(..)`
  - Move all comments from the constants module into the const whitelist
    in `VersionedConstants`.
- Add gas cost getter to `EntryPointExecutionContext`, for readability
  - enclose in closure inside hint_processor.rs for even more brevity.
- Move `Transaction::Initial_gas` into `VersionedConstants`: it is now
  derived from the constants json.
- No other logic changes.

* feat(fee): add os resources to versioned constants

- Move hardcoded os resources json into the versioned constants json,
  move `OsResources` into into `OsResources` module.
- Indent versioned_constants.json at 4.
- Delete os_usage.rs: all logic is now called from methods in
  `VersionedConstants`(todo: extract into submodules, currently
  just a big module).
- Delete os_usage_test.rs: these are not post-deserialize validations of
  `OsResources`.
- sorted versioned_constants (where possible, which is everywhere except
  for inside os_constants keys).

* chore(execution): fix the tests using create_state_changes_for_test for readability and correctness (starkware-libs#1430)

* refactor(execution, native_blockifier): make TransactionExecutor.execute() work w/o Py objects (starkware-libs#1414)

* refactor(execution, native_blockifier): make TransactionExecutor.finalize() work without Py objects (starkware-libs#1418)

* refactor(execution, native_blockifier): decouple BouncerInfo from Python (starkware-libs#1428)

* refactor: split InnerCallExecutionError into CallContractExecutionError & LibraryCallExecutionError.
Add storage_address to both new types and class_hash to LibraryCallExecutionError (starkware-libs#1432)

* feat(execution): moved global max step limit to input validation (starkware-libs#1415)

Signed-off-by: Dori Medini <dori@starkware.co>

* test(execution): improve covrage of test_count_actual_storage_changes (starkware-libs#1436)

* refactor(execution): the struct state changes and state changes count (starkware-libs#1417)

* chore(fee): update os resources and fix expected

* fix(native_blockifier): fix unused imports (starkware-libs#1442)

* chore(execution): consume state_changes on state_changes_count from (starkware-libs#1443)

* feat(execution): calculate the syscall resources for every entry point (starkware-libs#1437)

* feat(execution): add entry_point_syscall_counter (starkware-libs#1407)

* refactor(execution): split get_additional_os_resources (starkware-libs#1411)

* feat(execution): add the syscall resources to vm_resources (starkware-libs#1412)

* fix(fee): fix doc of usize u128 conversions (starkware-libs#1446)

* fix: memory holes tests (starkware-libs#1451)

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* fix: remove unused dep ctor (starkware-libs#1455)

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* refactor(execution): replace StateChangesCount::from with state_changes.count() (starkware-libs#1452)

* refactor(transaction): make DeclareTransaction fields public (starkware-libs#1441)

* refactor: remove enum VirtualMachineExecutionError (starkware-libs#1453)

* fix(native_blockifier): fix build for mac + unused import (starkware-libs#1406)

* feat(fee): add ClassInfo struct (starkware-libs#1434)

* Merge `main-v0.13.1` into `main` (resolve conflicts).

* feat(fee): add GasVector to TransactionExecutionInfo (starkware-libs#1399)

* feat(fee): add DA GasVector to TransactionExecutionInfo

Signed-off-by: Dori Medini <dori@starkware.co>

* chore(fee): rename blob_gas to l1_data_gas

Signed-off-by: Dori Medini <dori@starkware.co>

* chore(fee): delete unused PyGasVector

Signed-off-by: Dori Medini <dori@starkware.co>

* refactor(execution): remove syscall_counter from ExecutionResources (starkware-libs#1424)

* Update syscall resources (starkware-libs#1459)

* fix: os resources (starkware-libs#1465)

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* feat(state): bouncer DA count: define StateWriteKeys (starkware-libs#1461)

* chore: alphabetize top level keys in versioned constants (starkware-libs#1466)

Co-authored-by: Gilad Chase <gilad@starkware.com>

* feat(state): bouncer DA count: use StateWriteKeys to count state diff size (starkware-libs#1462)

* refactor(execution, native_blockifier): move Bouncer to blockifier crate (starkware-libs#1445)

* refactor(transaction): use ClassInfo in Declare transaction (starkware-libs#1456)

* feat(fee): add gas cost for calldata bytes (starkware-libs#1426)

* chore: bump CI versions (starkware-libs#1473)

- Remove the deprecated `actions-rs/toolchain@v1` wherever it is still
  used.
- Bump everything to latest versions.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* refactor(execution): delete enrty point ExecutionResources (starkware-libs#1447)

* fix: limit the syscall emit_event resources, keys, data and number of events (starkware-libs#1463)

* fix(transaction): set l1 hanlder payload size to None for non-l1 hanlder txs (bouncer count) (starkware-libs#1470)

* feat(fee): charge for signatures per byte (starkware-libs#1474)

* Merge `main-v0.13.0-hotfix` into `main-v0.13.1` (resolve conflicts).

* feat(execution): round block_number and timestamp for the execution info syscall in validate_mode (starkware-libs#1467)

* feat(fee): add gas cost for code bytes (starkware-libs#1475)

* feat(fee): add events gas cost (starkware-libs#1458)

* chore(execution): increase invoke_tx_max_n_steps (starkware-libs#1477)

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* chore(fee): update vm resource and l2 resource costs (starkware-libs#1479)

* chore: upgrade cairo and sn-api versions (starkware-libs#1454)

* chore: upgrade cairo and sn-api versions

* chore: upgrade cairo and sn-api versions (starkware-libs#1460)

* fix: fix cargo.lock - remove cargo update (starkware-libs#1481)

* fix: undo cargo update from previous commit 984431a

* fix: update cargo.lock to take into account changes in 984431a

they wered reverted in previous commit.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* fix: fix versioned_constants.json

* chore(fee): small fixes (starkware-libs#1484)

* refactor(fee): moving calculate_tx_gas_usage_vector to ActualCostBuilder (starkware-libs#1478)

* Release v0.5.0-rc.1

* refactor(execution): remove fee weights from config (starkware-libs#1487)

* fix: update the event limit constants (starkware-libs#1483)

* Merge `main-v0.13.0-hotfix` into `main-v0.13.1` (resolve conflicts).

* refactor(execution, native_blockifier): move TransactionExecutor to blockifier (starkware-libs#1497)

* refactor(state): remove unnecessary derive (starkware-libs#1505)

* refactor(execution): move bouncer, block and block_test to blockifier dir (starkware-libs#1502)

* feat(fee): add n_events field to BouncerInfo (starkware-libs#1489)

* refactor: move the event size limit constants into versioned_constants (starkware-libs#1490)

* refactor(transaction): remove public fields, add new function (starkware-libs#1480)

* refactor(execution): type of additional_data in test_validate_accounts_tx (starkware-libs#1485)

* refactor: move validate_block_number_rounding and validate_timestamp_rounding to versioned_constants (starkware-libs#1506)

* test(execution): add get_execution_info scenario to test_validate_accounts_tx (starkware-libs#1486)

* test(execution): get_sequencer_address in test_validate_accounts_tx (starkware-libs#1496)

* refactor: use versioned_constants function when possible (starkware-libs#1508)

* test(execution): get_block_number and get_block_timestamp in test_validate_accounts_tx for Cairo0 (starkware-libs#1512)

* chore(fee): Update os resources to include 4844 calculations (starkware-libs#1498)

* refactor: replaced default_invoke_tx_args with macro that enforces to explicitly define max_fee (starkware-libs#1510)

* feat(fee): calculate n_events field for BouncerInfo in a naive way (starkware-libs#1499)

* fix: state trait does not require &mut self (starkware-libs#1325)

* fix: state trait does not require &mut self

fix: sachedState use interior mutability

* chore: remove clippy allow

* fix: small review fix

* fix: remove to_state_diff from StateApi (starkware-libs#1326)

* chore: merge branch main-v0.13.1 into main (resolve conflicts)

* feat: add merge_branches script (starkware-libs#1513)

* chore: bump compiler version to 2.6.0-rc.1 (starkware-libs#1525)

Signed-off-by: Dori Medini <dori@starkware.co>

* chore: move validate consts into os constants (starkware-libs#1523)

* chore: move validate consts into os constants

Had to bump serde_json due to some apparent bug with the recent stable
rust.

* feat: make 13_1 consts backwards compatible

Assign the following defaults for keys missing in versioned_constants
files (for example, they are missing in versioned_constants_13_0.json).

- EventSizeLimit => max values
- L2ResourceGasCosts => 0 values
- validate rounding numbers => 1 (to preserve past no-rounding
  behavior). This required bundling them up in a specialized struct in
  order to define a custom default as 1 (rather than 0).
- Add test for default values: required extracting validation logic of
  `OsResources` so it won't trigger automatically in tests.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* chore!: limit pointer width in both crates (starkware-libs#1527)

Signed-off-by: Dori Medini <dori@starkware.co>

* feat: backwards compatibility for calldata_factor

If os resources' inner tx is missing `constant` and `calldata_factor`
keys, then:
- assume it is a flat `ExecutionResources` instance, and put its
  contents inside a `constant` key.
- initialize calldata_factor as default (all zeros).

* feat: add versioned_constants_13_0.json (starkware-libs#1520)

Changes between this and the current versioned_constants:
- no `event_size_limit`
- invoke_tx_max_n_steps is 3_000_000 instead of 4_000_000
- no `l2_resource_gas_costs`
- multiple value changes in os_resources
- `vm_resource_fee_cost` is X2 for all values
- no constants for validate's block number and timestamp rounding

No other changes (in particular, os constants is indeed _unchanged_)

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* Merge `main-v0.13.1` into `main` (resolve conflicts).

* fix: versioned constants validate (starkware-libs#1537)

- validate invocation had a syntax error, which was hidden by the cfg
- remove `testing` from the cfg, since we all compile with `testing`
  during development, and it's just not what we want.

Co-Authored-By: Gilad Chase <gilad@starkware.com>

* fix: update the tx event limit constants (starkware-libs#1517)

* chore: release 0.5.0-rc.2: 13_0.json support

* fix: generate_starknet_os_resources python target fix (starkware-libs#1541)

* No conflicts in main-v0.13.1 -> main merge, this commit is for any change needed to pass the CI.

* fix: generate_starknet_os_resources python target fix (starkware-libs#1548)

* refactor: remove some magic numbers from tests (starkware-libs#1547)

* fix!: update max_contract_bytecode_size to 81k (starkware-libs#1549)

* chore: add merge_status.py script (starkware-libs#1543)

Signed-off-by: Dori Medini <dori@starkware.co>

* chore: merge branch main-v0.13.1 into main (resolve conflicts)

* ci: advance setup-python action version (starkware-libs#1555)

* refactor(native_blockifier): rename l1_gas_amount to gas_weight in bouncerInfo (starkware-libs#1524)

* chore: release v0.5.0-rc.3

* chore: panic if u128_from_usize fails (starkware-libs#1522)

Signed-off-by: Dori Medini <dori@starkware.co>

* refactor: make StateError::UndeclaredClassHash a one-line error (starkware-libs#1563)

Change previous error which prints out as:
```
Class with hash ClassHash(
    StarkFelt(
        "0x00000000000000000000000000000000000000000000000000000000000004d2",
    ),
) is not declared.
```
into:
```
Class with hash 0x00000000000000000000000000000000000000000000000000000000000004d2 is not declared.
```

* test(fee): test get_message_segment_length function (starkware-libs#1471)

* refactor: add BlockWeights for the future bouncer (starkware-libs#1444)

* refactor: remove f64 usage at blockifier (starkware-libs#1519)

* fix: unwrap called on u128 (starkware-libs#1570)

Signed-off-by: Dori Medini <dori@starkware.co>

* test: use FeatureContract in create_test_state and tests which uses it (starkware-libs#1556)

* No conflicts in main-v0.13.1 -> main merge, this commit is for any change needed to pass the CI.

* chore: remove final f64s (starkware-libs#1574)

Signed-off-by: Dori Medini <dori@starkware.co>

* chore: remove some 'as' conversions and enforce no 'as' conversions (starkware-libs#1575)

* chore: remove final f64s

Signed-off-by: Dori Medini <dori@starkware.co>

* chore: remove some 'as' conversions and enforce no 'as' conversions

Signed-off-by: Dori Medini <dori@starkware.co>

* Restore workflow

* Rename ci

* Add commit lint rules

* nightly format

* update .env

* Test global env and composite action

* attemp fix to composite action

* Minor fix

* Modify .github structure

* Add composite action to jobs?

* Minor yaml fix

* Fix compilation errors

* WIP

* Storage_read_write_gas

* add tests for vm & native

* modify test_get_execution_info

* Review tests

* rename syscalls_test.rs -> syscall_tests.rs

* Fix minor compilation bug

* Update dependencies

---------

Signed-off-by: Dori Medini <dori@starkware.co>
Co-authored-by: Dori Medini <dori@starkware.co>
Co-authored-by: Gilad Chase <gilad@starkware.com>
Co-authored-by: giladchase <gilad@starkware.co>
Co-authored-by: Lior Goldberg <lior@starkware.co>
Co-authored-by: liorgold2 <38202661+liorgold2@users.noreply.github.com>
Co-authored-by: Arnon Hod <arnon@starkware.co>
Co-authored-by: Elin Tulchinsky <elin@starkware.co>
Co-authored-by: OriStarkware <76900983+OriStarkware@users.noreply.github.com>
Co-authored-by: Ayelet Zilber <138376632+ayeletstarkware@users.noreply.github.com>
Co-authored-by: zuphitf <51879558+zuphitf@users.noreply.github.com>
Co-authored-by: aner-starkware <147302140+aner-starkware@users.noreply.github.com>
Co-authored-by: Noa Oved <104720318+noaov1@users.noreply.github.com>
Co-authored-by: Yoni <78365039+Yoni-Starkware@users.noreply.github.com>
Co-authored-by: YaelD <70628564+Yael-Starkware@users.noreply.github.com>
Co-authored-by: mohammad-starkware <130282237+MohammadNassar1@users.noreply.github.com>
Co-authored-by: barak-b-starkware <110763103+barak-b-starkware@users.noreply.github.com>
Co-authored-by: Lucas @ StarkWare <70894690+LucasLvy@users.noreply.github.com>
Co-authored-by: dafnamatsry <92669167+dafnamatsry@users.noreply.github.com>
Co-authored-by: avi-starkware <avi.cohen@starkware.co>
Co-authored-by: Tzahi <tzahi@starkware.co>
Co-authored-by: nimrod-starkware <143319383+nimrod-starkware@users.noreply.github.com>
Co-authored-by: Timothée Delabrouille <34384633+tdelabro@users.noreply.github.com>
Co-authored-by: Barak <barakb@starkware.co>
Co-authored-by: Yonatan Iluz <yonatan@starkware.co>
Co-authored-by: Rodrigo <rodrodpino@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants