Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare air_public_inputs against python vm + Fix how public memory is built #1391

Merged
merged 28 commits into from
Aug 29, 2023

Conversation

fmoletta
Copy link
Contributor

@fmoletta fmoletta commented Aug 22, 2023

Fixes various bugs related to Public Memory
Adds small modifications to make air public input files equivalent to the ones outputted by cairo-lang
Adds targets to output and compare air_public_input files across vms
Missing in this PR: Adding air_public_input file comparison to CI (This will be added in a later PR due to the complicated nature of the current workflows)

Checklist

  • Linked to Github Issue
  • Unit tests added
  • Integration tests added.
  • This change requires new documentation.
    • Documentation has been added/updated.
    • CHANGELOG has been updated.

@fmoletta fmoletta linked an issue Aug 22, 2023 that may be closed by this pull request
@codecov
Copy link

codecov bot commented Aug 22, 2023

Codecov Report

Merging #1391 (e860e0e) into main (2aed1ba) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1391   +/-   ##
=======================================
  Coverage   96.96%   96.96%           
=======================================
  Files          94       94           
  Lines       38645    38662   +17     
=======================================
+ Hits        37473    37490   +17     
  Misses       1172     1172           
Files Changed Coverage Δ
vm/src/air_public_input.rs 96.92% <100.00%> (ø)
vm/src/vm/runners/cairo_runner.rs 98.04% <100.00%> (+<0.01%) ⬆️
vm/src/vm/vm_core.rs 97.61% <100.00%> (+<0.01%) ⬆️
vm/src/vm/vm_memory/memory_segments.rs 99.02% <100.00%> (+0.01%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@github-actions
Copy link

github-actions bot commented Aug 22, 2023

Benchmark Results for unmodified programs 🚀

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 4.060 ± 0.011 4.045 4.080 1.00 ± 0.00
head big_factorial 4.040 ± 0.010 4.026 4.056 1.00
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 3.421 ± 0.005 3.416 3.428 1.01 ± 0.00
head big_fibonacci 3.401 ± 0.008 3.393 3.416 1.00
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 9.601 ± 0.054 9.554 9.703 1.01 ± 0.01
head blake2s_integration_benchmark 9.520 ± 0.043 9.499 9.640 1.00
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 3.527 ± 0.057 3.498 3.689 1.01 ± 0.02
head compare_arrays_200000 3.491 ± 0.015 3.478 3.517 1.00
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 2.334 ± 0.006 2.326 2.344 1.01 ± 0.00
head dict_integration_benchmark 2.318 ± 0.005 2.312 2.324 1.00
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.718 ± 0.004 1.711 1.723 1.00 ± 0.01
head field_arithmetic_get_square_benchmark 1.713 ± 0.011 1.704 1.737 1.00
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 9.425 ± 0.018 9.393 9.455 1.00 ± 0.00
head integration_builtins 9.415 ± 0.030 9.389 9.481 1.00
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 9.822 ± 0.288 9.716 10.640 1.02 ± 0.03
head keccak_integration_benchmark 9.663 ± 0.008 9.653 9.677 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search 3.505 ± 0.009 3.493 3.522 1.01 ± 0.00
head linear_search 3.483 ± 0.008 3.472 3.493 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 2.579 ± 0.056 2.553 2.736 1.01 ± 0.02
head math_cmp_and_pow_integration_benchmark 2.551 ± 0.003 2.547 2.556 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 2.291 ± 0.005 2.285 2.302 1.01 ± 0.00
head math_integration_benchmark 2.276 ± 0.006 2.270 2.286 1.00
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 2.034 ± 0.010 2.025 2.060 1.01 ± 0.01
head memory_integration_benchmark 2.016 ± 0.004 2.011 2.023 1.00
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 2.241 ± 0.004 2.236 2.251 1.01 ± 0.00
head operations_with_data_structures_benchmarks 2.227 ± 0.010 2.216 2.250 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 798.2 ± 1.5 796.4 800.7 1.01 ± 0.00
head pedersen 792.8 ± 1.5 790.6 794.9 1.00
Command Mean [s] Min [s] Max [s] Relative
base poseidon_integration_benchmark 1.489 ± 0.005 1.479 1.499 1.01 ± 0.00
head poseidon_integration_benchmark 1.471 ± 0.002 1.468 1.476 1.00
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 2.828 ± 0.005 2.821 2.837 1.00 ± 0.00
head secp_integration_benchmark 2.825 ± 0.005 2.818 2.834 1.00
Command Mean [s] Min [s] Max [s] Relative
base set_integration_benchmark 1.622 ± 0.004 1.618 1.630 1.00
head set_integration_benchmark 1.697 ± 0.004 1.692 1.705 1.05 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 6.312 ± 0.130 6.251 6.680 1.01 ± 0.02
head uint256_integration_benchmark 6.248 ± 0.016 6.235 6.288 1.00

@fmoletta fmoletta changed the title Compare air_public_inputs against python vm + fixes Compare air_public_inputs against python vm + Fix how public memory is built Aug 22, 2023
@fmoletta fmoletta marked this pull request as ready for review August 23, 2023 13:32
@fmoletta fmoletta added bug Something isn't working enhancement New feature or request pipelines This PR/issue is exclusively about improving our CI labels Aug 23, 2023
@fmoletta fmoletta mentioned this pull request Aug 23, 2023
6 tasks
* Fix typo

* generate and compare air_public inputs in ci

* fix

* Fix syntax

* fix

* fix

* Revert "Fix typo"

This reverts commit 532dfca.

* Fix workflow
Copy link
Collaborator

@pefontana pefontana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one @fmoletta !
Can we add the *.air_public_input to the .gitignore file?

@pefontana pefontana enabled auto-merge August 29, 2023 20:18
@pefontana pefontana added this pull request to the merge queue Aug 29, 2023
Merged via the queue into main with commit 27bcbeb Aug 29, 2023
@pefontana pefontana deleted the compare-public-inputs branch August 29, 2023 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request pipelines This PR/issue is exclusively about improving our CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Check public inputs of the cairo-vm vs python VM
3 participants