Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more tests and port old tests #288

Merged
merged 16 commits into from
Aug 14, 2023
Merged

more tests and port old tests #288

merged 16 commits into from
Aug 14, 2023

Conversation

edg-l
Copy link
Member

@edg-l edg-l commented Aug 8, 2023

This PR adds more tests and ports the missing tests from pre-refactor, however, there are some tests that fail and show current bugs in the compiler, which they are not addressed in this PR:

test utils::test::test_alignment_compatibility_felt252 ... ignored
test utils::test::test_alignment_compatibility_u128 ... ignored
test felt252_to_bool_bug ... ignored, TODO: comparing a felt252 == 1 will lead to wrong results
test test_cases::_tests_cases_array_slice_cairo_expects_inconclusiveempty ... ignored, TODO: fails
test test_cases::_tests_cases_dict_insert_get_cairo_expects_inconclusiveempty ... ignored, gas mismatch
test test_cases::_tests_cases_felt_ops_div_cairo_expects_inconclusiveempty ... ignored, not implemented yet
test test_cases::_tests_cases_gas_available_gas_cairo_expects_inconclusiveempty ... ignored, unimplemented
test test_cases::_tests_cases_nullable_test_nullable_cairo_expects_inconclusiveempty ... ignored, unimplemented
test test_cases::_tests_cases_poseidon_cairo_expects_inconclusiveempty ... ignored, not implemented yet
test test_cases::_tests_cases_uint_safe_divmod_cairo_expects_inconclusiveempty ... ignored, TODO: cairo program is outdated
test dict_get_insert_proptest ... ignored, gas mismatch in dicts
test ec_point_from_x_big ... ignored, TODO: possible bug in ec_point_from_x_nz
test ec_point_from_x_proptest ... ignored, TODO: possible bug in ec_point_from_x_nz
brainfuck example program hangs indefinitly

fixes #242

@github-actions
Copy link

github-actions bot commented Aug 8, 2023

Benchmarking results

Benchmark for program factorial_100

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
cairo-run --available-gas 18446744073709551615 -s programs/benches/factorial_100.cairo 2.527 ± 0.020 2.495 2.562 2755.02 ± 759.96
echo '[null, 18446744073709551615]' | ./target/release/cairo-native-run programs/benches/factorial_100.cairo factorial_100::factorial_100::main --inputs - 0.002 ± 0.000 0.001 0.006 1.72 ± 0.61
./target/bench-outputs/factorial_100 0.001 ± 0.000 0.001 0.004 1.00
./target/bench-outputs/factorial_100-march-native 0.001 ± 0.000 0.001 0.006 1.00 ± 0.46

Benchmark for program fib_1k

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
cairo-run --available-gas 18446744073709551615 -s programs/benches/fib_1k.cairo 2.511 ± 0.031 2.469 2.556 2770.75 ± 814.65
echo '[null, 18446744073709551615]' | ./target/release/cairo-native-run programs/benches/fib_1k.cairo fib_1k::fib_1k::main --inputs - 0.002 ± 0.000 0.001 0.011 1.72 ± 0.66
./target/bench-outputs/fib_1k 0.001 ± 0.000 0.001 0.006 1.00
./target/bench-outputs/fib_1k-march-native 0.001 ± 0.000 0.001 0.010 1.02 ± 0.50

Benchmark for program map

Open benchmarks
Command Mean [s] Min [s] Max [s] Relative
cairo-run --available-gas 18446744073709551615 -s programs/benches/map.cairo 2.704 ± 0.036 2.664 2.789 1744.67 ± 442.17
echo '[null, 18446744073709551615]' | ./target/release/cairo-native-run programs/benches/map.cairo map::map::main --inputs - 0.002 ± 0.000 0.001 0.007 1.00
./target/bench-outputs/map 0.076 ± 0.002 0.073 0.080 49.03 ± 12.46
./target/bench-outputs/map-march-native 0.077 ± 0.002 0.073 0.081 49.46 ± 12.58

@codecov-commenter
Copy link

codecov-commenter commented Aug 8, 2023

Codecov Report

Merging #288 (962930b) into main (395534b) will increase coverage by 0.23%.
The diff coverage is 97.32%.

❗ Current head 962930b differs from pull request most recent head 0f3f2ae. Consider uploading reports for the commit 0f3f2ae to get more accurate results

@@            Coverage Diff             @@
##             main     #288      +/-   ##
==========================================
+ Coverage   67.85%   68.08%   +0.23%     
==========================================
  Files         110      110              
  Lines       20243    20327      +84     
==========================================
+ Hits        13736    13840     +104     
+ Misses       6507     6487      -20     
Files Changed Coverage Δ
src/jit_runner.rs 84.73% <25.00%> (-1.89%) ⬇️
src/libfuncs/array.rs 87.71% <100.00%> (+0.51%) ⬆️
src/libfuncs/felt252.rs 94.42% <100.00%> (+0.10%) ⬆️
src/libfuncs/felt252_dict_entry.rs 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@edg-l edg-l changed the title more tests more tests and port old tests Aug 10, 2023
@edg-l edg-l marked this pull request as ready for review August 10, 2023 15:57
@igaray
Copy link
Collaborator

igaray commented Aug 11, 2023

Closes #242

@igaray
Copy link
Collaborator

igaray commented Aug 11, 2023

Issues #290, #291, #292, #293, #294, #295 created to track remaining failing or missing tests.

@igaray igaray merged commit 13117d1 into main Aug 14, 2023
@igaray igaray deleted the more_tests branch August 14, 2023 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bring back comparison tests
3 participants