Skip to content
This repository has been archived by the owner on Nov 20, 2023. It is now read-only.

fix: skip test cases failing in geth #22

Merged

Conversation

vimpunk
Copy link
Contributor

@vimpunk vimpunk commented May 15, 2023

This patch adds filtering logic to jsontests/tests/state.rs to skip individual test cases (rather than entire test directories), which is used to skip cases that are failing both in geth and here. These can't currently be fixed until we have a correct reference implementation. These cases are:

  • stTransactionTest/HighGasPrice
  • stCreateTest/CreateTransactionHighNonce

Further, another test case is skipped due to different implementation details of the test suite (see code comments):

  • stTransactionTest/ValueOverflow

This PR along with rust-ethereum/evm#162 and rust-ethereum/evm#163 brings sputnikvm and the testsuite to parity with geth.

@vimpunk
Copy link
Contributor Author

vimpunk commented May 15, 2023

All active tests passing, when ran against sputnikvm with above 2 PRs included.

running 2 tests
test tests::test_blake2_f ... ok
test tests::test_vectors_from_eip ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/ethcore_builtin-7fa2745493ebd4a7)

running 40 tests
test tests::blake2_f_test_vector_8 ... ignored
test tests::blake2_f_is_err_on_invalid_length_2 ... ok
test tests::blake2_f_is_err_on_invalid_length ... ok
test tests::blake2_f_is_err_on_bad_finalization_flag ... ok
test tests::blake2_f_test_vector_5 ... ok
test tests::blake2_f_test_vector_7 ... ok
test tests::blake2_f_test_vector_6 ... ok
test tests::blake2_f_zero_rounds_is_ok_test_vector_4 ... ok
test tests::blake2f_cost_on_invalid_length ... ok
test tests::blake2f_cost ... ok
test tests::bls12_381_g1_multiexp_init_from_spec ... ok
test tests::bls12_381_g1_add ... ok
test tests::bls12_381_g2_multiexp_init_from_spec ... ok
test tests::bls12_381_g2_add ... ok
test tests::bn128_add_eip1108_transition ... ok
test tests::bn128_add ... ok
test tests::bls12_381_fp_to_g1 ... ok
test tests::bn128_mul_eip1108_transition ... ok
test tests::bls12_381_g1_mul ... ok
test tests::bn128_pairing_eip1108_transition ... ok
test tests::bn128_pairing_empty ... ok
test tests::bn128_pairing_fragmented ... ok
test tests::bn128_pairing_notcurve ... ok
test tests::from_json ... ok
test tests::from_named_linear ... ok
test tests::from_unknown_linear - should panic ... ok
test tests::identity ... ok
test tests::is_active ... ok
test tests::bn128_mul ... ok
test tests::multimap_use_last_with_same_activate_at ... ok
test tests::multimap_use_most_recent_on_activate ... ok
test tests::ripemd160 ... ok
test tests::sha256 ... ok
test tests::ecrecover ... ok
test tests::modexp ... ok
test tests::bls12_381_g2_mul ... ok
test tests::bls12_381_g1_multiexp ... ok
test tests::bls12_381_fp2_to_g2 ... ok
test tests::bls12_381_pairing ... ok
test tests::bls12_381_g2_multiexp ... ok

test result: ok. 39 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.13s

     Running unittests src/lib.rs (target/debug/deps/ethjson-5addc18450f8851b)

running 51 tests
test hash::test::hash_into ... ok
test bytes::test::bytes_into ... ok
test hash::test::hash_deserialization ... ok
test maybe::tests::maybe_deserialization ... ok
test bytes::test::bytes_deserialization ... ok
test spec::account::tests::account_balance_missing_not_empty ... ok
test spec::account::tests::account_code_missing_not_empty ... ok
test spec::account::tests::account_deserialization ... ok
test spec::account::tests::account_empty ... ok
test spec::account::tests::account_nonce_missing_not_empty ... ok
test spec::account::tests::account_storage_deserialization ... ok
test spec::account::tests::account_storage_missing_not_empty ... ok
test spec::authority_round::tests::authority_round_deserialization ... ok
test spec::basic_authority::tests::basic_authority_deserialization ... ok
test spec::builtin::tests::activate_at ... ok
test spec::builtin::tests::builtin_deserialization ... ok
test spec::builtin::tests::deserialization_blake2_f_builtin ... ok
test spec::builtin::tests::deserialization_alt_bn128_const_operations ... ok
test spec::builtin::tests::deserialization_bls12_381_multiexp_operation ... ok
test spec::builtin::tests::deserialization_bls12_381_multiexp_operation_in_g2 ... ok
test spec::clique::tests::clique_deserialization ... ok
test spec::builtin::tests::deserialize_multiple_pricings ... ok
test spec::ethash::tests::ethash_deserialization_missing_optionals ... ok
test spec::engine::tests::engine_deserialization ... ok
test spec::genesis::tests::genesis_deserialization ... ok
test spec::ethash::tests::test_zero_value_divisor - should panic ... ok
test spec::null_engine::tests::null_engine_deserialization ... ok
test spec::ethash::tests::ethash_deserialization ... ok
test spec::params::tests::params_deserialization ... ok
test spec::params::tests::test_zero_value_divisor - should panic ... ok
test spec::seal::tests::seal_deserialization ... ok
test spec::hardcoded_sync::tests::hardcoded_sync_deserialization ... ok
test spec::spec::tests::should_error_on_unknown_fields ... ok
test state::tests::log_deserialization ... ok
test spec::validator_set::tests::validator_set_deserialization ... ok
test test_helpers::state::tests::multi_transaction_deserialization ... ok
test test_helpers::blockchain::header::tests::header_deserialization ... ok
test test_helpers::blockchain::block::tests::block_deserialization ... ok
test spec::spec::tests::spec_deserialization ... ok
test test_helpers::trie::input::tests::input_deserialization_from_map ... ok
test transaction::tests::transaction_deserialization ... ok
test test_helpers::trie::input::tests::input_deserialization_from_array ... ok
test test_helpers::transaction::tests::transaction_deserialization ... ok
test uint::test::uint_deserialization ... ok
test uint::test::uint_deserialization_error_for_hex_too_large ... ok
test uint::test::uint_into ... ok
test vm::tests::call_deserialization_empty_dest ... ok
test vm::tests::call_deserialization_full_dest ... ok
test test_helpers::state::tests::state_deserialization ... ok
test test_helpers::blockchain::tests::blockchain_deserialization ... ok
test vm::tests::vm_deserialization ... ok

test result: ok. 51 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/evm_jsontests-ff298748b48eae1a)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/main.rs (target/debug/deps/evm_jsontests-9c7c88a75975592b)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/state.rs (target/debug/deps/state-15f25dee12ba1ee7)

running 53 tests
test st_chain_id ... ok
test st_code_copy ... ok
test st_bugs ... ok
test st_create2 ... ignored
test st_code_size_limit ... ok
test st_args_zero_one_balance ... ok
test st_eip150_specific ... ok
test st_call_delegate_codes_homestead ... ok
test st_eip158_specific ... ok
test st_call_delegate_codes_call_code_homestead ... ok
test st_call_code ... ok
test st_example ... ok
test st_homestead_specific ... ok
test st_init_code ... ok
test st_ext_code_hash ... ok
test st_mem_expanding_eip_150_calls ... ok
test st_log ... ok
test st_eip2930 ... ok
test st_eip150_single_code_gas_prices ... ok
test st_non_zero_calls ... ok
test st_precompiled_contracts2 ... ignored
test st_quadratic_complexity ... ignored
test st_eip1559 ... ok
test st_memory_stress ... ok
test st_recursive_create ... ok
test st_refund ... ok
test st_return_data ... ok
test st_revert ... ignored
test st_precompiled_contracts ... ok
test st_self_balance ... ok
test st_sload ... ok
test st_solidity ... ok
test st_special ... ignored
test st_sstore ... ignored
test st_random2 ... ok
test st_static_call ... ignored
test st_shift ... ok
test st_memory ... ok
test st_transition ... ok
test st_transaction ... ok
test st_zero_calls ... ok
test st_wallet ... ok
test st_zero_calls_revert ... ok
test st_create ... ok
test st_system_operations ... ok
test st_random ... ok
test st_delegate_call_homestead ... ok
test st_call_create_call_code ... ok
test st_zero_knowledge2 ... ok
test st_attack ... ok
test st_stack ... ok
test st_bad_opcode ... ok
test st_zero_knowledge ... ok

test result: ok. 46 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 24.64s

     Running tests/vm.rs (target/debug/deps/vm-2c62990bb6eef749)

running 12 tests
test vm_performance ... ignored
test vm_other ... ok
test vm_random ... ok
test vm_block_info ... ok
test vm_system ... ok
test vm_sha3 ... ok
test vm_environmental_info ... ok
test vm_log ... ok
test vm_bitwise_logic ... ok
test vm_push_dup_swap ... ok
test vm_arithmetic ... ok
test vm_io_and_flow ... ok

test result: ok. 11 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.11s

   Doc-tests eip-152

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests ethcore-builtin

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests ethjson

running 1 test
test src/spec/authority_round.rs - spec::authority_round (line 24) ... ignored

test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests evm-jsontests

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

@sorpaas sorpaas merged commit 9b60146 into rust-blockchain:master May 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants