Test Framework / Fuzzing for ABI Package vs. Solidity Packing / Unpacking

[aaronbuchwald]

After migrating an upstream change from geth here: #750 (upstream PR here: ethereum/go-ethereum#26568) we discussed adding better testing coverage to the ABI package.

We should set up a test framework that packs/unpacks Solidity types using the ABI vs. an actual Solidity contract returning an encoded type vs. unpacking a byte array into a specific type to ensure that the behaviors match.

We should set this framework up, so that we can easily run differential fuzzing between the ABI package and a generated Solidity contract that packs / unpacks the relevant types to search for any further edge cases / discrepancies that may exist.

[ceyonur]

We have discussed it with @aaronbuchwald and we lean towards these two solutions:

Test with Web3 Libraries:

• Web3JS/Ethers libraries support encode/decode function. We can test encoding/decoding without actually deploy any contract.
• Encode Tests: A set of inputs with ABI parameter names (type: "uint8", val: 5) will be used in both Go + JS tests.
• A generator program will generate different set of inputs.
• Go tests will be done with abi package encoders. Golang tests will take params generated and encode with abi package. Results will be written to a json file.
• The same generated parameters will be used with web3js.encode function and results will be written again into another json file.
• Results will be compared for Golang vs JS tests.
• Decoding will be done in similar fashion.
• Additional step: we can create Solidity functions for generated inputs and test it out as a third test.

Pros:

• Easy to generate different set of inputs dynamically
• No need to create solidity contracts/deploy contracts (unless want to perform the additional step)
• No need to run an actual network (unless want to perform the additional step)

Cons:

• Doesn't test out actual solidity encode/decode
• Requires 2 different test environments

Call precompile contract inside solidity contract:

- We can create a solidity function for different inputs as following:
function encode(string memory _string1, uint _uint, string memory _string2) public pure returns (bytes memory) {
        result = abi.encode(_string1, _uint, _string2)
        precompileResult = precompile.encode(_string1, _uint, _string2)
        require(result == precompileResult)
    }

where the precompile contract is also generated to perform same encoding with abi packers.

• Tests can be run directly in JS/Ds-Tests (foundry).

Pros:

• Tests actual solidity encoder and a generate precompile
• Closer to real world as it uses an actual solidity contract and actual generated precompile
• Requires only 1 testing environment (JS)

Cons:

• Harder implementation to dynamically generate different set of inputs for solidity + precompile
• Not very suitable for fuzz testing
• Requires actual networks to be run, which can take longer time