Corrected abi_decode function, used it when compiling function calls and changed how we were handling the tuple/struct return

[saikat041]

What I did

I fixed the issue: #2155

How I did it

• Both internal and external function return abi-encoded data. But we were not decoding the return data. So I corrected the abi_decode function and then used it when parsing function calls
  

  vyper/vyper/codegen/abi.py

  Line 408 in 39cbd40

  def abi_decode(lll_node, src, pos=None):

• abi-decode function was not being used earlier so to pass the test cases of tuple return we were partially decoding tuples in make_setters util function. But now that we are using the abi-decode function properly, so to avoid decoding already decoded data I had to make changes here too.
  

  vyper/vyper/parser/parser_utils.py

  Line 713 in 39cbd40

  for left_arg, right_arg, loc in zipped_components:

• Changed how tuple return statements were being compiled.
  

  vyper/vyper/codegen/return_.py

  Line 67 in 39cbd40

  def gen_tuple_return(stmt, context, sub):

How to verify it

Run the following test case:

def test_struct_return_external_contract_call_2(get_contract_with_gas_estimation):
    contract_1 = """
struct X:
    x: int128
    y: String[6]
@external
def out_literals() -> X:
    return X({x: 1, y: "abcdef"})
    """

    contract_2 = """
struct X:
    x: int128
    y: String[6]
interface Test:
    def out_literals() -> X : view

@external
def test(addr: address) -> (int128, String[6]):
    ret: X = Test(addr).out_literals()
    return ret.x, ret.y

    """
    c1 = get_contract_with_gas_estimation(contract_1)
    c2 = get_contract_with_gas_estimation(contract_2)

    assert c1.out_literals() == [1, "abcdef"]
    assert c2.test(c1.address) == list(c1.out_literals())

Description for the changelog

[codecov-commenter]

Codecov Report

Merging #2164 into master will increase coverage by 0.24%.
The diff coverage is 95.23%.

@@            Coverage Diff             @@
##           master    #2164      +/-   ##
==========================================
+ Coverage   84.94%   85.18%   +0.24%     
==========================================
  Files          83       83              
  Lines        8334     8337       +3     
  Branches     2013     2012       -1     
==========================================
+ Hits         7079     7102      +23     
+ Misses        753      733      -20     
  Partials      502      502              

Impacted Files	Coverage Δ
vyper/parser/self_call.py	90.00% <88.23%> (+1.11%)	⬆️
vyper/codegen/abi.py	74.24% <100.00%> (+7.44%)	⬆️
vyper/codegen/return_.py	100.00% <100.00%> (ø)
vyper/parser/external_call.py	84.37% <100.00%> (+1.04%)	⬆️
vyper/parser/parser_utils.py	80.65% <100.00%> (-0.37%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39cbd40...69fa68e. Read the comment docs.

[lgtm-com]

This pull request introduces 3 alerts and fixes 1 when merging aacff09 into 39cbd40 - view on LGTM.com

new alerts:

• 3 for Unused import

fixed alerts:

• 1 for Variable defined multiple times

[lgtm-com]

This pull request fixes 1 alert when merging 2769955 into 39cbd40 - view on LGTM.com

fixed alerts:

• 1 for Variable defined multiple times

[fubuloubu]

@charles-cooper this mucks with your abi generator code, can you take a look?

[charles-cooper]

@saikat041 I think this is a good approach. Thanks for fixing some things and simplifying the code structure. Since this modifies the return format for some kinds of things, I think we could do with some more tests though. Would you be able to add some tests? I think, specifically, we want to test nested dynamic data. For instance,

struct X1:
  a: uint256
  x1: Bytes[10]
struct Y1:
  a: uint256
  y1: X1

This should be tested for: external call, self-call to public, self-call to private. We also might want to test bytes with len > 31 because those cases are sometimes handled differently. Maybe @fubuloubu can help with this.

Also, could you please change the title of the pull request to summarize what the PR does? Thanks again!!!

[fubuloubu]

Looks good to me, I appreciate the comments.

Please add like 10ish new cases as @charles-cooper described, then we are good to go here!

[lgtm-com]

This pull request fixes 1 alert when merging 69fa68e into 39cbd40 - view on LGTM.com

fixed alerts:

• 1 for Variable defined multiple times

[saikat041]

@fubuloubu @charles-cooper I have added more test cases for internal/external calls returning struct, But the test cases for nested structs are failing(as before). They are failing because of this function:

vyper/vyper/types/types.py

Line 267 in 39cbd40

def get_size_of_type(typ):

The corrected version should be something like this:

def get_size_of_type(typ):
    if isinstance(typ, BaseType):
        return 1
    elif isinstance(typ, ByteArrayLike):
        # 1 word for length,
        # up to maxlen words for actual data.
        return ceil32(typ.maxlen) // 32 + 1
    elif isinstance(typ, ListType):
        size = get_size_of_type(typ.subtype) * typ.count
        return size + 1 if has_dynamic_data(typ) else size
    elif isinstance(typ, MappingType):
        raise InvalidType("Maps are not supported for function arguments or outputs.")
    elif isinstance(typ, TupleLike):
        size = sum([get_size_of_type(v) for v in typ.tuple_members()])
        return size + 1 if has_dynamic_data(typ) else size
    else:
        raise InvalidType(f"Can not get size of type, Unexpected type: {repr(typ)}")

We are using the above function for calculating offsets of keys in a normal struct but there should be a different function for that. Because normal structs in memory are not stored in abi-encoded form.

vyper/vyper/parser/parser_utils.py

Line 323 in 39cbd40

offset += 32 * get_size_of_type(typ.members[attrs[i]])

This function is being called in some other places and also being tested in some individual test cases. We have to make changes in those places too if we want to change this function.

I think there should be a separate issue and pull request for changing this get_size_of_type function as this pull request is already making a lot of changes. And also nested structs were failing before this pull request.