Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: immutable variables #2466

Merged
merged 56 commits into from
Nov 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
f4f3ea8
chore: replace `is_immutable` -> `is_constant`
skellet0r Sep 30, 2021
b06886e
feat: add `immutable` as a reserved keyword
skellet0r Sep 30, 2021
d967cf2
feat: handle immutable defs in ModuleNodeVisitor
skellet0r Sep 30, 2021
9a407b6
fix: add `is_immutable` kwarg to get_type_from_annotation
skellet0r Sep 30, 2021
ba27726
feat: add `is_immutable` keyword to primitive classes
skellet0r Sep 30, 2021
a246ca4
feat: set positions of immutable vars
skellet0r Sep 30, 2021
62d06e3
fix: parse_type of immutable var
skellet0r Sep 30, 2021
859c935
fix: prevent immutable modification outside constructor
skellet0r Sep 30, 2021
2f0e78b
WIP: add branch for immutable vars in expr.py
skellet0r Sep 30, 2021
e980523
fix: DataPosition class ImmutableSlot -> CodeOffset
skellet0r Oct 2, 2021
9645b1b
fix: runtime loading of immutables
skellet0r Oct 2, 2021
66c225c
feat: append to init lll storage of immutables
skellet0r Oct 3, 2021
a975a6c
fix: update VariableRecord class + add immutables to global ctx
skellet0r Oct 3, 2021
538d31e
fix: modify constructor to return runtime + immutables
skellet0r Oct 3, 2021
da098cd
fix: constructor handles data section
skellet0r Oct 3, 2021
12fe4d1
fix: account for immutables at end of runtime code
skellet0r Oct 3, 2021
5b50471
fix: vyper grammar include immutable_def
skellet0r Oct 3, 2021
ac55173
test: simple usage of immutable keyword with uint256
skellet0r Oct 3, 2021
b9542dc
fix: remove copying of immutables from external_function.py
skellet0r Oct 3, 2021
0f0bc1b
fix: add `is_immutable` kwarg to more Definition classes
skellet0r Oct 3, 2021
7dbe7c3
fix: raise syntax exception if immutable not assigned a value
skellet0r Oct 3, 2021
e63cdf1
test: immutable syntax, simple cases
skellet0r Oct 3, 2021
cd2cb98
test: accessing stored immutable
skellet0r Oct 3, 2021
c0eeeaf
test: verify immutables of dynamic length are disallowed
skellet0r Oct 3, 2021
6a3c946
fix: disallow bytes/string immutables (momentarily)
skellet0r Oct 3, 2021
b6c9279
fix: use make_setter for memory cp operation of immutables
skellet0r Oct 4, 2021
02dd076
fix: store memory loc + offset in data section in metadata
skellet0r Oct 4, 2021
2118af0
fix: use data offset + memory loc from metadata section
skellet0r Oct 4, 2021
9f144a0
fix: remove restriction using strings/bytes immutables
skellet0r Oct 4, 2021
11dddb4
fix(test): verify usage of string/bytes immutables
skellet0r Oct 4, 2021
2e32119
fix: only set _metadata on immutable during first pass
skellet0r Oct 4, 2021
d05d987
fix: return size of runtime code to account for data section
skellet0r Oct 4, 2021
e1a9508
chore: fix test parametrization
skellet0r Oct 4, 2021
15dd832
test: multiple immutable values
skellet0r Oct 4, 2021
86946bf
fix(test): change dummy address used
skellet0r Oct 4, 2021
e594099
fix: allocate a new var not internal var
skellet0r Oct 4, 2021
77c4c01
test: user defined struct immutable
skellet0r Oct 4, 2021
d358c07
fix: allow immutable sequences (add kwarg to classes)
skellet0r Oct 4, 2021
f1b6d6d
test: immutable list usage
skellet0r Oct 4, 2021
ec13acf
chore: modify test parametrization
skellet0r Oct 4, 2021
b95bef8
docs: add immutable usage docs
skellet0r Oct 4, 2021
14b2edb
fix: allocate memory of immutable in constructor
skellet0r Oct 4, 2021
10c73e4
fix: disallow multiple assignments to immutable
skellet0r Oct 4, 2021
42981ab
test: multiple assignments blocked
skellet0r Oct 4, 2021
3e4de6e
fix: set immutable data location to CODE
skellet0r Oct 13, 2021
48d4691
fix: verify immutable is given a single argument
skellet0r Oct 13, 2021
b0fe010
fix: if stmt use bool type as condition
skellet0r Oct 20, 2021
b60ae2e
fix: make immutable data size a cached prop on global ctx
skellet0r Oct 20, 2021
9d04be1
fix: swap 'lll' arguments, offset is first code is second
skellet0r Oct 21, 2021
99217d9
fix: remove _metadata dict on LLLnode for immutables
skellet0r Oct 21, 2021
7c01275
fix: mypy typing error
skellet0r Nov 12, 2021
b55dfe1
Update vyper/semantics/types/indexable/sequence.py
skellet0r Nov 12, 2021
502bb6a
Update vyper/semantics/types/bases.py
skellet0r Nov 12, 2021
90c8594
fix: use cached property from vyper.utils
skellet0r Nov 12, 2021
713a7bd
chore: leave todo, come back and resolve immutable offsets at compile…
skellet0r Nov 12, 2021
9bd3b45
update a comment about lll macro
charles-cooper Nov 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/scoping-and-declarations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,23 @@ The compiler automatically creates getter functions for all public storage varia

For public arrays, you can only retrieve a single element via the generated getter. This mechanism exists to avoid high gas costs when returning an entire array. The getter will accept an argument to specity which element to return, for example ``data(0)``.

Declaring Immutable Variables
--------------------------

Variables can be marked as ``immutable`` during declaration:

.. code-block:: python

DATA: immutable(uint256)

@external
def __init__(_data: uint256):
DATA = _data

Variables declared as immutable are similar to constants, except they are assigned a value in the constructor of the contract. Immutable values must be assigned a value at construction and cannot be assigned a value after construction.

The contract creation code generated by the compiler will modify the contract’s runtime code before it is returned by appending all values assigned to immutables to the runtime code returned by the constructor. This is important if you are comparing the runtime code generated by the compiler with the one actually stored in the blockchain.

Tuple Assignment
----------------

Expand Down
3 changes: 2 additions & 1 deletion tests/compiler/LLL/test_compile_lll.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,13 @@ def test_lll_from_s_expression(get_contract_from_lll):
(return
0
(lll ; just return 32 byte of calldata back
0
(seq
(calldatacopy 0 4 32)
(return 0 32)
stop
)
0)))
)))
"""
abi = [
{
Expand Down
5 changes: 5 additions & 0 deletions tests/grammar/vyper.lark
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ module: ( DOCSTRING
| variable_def
| event_def
| function_def
| immutable_def
| _NEWLINE )*


Expand All @@ -36,6 +37,10 @@ import: _IMPORT DOT* _import_path [import_alias]
// NOTE: Temporary until decorators used
constant_def: NAME ":" "constant" "(" type ")" "=" _expr

// immutable definitions
// NOTE: Temporary until decorators used
immutable_def: NAME ":" "immutable" "(" type ")"

variable: NAME ":" type
// NOTE: Temporary until decorators used
variable_with_getter: NAME ":" "public" "(" type ")"
Expand Down
100 changes: 100 additions & 0 deletions tests/parser/features/test_immutable.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
import pytest

@pytest.mark.parametrize(
"typ,value",
[
("uint256", 42),
("int256", -(2 ** 200)),
("int128", -(2 ** 126)),
("address", "0xEeeeeEeeeEeEeeEeEeEeeEEEeeeeEeeeeeeeEEeE"),
("bytes32", b"deadbeef" * 4),
("bool", True),
("String[10]", "Vyper hiss"),
("Bytes[10]", b"Vyper hiss"),
],
)
def test_value_storage_retrieval(typ, value, get_contract):
code = f"""
VALUE: immutable({typ})

@external
def __init__(_value: {typ}):
VALUE = _value

@view
@external
def get_value() -> {typ}:
return VALUE
"""

c = get_contract(code, value)
assert c.get_value() == value


def test_multiple_immutable_values(get_contract):
code = """
a: immutable(uint256)
b: immutable(address)
c: immutable(String[64])

@external
def __init__(_a: uint256, _b: address, _c: String[64]):
a = _a
b = _b
c = _c

@view
@external
def get_values() -> (uint256, address, String[64]):
return a, b, c
"""
values = (3, "0xEeeeeEeeeEeEeeEeEeEeeEEEeeeeEeeeeeeeEEeE", "Hello world")
c = get_contract(code, *values)
assert c.get_values() == list(values)


def test_struct_immutable(get_contract):
code = """
struct MyStruct:
a: uint256
b: uint256
c: address
d: int256

my_struct: immutable(MyStruct)

@external
def __init__(_a: uint256, _b: uint256, _c: address, _d: int256):
my_struct = MyStruct({
a: _a,
b: _b,
c: _c,
d: _d
})

@view
@external
def get_my_struct() -> MyStruct:
return my_struct
"""
values = (100, 42, "0xEeeeeEeeeEeEeeEeEeEeeEEEeeeeEeeeeeeeEEeE", -(2 ** 200))
c = get_contract(code, *values)
assert c.get_my_struct() == values


def test_list_immutable(get_contract):
code = """
my_list: immutable(uint256[3])

@external
def __init__(_a: uint256, _b: uint256, _c: uint256):
my_list = [_a, _b, _c]

@view
@external
def get_my_list() -> uint256[3]:
return my_list
"""
values = (100, 42, 23230)
c = get_contract(code, *values)
assert c.get_my_list() == list(values)
108 changes: 108 additions & 0 deletions tests/parser/syntax/test_immutables.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
import pytest

from vyper import compile_code

fail_list = [
# VALUE is not set in the constructor
"""
VALUE: immutable(uint256)

@external
def __init__():
pass
""",
# no `__init__` function, VALUE not set
"""
VALUE: immutable(uint256)

@view
@external
def get_value() -> uint256:
return VALUE
""",
# VALUE given an initial value
"""
VALUE: immutable(uint256) = 3

@external
def __init__():
pass
""",
# setting value outside of constructor
"""
VALUE: immutable(uint256)

@external
def __init__():
VALUE = 0

@external
def set_value(_value: uint256):
VALUE = _value
""",
# modifying immutable multiple times in constructor
"""
VALUE: immutable(uint256)

@external
def __init__(_value: uint256):
VALUE = _value * 3
VALUE = VALUE + 1
"""

]


@pytest.mark.parametrize("bad_code", fail_list)
def test_compilation_fails_with_exception(bad_code):
with pytest.raises(Exception):
compile_code(bad_code)


types_list = (
"uint256",
"int256",
"int128",
"address",
"bytes32",
"decimal",
"bool",
"Bytes[64]",
"String[10]",
)


@pytest.mark.parametrize("typ", types_list)
def test_compilation_simple_usage(typ):
code = f"""
VALUE: immutable({typ})

@external
def __init__(_value: {typ}):
VALUE = _value

@view
@external
def get_value() -> {typ}:
return VALUE
"""

assert compile_code(code)


pass_list = [
# using immutable allowed in constructor
"""
VALUE: immutable(uint256)

@external
def __init__(_value: uint256):
VALUE = _value * 3
x: uint256 = VALUE + 1
"""
]


@pytest.mark.parametrize("good_code", pass_list)
def test_compilation_success(good_code):
assert compile_code(good_code)
7 changes: 6 additions & 1 deletion vyper/ast/signatures/function_signature.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import math
from dataclasses import dataclass
from typing import Optional

from vyper import ast as vy_ast
from vyper.exceptions import StructureException
Expand All @@ -12,7 +13,7 @@
# TODO move to context.py
# TODO use dataclass
class VariableRecord:
def __init__(
def __init__( # type: ignore
self,
name,
pos,
Expand All @@ -23,6 +24,8 @@ def __init__(
blockscopes=None,
defined_at=None,
is_internal=False,
is_immutable=False,
data_offset: Optional[int] = None,
):
self.name = name
self.pos = pos
Expand All @@ -33,6 +36,8 @@ def __init__(
self.blockscopes = [] if blockscopes is None else blockscopes
self.defined_at = defined_at # source code location variable record was defined.
self.is_internal = is_internal
self.is_immutable = is_immutable
self.data_offset = data_offset # location in data section

def __repr__(self):
ret = vars(self)
Expand Down
2 changes: 1 addition & 1 deletion vyper/compiler/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ def build_gas_estimates(lll_nodes: LLLnode) -> dict:
and len(lll_nodes.args) > 0
and lll_nodes.args[-1].value == "return"
):
lll_nodes = lll_nodes.args[-1].args[1].args[0]
lll_nodes = lll_nodes.args[-1].args[1].args[1]

external_sub = next((i for i in lll_nodes.args if i.value == "with"), None)
if external_sub:
Expand Down
4 changes: 2 additions & 2 deletions vyper/lll/compile_lll.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@ def _compile_to_assembly(code, withargs=None, existing_labels=None, break_dest=N
endcode = mksymbol()
o.extend([endcode, "JUMP", begincode, "BLANK"])

lll = _compile_to_assembly(code.args[0], {}, existing_labels, None, 0)
lll = _compile_to_assembly(code.args[1], {}, existing_labels, None, 0)

# `append(...)` call here is intentional.
# each sublist is essentially its own program with its
Expand All @@ -303,7 +303,7 @@ def _compile_to_assembly(code, withargs=None, existing_labels=None, break_dest=N
o.append(lll)

o.extend([endcode, "JUMPDEST", begincode, endcode, "SUB", begincode])
o.extend(_compile_to_assembly(code.args[1], withargs, existing_labels, break_dest, height))
o.extend(_compile_to_assembly(code.args[0], withargs, existing_labels, break_dest, height))

# COPY the code to memory for deploy
o.extend(["CODECOPY", begincode, endcode, "SUB"])
Expand Down
33 changes: 33 additions & 0 deletions vyper/old_codegen/expr.py
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,39 @@ def parse_Name(self):
return LLLnode.from_list(
[obj], typ=BaseType(typ, is_literal=True), pos=getpos(self.expr)
)
elif self.expr._metadata["type"].is_immutable:
# immutable variable
# need to handle constructor and outside constructor
var = self.context.globals[self.expr.id]
is_constructor = self.expr.get_ancestor(vy_ast.FunctionDef).get("name") == "__init__"
if is_constructor:
# store memory position for later access in parser.py in the variable record
memory_loc = self.context.new_variable(f"#immutable_{self.expr.id}", var.typ)
self.context.global_ctx._globals[self.expr.id].pos = memory_loc
# store the data offset in the variable record as well for accessing
data_offset = self.expr._metadata["type"].position.offset
self.context.global_ctx._globals[self.expr.id].data_offset = data_offset

return LLLnode.from_list(
memory_loc,
typ=var.typ,
location="memory",
pos=getpos(self.expr),
annotation=self.expr.id,
mutable=True,
)
else:
immutable_section_size = self.context.global_ctx.immutable_section_size
offset = self.expr._metadata["type"].position.offset
# TODO: resolve code offsets for immutables at compile time
return LLLnode.from_list(
["sub", "codesize", immutable_section_size - offset],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting. I wonder if we can get the location of the data section into the runtime code during deploy time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Thinking through the approach, it may not be worth it, but if you have some clever ideas here I am all ears.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an idea: our assembly-to-evm compilation phase has a step which resolves symbols to locations. Since the locations of immutables are statically known (no matter what value they are set to), we can insert symbols into the LLL which directly precede blank space. Basically, _sym_foo doesn't actually take any space in the code. So the runtime data section itself would have something like

_sym_datasection_start
BLANK
_sym_immutable_uint256
JUMPDEST... <32 JUMPDESTs>
_sym_immutable_bytes6
JUMPDEST... <32+6 jumpdests>

Then your code for copying items to LLL would stay the same (it would just overwrite all the jumpdests), but we would know the locations statically.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK let's add a TODO here for now: TODO: resolve code offsets for immutables at compile time

typ=var.typ,
location="code",
pos=getpos(self.expr),
annotation=self.expr.id,
mutable=False,
)

# x.y or x[5]
def parse_Attribute(self):
Expand Down
Loading