Memory scoping #2197
Memory scoping #2197
Conversation
|
This pull request introduces 2 alerts when merging 1ba75c5 into 192d991 - view on LGTM.com new alerts:
|
|
p.s. is there a way to use an infinite generator without lgtm complaining? |
iamdefinitelyahuman
force-pushed
the
feat-mem-scoping
branch
from
1ba75c5
to
40b23d5
Compare
Codecov Report
@@ Coverage Diff @@
## master #2197 +/- ##
==========================================
- Coverage 85.47% 85.45% -0.03%
==========================================
Files 83 83
Lines 8503 8511 +8
Branches 2053 2057 +4
==========================================
+ Hits 7268 7273 +5
- Misses 733 735 +2
- Partials 502 503 +1
Continue to review full report at Codecov.
|
|
This pull request introduces 2 alerts when merging 40b23d5 into 192d991 - view on LGTM.com new alerts:
|
iamdefinitelyahuman
force-pushed
the
feat-mem-scoping
branch
from
40b23d5
to
d31231c
Compare
Codecov Report
@@ Coverage Diff @@
## master #2197 +/- ##
==========================================
- Coverage 85.47% 85.45% -0.03%
==========================================
Files 83 83
Lines 8503 8511 +8
Branches 2053 2057 +4
==========================================
+ Hits 7268 7273 +5
- Misses 733 735 +2
- Partials 502 503 +1
Continue to review full report at Codecov.
|
iamdefinitelyahuman
force-pushed
the
feat-mem-scoping
branch
from
d31231c
to
10a5211
Compare
Codecov Report
@@ Coverage Diff @@
## master #2197 +/- ##
==========================================
- Coverage 85.47% 85.30% -0.18%
==========================================
Files 83 83
Lines 8503 8524 +21
Branches 2053 2061 +8
==========================================
+ Hits 7268 7271 +3
- Misses 733 747 +14
- Partials 502 506 +4
Continue to review full report at Codecov.
|
|
This pull request introduces 2 alerts when merging 10a5211 into 192d991 - view on LGTM.com new alerts:
|
iamdefinitelyahuman
force-pushed
the
feat-mem-scoping
branch
from
10a5211
to
12b388a
Compare
|
This pull request introduces 2 alerts when merging 12b388a into 192d991 - view on LGTM.com new alerts:
|
|
This pull request introduces 2 alerts when merging 380cac7 into 192d991 - view on LGTM.com new alerts:
|
|
Shut the hell up lgtm |
| int | ||
| Position of the newly allocated memory | ||
| """ | ||
| if size > self.size: |
There was a problem hiding this comment.
Good catch!
| self.position = position | ||
| self.size = size | ||
|
|
||
| def __repr__(self): | ||
| return f"(FreeMemory: pos={self.position}, size={self.size})" | ||
|
|
||
| def partially_allocate(self, size: int) -> int: |
There was a problem hiding this comment.
should this be a private function?
| @@ -124,3 +154,8 @@ def deallocate_memory(self, pos: int, size: int) -> None: | |||
| else: | |||
| active = next_slot | |||
| i += 1 | |||
|
|
|||
| last = self.deallocated_mem[-1] | |||
There was a problem hiding this comment.
could add a comment here
|
|
||
| if not self.deallocated_mem or self.deallocated_mem[-1].position < pos: | ||
| elif not self.deallocated_mem or self.deallocated_mem[-1].position < pos: | ||
| # no previously deallocated memory, or this is the highest position deallocated | ||
| self.deallocated_mem.append(FreeMemory(position=pos, size=size)) | ||
| else: |
There was a problem hiding this comment.
i feel like instead of maintaining the sort invariant ourselves, we could just make a call to list.sort().
| before_value = self.next_mem | ||
| self.next_mem += size | ||
| self.size_of_mem = max(self.size_of_mem, self.next_mem) | ||
| return before_value | ||
|
|
||
| def deallocate_memory(self, pos: int, size: int) -> None: |
There was a problem hiding this comment.
how about def free()? :)
There was a problem hiding this comment.
actually one change i've wanted to make for a long time is to include the units in the name. so, def free_bytes(). makes it easier for the API user to know what is being freed without reading the allocator source.
| @@ -101,9 +129,8 @@ def deallocate_memory(self, pos: int, size: int) -> None: | |||
| # releasing from the end of the allocated memory - reduce the free memory pointer | |||
| if pos + size == self.next_mem: | |||
There was a problem hiding this comment.
i had slight confusion reading this because it overlaps with the final clause in this function. i think this function would read better if we removed this branch.
There was a problem hiding this comment.
Yeah, all of this logic can be cleaned up quite a bit.
| @@ -59,6 +61,6 @@ def parse_function(code, sigs, origcode, global_ctx, is_contract_payable, _vars= | |||
| ) | |||
|
|
|||
| o.context = context | |||
| o.total_gas = o.gas + calc_mem_gas(o.context.memory_allocator.get_next_memory_position()) | |||
| o.total_gas = o.gas + calc_mem_gas(o.context.memory_allocator.size_of_mem) | |||
There was a problem hiding this comment.
this should be part of the allocator API - allocator.memsize().
| Internal memory scope context manager. | ||
|
|
||
| Internal variables that are declared within this context are de-allocated | ||
| upon exitting the context. |
| def _new_variable(self, name: str, typ: NodeType, var_pos: int) -> int: | ||
| def _new_variable(self, name: str, typ: NodeType, var_size: int, is_internal: bool) -> int: | ||
| if is_internal: | ||
| var_pos = self.memory_allocator.expand_memory(var_size) |
There was a problem hiding this comment.
why do internal variables get a different allocation strategy?
There was a problem hiding this comment.
There are places in parser where an assumption is made that internal variables will be allocated sequentially within memory. If we don't do this, we end up with memory corruption. I tried to unravel / refactor that logic but eventually decided it wasn't safe - I can't be sure I've found them all.
By always allocating new internal variables on the edge of memory like this, we don't risk memory corruption but still gain some efficiency by not expanding memory needlessly. In the future when we refactor parser we should eliminate the assumption about how memory is sequenced, and then we can do away with this logic.
| pos=var_pos, | ||
| typ=typ, | ||
| mutable=True, | ||
| blockscopes=self._scopes.copy(), |
There was a problem hiding this comment.
i'm not sure why a variable needs to have pointers to all parent scopes. this seems like it should just be a single scope_id.
There was a problem hiding this comment.
Good point. I just copied this from existing code without thinking 😬
What I did
Deallocate internal memory variables between statements.
How I did it
Upon exiting
internal_memory_scope, all internal variables declared within the scope are deallocated. The scope is applied prior to parsing each statement node.How to verify it
Run tests.
Cute Animal Picture