Check access restrictions during tx execution #276

jannikluhn · 2018-01-16T13:48:52Z

What was wrong?

How was it fixed?

Fix Computation.state_db
Convert transaction.access_list to prefix list form during initialization and store the result in transaction.prefix_list.
Whenever a Computation object is available (i.e. in all opcode functions) use computation.state_db instead of computation.vm_state.state_db to access state.
In all other places use vm.state.state_db but pass transaction.prefix_list explicitly.

Cute Animal Picture

Also don't convert read/write list format in message constructor -- this should happen only once right after transaction decoding.

1. Set `reads` and `writes` properties for accessing `_reads` and `writes` 2. Modified the `exsits` and `deletes` to store the accessed key-value result in dicts

This operates in a O(n*logn) time complexity because of the sort. Closes ethereum#212

Encode integers as big endian and code as bytes, instead both as RLP

two layer trie backend -> nested trie backend (storage tries are referenced in account trie, so we have a nested structure) one layer trie backend -> flat trie backend (just one trie, no additional structure)

The methods are - account_exists - delete_account - delete_storage Those would require additional trie methods, but are likely not needed anyway, so they are explicitly NotImplemented for now.

hwwhww · 2018-01-17T14:30:43Z

@jannikluhn oh, sad. #269 also added some new test cases, just wondering if we merge #269 first, could help for reducing conflicts for this PR?

jannikluhn · 2018-01-17T14:39:59Z

Looks like the tests in #269 would run into the same problem if this PR (#276) is merged first. So I'd vote to merge #269 first, then I can fix these issues here (specifying correct access lists and applying @xfail's).

NIC619 · 2018-01-17T15:37:33Z

I will get the #269 merged as soon as tests passed.

Add create2

jannikluhn · 2018-01-18T12:03:09Z

Rebased and added access lists to the tests of #269 . Also marked them with xfail because of #281 and old gas payment style. The tests pass if the following lines are commented out:

I'll apply similar fixes to #230 once it is merged, that's why I'm setting this as work in progress again.

pipermerriam · 2018-01-18T20:47:31Z

@jannikluhn FYI

This is a common pytest pitfall. xfail is intended for tests that are unreliable, rather than tests that are required to fail. A test marked with xfail that passes doesn't cause the test suite to fail.

In cases where we have an explicit failure, I'd rather stay away from xfail and instead write the test with a with pytest.raises(TheExpectedError) in the place that it's expected to fail. Then at least when we fix the error, we'll have a clearly failing test so that we know that section needs to be updated as well.

In general, I think we should stay way from xfail entirely because it's not that useful.

jannikluhn · 2018-01-19T08:52:17Z

The pytest documentation says the following:

A common example [for xfail] is a test for a feature not yet implemented, or a bug not yet fixed.

so I think xfail is correct here. I'm not a fan of adding pytest.raises because that would change the test itself, although it is correct (it's the tested code that's broken).

I agree though that it would be nicer if the test would fail if it passes unexpectedly. xfail seems to have a strict flag which does exactly that. What do you think about keeping the xfails and adding the strict flag?

pipermerriam · 2018-01-24T21:01:18Z

@jannikluhn eating my words. Looks like xfail has been improved since I last used it and now does what we/you need. Feel free to continue using it. I think we should add the xfail_strict=true to our pytest.ini to ensure that this is always the case.

Also, I have a preference towards using raises in conjunction with reason since it's more explicit about how the test is expected to fail.

jannikluhn · 2018-01-26T17:26:51Z

Replaced by #313.

jannikluhn and others added 30 commits January 4, 2018 12:06

Add access list logic

dc353b7

Restrict state access when access lists are given

e5b3335

Use access lists in EVM if present in message

a36dca7

Fix imports and default parameter

329e324

Added StateDB

d3bdc0a

Various minor improvements

0b8bcae

Use single read/write list parameter

fb646ac

Also don't convert read/write list format in message constructor -- this should happen only once right after transaction decoding.

Access state db through computation object

484db78

Restrict access for account deletion and checks

c4df1f2

flake8

f84b4b2

Renamed StateDB to TrackedDB and move to its own module

d822f21

Updated TrackedDB

5bc34e3

1. Set `reads` and `writes` properties for accessing `_reads` and `writes` 2. Modified the `exsits` and `deletes` to store the accessed key-value result in dicts

Use single access list instead of read+write lists

fd8f257

Naming nitpick

f4a2d45

Remove redundant prefixes in prefix list form

e61db4b

This operates in a O(n*logn) time complexity because of the sort. Closes ethereum#212

convert test to use pytest.mark.parametrize

492cf46

handle edge case of inclusion of empty string

266e26f

Add backend class defining state storing logic

bb6b7c9

Add single layer state backend

e864f5f

Use one layer backend in access restriction tests

e516b71

Fix account existence check

78c6d6f

Simplify state encoding

4bee002

Encode integers as big endian and code as bytes, instead both as RLP

Add state tests

52c8471

Renamed trie backends

2d84fb9

two layer trie backend -> nested trie backend (storage tries are referenced in account trie, so we have a nested structure) one layer trie backend -> flat trie backend (just one trie, no additional structure)

Remove superfluous methods from flat trie

fb9154f

The methods are - account_exists - delete_account - delete_storage Those would require additional trie methods, but are likely not needed anyway, so they are explicitly NotImplemented for now.

Implement RLP sedes class for access lists

1967232

Renamed access list test file

03669bf

Add sharding transaction class

8bd686b

Fix address list validation

3707a1d

Fix access list tests

62e4934

NIC619 and others added 5 commits January 18, 2018 11:14

Merge pull request ethereum#269 from NIC619/add_create2

01a07cc

Add create2

Check access restrictions during tx execution

1662d85

Use FlatTrieBackend in sharding VM tests

634ba22

Provide proper access lists in sharding VM tests

214f5cb

Mark sharding VM tests as xfail

340e978

jannikluhn force-pushed the fix-access-restrictions branch from 34d6a29 to 340e978 Compare January 18, 2018 11:50

jannikluhn added PR state: WIP and removed Needs Review labels Jan 18, 2018

jannikluhn mentioned this pull request Jan 24, 2018

Avoid usage of account_exists #291

Merged

3 tasks

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Jan 24, 2018

update state_db usage in CREATE2 based on ethereum#276

708f317

hwwhww force-pushed the sharding branch from c0c1008 to 7a1db70 Compare January 25, 2018 08:23

hwwhww pushed a commit that referenced this pull request Jan 25, 2018

update state_db usage in CREATE2 based on #276

b279b9f

jannikluhn mentioned this pull request Jan 26, 2018

Fix transaction execution #313

Merged

jannikluhn closed this Jan 26, 2018

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Feb 1, 2018

update state_db usage in CREATE2 based on ethereum#276

d26813f

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Feb 2, 2018

update state_db usage in CREATE2 based on ethereum#276

141ee85

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Feb 6, 2018

update state_db usage in CREATE2 based on ethereum#276

5f6641f

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Feb 18, 2018

update state_db usage in CREATE2 based on ethereum#276

d8faa62

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Feb 22, 2018

update state_db usage in CREATE2 based on ethereum#276

e2d9bc0

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Feb 23, 2018

update state_db usage in CREATE2 based on ethereum#276

60b91d7

hwwhww pushed a commit to hwwhww/py-evm that referenced this pull request Mar 2, 2018

update state_db usage in CREATE2 based on ethereum#276

7d0ab08

hwwhww pushed a commit that referenced this pull request Mar 2, 2018

update state_db usage in CREATE2 based on #276

d18ebb8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check access restrictions during tx execution #276

Check access restrictions during tx execution #276

jannikluhn commented Jan 16, 2018 •

edited

Loading

hwwhww commented Jan 17, 2018

jannikluhn commented Jan 17, 2018

NIC619 commented Jan 17, 2018

jannikluhn commented Jan 18, 2018

pipermerriam commented Jan 18, 2018

jannikluhn commented Jan 19, 2018

pipermerriam commented Jan 24, 2018

jannikluhn commented Jan 26, 2018

Check access restrictions during tx execution #276

Check access restrictions during tx execution #276

Conversation

jannikluhn commented Jan 16, 2018 • edited Loading

What was wrong?

How was it fixed?

Cute Animal Picture

hwwhww commented Jan 17, 2018

jannikluhn commented Jan 17, 2018

NIC619 commented Jan 17, 2018

jannikluhn commented Jan 18, 2018

pipermerriam commented Jan 18, 2018

jannikluhn commented Jan 19, 2018

pipermerriam commented Jan 24, 2018

jannikluhn commented Jan 26, 2018

jannikluhn commented Jan 16, 2018 •

edited

Loading