Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New DALC API #341

Merged
merged 14 commits into from
Apr 10, 2022
Merged

New DALC API #341

merged 14 commits into from
Apr 10, 2022

Conversation

tzdybal
Copy link
Member

@tzdybal tzdybal commented Mar 25, 2022

This PR introduces new DALC API into optimint.

Most important changes:

  • BlockManager needs to keep track of DA layer height
  • single DA layer block can contain multiple Optimint blocks
  • sync is now more complicated - but I tried to keep it as simple as possible

Actually, there's no need to explicitly map DA layer height to Optimint height.

Resolves #281.

@codecov-commenter
Copy link

codecov-commenter commented Mar 25, 2022

Codecov Report

Merging #341 (7dae373) into main (404e426) will increase coverage by 0.50%.
The diff coverage is 77.81%.

@@            Coverage Diff             @@
##             main     #341      +/-   ##
==========================================
+ Coverage   58.61%   59.12%   +0.50%     
==========================================
  Files          44       44              
  Lines        7092     7180      +88     
==========================================
+ Hits         4157     4245      +88     
- Misses       2363     2364       +1     
+ Partials      572      571       -1     
Impacted Files Coverage Δ
config/config.go 84.21% <ø> (ø)
types/pb/optimint/optimint.pb.go 43.99% <ø> (ø)
types/pb/dalc/dalc.pb.go 45.18% <62.26%> (+0.36%) ⬆️
da/grpc/mockserv/mockserv.go 82.35% <83.33%> (+1.50%) ⬆️
da/grpc/grpc.go 73.13% <84.00%> (+5.27%) ⬆️
da/mock/mock.go 79.54% <84.21%> (+13.97%) ⬆️
block/manager.go 64.11% <91.37%> (+4.58%) ⬆️
log/test/loggers.go 84.61% <100.00%> (+3.66%) ⬆️
state/state.go 92.10% <100.00%> (+0.43%) ⬆️
store/store.go 68.05% <100.00%> (-1.49%) ⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 404e426...7dae373. Read the comment docs.

block/manager.go Outdated Show resolved Hide resolved
block/manager.go Show resolved Hide resolved
block/manager.go Outdated Show resolved Hide resolved
block/manager.go Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
Copy link
Contributor

@jbowen93 jbowen93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM. I've run the simple ethermint RPC tests against the CI built evmos image using the ephemeral cluster and they pass.

I'm confused how the tests are passing since the ephemeral cluster setup using an older DALC image which shouldn't have a compatible API...

block/manager.go Outdated Show resolved Hide resolved
block/manager.go Outdated Show resolved Hide resolved
@tzdybal
Copy link
Member Author

tzdybal commented Mar 29, 2022

I'm confused how the tests are passing since the ephemeral cluster setup using an older DALC image which shouldn't have a compatible API...

IIRC in ephemeral cluster we only have single optimint node, which is an aggregator. It shouldn't sync, so it's not using changed API. Only change in part related to block submission was adding single field in DAResult - this will just have zero-value, and this shouldn't be crucial for Optimint operation.

@tzdybal
Copy link
Member Author

tzdybal commented Apr 7, 2022

Pinging all approvors (@liamsi @jbowen93 @adlerjohn @evan-forbes) ;) PR is updated. All comments addressed.

This mock DA implementation reflects all new requirements:
 - use new DA API (DA height oriented)
 - store multiple Optimint block in single DA block (at same height)
 - optimint blocks may be located in non-consecutive DA blocks

New mock requires Optimint to implement block height mapping, improved
sync logic. Many unit tests are going to fail now. Implementing
mentioned features is required to fix the tests.
This time mock mimicks actual DA layer completely. It "creates blocks"
at given intervals. Block height can be incremented by more than 1.
With new DALC API, syncing logic has to be changed.
Block manager has to keep track of DA height (it's also saved in state).
m.logger.Error("failed to retrieve block from DALC", "daHeight", daHeight, "errors", err.Error())
break
}
atomic.AddUint64(&m.daHeight, 1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, are you sure this is the right use of atomics?

What if &m.daHeight changes in between loading it and adding 1 to it here? Then you'd be adding 2 to it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only place where m.daHeight is changed. It's atomic change, because we're reading this variable in several other places (in other goroutines). TBH the LoadUint64 in the beginning of the loop is not necessary (normal read should be fine), but I decided to handle this variable "atomically" everywhere for consistency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is if e.g. &m.daHeight is N, then it could be read as N in two separate threads and incremented twice to N+2, completely bypassing N+1. Is that ever possible, if is so, is it a problem?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not possible right now, as there is only one thread incrementing this variable, and I don't see a reason to add more threads/goroutines.

block/manager.go Outdated Show resolved Hide resolved
block/manager.go Show resolved Hide resolved
da/da.go Outdated Show resolved Hide resolved
@tzdybal tzdybal requested review from adlerjohn and liamsi April 8, 2022 08:38
Copy link
Member

@adlerjohn adlerjohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not going to block on #341 (comment), but if it's an issue can you open an PR to further investigate.

@tzdybal tzdybal merged commit cf71b4d into main Apr 10, 2022
@tzdybal tzdybal deleted the tzdybal/new_dalc_api branch April 10, 2022 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DA API Improvements
5 participants