-
Notifications
You must be signed in to change notification settings - Fork 638
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: make RuntimeExt: Send #11634
Conversation
In a world where we have pipelined compilation, instantiation and execution, `VMLogic` will have to move between threads, which requires that it becomes `Send`. It in turn has required some other types to become not only `Send` but also `Sync` due to them currently being stored as a `&` reference (which allows for multiple copies, there are better places to explain why `Sync` becomes necessary here...) I think the changes here are largely straightforward enough, but overall things are shaping to be pretty involved, eh?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changing ownership for TrieUpdate
looks challenging. I didn't fully understand the plan though. Before execution it is not needed; if execution for the same shard will happen in parallel, then one must add support of parallel read-write access to prospective
and committed
changes.
I don't anticipate parallel writes to In fact ideally we wouldn't deal with |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #11634 +/- ##
==========================================
+ Coverage 71.46% 71.64% +0.17%
==========================================
Files 788 787 -1
Lines 160921 160945 +24
Branches 160921 160945 +24
==========================================
+ Hits 115008 115308 +300
+ Misses 40893 40594 -299
- Partials 5020 5043 +23
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seeing the new locks makes me a bit sad, but we can come back to them later and see whether there’s a chance to remove them when we have more time :)
@@ -307,7 +306,7 @@ pub trait TrieStorage { | |||
#[derive(Default)] | |||
pub struct TrieMemoryPartialStorage { | |||
pub(crate) recorded_storage: HashMap<CryptoHash, Arc<[u8]>>, | |||
pub(crate) visited_nodes: RefCell<HashSet<CryptoHash>>, | |||
pub(crate) visited_nodes: std::sync::RwLock<HashSet<CryptoHash>>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My gut feeling would have been Mutex
to be a better match here, as it’s probably write-many-read-once. But I didn’t check all the uses and it’s likely unrelevant optimization anyway, so I’ll just leave that as an informational message if you want to think more about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My reasoning here was that RwLock
is a direct match to RefCell
with regards to its API surface (borrow
=> read
/borrow_mut
=> write
.) We may indeed be able to revert this eventually; we'll see.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SGTM :)
In a world where we have pipelined compilation, instantiation and execution,
VMLogic
will have to move between threads, which requires that it becomesSend
. It in turn has required some other types to become not onlySend
but alsoSync
due to them currently being stored as a&
reference (which allows for multiple copies, there are better places to explain whySync
becomes necessary here...)I'm not sure if all of these types will continue requiring
Sync
. In particularTrieUpdate
that's stored inRuntimeExt
is now by reference, but I eventually want to also makeVMLogic: 'static
, which would require finding some owning pointer structure that would work forTrieUpdate
... Or I might be able to use scoped threads... in which case we're looking atSync
anyway...I think the changes here are largely straightforward enough, but overall things are shaping to be pretty involved, eh?
Part of #11319