refactor: make RuntimeExt: Send #11634
Conversation
In a world where we have pipelined compilation, instantiation and execution, `VMLogic` will have to move between threads, which requires that it becomes `Send`. It in turn has required some other types to become not only `Send` but also `Sync` due to them currently being stored as a `&` reference (which allows for multiple copies, there are better places to explain why `Sync` becomes necessary here...) I think the changes here are largely straightforward enough, but overall things are shaping to be pretty involved, eh?
There was a problem hiding this comment.
Changing ownership for TrieUpdate looks challenging. I didn't fully understand the plan though. Before execution it is not needed; if execution for the same shard will happen in parallel, then one must add support of parallel read-write access to prospective and committed changes.
|
I don't anticipate parallel writes to In fact ideally we wouldn't deal with |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #11634 +/- ##
==========================================
+ Coverage 71.46% 71.64% +0.17%
==========================================
Files 788 787 -1
Lines 160921 160945 +24
Branches 160921 160945 +24
==========================================
+ Hits 115008 115308 +300
+ Misses 40893 40594 -299
- Partials 5020 5043 +23
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
| @@ -307,7 +306,7 @@ pub trait TrieStorage { | |||
| #[derive(Default)] | |||
| pub struct TrieMemoryPartialStorage { | |||
| pub(crate) recorded_storage: HashMap<CryptoHash, Arc<[u8]>>, | |||
| pub(crate) visited_nodes: RefCell<HashSet<CryptoHash>>, | |||
| pub(crate) visited_nodes: std::sync::RwLock<HashSet<CryptoHash>>, | |||
There was a problem hiding this comment.
My gut feeling would have been Mutex to be a better match here, as it’s probably write-many-read-once. But I didn’t check all the uses and it’s likely unrelevant optimization anyway, so I’ll just leave that as an informational message if you want to think more about it.
There was a problem hiding this comment.
My reasoning here was that RwLock is a direct match to RefCell with regards to its API surface (borrow => read/borrow_mut => write.) We may indeed be able to revert this eventually; we'll see.
In a world where we have pipelined compilation, instantiation and execution,
VMLogicwill have to move between threads, which requires that it becomesSend. It in turn has required some other types to become not onlySendbut alsoSyncdue to them currently being stored as a&reference (which allows for multiple copies, there are better places to explain whySyncbecomes necessary here...)I'm not sure if all of these types will continue requiring
Sync. In particularTrieUpdatethat's stored inRuntimeExtis now by reference, but I eventually want to also makeVMLogic: 'static, which would require finding some owning pointer structure that would work forTrieUpdate... Or I might be able to use scoped threads... in which case we're looking atSyncanyway...I think the changes here are largely straightforward enough, but overall things are shaping to be pretty involved, eh?
Part of #11319