Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert shim-shared-memory to Rust #2462

Merged
merged 11 commits into from
Oct 17, 2022
Merged

Conversation

sporksmith
Copy link
Contributor

@sporksmith sporksmith commented Oct 12, 2022

  • Import rootedcell from https://github.com/sporksmith/objgraph/. This is a safe abstraction around the thread safety model used in Shadow, where access to a single Root object (which may be protected at a coarse level, such as by a per-Host lock) is used to prove that non-atomic operations are safe for "child" objects. This is the model used by the current shim shared memory API.

  • Implement the VirtualAddressSpaceIndependent trait and Derive-macro. The former marks types as usable outside of their original virtual address space. The corresponding Derive macro is a safer way of implementing the trait for more-complex structs, by validating that all fields of that struct are VirtualAddressSpaceIndependent.

  • Convert the shim shared memory structs to Rust. For now this is a fairly direct translation into Rust, using the above to help ensure safety. We'll probably want more safe Rust APIs for accessing these, but for now the focus was just on reimplementing the C APIs.

@github-actions github-actions bot added Component: Build Build/install tools and dependencies Component: Libraries Support functions like LD_PRELOAD and logging Component: Main Composing the core Shadow executable labels Oct 12, 2022
This was referenced Oct 12, 2022
@sporksmith sporksmith changed the title Shmem v2 Convert shim-shared-memory to Rust Oct 12, 2022
@sporksmith sporksmith force-pushed the shmem-v2 branch 2 times, most recently from 3802858 to c847455 Compare October 12, 2022 19:52
@codecov
Copy link

codecov bot commented Oct 12, 2022

Codecov Report

Base: 42.48% // Head: 43.39% // Increases project coverage by +0.91% 🎉

Coverage data is based on head (21e3f01) compared to base (5442bfb).
Patch coverage: 85.09% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2462      +/-   ##
==========================================
+ Coverage   42.48%   43.39%   +0.91%     
==========================================
  Files         180      185       +5     
  Lines       26710    27246     +536     
  Branches     5379     5414      +35     
==========================================
+ Hits        11348    11824     +476     
- Misses      12736    12762      +26     
- Partials     2626     2660      +34     
Flag Coverage Δ
tests 43.39% <85.09%> (+0.91%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/lib/shadow-shim-helper-rs/src/scmutex.rs 86.74% <ø> (-0.61%) ⬇️
src/lib/shim/shim_rdtsc.c 62.71% <ø> (ø)
src/lib/shim/shim_signals.c 67.96% <ø> (ø)
src/lib/shim/shim_syscall.c 80.80% <ø> (ø)
...rc/lib/shadow-shim-helper-rs/src/rootedcell/mod.rs 56.00% <56.00%> (ø)
src/lib/shadow-shim-helper-rs/src/shim_shmem.rs 82.79% <82.79%> (ø)
src/lib/shadow-shim-helper-rs/src/signals.rs 70.06% <85.71%> (-4.04%) ⬇️
...ib/shadow-shim-helper-rs/src/rootedcell/refcell.rs 89.28% <89.28%> (ø)
src/lib/shadow-shim-helper-rs/src/rootedcell/rc.rs 94.11% <94.11%> (ø)
src/lib/shadow-shim-helper-rs/src/emulated_time.rs 71.68% <100.00%> (+2.45%) ⬆️
... and 26 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@sporksmith sporksmith force-pushed the shmem-v2 branch 3 times, most recently from 1ba3e7b to af5d10b Compare October 12, 2022 21:09
@sporksmith sporksmith marked this pull request as ready for review October 12, 2022 21:09
@sporksmith
Copy link
Contributor Author

Just started a benchmark: https://github.com/shadow/benchmark/actions/runs/3237922848

I don't expect this to change performance, but good to double-check since this is somewhat in the hot path

Copy link
Contributor

@stevenengler stevenengler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks cool! While I think this approach takes a little more mental effort to make sure everything is safe, I think it's a lot easier to reason about and understand than the rkyv approach.

src/lib/shadow-shim-helper-rs/src/rootedcell/mod.rs Outdated Show resolved Hide resolved
src/lib/shadow-shim-helper-rs/src/rootedcell/mod.rs Outdated Show resolved Hide resolved
src/lib/shadow-shim-helper-rs/src/rootedcell/rc.rs Outdated Show resolved Hide resolved
src/lib/vasi-macro/src/lib.rs Outdated Show resolved Hide resolved
src/lib/vasi-macro/src/lib.rs Outdated Show resolved Hide resolved
src/lib/vasi/src/lib.rs Outdated Show resolved Hide resolved
src/lib/shadow-shim-helper-rs/src/rootedcell/mod.rs Outdated Show resolved Hide resolved
src/lib/shadow-shim-helper-rs/src/signals.rs Outdated Show resolved Hide resolved
src/lib/shadow-shim-helper-rs/src/shim_shmem.rs Outdated Show resolved Hide resolved
@sporksmith sporksmith enabled auto-merge October 17, 2022 18:41
@sporksmith sporksmith merged commit f03e879 into shadow:main Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Build Build/install tools and dependencies Component: Libraries Support functions like LD_PRELOAD and logging Component: Main Composing the core Shadow executable
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants