Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binary serialization over pipes #741

Merged
merged 1 commit into from
Sep 7, 2023

Conversation

xy2i
Copy link
Contributor

@xy2i xy2i commented Aug 25, 2023

We use UnixStream::pair() a few times and reimplement binary ser/de each time. Add an abstraction that does the serde (BinSerDe) and buffer ceremony (BinPipe) for us.

Fixes #471.

@codecov
Copy link

codecov bot commented Aug 29, 2023

Codecov Report

❗ No coverage uploaded for pull request base (main@6d40763). Click here to learn what that means.
Patch coverage: 51.63% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #741   +/-   ##
=======================================
  Coverage        ?   55.28%           
=======================================
  Files           ?       72           
  Lines           ?     9835           
  Branches        ?        0           
=======================================
  Hits            ?     5437           
  Misses          ?     4398           
  Partials        ?        0           
Files Changed Coverage Δ
src/exec/no_pty.rs 0.00% <0.00%> (ø)
src/exec/use_pty/backchannel.rs 0.00% <0.00%> (ø)
src/exec/use_pty/monitor.rs 0.00% <0.00%> (ø)
src/common/bin_serde.rs 91.30% <91.30%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@xy2i xy2i force-pushed the 471-bin-pipes branch 2 times, most recently from 61f709d to a62a216 Compare September 5, 2023 17:29
@rnijveld rnijveld requested a review from pvdrz September 5, 2023 17:30
src/common/bin_serde.rs Outdated Show resolved Hide resolved
src/common/bin_serde.rs Outdated Show resolved Hide resolved
src/common/bin_serde.rs Outdated Show resolved Hide resolved
src/exec/use_pty/monitor.rs Outdated Show resolved Hide resolved
src/common/bin_serde.rs Outdated Show resolved Hide resolved
@xy2i
Copy link
Contributor Author

xy2i commented Sep 6, 2023

Thank you for your review !

I ran into an issue I feel like is blocking, which I couldn't work around (see the review above). If there's no good work around I'm okay closing this PR, but let me know.

@pvdrz
Copy link
Collaborator

pvdrz commented Sep 6, 2023

@xy2i oh silly me... Yeah I forgot that const generics is very minimal right now. But this is no reason to not merge your PR. There might be an alternative but I'm not sure how well it works:

mod sealed {
    pub trait DeSerializeBytes {}
    
    impl<const N: usize> DeSerializeBytes for [u8; N] {}
}

pub trait DeSerialize {
    type Bytes: sealed::DeSerializeBytes;
    fn serialize(&self) -> Self::Bytes;
    fn deserialize(bytes: Self::Bytes) -> Self;
}

impl DeSerialize for i32 {
    type Bytes = [u8; std::mem::size_of::<Self>()];
    
    fn serialize(&self) -> Self::Bytes {
        self.to_ne_bytes()    
    }
    
    fn deserialize(bytes: Self::Bytes) -> Self {
        Self::from_ne_bytes(bytes)
    }
}

Let me know if you think that's good enough. Otherwise, we can merge this as it is I think :)

Thanks!

@xy2i
Copy link
Contributor Author

xy2i commented Sep 7, 2023

That works, thanks !

We use `UnixStream::pair()` a few times and reimplement binary ser/de
each time. Add an abstraction that does the serde steps (`DeSerialize`) and
buffer ceremony (`BinPipe`) for us.

Fixes trifectatechfoundation#471.
@pvdrz pvdrz added this pull request to the merge queue Sep 7, 2023
@pvdrz
Copy link
Collaborator

pvdrz commented Sep 7, 2023

Thanks for this!

Merged via the queue into trifectatechfoundation:main with commit 5165038 Sep 7, 2023
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Binary serialization over pipes
2 participants