Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize Rust Vector to JS typed array (Float32Array, Uint32Array, Uint16Array ...) #2

Closed
lanceschi opened this issue Aug 12, 2019 · 4 comments

Comments

@lanceschi
Copy link

lanceschi commented Aug 12, 2019

Hi,

I spent a few time with this great lib although I would like to know if there's a workaround in order to pass back to javascript typed arrays (Float32Array, Uint32Array, Uint16Array ...)?

Something like what is feasible with https://github.com/serde-rs/bytes for Rust Vec<u8>.

Thanks!

@Pauan
Copy link

Pauan commented Aug 12, 2019

web-sys already has this functionality built-in:

https://rustwasm.github.io/wasm-bindgen/api/js_sys/struct.Uint8Array.html#impl-From%3C%26%27a%20%5Bu8%5D%3E

(And similarly for the other typed arrays)

Keep in mind this will make a O(n) copy of the data.

@lanceschi
Copy link
Author

Hi @Pauan ,

thanks for reaching out and for the precious tip.

I followed your advice and flicking through docs, issues and reddit threads, I learned a little bit more regarding Rust/Wasm interaction and I ended up successfully using js_sys::Float32Array subarray method.

I got your point regarding O(n) classification of data copying function, but we're talking about memory.
It should be fast enough (correct me if I'm wrong)? Are there better approaches that can implemented instead?

Thanks a lot!

@Pauan
Copy link

Pauan commented Aug 14, 2019

It should be fast enough (correct me if I'm wrong)?

Yeah, it's probably pretty fast.

Are there better approaches that can implemented instead?

No, in order to maintain memory safety it has to make a copy.

There is a view function which doesn't do any copying, but it is unsafe, so you have to be careful to uphold its guarantees.

In particular, as long as the typed array is alive you must not do any Rust allocations whatsoever. That means no Vec, no Box, no String, etc.

So if you're passing the typed array to JS, and you don't call any Rust functions until after it's been discarded in JS, then yeah you can use it.

But I recommend using the safe From impl, because it doesn't have any of those problems.

flicking through docs, issues and reddit threads

That code is unsafe. It is using the raw WebAssembly memory, which can be invalidated at any time.

It is the same as the view function, except that post was made before view existed. Nowadays you should just use view.

But as I said above, if you use view then you need to carefully uphold the memory guarantees. So it's better to use From unless you really know what you're doing.

It's always possible to switch from From to view later, if you find out that it's a performance bottleneck.

@lanceschi
Copy link
Author

lanceschi commented Aug 14, 2019

Hi @Pauan ,

I got your concern regarding unsafe memory guarantees.

I took the time to check source code and see how the From trait was implemented for Rust slices argument. The comment is pretty explanatory:

impl<'a> From<&'a [$ty]> for $name {
  #[inline]
  fn from(slice: &'a [$ty]) -> $name {
    // This is safe because the `new` function makes a copy if its argument is a TypedArray
    unsafe { $name::new(&$name::view(slice)) }
  }
}

I'll definitely follow your advice!

Thanks again! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants