Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace unsound PyByteArray::data with PyByteArray::to_vec #545

Merged
merged 3 commits into from
Jul 20, 2019

Conversation

konstin
Copy link
Member

@konstin konstin commented Jul 18, 2019

Thanks to @ExpHP's comment in #373 pointing out the existing unsoundness of PyByteArray::data I've replace the function with to_vec, which copies the bytearray to a Vec. In the long run we likely want something like ReadOnlyCell for an as_slice, but this at least fixes the UB for now.

@kngwyu
Copy link
Member

kngwyu commented Jul 19, 2019

I'm not against this PR, but to think about what kind of safety we can provide under the current design, we should keep in our mind that we can do mutate operation to all &Py~ types.
It means we cannot follow the Rust's covariant rule about &'a types.
So, to get surface safety, one possible approach we(and rust-cpython team) can do is to use cell/pointer types when exposing data underlying PyObect pointer to Rust world, which makes mutate operation unsafe.
Yeah, I think this approach is fine, but I also feel it makes our codes more complicated and not offers many benefits...

@birkenfeld
Copy link
Member

In the end, to be taken seriously by the Rust community, and to deserve the name of a safe wrapper, PyO3 cannot leave UB possible in safe code. This is the benefit that outweighs any drawback of more complicated code.

@konstin
Copy link
Member Author

konstin commented Jul 19, 2019

For all I can tell pyo3 is in its general design sound. The known exceptions are:

I'd be happy about pull requests for those issues and I'm willing to help with questions arising when implementing those.

Given the size of pyo3's codebase and the cpython ffi, there are surely more bugs that can produce UB in the code, but I don't think that there are fundamental problems with pyo3's current design.

I'm not against this PR, but to think about what kind of safety we can provide under the current design, we should keep in our mind that we can do mutate operation to all &Py~ types. It means we cannot follow the Rust's covariant rule about &'a types.

Could you explain that a bit more? For my understanding what we're doing is well defined because every modification happens through a call to a c function with a raw pointer, which are both opaque to rustc.

@kngwyu
Copy link
Member

kngwyu commented Jul 20, 2019

Could you explain that a bit more?

Sorry for my poor English and explanation, ...
In Rust, fn(&A)->&B is generally safe(so &~ types are covariant), but in our case, it is sometimes not safe since &Py~ types are actually mutable(e.g., the case A is PyByteArray and B is [u8]).
I mean this fact makes it difficult to define what kind of safety we can provide.
We don't explicitly follow Rust's rule about references.
To provide safe interfaces, one realistic way is to wrap internal data(say, &[u8]) by *Cell when we expose it Rust types, since *Cell types are considered as invariant(≒ we can retrieve &mut from &) in Rust's manner.

For my understanding what we're doing is well defined because every modification happens through a call to a c function with a raw pointer, which are both opaque to rustc.

Yeah, I think this is a good interpretation of what kind of safety we can provide.
But there still remain unsafety in that we can do mutate opretation via py.run without unsafe block.

@konstin konstin merged commit 58e17be into master Jul 20, 2019
@konstin konstin deleted the replace-PyByteArray-data branch July 20, 2019 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants