-
Notifications
You must be signed in to change notification settings - Fork 471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Support DSTs #209
[WIP] Support DSTs #209
Conversation
b75583d
to
8295e89
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was going to write a long answer, but then realized that maaaaybe there's a cleaner way to support DSTs without changing the public API or introducing any new types at all. We'd only generalize struct Atomic<T>
to struct Atomic<T: ?Sized>
and that's it. Same with Shared
and Owned
. I'll give it some more thought and then reply with another comment.
By the way, there are two important RFCs to watch:
- Unsized Rvalues: Unsized Rvalues rust-lang/rfcs#1909
- Pointer metadata & VTable: RFC: Pointer metadata & VTable rust-lang/rfcs#2580
I expect in 2019 we'll get new Rust features (unsized rvalues and possibly a way to get metadata from a DST pointer) that will simplify DSTs for us a whole lot. Whatever we do today to support DSTs should be compatible with the upcoming language-level features. Just something to think consider.
@stjepang I'm looking forward to reading your long answer :) Changing |
Right. So perhaps we could do the following. If If impl<T: ?Sized> Owned<T> {
fn new(value: T) -> Owned<T> { // Assumes feature `unsized_locals` is available.
unsafe {
if is_sized::<T>() {
// TODO: Allocate as usual.
} else {
let (layout, offset) = Layout::<&T>::new().extend(Layout::for_value(&value)).unwrap();
let dest = Layout::alloc(layout);
// Make a fat pointer to `value`.
let mut ptr = &value as *const T;
// Change the address field in `ptr` to point to `dest.add(offset)`.
let ptr_ptr = &mut ptr as *mut _ as *mut usize;
ptr_ptr.write(dest.add(offset) as usize);
// TODO: Write `value` into `dest.add(offset)`.
// TODO: Write `ptr` into `dest`.
Owned {
data: dest as usize,
_marker: PhantomData,
}
}
}
}
} Now, in order to dereference an The |
I think that's a great idea. Do you know if it's implementable in today's stable? If so, I'll try to get it done. |
I believe it is. Just a tip: you might need this function when implementing DST support: |
@stjepang I think we cannot implement |
@jeehoonkang You could hack it with |
@stjepang Thanks for for hack :) I tried to dig a little bit more, and here is my initial thought:
|
True. We'd have to wait until unsized locals get stabilized.
I'd be okay with supporting
It isn't, the layout of DSTs hasn't been stabilized. Although, I believe it's very unlikely to change in the future - it'll probably always be an address followed by some metadata (array length or virtual table). |
@stjepang I implemented a prototype here: https://github.com/jeehoonkang/crossbeam/tree/dst
|
3c468f0
to
32cf6bb
Compare
Now I think introducing the storage trait (this PR) is better than utilizing unsized types (https://github.com/jeehoonkang/crossbeam/tree/dst) in supporting dynamically sized types in Crossbeam, even after all necessary features for unsized types are stabilized. First, storage trait is more general. In theory, it can support not only dynamically sized types but also allocations in different regions (e.g. those for CPU memory and those for GPU memory) and other use cases. I believe it's better to treat allocation and storage as a first-class citizen. Second, the implementation based on the storage trait will be simpler. In essence, somewhere we need to apply different logics to sized and unsized types. This PR does that using a trait, which is exactly designed for this purpose, while the implementation based on unsized types will do that using specialization, which is non-parametric and likely introduces more cognitive loads. For those reasons, I'd like to pursue this PR further. @stjepang, what do you think? |
8529558
to
cd4dcf5
Compare
@stjepang any opinions on this issue? |
I'm still unsure whether adding this feature is necessary since the drawbacks currently outweight the benefits. The balance in my mind goes like this: Benefits:
Drawbacks:
I'm especially hesitant because this crate is not very easy to use already and would therefore try to avoid adding more complexity to it, if possible. Feature |
4b653a5
to
0845f96
Compare
Superseded by #452 |
452: Support dynamically sized types r=jeehoonkang a=jeehoonkang It supersedes #209 for supporting dynamically sized types (DST) in Crossbeam. It's much cleaner and less intrusive than the previous attempts in that it doesn't introduce another `Atomic` type. `Atomic<T>` is still there without significant changes. The key idea is (1) instead of requiring the condition on `T: Sized`, (2) requiring `T: Pointable` that means an object of type `T` can be pointed to by a single word. For instance, `Atomic` becomes: `pub struct Atomic<T: ?Sized + Pointable> { ... }`. It is breaking the backward compatibility in that it ~~(1) increases the minimum required Rust version to 1.36 (for `MaybeUninit`) and (2)~~ now `const fn Atomic::null()` is Nightly only. Co-authored-by: Jeehoon Kang <jeehoon.kang@kaist.ac.kr>
NOTE: this is a copy of crossbeam-rs/crossbeam-epoch#87
Currently, crossbeam-epoch doesn't support dynamically-sized types (DST): https://github.com/crossbeam-rs/rfcs/blob/master/text/2017-05-02-atomic-api.md#fat-pointers-and-dsts
This PR adds the support for DST by generalizing
Box<T>
, which was used as the underlying storage type for atomic pointers, to theStorage<T>
trait. For more details, see the comment on theStorage
trait. As an example, I added support for dynamically-sized arrays. Other kinds of DSTs can be supported by implementingStorage<T>
, even in the client code.This PR was motivated by the fact that there's an unnecessary indirection in
crossbeam-deque
when accessing its underlying buffer. As a proof of concept, I removed that indirection on top of PR in my local development:concurrent-circbuf
.This is a breaking change, but I tried to keep it as small as possible. As far as I understand, the only breaking change is renaming
Atomic::from(*const T)
intoAtomic::from_raw(*const T)
andShared::from(*const T)
intoShared::from_raw(*const T)
. (Question: I ended up changing the underlying types. Is it a breaking change?)This is a WIP. Probably we want to discuss this PR's design and even write up an RFC. I just share it for starting a discussion. Thanks!