Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move queues into a new crate #292

Closed
ghost opened this issue Jan 18, 2019 · 5 comments
Closed

Move queues into a new crate #292

ghost opened this issue Jan 18, 2019 · 5 comments

Comments

@ghost
Copy link

ghost commented Jan 18, 2019

So far, I've been hesitating splitting Crossbeam into multiple crates any further, at least until there comes a need for it. The time has now come since we'd like to use SegQueue in Tokio without pulling in the whole crossbeam crate as a dependency. Rayon needs SegQueue for FIFO scopes as well.

So the question is where should we move SegQueue, MsQueue, and soon to be merged ArrayQueue? I suggest taking the simplest possible option - just put them into a new crossbeam-queue crate. In the future we'd probably like to add more types of queues and parts of circbuf (#277) that make sense.

How does everyone feel about this?

@BurntSushi
Copy link

The time has now come since we'd like to use SegQueue in Tokio without pulling in the whole crossbeam crate as a dependency.

Could you say more about the motivation for this?

@ghost
Copy link
Author

ghost commented Jan 18, 2019

Could you say more about the motivation for this?

Sure. There are places where we really only need SegQueue and the only way to get it is to add crossbeam as a dependency. However, that transitively also adds crossbeam-channel, crossbeam-deque (and soon also crossbeam-epoch), which are pulled in despite not being needed at all. It'd be nice to pull in only what you need.

The crossbeam crate is a batteries-included library with tools for concurrency. It's handy when you're writing an application, but libraries prefer to have as few dependencies as possible, so we'd like to give them a choice to pick the parts they need.

Tokio is for the same reason split into tokio, tokio-executor, tokio-threadpool, tokio-fs, and so on. Applications are expected to use the main tokio crate, but libraries should only pick the parts they actually need.

@BurntSushi
Copy link

Ah, thanks for explaining. I guess it would be nice to see that philosophy applied elsewhere. For example, crossbeam-channel is definitely something that other libraries might want to use, but today it brings in 32 (I think?) crates. I'm guilty of this myself as well. I don't quite know how to combat it unfortunately.

Anyway, I think your proposal makes sense!

@ghost
Copy link
Author

ghost commented Jan 18, 2019

I have plans to reduce dependencies of crossbeam-channel as well. Most of those probably come from parking_lot, which will hopefully soon get into the standard library.

@ghost
Copy link
Author

ghost commented Jan 19, 2019

@jeehoonkang @Vtec234 I'm especially interested in your thoughts on this. :)

bors bot added a commit that referenced this issue Jan 28, 2019
303: Move queues into crossbeam-queue r=stjepang a=stjepang

This PR moves `ArrayQueue` and `SegQueue` into a new crate `crossbeam-queue`.

We need queues in a crate that doesn't pull all other Crossbeam crates as a dependency. In particular, Tokio wants to use MPMC queues but cares a lot about keeping its dependency tree as small as possible. See #292 for more information.

Since we already have the `crossbeam::queue` module, and this simply extracts it into `crossbeam-queue`, and there's a real need for moving queues out of `crossbeam`, I believe the introduction of a new crate should be uncontroversial.

I'll leave this PR open for a few days and merge if there are no objections. If anyone feels this might be a wrong decision, please speak up!

Closes #292 

Co-authored-by: Stjepan Glavina <stjepang@gmail.com>
@bors bors bot closed this as completed in #303 Jan 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

1 participant