Add thread-pool capabilities to scope

[TyOverby]

Maybe with an API like

let pool = Pool::new(8); // 8 threads in the pool.
pool.scoped(|scope| {
    scope.spawn(...);
});

[teburd]

+1

[upsuper]

It would probably also be useful to allow user to control which task goes to which thread, for example, something like

let pool = Pool::new(8);
pool.scoped(|threads| {
    for (thread, task) in threads.iter().zip(tasks.iter()) {
        thread.dispatch(task);
    }
});

[upsuper]

One use case of thread pool can be seen in bryant/argon2rs, where fixed number of threads need to be synchronized frequently, which isn't quite doable with crossbeam right now.

[jeehoonkang]

FYI, probably this is one of the most relevant crates to this issue: https://github.com/Kimundi/scoped-threadpool-rs . Also there is a discussion on various Rust libraries on cpu pools: rust-lang/futures-rs#411

[jeehoonkang]

Also related: https://docs.rs/rayon/0.9.0/rayon/fn.scope.html . At least, we need to write a better documentation as done in Rayon.

[ghost]

The library ecosystem has evolved quite a bit since this question was asked. Tokio is now the thread pool for async IO and Rayon is the thread pool for parallel CPU-intensive tasks.

Since we already have two universal thread pools that cover a wide range of use cases, I don't see a need for creating another one. Even if neither Tokio nor Rayon are good enough for a particular use case, then a simple thread pool provided by Crossbeam probably won't be either.

Should we close this issue?

[nhukc]

I don't think this is resolved.

Rayon does not work for cases where a mutex is required due to work stealing. A non-work-stealing scoped thread pool implementation would be extremely useful for writing userspace rust drivers.

See rayon-rs/rayon#1174 for additional details.