API proposal: tokio::task::try_spawn_blocking

[lkts]

Is your feature request related to a problem? Please describe.
Currently spawn_blocking queues tasks if there is no free threads to execute it in blocking thread pool. This behavior may not be suitable for some applications:

• Queuing is invisible to client code which can lead to surprising and hard to diagnose delays in blocking task execution. If there is a need to execute blocking tasks in latency-sensitive workflows, there needs to be a layer on top of Tokio to limit number of concurrent blocking tasks.
• Queue is not bounded which can in extreme cases lead to queue consuming all available memory.

Describe the solution you'd like
I propose a try_spawn_blocking API.

pub fn try_spawn_blocking<F, R>(f: F) -> Result<JoinHandle<R>, NoAvailableThreads>
where
    F: FnOnce() -> R + Send + 'static,
    R: Send + 'static 
{
}

struct NoAvailableThreads {}

Alternatively (i don't know if it makes more sense implementation-wise but i suspect so) try_spawn_blocking can return a special BlockingTaskJoinHandle which will expose the error.

Describe alternatives you've considered
Having a semaphore-based limiter on top of Tokio works well. Motivation for adding this API to Tokio is that:

• having such explicit API helps Tokio users to discover potential issues i am describing
• users don't have to handle (albeit not big) complexity of limiting concurrency on top of Tokio

[ipetkov]

Worth noting that with #4823 we have fallible spawn methods on task::Builder.

#4850 is tracking its stabilization

[hawkw]

Yeah, since, as @ipetkov pointed out, the task::Builder API will have fallible spawn methods, I think the best way to add support for attempting to spawn a blocking task only if it wouldn't be queued is something like this:

task::Builder::new()
    .disable_blocking_queue() // name, obviously, subject to bikeshedding)
    .spawn_blocking(|| { /* ... */ }

and, we would add a QueueDisabled (again, name subject to bikeshedding) variant to SpawnError, here:

tokio/tokio/src/runtime/blocking/pool.rs

Lines 86 to 92 in cf181e8

	pub(crate) enum SpawnError {
	/// Pool is shutting down and the task was not scheduled
	ShuttingDown,
	/// There are no worker threads available to take the task
	/// and the OS failed to spawn a new one
	NoThreads(io::Error),
	}