Excessive stack usage when using tokio::spawn in debug builds

[udoprog]

Description

When using tokio::spawn, the task to spawn is passed through a number of functions before it is moved to the heap:

• https://github.com/tokio-rs/tokio/blob/master/tokio/src/task/spawn.rs#L121
• https://github.com/tokio-rs/tokio/blob/master/tokio/src/runtime/spawner.rs#L23
• https://github.com/tokio-rs/tokio/blob/master/tokio/src/runtime/thread_pool/spawner.rs#L31
• https://github.com/tokio-rs/tokio/blob/master/tokio/src/runtime/thread_pool/slice.rs#L58
• https://github.com/tokio-rs/tokio/blob/master/tokio/src/task/mod.rs#L308

In my case, I had 150kb of stack space remaining (checked using stacker::remaining_stack), and I was passing a 50kb task. So for each stack frame above, 50kb of stack was used until the task was moved to the heap in the form of a RawTask.

This is typically not an issue with optimized builds since most of the stack frames are inlined and the task size is significantly decreased through optimizations. But it might be possible to be nicer towards non-optimized builds by moving the task to the heap as early as possible.

[blasrodri]

I made some changes where I tried inline the whole path from tokio::spawn up to where the future is stored in the heap (thanks @Darksonn )

This is the path the code follows:

tokio::spaw
    ::util::trace::task -> inlined already in `master`
        Spawner::spawn  -> inlined
            Spawner::ThreadPool(spawner) => spawner.spawn(future) -> inlined
                task::joinable -> inlined
                    RawTask::new -> inlined
                        Box::into_raw -> inlined
                            Cell::<_, S>::new -> inlined
                                UnsafeCell::new -> inlined
                                    (Stage::Running(future))

I then run a simple script to trigger the issue on debug mode, that doesn't happen in release mode:

use tokio;
use tokio::task;

const num_tasks: i32 = 1000;

#[tokio::main]
async fn main() {
    let data = [0; 110_224];
    dbg!(stacker::remaining_stack());
    task::spawn(async move {
        println!("{}", data[22]);
        tokio::time::delay_for(std::time::Duration::from_millis(100)).await;
    });
}

A gist of the results, and the different different versions of Tokio I'm using for both:
https://gist.github.com/blasrodri/54b9d43007370d780f8a0974b4833e98

All in all, I get to a point where I don't get a stack overflow for that original case.
Nevertheless, this isn't on par with release mode. For instance, I'm able to get it running without any panics with

data = [0; 200_000];

in release mode, but not on debug mode, even with the inlining changes applied. So I don't really think we're getting enough from these in-lines.

--
I also tried removing some layers and having the future placed in the heap before. However, I got bitten by https://github.com/tokio-rs/tokio/blob/master/tokio/src/runtime/task/mod.rs#L86. I needed to add it as a trait bound in the tokio:spawn, and that's not feasible since it's a pub(crate) trait.