SIGTERM is ignored when process has no TTY

[darakian]

Description

Ran into an issue killing a rocket process and tracked it down to a missing tty. SIGKILL still kills the process, but a graceful shutdown would be nice even when missing a tty. SIGQUIT works as expected.

To Reproduce

Launch a rocket web server as a background process then log out the active user.

Expected Behavior

The root user (or other management users) should be able to kill the process using the kill command as kill <pid> rather than needed to specify SIGQUIT.

Environment:

• OS Distribution and Kernel: CentOS 8 stream 4.18.0-269.el8.x86_64
• Rocket Version: 0.5.0-rc.1 via crates.io

Additional Context

[root@node ~]# ps -U web
    PID TTY          TIME CMD
 918027 pts/1    00:00:00 rocket_process
[root@node ~]# kill -s SIGTERM 918027
[root@node ~]# ps -U web
    PID TTY          TIME CMD
[root@node ~]# ps -U web
    PID TTY          TIME CMD
 918066 ?        00:00:00 rocket_process
[root@node ~]# kill -s SIGTERM 918066
[root@node ~]# ps -U web
    PID TTY          TIME CMD
 918066 ?        00:00:00 rocket_process

Launch config

Configured for release.
   >> address: 0.0.0.0
   >> port: 80
   >> workers: 1
   >> ident: Rocket
   >> keep-alive: 5s
   >> limits: bytes = 8KiB, data-form = 2MiB, file = 1MiB, form = 32KiB, forms = 32KiB, json = 1MiB, msgpack = 1MiB, string = 8KiB
   >> tls: disabled
   >> temp dir: /tmp
   >> log level: critical
   >> cli colors: true
   >> shutdown: ctrlc = true, force = true, signals = [SIGTERM], grace = 2s, mercy = 3s

[kolbma]

I can confirm this.
It works if you redirect the process output...

executable >out.log 2>&1

or

executable 2>&1 | tee out.log

Will see if this can be fixed in Rocket or it is a problem in tokio::signal.

[kolbma]

@darakian Does Rocket successful service requests in this detached state?

It panics in trying to warn!("Received {}. Requesting shutdown.", sig); because print to stdout is not possible anylonger.

[kolbma]

Mmh... with this pull in the year 2015 print/println shouldn't panic, only if stdout is requested somehow via std::io::stdout.
rust-lang/rfcs#1014

But it is documented that it will panic, for println!:

Panics if writing to [io::stdout] fails.

[kolbma]

sorry. shot too fast... I'd tested it with redirecting the output ;-)

[kolbma]

It panics at std::io::stdio::print_to...

rust_panic (@rust_panic:7)
rust_panic_with_hook (@std::panicking::rust_panic_with_hook::h71e6a073d87de1f5:105)
{{closure}} (@std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::hd549436f6bb6dbb8:42)
__rust_end_short_backtrace<closure-0,!> (@std::sys_common::backtrace::__rust_end_short_backtrace::h4e5f4b72b04174c3:10)
begin_panic_handler (@rust_begin_unwind:21)
begin_panic_fmt (@std::panicking::begin_panic_fmt::h818c3c917eaeb432:16)
std::io::stdio::print_to (@std::io::stdio::_print::hfdac4ecf8a146755:171)
_print (@std::io::stdio::_print::hfdac4ecf8a146755:77)
log (/home/makolb/Source/Repos/rocket-issues/issue2019/Rocket/core/lib/src/log.rs:118)
__private_api_log (/home/makolb/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/log-0.4.14/src/lib.rs:1460)
{{closure}}<tokio::net::tcp::listener::TcpListener> (/home/makolb/Source/Repos/rocket-issues/issue2019/Rocket/core/lib/src/server.rs:433)
poll<generator-1> (/home/makolb/.rustup/toolchains/1.51.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80)
{{closure}}<core::future::from_generator::GenFuture<generator-1>> (/home/makolb/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:161)
with_mut<tokio::runtime::task::core::Stage<core::future::from_generator::GenFuture<generator-1>>,core::task::poll::Poll<()>,closure-0> (/home/makolb/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/tokio-1.15.0/src/loom/std/unsafe_cell.rs:14)
poll<core::future::from_generator::GenFuture<generator-1>> (/home/makolb/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:151)
{{closure}}<core::future::from_generator::GenFuture<generator-1>> (/home/makolb/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:461)
call_once<core::task::poll::Poll<()>,closure-0> (/home/makolb/.rustup/toolchains/1.51.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:344)
do_call<std::panic::AssertUnwindSafe<closure-0>,core::task::poll::Poll<()>> (/home/makolb/.rustup/toolchains/1.51.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:379)
__rust_try (@__rust_try:10)
try<core::task::poll::Poll<()>,std::panic::AssertUnwindSafe<closure-0>> (/home/makolb/.rustup/toolchains/1.51.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:343)

log.rs:118 is official this https://github.com/SergioBenitez/Rocket/blob/8cae077ba1d54b92cdef3e171a730b819d5eeb8e/core/lib/src/log.rs#L104

I've replaced this println! withprintln!("Warning"); and still panics.
But with io::stdout().write(b"Warning").unwrap(); there is no panic.

What are the print-macros doing here? Do they cache somewhere stdout?

[kolbma]

rust-lang/rust#92989

[darakian]

@darakian Does Rocket successful service requests in this detached state?

It does. The only noticeable behavior has been that I needed to alter a control script. Thanks for digging into this, it was a fascinating read!