Segfault with stack overflow and nested exception handlers

[c42f]

The following recursively nested exception handling causes a segfault on linux x86_64, with both julia 0.6 and julia 1.0. This seems to happen at the boundary of a stack overflow, as it's fine for all n < 27542 on my system, and seems to segfault above that.

This could be related to #17109, though running the first example code from that issue works fine for me.

@noinline boom() = error("Boom")

function evil(n)
    if n == 0 
        return 1
    end
    try
        evil(n-1)
    catch
        boom()
    end
end

evil(1000000)

To be clear, I didn't find this in real code - it's an intentional stress test for a WIP fix for #19979.

[KristofferC]

Segfault on stack overflow on mac is a known issue, I think.

JuliaLang/LinearAlgebra.jl#517
#25715
etc

[yuyichao]

This is unrelated to the mac issue. It's sort of unclear though what is the correct thing to do if the user code tries this hard to break exception handling (the handler of the stack overflow also overflows the stack). The only thing that could be done better without a very different exception handling mechanism AFAICT is to detect this and give a better error message before quitting instead of a segfault...

[fisiognomico]

We discussed something similar in #25523. This seems a classic recursive function which floods each stack page. Putting a guard page that returns a StackOverflow if overwritten can be an answer. (This is what Rust does).
LLVM provides a probe-stack attribute, but I think that it does not include an exit from the guard page.
For example I tried a quick implementation

diff --git a/src/codegen.cpp b/src/codegen.cpp
index 1290d3a599..739317a8e1 100644
--- a/src/codegen.cpp
+++ b/src/codegen.cpp
@@ -5475,6 +5475,13 @@ static std::unique_ptr<Module> emit_function(
 #ifdef JL_DEBUG_BUILD
     f->addFnAttr(Attribute::StackProtectStrong);
 #endif
+
+#if !defined(JL_ASAN_ENABLED)
+#if !defined(_OS_WINDOWS_) && (defined(_CPU_X86_64_) || defined(_CPU_X86_))
+    f->addFnAttr("probe-stack");
+#endif // !_OS_WINDOWS_ && (_CPU_X86_64_ || _CPU_X86_)
+#endif //! JL_ASAN_ENABLED
+
     ctx.f = f;
 
     // Step 4b. determine debug info signature and other type info for locals

With this patch, Julia freezes with the example code, instead of segfaulting.
I have not checked out if this due to an error in codegen, but it seems more probable that LLVM does not provide natively a safer exit.

[c42f]

It looks like the probe-stack attribute is meant to have an argument supplied, namely the function which will do the actual probing, to be called by LLVM in the preamble of functions with large stack frames. For rust this is __rust_probestack and we'd need something equivalent.

I guess if we did that, we'd potentially be able to fix this. Depends on whether we can ensure enough stack space to at least cleanly restore the next outer error handler?

the handler of the stack overflow also overflows the stack

Right. Though I would have though that would also result in a new SEGV -> signal handler call_in_ctx -> rethrow of StackOverflowError (executing on signal stack) -> caught in the next outer frame -> ...

And hence peeling off one frame each time until the handler runs correctly. Obviously I'm missing something or we wouldn't be seeing this problem.

[fisiognomico]

It looks like the probe-stack attribute is meant to have an argument supplied, namely the function which will do the actual probing, to be called by LLVM in the preamble of functions with large stack frames. For rust this is __rust_probestack and we'd need something equivalent.

Oh yeah I didn't noticed that. Well function attributes are not documented very well, anyway it seems that LLVM guarantees that for if a function with the probe-stack attribute is inlined into one with none, then the caller "inherits" the attribute of the called.

I will continue to check on that.