handle infinite include loops more gracefully

[mfherbst]

This issue summarises the behaviour of julia when exposed to circular includes.
The observed behaviour varies between julia versions from segfaults, irresponsive REPL or execution to rather intractable error messages.

Even though circular includes are without a doubt a sign of bad code design, I'd find it desirable to see a more helpful error message and to avoid julia to run into a stack overflow or become irresponsive 😄.

To reproduce

• File a.jl:

include("b.jl")

• File b.jl:

include("a.jl")

Test cases

• REPL: A plain julia> include("a.jl") was issued inside the REPL.
• script: I executed from a shell julia a.jl.

Result summary

julia version	REPL	script
1.1	segfault	lengthy StackOverflowError
1.2	lengthy StackOverflowError + irresponsive	irresponsive
1.3	lengthy StackOverflowError + irresponsive	irresponsive

The behaviour changes between 1.1 and 1.2. The 1.2+ REPL behaviour is to print the error message

LoadError: LoadError: LoadError: LoadError: LoadError: LoadError:

... (many more times and finally) ...

StackOverflowError:

followed by an irresponsive REPL. When run as a script on 1.2+ the only printed output is

ERROR: 

followed by an irresponsive julia process.

Update: In fact "irresponsive" is not correct. The REPL/process just takes long (order of 2 minutes) to respond.

[yuyichao]

There's no way to detect this. Circular include is as legal as recursive functions: you get a stack overflow if you don't add a proper termination condition.

It is definately not desired to guess what error the user make rather than just reporting the error. It's also not very nice to arbitrarily truncate the backtrace since it's almost guaranteed that you'll throw out exactly the useful one in some cases while not actually saving time in a lot of cases.

If this is on a mac the the crash is dup of #17109. The irresponsiveness is probably just printing error (or a variant of #17109).

AFAICT, the backtrace printing does finish on linux. It is indeed slower than 0.6 but that's likely simply because we are collecting much more information compared to an older version.

[mfherbst]

@yuyichao Thanks for your comments. I'm not sure the mac issue applies, since I tested this on my linux laptop, versioninfo() (for the 1.3 version):

Julia Version 1.3.0-alpha.0
Commit 6c11e7c2c4 (2019-07-23 01:46 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 4

I get why the error happens and surely a stack overflow is bound to happen in such a case. My main concerns are:

• In contrast to a recursive function calls, julia does not terminate gracefully, nor does it throw a tractable exception. Instead the process (on my machine) freezes completely until I manually interrupt it. For producing a similar situation with recursive function calls, I used the code

a(x) = b(x)
b(x) = a(x)
a(1)

which fails with

ERROR: StackOverflowError:
Stacktrace:
 [1] a(::Int64) at ./REPL[1]:1
 [2] b(::Int64) at ./REPL[2]:1
 ... (the last 2 lines are repeated 39998 more times)
 [79999] a(::Int64) at ./REPL[1]:1

however, without causing the REPL or my script execution process to freeze. Afterwards I get the REPL back or the julia process terminates with an error exit code.

• Issues related to circular include can be subtle in larger projects and it might be nice to think about providing a helpful hint for the user. I do not know how this could be sensibly done and if it's worth it, however. If the StackOverflowError output ends up appearing collapsed as in the recursive function example, that would be fine from my point of view.

[yuyichao]

How long did you wait? It took significantly longer to print for sure ( a few seconds on my laptop) but it does finish. Without checking carefully the long time is probably due to the interpreter backtrace and the n^2 number of include filename.

[StefanKarpinski]

This could probably be handled more gracefully.

[mfherbst]

It took significantly longer to print for sure ( a few seconds on my laptop) but it does finish.

@yuyichao I tested it again and indeed you are right. In all cases julia 1.2+ becomes responsive after a while. This took a bit longer on my machine (order of 2 minutes). I guess I simply was not patient enough. Thanks for looking into this nevertheless 👍.

So the remaining aspect of this issue is, that the output is a bit cumbersome. This probably could be improved, but personally I would consider this fair enough in such a circumstance. Feel free to close this issue if you agree.

[vtjnash]

Duplicate of #17109, as mentioned above (removing LoadError might help with the output also, via #31882, since we're several thousand deep)