Backtrace filtering for testsets no longer work on 0.7

[KristofferC]

0.6:

Arrays: Test Failed
  Expression: foo(ones(4)) == 15
   Evaluated: 16 == 15
Stacktrace:
 [1] macro expansion at ./REPL[3]:12 [inlined]
 [2] macro expansion at ./test.jl:860 [inlined]
 [3] macro expansion at ./REPL[3]:10 [inlined]
 [4] macro expansion at ./test.jl:860 [inlined]
 [5] anonymous at ./<missing>:?

0.7:

Arrays: Test Failed
  Expression: foo(ones(4)) == 15
   Evaluated: 16 == 15
Stacktrace:
 [1] record(::Test.DefaultTestSet, ::Test.Fail) at /home/kristoffer/julia/usr/share/julia/site/v0.7/Test/src/Test.jl:650
 [2] do_test(::Test.Returned, ::Expr) at /home/kristoffer/julia/usr/share/julia/site/v0.7/Test/src/Test.jl:418
 [3] macro expansion at ./REPL[4]:12 [inlined]
 [4] macro expansion at /home/kristoffer/julia/usr/share/julia/site/v0.7/Test/src/Test.jl:942 [inlined]
 [5] macro expansion at ./REPL[4]:10 [inlined]
 [6] macro expansion at /home/kristoffer/julia/usr/share/julia/site/v0.7/Test/src/Test.jl:942 [inlined]
 [7] anonymous at ./<missing>:?
 [8] eval(::Module, ::Expr) at ./repl/REPL.jl:3
 [9] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./repl/REPL.jl:69
 [10] macro expansion at ./repl/REPL.jl:100 [inlined]
 [11] (::getfield(Base.REPL, Symbol("##1#2")){Base.REPL.REPLBackend})() at ./event.jl:96

I think this happened when the test files were moved to stdlib. I will look into it.

[c42f]

It seems to me that we shouldn't be printing a stacktrace at all for individual failed tests which fail the test predicate. Wouldn't it be better to precisely capture the file and line number of the @test macro itself? This should now be easily possible with the implicit __source__ argument to macros.

[KristofferC]

Yeah, the question is if the stacktrace is useful in situations like:

julia> function test_1(x)
           @test x == 3
       end;

julia> @noinline test_2(x) = test_1(x);

julia> @testset begin
           test_2(2)
       end
test set: Test Failed
  Expression: x == 3
   Evaluated: 2 == 3
Stacktrace:
 [1] test_1(::Int64) at ./REPL[11]:2
 [2] test_2(::Int64) at ./REPL[12]:1
 [3] macro expansion at ./REPL[13]:2 [inlined]
 [4] macro expansion at ./test.jl:860 [inlined]
 [5] anonymous at ./<missing>:?
Test Summary: | Fail  Total
test set      |    1      1

Here you can see how you got to the test.

[KristofferC]

Ok, this is what I want to do:

1: Change the line:

test set: Test Failed

to

test set: Test Failed at path/subpath/testfile.jl:32

2. Stop printing the stacktrace if there are no functions call to reach the @test from the @testset.

[c42f]

Sounds great to me, this will be a massive improvement in readability.

[KristofferC]

One problem is that some functions e.g test_approx_eq_modphase uses @test internally which will then give the wrong line information since the macro is called in the function.

[andreasnoack]

I think we should just get rid of test_approx_eq_modphase, see JuliaLang/LinearAlgebra.jl#393.

[KristofferC]

Actually the situation withtest_approx_eq_modphase is completely analogous to a user calling @test in a function so I'll need to handle it anyway. But should be no big problem.

[KristofferC]

This issue was fixed by #23998