[Rust] no stderr stream

[ShuiRuTian]

VSCode Version: 1.78.2 (user setup)
OS: Windows_NT x64 10.0.22621
CodeLLDB version: 1.9.2
Compiler: cargo 1.70.0-nightly (0e474cfd7 2023-03-31)
Debuggee: x86_64-windows

Problem: The output from stderr seems disappear.

How do I get there: It's by default I think, although it is weird, because this lib is popular and no one report it.

When debugging test by using "cargo test some::test::path",
the output should be like

    Finished test [unoptimized + debuginfo] target(s) in 0.05s                       // from stderr
     Running unittests src\lib.rs (target\debug\deps\hashbrown-3547e1bc587fc63a.exe) // from stderr

running 1 test
test raw::test_map::rehash5 ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 90 filtered out; finished in 0.00s

However, when I use some config like

        {
            "type": "lldb",
            "request": "launch",
            "name": "Debug unit tests in library 'hashbrown'",
            "cargo": {
                "args": [
                    "test",
                    "--no-run",
                    "--lib",
                    "--package=hashbrown"
                ],
                "filter": {
                    "name": "hashbrown",
                    "kind": "lib"
                }
            },
            "args": [],
            "cwd": "${workspaceFolder}"
        },

The output does not contain the content from stderr.

And when I want to redirect though stdio, like stdio: [null, "tmp1.txt", "tmp2.txt"], the tmp2.txt(the content of stderr) is empty.

Verbose log

[ShuiRuTian]

Friendly ping @vadimcn ~

No pressure, just want to make sure this issue is not missed somehow :)

[vadimcn]

These lines are output by cargo, not by the debuggee. You can still see the first one in the terminal used to execute cargo. The second one is, of course, not output, since we pass --no-run.

[ShuiRuTian]

Thanks, may I ask some more questions?

1. It does make sense, if we pass "--no-run", the test should not be run. But, this should also means if we set a breakpoint, it should not be hint, because the test is not ran, right? However, breakpoint does work.

2. I could see the first line in the output of "cargo", however, the second line Running unittests src\lib.rs (target\debug\deps\hashbrown-3547e1bc587fc63a.exe) makes more sense for me. I need this line to get the package name that the test belongs to. Is this a bug of LLVM? It only captures the stdout rather than stderr.

[vadimcn]

Normally, cargo compiles and then immediately runs the unit tests executable. This is not suitable for debugging, we want the debugger to be the immediate parent of the debuggee process. A secondary problem with cargo is that it generates pseudo-random names of the test executable, so we cannot simply specify it's name in program. Therefore, we ask cargo to compile, but not run the tests. While compiling, it also outputs name of the executable, which we capture and then pass to the debugger for launch.

I need this line to get the package name that the test belongs to.

Why are you trying to do this as a part of a debugging session?

[ShuiRuTian]

Thanks for the details.

Why are you trying to do this as a part of a debugging session?

I do not need it as part of a debugging session, I just need the output, but the output is managed by the debugging session(or in fact, LLDB, if I understand it correctly).

Here is the background:
I am implementing TestExplorer for VSCode based on Rust-Analyzer and this plugin: rust-lang/rust-analyzer#14589

We need to know the result of a running of test, and for now, we do it through analyzing the output.(not taht good, but we have not choice, because rustc does not provide any callback mechanism)

If user clicks "Run" button, it's good, we could run cargo and redirect stdout/stderr through js. But if user clicks "debug" button, there is not much we could do but relay on this plugin.

And I need all of the output of cargo to decide which test was ran if user runs all tests under workspace/package level. Because the test case might have same name, although they are under different targets.

[vadimcn]

If I were you, I wouldn't try to intercept all possible ways of executing tests. Updating test results only when the user clicks 'Run' seems entirely reasonable. After all, you can't know whether test execution was tampered with while debugging (by altering the values of variables and registers, skipping over statements, etc).

We need to know the result of a running of test, and for now, we do it through analyzing the output.(not taht good, but we have not choice, because rustc does not provide any callback mechanism)

BTW, if you aren't already, you should be using cargo's --message-format option to output results as JSON.

[ShuiRuTian]

Good point, it should help, thanks!