Stream build backend output to debug!

[charliermarsh]

Summary

We need to decide whether we want this in debug! or tracing!. We also probably (?) want to show this by default in uv build.

Closes #1567.

Closes #5893.

[charliermarsh]

[charliermarsh]

Confusingly these failing tests pass on my machine.

[konstin]

Do we need to prefix builds when multiple ones are running in parallel, or generally to identify which one is which?

[konstin]

Tests are also passing on my linux machine

[charliermarsh]

Do we need to prefix builds when multiple ones are running in parallel, or generally to identify which one is which?

I thought about this, but if you run -vvv, the tree structure seems fine to me.

[charliermarsh]

Ok, figured it out -- I was dropping output from one stream as soon as the other completed.

[henryiii]

We also probably (?) want to show this by default in uv build.

Remember, this is the only way for a build backend like scikit-build-core, maturin, hatching, etc. to communicate with the user. Modern build backends are fairly careful with output, trying to just display useful information to the user, like compiler warnings, enabled components, etc. In my opinion, uv build . and uv pip install . should show output. I even think uv pip install numpy should show output if it's building a wheel, but that's probably too large a departure from pip and what people know. (Remember, most of the time it won't build!)

I think the reason it's hidden by default is a) setuptools produces a ton of pretty useless output, like every file it copies, b) pip was designed in a time when pre-compiled wheels for everything didn't exist, which meant every install produced stuff, while today most packages have wheels, and c) dependency output could get confusing. Very little of this makes sense, though, in the modern most-packages-have-wheels world, where most SDist-only packages are compiled packages where output is both helpful and keeps the terminal from just sitting idle for seconds to minutes (to hours in rare cases). But in the case of building from a source directory, I think it's easy to say hiding the build-backend output is never helpful. And again, this is the only way to see the build-backend produce output. You only interact to build backend via pip install . and build .!

Two issues with this as well, by the way. It looks like every line gets a long prefix with logging info (Correction: it appears to just be "DEBUG", which isn't bad), and it looks like the build backend can't tell things like the terminal width and color support. scikit-build-core, cmake, and most compilers all support color output, but it's gone with uv. What about also providing a flag that disables capturing the subprocess output? Pip has the same problem, but it only adds a couple of spaces as a prefix. This is why build's output is so much better than pips, as it doesn't buffer it and modify it.

Here's an example of build's output for boost-histogram, with scikit-build-core's logging enabled:

Now here's uv's current support:

There's about two pages of uv logging before the build backend info starts, it's got a DEBUG prefix on everything, and all color is gone.

Also, I noticed another problem. Without scikit-build-core's logging enabled, this is the entire output from pipx run build --wheel:

Notice that Ninja is rewriting its output if each file is successfully compiled, keeping the log short, because it knows it's writing to a terminal. However, you can't do that when you buffer! Here's the output part from uv:

Having a special flag to disable capturing the output would fix that too.

Edit: #6912 looks better, not sure about it supporting terminal detection though.

[henryiii]

Actually, this is what it looks like for me now with 0.4.5:

Every line is three lines, two of them with DEBUG prefixes.

[charliermarsh]

That’s strange, I’ll take a look. I don’t see that for other backends.