rework std.Progress #20059
rework std.Progress #20059
Commits on May 28, 2024
-
New design ideas: * One global instance, don't try to play nicely with other instances except via IPC. * One process owns the terminal and the other processes communicate via IPC. * Clear the whole terminal and use multiple lines. What's implemented so far: * Query the terminal for size. * Register a SIGWINCH handler. * Use a thread for redraws. To be done: * IPC * Handling single threaded targets * Porting to Windows * More intelligent display of the progress tree rather than only using one line.
-
Move the mutex into the nodes Track the whole tree instead of only recently activated node
-
This time, we preallocate a fixed set of nodes and have the user-visible Node only be an index into them. This allows for lock-free management of the node storage. Only the parent indexes are stored, and the update thread makes a serialized copy of the state before trying to compute children lists. The update thread then walks the tree and outputs an entire tree of progress rather than only one line. There is a problem with clearing from the cursor to the end of the screen when the cursor is at the bottom of the terminal.
-
the clear, save, restore thing doesn't work when the terminal is at the bottom
-
-
-
-
-
-
std.Progress: save a copy of IPC data
so that the previous message can be used when the pipe is empty. prevents flickering
-
std.process.Child: fix ZIG_PROGRESS env var handling
and properly dup2 the file descriptor to make it handle the case when other files are already open
-
std.Progress: fix using saved IPC data
also fix handling of BrokenPipe also fix continuing wrong loop in error conditions
-
-
-
-
-
-
* bump default statically allocated resources * debug help when multiple instances of std.Progress are initialized * only handle sigwinch on supported operating systems * handle when reading from the pipe returns 0 bytes * avoid printing more lines than rows
-
-
-
-
std.Progress: fix race condition with IPC nodes
It stored some metadata into the canonical node storage data but that is a race condition because another thread recycles those nodes. Also, keep the parent name for empty child root node names.
-
-
std.process.Child: prevent racing children from inheriting progress p…
…ipes This fix is already in master branch for stdin, stdout, and stderr; this commit solves the same problem but for the progress pipe. Both fixes were originally included in one commit on this branch, however it was split it into two so that master branch could receive the fix before the progress branch is merged.
-
Module: fix and improve progress reporting
* correctly report time spent analyzing function bodies * print fully qualified decl names * also have a progress node for codegen The downside of these changes is that it's a bit flickerey, but the upside is that it's accurate; you can see what the compiler's doing!
-
Progress: fix compile errors on windows
Works for `zig build-exe`, IPC still not implemented yet.
-
-
std.process.Child: comptime assert to protect prog_fileno
documenting my assumptions via comptime assertion
-
-
-
frontend: fix use of undefined progress node
This was causing a crash when running `zig test`.
-
std.Progress: move global preallocations to thread memory
Instead of making static buffers configurable, let's pick strong defaults and then use the update thread's stack memory to store the preallocations. The thread uses a fairly shallow stack so this memory is otherwise unused. This also makes the data section of the executable smaller since it runtime allocates the memory when a `std.Progress` instance is allocated, and in the case that the process is not connected to a terminal, it never allocates the memory.
-
std.Progress: move more global preallocations to thread memory
Same idea as previous commit
-
std.Progress: truncate IPC data exceeding preallocated buffers
This accomplishes 2 things simultaneously: 1. Don't trust child process data; if the data is outside the expected range, ignore the data. 2. If there is too much data to fit in the preallocated buffers, drop the data.
-
-
std.Progress: smaller type for parents and robustify
Switch Node.Parent, Node.Index, and Node.OptionalIndex to be backed by u8 rather than u16. This works fine since we use 200 as the preallocated node buffer. This has the nice property that scanning the entire parents array for allocated nodes fits in 4 cache lines, even if we bumped the 200 up to 254 (leaving room for the two special states). The thread that reads progress updates from the pipe now handles short reads by ignoring messages that are sent in multiple reads. When checking the terminal size, if there is a failure, fall back to a conservative guess of 80x25 rather than panicking. A debug message is also emitted which would be displayed only in a debug build.
-
std.Progress: use std.log.debug rather than warn
when the errors could possibly be spammed many times
-
zig build: give a root progress name
Now it's more clear when zig is building the build script vs building the actual project.
-
std.Progress: keep cursor on newline
Don't truncate trailing newline. This better handles stray writes to stderr that are not std.Progress-aware, such as from non-zig child processes. This commit also makes `Node.start` and `Node.end` bail out early with a comptime branch when it is known the target will not be spawning an update thread.
-
-
std.Progress: count newlines more accurately
Split newline_count into written_newline_count and accumulated_newline_count. This handle the case when the tryLock() fails to obtain the lock, because in such case there would not be any newlines written to the terminal but the system would incorrectly think there were. Now, written_newline_count is only adjusted when the write() call succeeds. Furthermore, write() call failure is handled by exiting the update thread.
-
std.Progress: elide root node if empty
when the root progress node has a zero length name, the sub-tree is flattened one layer, reducing visual noise, as well as bytes written to the terminal.
-
CLI: add missing call to root progress node end()
cleans up unwanted "LLVM Emit Object" being left on the screen
-
std.process.cleanExit: lock stderr before exiting
This makes it so that any other threads which are writing to stderr have a chance to finish before the process terminates. It also clears the terminal in case any progress has been written to stderr, while still accomplishing the goal of not waiting until the update thread exits.
-
-
-
std.Build.Step.Run: account for new environment variable
Introduces `disable_zig_progress` which prevents the build runner from assigning the child process a progress node. This is needed for the empty_env test which requires the environment to be completely empty.
-
std.Progress: adjust the timings a little bit
Slightly slower refresh rate. It's still updating very quickly. Lower the initial delay so that CLI applications feel more responsive. Even though the application is doing work for the full 500ms until something is displayed on the screen, it feels nicer to get the progress earlier.
-
std.Progress: handle big-endian targets
We cannot rely on host endianness because the parent or child process may be executing inside QEMU.
-
-
-
-
std.Progress: fix race condition with setIpcFd
The update thread was sometimes reading the special state and then incorrectly getting 0 for the file descriptor, making it hang since it tried to read from stdin.
-
zig build: respect --color argument
`--color off` now disables the CLI progress bar both in the parent process and the build runner process.
-
std.process.Child: fix spawning child proc with new cwd fd
Before this fix, the dup2 of the progress pipe was clobbering the cwd fd, causing the fchdir to return ENOTDIR in between fork() and exec().
-
CLI: actually fix translate-c creating root progress node twice
7281cc1 did not solve the problem because even when Node.index is none, it still counts as initializing the global Progress object. Just use a normal zig optional, and all is good.
-
std.Progress: relax some of the atomic orderings
Generates better machine code, particularly on ARM
-
std.Progress: better Windows support
* Merge a bunch of related state together into TerminalMode. Windows sometimes follows the same path as posix via ansi_escape_codes, sometimes not. * Use a different thread entry point for Windows API but share the same entry point on Windows when the terminal is in ansi_escape_codes mode. * Only clear the terminal when the stderr lock is held. * Don't try to clear the terminal when nothing has been written yet. * Don't try to clear the terminal in IPC mode. * Fix size detection logic bug under error conditions.
