Redesigning pip's output (for install, wheel and download commands) #4649
Comments
pradyunsg
added
type: enhancement
|
I think it's understood but stating anyway, because it won't hurt. If we decide to integrate only some parts (or none) of that proposal, I'm obviously fine with that. :) Looking at this again, "0" looks a little dirty at the top, in the Fetching portion, due to the large amounts of text there. Would dropping dependency information there be fine? (gem does that) |
|
Also there is no reason why the wheels should not be built in parallel. |
|
@xoviat I don't disagree with that; it's just that no one has had the time to go ahead and implement it yet. #825 is probably where discussions on the parallelisation (of any part IMO) of pip should probably happen. That said, I think it'd be best to defer any discussion on that until someone actually makes a PR for it. |
|
If we go the |
|
Related: tqdm/tqdm#427. |
|
@xoviat That's for when there are parallel downloads, correct? |
|
The UI problem should be addressed all at once so that the UI is parallel capable and runs on another thread rather than the call_subprocess function. |
|
@xoviat I'll assume you meant yes to that yes/no question I asked. :) I'm averse to adding that to this discussion because it mixes multiple interests and that makes stuff harder to discuss and keep on track. My aversion comes from the fact that this probably won't happen very soon and that functionality change is big enough that when it lands a UI change would be justified. I agree that a UI that extends cleanly would be nice. I don't want to discuss how some future pip with more features should or would look. I want to limit this discussion to the current functionality. If we end up such that a future feature's UI would fit in seemlessly, all good but I don’t want that to be the talking point. That stuff can (and should) be discussed when there's someone writing code for it or If you disagree with that, I kindly request you to open a new issue to discuss how a pip with parallel building or downloading should look - please don’t ask for that discussion in this issue. |
|
+1. Parallel processing will be a pretty big change, so it should be a separate issue/PR of its own. But please don't open another issue for that - as has already been noted, that discussion is ongoing on #825. (Technically #825 is about parallel downloads only, but any other use of parallelism in pip can initially be brought up on that thread as well, and spun off into a separate discussion from there if that seems appropriate). Until #825 results in a PR and that PR has landed, we should assume pip runs processes serially. |
|
@pradyunsg I can't look at gists easily as inline content, because access to gists is blocked at work. So I haven't had a chance to look yet. |
|
Would pradyunsg.github.io be accessible? Otherwise, I don't mind posting it inline. |
|
Don't worry, I'll get to it in the end... (I just have to remember/have time when I'm at home, which is when I should be looking at this stuff anyway 😄 ) |
|
Haha. Okay. :) |
|
@xavfernandez Hi! Sorry for being nosy. Could you please take a look at this? I'd be grateful if you tell me what you think. :) |
|
@pradyunsg Well I really like "Step 0" and its clear "Fetching / Wheeling / Installing" steps split. Having a level between our current very verbose The resolver and its backtracking algorithm will also be quite interesting to output ^^ |
|
I know this has been here awhile. I do like the changes and I do like the example that @xavfernandez has indicated. One thing that I'm not entirely sure about here is that this output assumes that there are a number of distinct steps where we first download all of the packages, then build wheels, then install them. That isn't completely correct though, particularly with PEP 517 where we might have to build a wheel to get access to things like dependencies of the project (the hook that allows you to avoid building a wheel is optional). |
+1
Oh right. That means pip would either have to build a source tree into a wheel or use the hook to get the metadata, immediately after downloading it. Correct? One way to handle this could be to get rid of the Wheeling section (as @xavfernandez calls it) and still follow the same theme as above the current proposal by having the building spinners on the next line, indented, in the Fetching section instead. |
Yeah. FWIW, I think it'd just come down to nicely showing "no compatible version found, backtracking to change a previous choice" during the downloading phase. I was giving it some thought and have some ideas but honestly, I don't want to talk too much about it until I have something to show for it. |
|
This design looks much better than the current one, but I feel this design is still somewhat verbose. For example: As for it would be better to move dependencies like So it becomes: I also created two conceptual designs that may be helpful on this. |
There's no way to vertically align this since at this stage, pip doesn't know what the package names are going to be. |
Yea, I was thinking about this. To be clear, it's about where the requirement "came from", rather than dependencies. |
Yes, pip learns more packages as it downloads them. But it doesn't mean that there's no way to align data.
class Stdout:
def __init__(self):
self.active_lines = 0
@staticmethod
def get_twidth():
return shutil.get_terminal_size().columns
@staticmethod
def print_fill_line(s):
print(s + " " * (Stdout.get_twidth() - len(s)))
def moveback(self):
for _ in range(0, self.active_lines):
sys.stdout.write("\033[F")
sys.stdout.flush()
def update_active_lines(self, lines):
if len(lines) < self.active_lines:
raise ValueError
self.moveback()
for line in lines:
self.print_fill_line(line)
self.active_lines = len(lines)
sys.stdout.flush()
def print_permanent_line(self, s):
self.moveback()
sys.stdout.write("\033[K")
print(s.rstrip())
self.active_lines = 0
sys.stdout.flush()
stdout = Stdout()
all_pkgs = [
('Flask', '0.12.2', 'sdist (83kB)'),
('click', '6.7', 'wheel (cached)'),
('Jinja2', '2.9.6', 'wheel (340kB)'),
('itsdangerous', '0.24', 'sdist (46kB)'),
('Werkzeug', '0.12.2', 'wheel (312kB)'),
('MarkupSafe', '1.0', 'wheel (cache)'),
]
stdout.print_permanent_line("Fetching Package(s):")
for i in range(0, len(all_pkgs)):
lens = []
lines = []
now_pkg = all_pkgs[:i + 1]
for pkg in now_pkg:
for index, f in enumerate(pkg):
try:
lens[index] = lens[index] \
if len(f) < lens[index] else len(f)
except:
lens.append(len(f))
for pkg in now_pkg:
fs = []
for index, f in enumerate(pkg):
fs.append(f + " " * (lens[index] - len(f)))
lines.append(" " + fs[0] + " == " + fs[1] + " - " + fs[2])
stdout.update_active_lines(lines)
time.sleep(.3)prints
This one has some limitations. Content will be pushed into the next column if the last one is too long. But it still looks better than the original one.
|
|
Yea, I'm not super warm to the idea of using ANSI escapes. That said, I do want us to retain this information, as it is useful information when using pip, to debug how things are happening and reporting of how things are being done. As of now, I want to keep that column, even though I'm not happy with how it looks myself. ;) |
Apart from anything else it's not obvious they would work on Windows (colorama has some ANSI support, but I don't know how complete or robust it is). |
|
That said, thanks @futursolo for the ideas (and code)! |
|
One thing to consider when doing this -- move warning and error messages to the end of the output instead of being in the middle or something. (eg -- warnings about broken dependencies as added in #5000 or warnings about not-on-PATH script locations) |
pradyunsg
changed the title
nlhkabu
added
UX
|
Alright. So, one of the "trends" from the UX studies seems to be that users think most of pip's output isn't relevant most of the time. It's a lot of output with not enough relevant information. I made a thing to make visualizing the CLI stuff easier. Don't @ me for feature requests on that. :P Let's call this one the "Oct 9 2020 iteration"This is just a toy example I've made, of what we could output in a hypothetical pip with revamped outputs on the CLI. What you want to do to see this properly is copy the entire thing below and paste it into the left text box of the page above. Once you click outside of the text box, the other elements will process that and you can press "Play" to see it in action. :) |
|
Um, yes I like this? When's it going to be ready? 🙂 More seriously, I think it would be great to implement the console output mechanisms needed to implement this type of output. Iterating on the precise form of the reporting will be relatively easy once we have the machinery in place. And overall, I like the look of the "Oct 9 2020 iteration". However, we need to be careful not to vendor in a big bunch of of console rendering libraries in pursuit of prettier output - pip is big enough already 🙁 |
|
That output looks really good, and I'm another +1 on it. I think ideally it'd support pushing logging messages "above" that, so that all the relevant "Stuff I'm doing" stays at the bottom, and the top is just important logging information like.. deprecation notices or what have you. |
|
/cc @ei8fdb, since this is the catch-all for the pip-output discussion. IDK if you saw what I posted earlier this week, based off of our discussion. :) |
I have written https://gist.github.com/pradyunsg/6eb63fa4571a9723f4d42166e8abb417 as kind of a proposal. :)
Thoughts @pypa/pip-committers?
The text was updated successfully, but these errors were encountered: