travis: print some statistics from "top".

[matthiaskrgr]

cc #48828

[matthiaskrgr]

First results: https://gist.github.com/matthiaskrgr/7ab3c5998dc6943069927683e3a26a2e
There seem to be a bunch of times where it looks like only one core is used (indicating by falling load average).
I modified the script to also print directly into stdout so we can easily see what is happening in the foreground.
https://travis-ci.org/rust-lang/rust/builds/350827434

[ishitatsuyuki]

The thing you mentioned is caused by large, build-blocking crates like libsyntax_ext or librustc. Most of the time is spent doing codegen, and as long as we don't use codegen-units for release build, there is not really a way to resolve this issue.

[matthiaskrgr]

If the purpose of stage0 is solely to bootstrap stage1, maybe we can compile stage0 with several codegen units?

[ishitatsuyuki]

Maybe it works. The best way is to open a PoC PR and check the time difference.

[matthiaskrgr]

I tried to make this work in #48843 , let's see if it works...

[matthiaskrgr]

I updated the commit so that the .travis.yml line that enables the logging is commented out.
Remove the "#" to enable it.

[shepmaster]

I think this wants to be merged now...

r? @kennytm

[kennytm]

cc @alexcrichton @rust-lang/infra #48828. This PR implements a very simple script which polls top every 30 seconds to obtain a (1-minute) load-average time graph plus other information. The script is disabled so we will need to enable it and figure out where to upload the script afterwards.

We've got some data out of it in #48841 (comment).

Personally I incline to close this, because

1. I think load average is not the metric we want to gather. I expect something more like the CPU usage graph in the Task Manager, with a percentage per core.

2. This output does not show which step are we in. Perhaps we could analyze together with ci: Print out how long each step takes on CI #49094, though ci: Print out how long each step takes on CI #49094 will need to mark the starting point of each step as well, to make parsing easier.

3. The script is disabled here, so merging it at this state won't have any effect anyway.

(BTW, top -ibn 1 is not a valid command on macOS. I suggest you use uptime instead if we only care about the load average. Also I'm not sure if top/uptime exists on AppVeyor.

invalid interval number (not an integer): bn
top usage: top
		[-a | -d | -e | -c <mode>]
		[-F | -f]
		[-h]
		[-i <interval>]
		[-l <samples>]
		[-ncols <columns>]
		[-o <key>] [-O <secondaryKey>]
			keys: pid (default), command, cpu, cpu_me, cpu_others, csw,
				time, threads, ports, mregion, mem, rprvt, purg, vsize, vprvt,
				kprvt, kshrd, pgrp, ppid, state, uid, wq, faults, cow, user,
				msgsent, msgrecv, sysbsd, sysmach, pageins, boosts, instrs, cycles
		[-R | -r]
		[-S]
		[-s <delay>]
		[-n <nprocs>]
		[-stats <key(s)>]
		[-pid <processid>]
		[-user <username>]
		[-U <username>]
		[-u]

)

[alexcrichton]

@kennytm I agree with everything you said, I'll continue with some more thoughts on #48828

[kennytm]

As discussed above, I'm closing this PR. Thanks for the exploration @matthiaskrgr!