Terminal escape characters pollute CI logs

[gnzlbg]

For some reason terminal escape characters are used in CI to color the terminal output, which ends up polluting the logs making them harder to read. For example, in https://dev.azure.com/rust-lang/e71b0ddf-dd27-435a-873c-e30f86eea377/_apis/build/builds/5993/logs/49 , we find:

2019-08-15T14:24:58.7750305Z �[0m�[0m�[1m�[32m   Compiling�[0m proc-macro2 v0.4.30
2019-08-15T14:24:58.7750678Z �[0m�[0m�[1m�[32m   Compiling�[0m unicode-xid v0.1.0
2019-08-15T14:24:58.9095875Z �[0m�[0m�[1m�[32m   Compiling�[0m syn v0.15.35
2019-08-15T14:24:59.7597252Z �[0m�[0m�[1m�[32m   Compiling�[0m memchr v2.2.0
2019-08-15T14:24:59.8279742Z �[0m�[0m�[1m�[32m   Compiling�[0m serde v1.0.92

and

2019-08-15T15:18:20.0468481Z running 8921 tests
2019-08-15T15:18:34.0401857Z �[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m�[32m.�(B�[m 100/8921
2019-08-15T15:18:41.6138768Z 

We probably should disable colored output when running on CI, not only for rust-lang/rust tools, but also for cargo. Probably cargo is detecting the terminal using in azure as supporting colors, so maybe a quick hack for this could be to just, e.g., TERM=1 in azure to remove this.

cc @rust-lang/infra

[shepmaster]

Personally, I like the control characters, but I download logs like that and look at them locally (using less -R, for example) to make use of the colors:

[Centril]

👍 -- These control characters are hugely annoying when managing rollups and makes me slower. I think by and far most people look at the dump that highfive/rust-log-analyzer emits. I think if we don't fix this in CI then we should at minimum fix this in RLA by doing post-processing.

[kennytm]

These were added because the colors worked in Travis CI and AppVeyor previously. If the control characters aren't supported in Azure Pipelines let's remove them.

[gnzlbg]

They are supported in Azure, their terminal advertises this and we use them. The problem is that people prefer to look / process the raw logs instead of inspecting the log web view that Azure supports because of various reasons.

Maybe fixing this by doing some post-processing in RLA like @Centril proposes is the better solution. Or maybe there is some service that we can pipe this logs into and renders them quickly with colors and we can just post a link to that ?

The main reason I don't use the Azure web API for reading the logs is that the link does not point to it. With travis, the other reason was that the web API only supported 10k Lines, and that it was very slow. I don't know if this also holds for Azure, but if it doesn't, maybe we can just link users there.

[Centril]

The problem is that people prefer to look / process the raw logs instead of inspecting the log web view that Azure supports because of various reasons.

Azure's UI is unergonomic and buggy; it takes much less time for me to check the RLA raw logs when doing rollups.

Maybe fixing this by doing some post-processing in RLA like @Centril proposes is the better solution. Or maybe there is some service that we can pipe this logs into and renders them quickly with colors and we can just post a link to that ?

I would do whatever can be fixed quickest because this problem is painful today.

[Mark-Simulacrum]

RLA does (at least try) strip them out from the GH comments -- e.g., rust-lang/rust#63991 (comment) -- but stripping from raw logs is somewhat harder in that we don't actually store those anywhere today (RLA's comment just points directly to the Azure raw log, e.g. https://dev.azure.com/rust-lang/e71b0ddf-dd27-435a-873c-e30f86eea377/_apis/build/builds/6949/logs/49).

If we want to remove the ANSI codes from the raw link then we could hypothetically implement a sanitizer service that'll strip out the codes before sending the bytes on without storing the logs... or maybe something like that exists already.

@Centril, do you think a sanitizer service is what's needed here?

Edit: Thinking about this a bit more, it actually seems like we could probably sanitize the logs locally in browser via JS which would then be fairly low cost for us (just a single static page) and fairly easy to implement (fetch request, regex replace). I can look into doing so if there's interest.

[Centril]

@Mark-Simulacrum The main thing I want is to be able to click the raw log link in the RLA comments and not see the escape codes. If doing it via a JS landing page that does the sanitation works then that's fine by me. If you can replace the codes by actual colors (also via regex) then even better.

[Mark-Simulacrum]

#31 is the PR implementing the client-side JS (and can technically be used today by prepending https://rust-lang.github.io/rust-log-analyzer/# to raw log URLs).

[Mark-Simulacrum]

I'm going to transfer this issue over to rust-log-analyzer since I think we do not want to change behavior in CI (though log-analyzer may want to expose different UI, which we're working on).