crater run to estimate impact of full NLL transition

[nikomatsakis]

In the @rust-lang/lang meeting today, we were saying it'd be a good idea to do a crater run against full NLL just to get a rough idea of how many affected crates there will be.

There are two ways we could do this and both have some value:

• change the "warning lints" to "deny" and then do the crater run. This has the benefit that if crate X fails to compile, dependencies of crate X will still be tested, giving a more accurate overall picture.
• change default borrowck mode to NLL. This has the advantage that some errors (notably regionck errors) that are no longer relevant are disabled. However, I think we will overall get an underestimate this way, plus there are still (maybe) some soundness holes.

[Centril]

I would suggest that we do both of the two ways since it's an important ticket.

[pnkfelix]

cc #43234

[pnkfelix]

triage: this is an important task, but not one that I find should be prioritized over the various bugs currently plaguing the compiler.

Marking as P-medium.

I'm not sure why @Centril nominated it. Its possible that they nominated it solely for prioritization. Or its possible they wanted it to get attention from the compiler team. I'm going to leave the nomination tag on for now, with the understanding, that if we do discuss it at the T-compiler meeting and manage to assign it to someone, then we can remove the nomination tag.

[lqd]

I will try to help @pnkfelix with this.

[pnkfelix]

@lqd looks like the easiest way to go is to open two distinct PR's corresponding to the two avenues outlined in the description, right?

[lqd]

(we talked about this in this Zulip thread) yes and I opened #60911 and #60914 for these 2 avenues

[lqd]

Copying triage results over from each PR:

#60914:

Of the 1877 regressions, these 87 crates caused 1859 crates to fail downstream, while these 54 crates themselves failed to build.

There is some overlap between these 2 categories: so this PR impacted around 130 root crates. I remember a lot of those from the previous crater runs, with legit errors — some of them fixed in more recent versions of the crates (including semver-compatible releases only requiring a cargo update to fix the NLL errors) .

#60911:

Here as well, of the 1998 regressions, these 99 crates caused 1892 failures downstream, while these 140 crates themselves failed to build.

That's 220 crates in total (all the 131 in #60914 + these 89 new ones).

[Centril]

Nominating for lang team (possibly also compiler team for execution) to discuss ^-- and how to proceed with that data.

[Centril]

We did not get to discuss this on the language team meeting. Revisiting next time.

[Centril]

Sadly, we did not have enough time to discuss this on the language team meeting this week either, rescheduling for next time.

[pnkfelix]

discussed in lang team meeting.

The overall set of regressions was not a deal breaker for the people present in the meeting.

One trend we noted with the listed regressions is that a lot of the problems are coming from ... "old crates", and the main point will be "yep, someone needs to update their dependencies."

So, as part of that, is that we want to write a blog post discussing the overall plan here. in particular, we want to discuss the overall migration to NLL, stating that NLL is coming for every Rust edition, and then saying that the final transition (to hard errors) is fixing soundness bugs. So, part of that post will present a representative sample of real world code with bugs that are being detected by NLL.

(I am taking on the job of writing the aforementioned post; @nikomatsakis is going to provide guidance about where the post will end up being published; i.e. what forum is most appropriate for it. I plan to have the post ready in time for the next Rust release.)

[Centril]

(With my release team hat on, we will need to talk about NLL in 2015 in the 1.35.0 blog post in any case, and so linking to the aforementioned to-write blogpost would be a ideal).

[Stargateur]

Can you only list crate >= 1.0.0 ? Maybe sort by last update too.

I mean if I understand correctly, these crates "should" not compile, there are probably not maintains. Do we really have the ressource to keep them working for almost no gain ?