RFC: Preview for Unstable Features #3120
Conversation
Making select unstable features available on stable Rust behind a preview flag. [Rendered](https://github.com/jonhoo/rfcs/blob/preview-features/text/0000-preview-features.md)
|
It's not clear to me why people can't use Stable to develop and then do a particular build trial with Nightly. This seems like the simplest solution since it doesn't even take an RFC. It's not clear to me why the opt-in-to-preview can't be done with Beta getting the opt-in. The RFC only seems to consider Beta in the context of allowing every feature ever. Since allowing every feature ever is obviously bad, that part feels like a strawman. As far as I know, Beta is about as bug free as Stable because stuff doesn't generally hit Beta in the first place until it's ready for stabilization. It's not clear to me what would prevent people from using a preview feature in 1.60 and then refusing to update to 1.61 if the preview ended without acceptance. It's unclear what happens when a preview feature is accepted. Since updates ride from Nightly to Beta to Stable, if something is accepted for stabilization, it should ride to Beta then Stable, so does it exist as a preview in 1.60, then not at all in 1.61, and then suddenly exists again in 1.62? Likely it would just exist as a preview for 1.61 as well, but I'd like that written down somewhere. Saying that a preview is available "usually a bit longer than one release cycle" seems very strange to me, since Stable either has or does not have the preview. Are you suggesting that we'd alter the 1.60 Stable release 7 weeks (or whatever duration) after it is first released and suddenly people would be downloading a different binary than they did yesterday? Because I sure don't like that idea. (all specific version numbers are for illustrative purposes only) |
| internals.rust-lang.org][thread]: | ||
|
|
||
| - `rustc -Zstrip`: https://github.com/rust-lang/rust/issues/72110 | ||
| - `rustc -Zoom=panic`: https://github.com/rust-lang/rust/issues/43596 |
There was a problem hiding this comment.
-Zoom does not exist
There was a problem hiding this comment.
Oh, yeah, good catch. How is oom=panic even expressed today @kornelski?
There was a problem hiding this comment.
AFAIK it doesn't exist. There was a misunderstanding around oom handlers in #51245 — they were supposed to be the mechanism for oom=panic, but in the end the handlers were designed to forbid panicking.
There was a problem hiding this comment.
Ah, thanks for the update! I still think oom=panic is a good candidate for inclusion here, so it'd just be a matter of adjusting based on whatever the enabling mechanism will end up being.
A build run with nightly doesn't really provide much supporting evidence for a feature like stripping binaries of debug symbols or a Cargo change like
I'm not sure exactly what you're proposing here, so I'll try to tackle both my interpretations:
This would mean that users cannot test only a subset of preview features, which opens up more risk as more unstable features are enabled. Furthermore, it makes beta different enough from stable that users may be hesitant to test on stable as it also exhibits behaviors that won't land in the resulting stable. Or, phrased differently, the fact that something works on beta no longer means that it will work on the resulting stable, which I think is a property we want to preserve.
This isn't a bad idea, and is one that's been proposed before, and wasn't outright discarded at the time. I think the biggest reason to not make it beta-only is that beta isn't as stable as stable, and is thus riskier to adopt (and thus less likely to be adopted). Concretely:
This is only kind of true. When a new beta is cut, it is exactly as (un)stable as the nightly it was cut from. There's no difference between the two. Then, over time as the beta receives backports, it matures and becomes more stable, until six weeks later when it is actually ready to become the next stable release.
They wouldn't get new features and bug fixes. This would be the same as pinning an old nightly. Of course, you're right that we generally want the barrier to staying with the latest release at all times to be as low as possible, otherwise the ecosystem ends up stagnating, which is exactly why I propose that preview features be of a very specific kind. Specifically, those that are only likely to be used at the "leaves" of the dependency graph. Pinning the compiler at the leaves is much less problematic than a crate in the "middle" requiring a particular stable version, since there aren't any dependencies (and thus chunks of the ecosystem) that are also held back. This property, in turn, adds pressure to keeping with the latest stable — as the ecosystem above such a pinned leaf crate evolves, and adopts features from newer stable releases, the leaf crate must either drop the pin (and the preview feature), or start pinning all their dependencies as well. And the latter is an option that I think it's fine to let users have — users should have the option to stay in the past if they really want to.
Right, this is why the RFC suggests that if a preview feature is stabilized, its preview period should be extended to the stable release in which it will land. That way, there won't be any gaps.
Ah, sorry, no, that's not quite what I meant. The intention wasn't to imply that the stable release would change mid-way through, but rather that the preview window should align with some release after the next, which ends up being "a bit longer than one release cycle" (1.5 on average). |
|
I like the idea of distinguishing almost-stable/release-candidate features from truly unstable features that are work in progress or rustc-internal. This distinction may be useful even just for documentation and encouraging users to try soon-to-be-stabilized features. |
|
I remember the mess of
|
|
If the concern is a downstream user pinning a compiler version to get an unstable feature on stable-channel Rust, why not make the expiry annotation take a timestamp rather than a compiler version number? That should make the existing option of abusing |
|
that would be an absolutely unacceptable system. |
|
@Lokathor Would you mind elaborating? |
|
The compiler should not look at the system clock when determining what to do with a piece of code. The compiler should instead look at as little external state as possible when compiling code. |
|
Normally, I'd agree, given the importance of reproducible builds, but this is being pitched as something akin to Mozilla Firefox Studies, and the code paths which query the time wouldn't even be invoked unless a downstream user specifies an option that wouldn't normally work anyway. |
That's an interesting idea! My first instinct is that features that can be conditionally compiled are unlikely to be good preview candidates since it might then be attractive to implement them in the middle of the crate graph rather than just in leaves, but if if we had a concrete unstable feature that might be featured as a @Mark-Simulacrum made the point that most of the features that fit the preview criteria are likely to be build configurations/options rather than language/library features. Things like |
Yeah this is what I was going for. Why should Stable have the previews and not Beta?
Okay, alright, I am not presuming that you would do exactly one build. I am presuming that you'd do whatever checking and testing over whatever time frame that you'd do with a Preview Feature. This can vary wildly based on the exact feature of course, details depend on the sitaution, etc etc. Essentially, while I broadly agree with you that some sort of preview system for "close to ready" features should become a norm of Rust, the text of this RFC makes no case at all for why
is somehow particularly better than
Because ultimately every Stable is a Beta which is a Nightly, and we know this because of Particularly, while Nightly does have issues, all those issues are nearly always to do with the parts of the language that aren't stable to begin with. In other words, if you have some bit of code that compiles and passes tests (and meets whatever other criteria) using the Stable compiler, then it is very unlikely that the same code will suddenly explode on you when you build it with In fact I know many people who target Stable but develop on Nightly because they want just want Nightly rustdoc or rustfmt abilities. I myself literally did not know that the whole "intra-doc links" thing had been unstable for 3 years because I've been using it this entire time because docs.rs always renders docs using Nightly anyway. And my most popular crates target 1.36 and 1.34, that's pretty Stable. So why are we building this system of previews into cargo and the compiler, and then having an FCP bureaucracy for when to do previews. Why can't people who want to preview something just build their current codebase with Again, I do think that not enough people trying out the slightly unstable stuff is a problem, and I do think that getting people to try out the nearly-stable stuff sooner is a good plan. I just don't think that the text of the RFC makes a compelling case about this particular solution. |
@Lokathor I beg to differ. There are over 1500 (closed) issues labeled as regression in the Rust repository. Which shouldn't be surprising; Neither automated tests nor careful code reviews can prove the absence of bugs. And sometimes bugs affect stable parts of the language. Most of these regressions are probably really rare, so you probably never encountered one. But they do matter to big corporations; When they deploy an important piece of software with thousands or millions of users, they want to be absolutely sure that the code wasn't miscompiled, or that the borrow checker didn't accidentally accept invalid code. Stable can still contain bugs, but they're much less likely on Stable than on Nightly. If you question this, you are questioning the very purpose of the Stable channel. |
|
I'm aware of regressions existing, and I'm aware that they slip through. I spent part of last summer completely unable to use Stable for two releases because of ARM regressions that took a long time to resolve. Yay. But if you need that maximum level of assurance then you shouldn't be deploying your preview builds anyway. Even if they're based on Stable rather than Beta. |
That sounds to me like these big corporations can't just willy-nilly take new stable compilers either. Presumably they have some centralized way to roll them out, and a way to run validation passes against the internal projects before actually applying the new version to everything. So couldn't that be done for a pinned nightly, too? It could even be the nightly that was copied exactly to be become beta. It might need to be rolled back. But it'll need to be rolled back in 6 weeks anyway when the preview ends. (And I would expect that CI would still need to pass against the stable compiler for normal merged code -- which immediately mitigates any concern about people trying to use extra |
I'm not sure I follow — I think both beta and stable should have preview features, and that's also what I tried to say in the RFC. Did that not come across?
This is definitely the crux of the discussion so far, so let me try to articulate it better. The primary concern is that nightly is not as stable as stable. While it's true that any fix that's on stable is also on nightly, nightly is also seeing additional changes on an ongoing basis, any one of which may cause new issues. Some of those are simple to debug — code suddenly fails to compile, which would be caught by any build. But some are not, such as miscompilations or borrow checker regressions. It's true that some of those could be caught by compiling everything twice (with stable and nightly), but not only is that wasteful, it also doesn't seem like it shouldn't be necessary — after all, I don't need or want those extra nightly changes, I just want the one feature I've signed up for testing, and everything else I want to be stable. I think, arguably, the better question is why is better than which also gets at your point
There are two reasons why the former is better in my mind. The first is that the This ties into your other remark that
I think this depends on what the preview feature in question is (e.g.,
I think the answer here is "it's complicated". Basically, while it's true that rolling out a new version of the compiler goes through a number of build checks and such, that can only catch some subset of issues. And while it's possible that the stable compiler triggers, say, a miscompilation, or has an algorithmic (i.e., runtime) error somewhere, that is much less likely than that such an error appears on a nightly. You are probably right that it's possible to find a nightly that has no problems that impact any project at a large org., but it would likely be a fairly involved and semi-manual process to discover such a nightly. We could totally just say that that is the process large orgs. should follow — find a nightly that works for you if you want to try a feature — but I'm hoping we can do better than that precisely by leveraging the process that already exists for finding a "good" version of the compiler: stable.
Yeah, rollbacks for when a preview ends with a decision not to stabilize are going to be a pain. But the hope would be that that is a somewhat rare occurrence. If most previews end in a decision not to stabilize, that suggests that too many features are being added to preview, or at least that they are being added prematurely. |
| alternatives][rationale-and-alternatives] for alternatives). But since | ||
| stabilization requires evidence of impact and maturity, these features | ||
| can get caught in a Catch-22 situation where evidence won't be provided | ||
| until they're stabilized, and they can't be stabilized without evidence. |
There was a problem hiding this comment.
My biggest meta-question here is whether we've sufficiently tried asking for more evidence on these things.
For example, the "enabled in one place in a build system" feature of cargo pipelining, where as far as I can tell, asking for people to try it out and give feedback in https://internals.rust-lang.org/t/evaluating-pipelined-rustc-compilation/10199 was highly successful without any special technical mechanism.
How much of the value here could we get from something that wouldn't need an RFC at all, such as an occasional blog post asking people to try out certain features on a particular known-to-be-reasonable nightly?
There was a problem hiding this comment.
It's a good point! I think a combination of a recommended, well-chosen nightly plus an explicit reminder to set -Zallow-features gets us quite far. But at the same time, I feel like "recommended, well-chosen nightly" is really just a suboptimal version of env RUSTC_BOOSTRAP=1 cargo +stable (or +beta maybe rather).
To be clear, I'm not opposed to recommending RUSTC_BOOSTRAP for these kinds of use-cases. My biggest grip with it is that it opts into everything, and then users have to know to opt out with -Zallow-features. And they have to make a (fairly tricky) choice for each feature whether or not to allow it. My thinking with preview features was basically that rather than relegate that choosing to users, it's made by those who are best equipped to decide the stability (and fragility) of a feature.
There was a problem hiding this comment.
And they have to make a (fairly tricky) choice for each feature whether or not to allow it.
I don't really see that. If I use the same example above, I don't get the impression that people started turning on other stuff as part of it. I would expect that a post saying "hey, try out this cool thing by running RUSTC_FLAGS=-Zstrip cargo +nightly build" would generally have people doing just that, not also suddenly adding some other random features like #[feature(link_llvm_intrinsics)] or something.
(Especially in a big corporate setting, where they're all the more likely to have strict CI and code reviews and static analysis and be using their own builds instead of shipped ones because of toolchain attack concerns and ...)
But at the same time, I feel like "recommended, well-chosen nightly" is really just a suboptimal version of [...]
There's a ≈9 week lag involved in that, though. Is waiting that much time really worth the slight increase to reliability of the compiler -- especially if something does go wrong and a fix is needed?
(Now is a particularly ironic time for that, too, given current stable has some stumbling blocks.)
It only clicked in my brain exactly as I read your sentence. So, no, it's not clear in the RFC that the preview would be on both Stable and Beta. I was comlpetely under the impression that the RFC intends for a preview to be set up on a particular Stable release only, and not on Beta at any time.
Absolutely agree. I do not think that people should be using
Technically true, but if your codebase builds with Stable it's naturally not using any features, so passing one extra top level flag to the build wouldn't suddenly start using any features even if they're allowed. My thinking is:
|
|
I don't have time to write a real comment here right this moment, but I am extremely uncomfortable with this idea. Hopefully I'll be able to return to it in a few days, but I'm honestly finding it a bit difficult to even put into words. |
|
Hmm, I'm conflicted on this. The RFC limits what features can be put in preview excluding almost all language and library changes (limiting its usefulness), while still risking that preview features get used so much that we have to stabilize them even if we find issues in their design.
To me this proposal seems to be targeted more at big corporations, who often have their own toolchain. I'm wondering if we can make some changes to rustbuild to better accomodate that use case. To build their toolchain they could grab a stable source code tarball and build it with this [rust]
# Already implemented. Would build the stable source code as a nightly.
channel = "nightly"
# *New* proposed configuration key. Would limit which features can be enabled in the toolchain at compile time.
allowed-features = ["patch-in-config", "const_generics"]That way you'd have the stability of the stable source code while allowing just a subset of features, preventing developers from using unapproved features. Would that work @jonhoo? |
|
@pietroalbini I like the proposal in theory, but I think in practice it buys little over Judging from the discussion thus far, it seems like the general take is that this kind of approach isn't the right one, and that those who wish to test unstable features should:
I'm going to close out the RFC at this point, but happy to keep discussing alternative approaches. I think my main opposition to the above 3 approaches are:
|
|
FWIW, I at least think we should aim to find some better solutions here. I think discussion thus far has focused largely on the case of features that can be materially tested by enabling them for some build (e.g., a one-off nightly build) and then providing an experience report. As Rust matures, though, I suspect that many features will really want a "testing ground" that encompasses a wider set of experiences. I basically agree with all three points of your "existing methods don't work" @jonhoo -- I think that's part of why I want us to invest in finding a solution that can be officially recommended and works well to not prevent upgrades and similar for most users. I think in many cases the features we'd consider for stabilization as part of this are indeed pretty likely to stabilize as-is, and if we are inclined to make breaking changes, we'd also be able to roll them out pretty smoothly to end users currently in the preview (e.g., gating the new behavior behind not opting in, and leaving the old behavior in for a few cycles). Ultimately the other side of this fence is users simply not upgrading to newer Rust versions because they're stuck on a particular nightly/snapshot of stable - that seems worse. I haven't had the time to engage much on this thread but I personally continue to be interested in working towards some answer to the problem, because I think it's worthwhile to solve and not well addressed by existing solutions. What that solution looks like is not yet entirely clear, though. |
Making select unstable features available on stable Rust behind a preview flag.
This is the culmination of a lot of previous discussion:
My hope is that all the discussion should already be encapsulated in the RFC text.
Rendered