From a18aafc9e59fe50dfb5fd79d0ca818e1c6174dfc Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Sun, 25 Aug 2019 20:12:31 +0200 Subject: [PATCH 01/10] note on consts --- README.md | 6 +++-- const.md | 40 ++++++++++++++---------------- promotion.md | 69 +++++++++++++++++++++++++++++++++++++--------------- static.md | 32 ++++++++++++++++++++++++ 4 files changed, 104 insertions(+), 43 deletions(-) create mode 100644 static.md diff --git a/README.md b/README.md index 1038bc4..91afa6a 100644 --- a/README.md +++ b/README.md @@ -21,8 +21,10 @@ which sort of is a virtual machine using `MIR` as "bytecode". ## Table of Contents * [Const Safety](const_safety.md) -* [Constants](const.md) -* [Promotion](promotion.md) +* The three "kinds" of compile-time evaluated data: + * [Statics](static.md) (`static`, `static mut`) + * [Constants](const.md) (`const`, array sizes, non-`Copy` array initializers) + * [Promoteds](promotion.md) (rvalue promotion) ## Related RFCs diff --git a/const.md b/const.md index 9c0173a..b7e3f9e 100644 --- a/const.md +++ b/const.md @@ -1,13 +1,15 @@ -# Further restrictions for constants +# Constants -The [const safety](const_safety.md) concerns apply to all computations happening -at compile-time, `static` and `const` alike. However, there are some additional -considerations about `const` specifically. These arise from the idea that +"Constants" in this document refers to `const` bodies, array sizes, and non-`Copy` array initializers. +On top of what applies to [statics](static.md), they are subject to an additional constraint: In code like ```rust const CONST: T = EXPR; ``` is supposed to behave as-if `EXPR` was written at every use site of `CONST`. +Based on this requirement, we allow other constants and [promoteds](promotion.md) to read from constants. +This is why the value of a `const` is subject to validity checks. + ## References One issue is constants of reference type: @@ -23,9 +25,9 @@ const REF: &u32 = { const _VAL = EXPR; static _STATIC = EXPR; &_STATIC }; (`EXPR` is assigned to a `const` first to make it subject to the restrictions discussed in this document.) -There are three reasons why this could be an issue. +There are various reasons why this could be an issue. -### Pointer equality +### 1. Pointer equality We effectively "deduplicate" all the allocations that would otherwise locally be created at each use site of `REF`. This is observable when the program compares @@ -33,13 +35,18 @@ these pointers for equality. We consider this okay, i.e., programs may not rely on such constants all getting distinct addresses. They may not rely on them all getting the same address either. -### Interior mutability +### 2. Interior mutability If the reference has type `&Cell` it is quite clear that the program can easily observe whether two references point to the same memory even without comparing their address: Changes through one reference will affect reads through the other. So, we cannot allow constant references to types that have interior -mutability (types that are not `Freeze`). +mutability (types that are not `Freeze`): + +```rust +const BAD: &Cell = &Cell::new(42); +// Inlining `BAD` everywhere clearly is not the same as them all pointing to the same thing. +``` However, we can do better than that: Even if a *type* is not `Freeze`, it can have *values* that do not exhibit any interior mutability. For example, `&None` @@ -47,7 +54,7 @@ at type `&Option>` would be rejected by the naive analysis above, but is actually accepted by the compiler because we know that there is no `UnsafeCell` here that would permit interior mutability. -### `Sync` +### 3. `Sync` Finally, the same constant reference is actually shared across threads. This is very similar to multiple threads having a shared reference to the same `static`, @@ -59,12 +66,9 @@ ecosystem that would break if we just started enforcing this now. See [this issue](https://github.com/rust-lang/rust/issues/49206) and the [PR attempting to fix this](https://github.com/rust-lang/rust/pull/54424/). -### `Drop` +### 4. Drop -Values of "needs drop" types -can only be used as the final initialization value of a `const` or `static` item. -They may not be used as intermediate values that would be dropped before the item -were initialized. As an example: +`Drop` is actually not an issue, at least not more so than for statics: ```rust struct Foo; @@ -76,19 +80,11 @@ impl Drop for Foo { } const FOO: Foo = Foo; // Ok, drop is run at each use site in runtime code -static FOOO: Foo = Foo; // Ok, drop is never run // Not ok, cannot run `Foo::drop` because it's not a const fn const BAR: i32 = (Foo, 42).1; ``` -This restriction might be lifted in the future after trait impls -may be declared `const` (https://github.com/rust-rfcs/const-eval/pull/8). - -Note that in promoteds this restriction can never be lifted, because -otherwise we would silently stop calling the `Drop` impl at runtime and -pull it to much earlier (compile-time). - ## Reading statics Beyond values of reference type, we have to be careful that *computing* a diff --git a/promotion.md b/promotion.md index 74a678a..5bf3f74 100644 --- a/promotion.md +++ b/promotion.md @@ -1,15 +1,21 @@ # Const promotion -"Promotion" is a mechanism that affects code like `&3`: Instead of putting it on -the stack, the `3` is allocated in global static memory and a reference with -lifetime `'static` is provided. This is essentially an automatic transformation -turning `&EXPR` into `{ const _PROMOTED = &EXPR; EXPR }`, but only if `EXPR` -qualifies. +["(Implicit) Promotion"][rfc] is a mechanism that affects code like `&3`: +Instead of putting it on the stack, the `3` is allocated in global static memory +and a reference with lifetime `'static` is provided. This is essentially an +automatic transformation turning `&EXPR` into `{ const _PROMOTED = &EXPR; EXPR +}`, but only if `EXPR` qualifies. Note that promotion happens on the MIR, not on surface-level syntax. This is relevant when discussing e.g. handling of panics caused by overflowing arithmetic. +On top of what applies to [consts](const.md), promoteds suffer from the additional issue that *the user did not ask for them to be evaluated at compile-time*. +Thus, if CTFE fails but the code would have worked fine at run-time, we broke the user's code for no good reason. +That's why we have to be very conservative with what can and cannot be promoted. + +[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1414-rvalue_static_promotion.md + ## Rules ### 1. Panics @@ -17,9 +23,8 @@ arithmetic. Promotion is not allowed to throw away side effects. This includes panicking. Let us look at what happens when we promote `&(0_usize - 1)` in a debug build: We have to avoid erroring at compile-time, because that would be promotion -breaking compilation (the code would have compiled just fine if we hadn't -promoted), but we must be sure to error correctly at run-time. In the MIR, this -looks roughly like +breaking compilation, but we must be sure to error correctly at run-time. In +the MIR, this looks roughly like ``` _tmp1 = CheckedSub (const 0usize) (const 1usize) @@ -89,18 +94,12 @@ but to abort compilation of a program that would have compiled fine if we would not have decided to promote. It is the responsibility of `foo` to not fail this way when working with const-safe arguments. -### 3. Constraints on constants - -All the [extra restrictions for constants](const.md) beyond const safety also -apply to promoteds, for the same reason: Evaluating the expression at -compile-time instead of run-time should not alter program behavior. - -### 4. Drop +### 3. Drop -Expressions containing "needs drop" types -can never be promoted. If such an expression were promoted, the `Drop` impl would -never get called on the value, even though the user did not explicitly request such -behavior by using an explicit `const` or `static` item. +Expressions returning "needs drop" types can never be promoted. If such an +expression were promoted, the `Drop` impl would never get called on the value, +even though the user did not explicitly request such behavior by using an +explicit `const` or `static` item. As expression promotion is essentially the silent insertion of a `static` item, and `static` items never have their `Drop` impl called, the `Drop` impl of the promoted @@ -111,6 +110,38 @@ it is unlikely to be the desired behavior in most cases and very likey to be con to the user. If such behavior is desired, the user can still use an explicit `static` or `const` item and refer to that. +## `&` in `const` and `static` + +Promotion is also responsible for making code like this work: + +```rust +const FOO: &'static i32 = { + let x = &13; + x +}; +``` + +However, since this is in explicit const context, we could be less strict about +promotion in this situation. + +Promotion is *not* involved in something like this: + +```rust +#![feature(const_vec_new)] +const EMPTY_BYTES: &Vec = &Vec::new(); + +const NESTED: &'static Vec = { + // This does not work when we have an inner scope: + let x = &Vec::new(); //~ ERROR: temporary value dropped while borrowed + x +}; +``` + +In `EMPTY_BYTES`, the reference obtains the lifetime of the "enclosing scope", +similar to how `let x = &mut x;` creates a reference whose lifetime lasts for +the enclosing scope. This is decided during MIR building already, and does not +involve promotion. + ## Open questions * There is a fourth kind of CTFE failure -- resource exhaustion. What do we do diff --git a/static.md b/static.md new file mode 100644 index 0000000..9b78e8c --- /dev/null +++ b/static.md @@ -0,0 +1,32 @@ +# Statics + +Statics (`static`, `static mut`) are the simplest kind of compile-time evaluated data: +The user explicitly requested them to be evaluated at compile-time, +so evaluation errors from computing the initial value of a static are no concern. +They observably get evaluated *once*, with the result being put at some address known at run-time, +so there are no fundamental restrictions on what statics can do. +The compiler checks that statics are `Sync`, justifying sharing their address across threads. +[Constants](const.md) and [promoteds](promotion.md) are not allowed to read from statics, +so their final value does not have have to be [const-valid](const_safety.md) in any meaningful way +(but as of 2019-08, we do check them for validity anyway, to be conservative). + +## `Drop` + +The compiler rejects intermediate values (created and discarded during the computation of a static initializer) that implement `Drop`. +The reason for this is simply that the `Drop` implementation might be non-`const fn`. +This restriction can be lifted once `const impl Drop for Type` (or something similar) is supported. + +```rust +struct Foo; + +impl Drop for Foo { + fn drop(&mut self) { + println!("foo dropped"); + } +} + +static FOOO: Foo = Foo; // Ok, drop is never run + +// Not ok, cannot run `Foo::drop` because it's not a const fn +static BAR: i32 = (Foo, 42).1; +``` From a9c7047c73398872ea5b8eb6bf027988a4a73b4e Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Sun, 25 Aug 2019 20:19:14 +0200 Subject: [PATCH 02/10] notes on dynamic checks --- const.md | 10 ++++++++++ promotion.md | 9 +++++++++ static.md | 3 +++ 3 files changed, 22 insertions(+) diff --git a/const.md b/const.md index b7e3f9e..dd73558 100644 --- a/const.md +++ b/const.md @@ -54,6 +54,11 @@ at type `&Option>` would be rejected by the naive analysis above, but is actually accepted by the compiler because we know that there is no `UnsafeCell` here that would permit interior mutability. +*Dynamic check.* The Miri engine could check this dynamically by ensuring that +the new data that is interned for a constant is all marked as +immutable. (Constants referring to already existing mutable data are not +inherently problematic.) + ### 3. `Sync` Finally, the same constant reference is actually shared across threads. This is @@ -66,6 +71,8 @@ ecosystem that would break if we just started enforcing this now. See [this issue](https://github.com/rust-lang/rust/issues/49206) and the [PR attempting to fix this](https://github.com/rust-lang/rust/pull/54424/). +*Dynamic check.* It is unclear how the Miri engine could dynamically check this. + ### 4. Drop `Drop` is actually not an issue, at least not more so than for statics: @@ -97,3 +104,6 @@ This is distinct to the concern about interior mutability above: That concern was about first computing a `&Cell` and then using it at run-time (and observing the fact that it has been "deduplicated"), this here is about using such a value at compile-time even though it might be changed at run-time. + +*Dynamic check.* The Miri engine could check this dynamically by refusing to +access mutable global memory when computing a const. diff --git a/promotion.md b/promotion.md index 5bf3f74..e62b3ef 100644 --- a/promotion.md +++ b/promotion.md @@ -51,6 +51,9 @@ earlier version of miri used to panic on arithmetic overflow even in release mode. This breaks promotion, because now promoting code that would work (and could not panic!) at run-time leads to a compile-time CTFE error. +*Dynamic check.* The Miri engine already dynamically detects panics, but the +main point of promoteds is ruling them out statically. + ### 2. Const safety We have explained what happens when evaluating a promoted panics, but what about @@ -94,6 +97,9 @@ but to abort compilation of a program that would have compiled fine if we would not have decided to promote. It is the responsibility of `foo` to not fail this way when working with const-safe arguments. +*Dynamic check.* The Miri engine already dynamically detects const safety +violations, but the main point of promoteds is ruling them out statically. + ### 3. Drop Expressions returning "needs drop" types can never be promoted. If such an @@ -110,6 +116,9 @@ it is unlikely to be the desired behavior in most cases and very likey to be con to the user. If such behavior is desired, the user can still use an explicit `static` or `const` item and refer to that. +*Dynamic check.* The Miri engine could dynamically check this by ensuring that + the result of computing a promoted is a value that does not need dropping. + ## `&` in `const` and `static` Promotion is also responsible for making code like this work: diff --git a/static.md b/static.md index 9b78e8c..678ca6a 100644 --- a/static.md +++ b/static.md @@ -30,3 +30,6 @@ static FOOO: Foo = Foo; // Ok, drop is never run // Not ok, cannot run `Foo::drop` because it's not a const fn static BAR: i32 = (Foo, 42).1; ``` + +*Dynamic check.* The Miri engine dynamically checks that this is done correctly +by not permitting calls of non-`const` functions. From 50a1829a5ccac525893ac2fc2a80879dad65f1b6 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Sun, 25 Aug 2019 20:30:10 +0200 Subject: [PATCH 03/10] link to the must-not-err RFC --- promotion.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/promotion.md b/promotion.md index e62b3ef..ed8ac21 100644 --- a/promotion.md +++ b/promotion.md @@ -1,10 +1,10 @@ # Const promotion -["(Implicit) Promotion"][rfc] is a mechanism that affects code like `&3`: +["(Implicit) Promotion"][promotion-rfc] is a mechanism that affects code like `&3`: Instead of putting it on the stack, the `3` is allocated in global static memory and a reference with lifetime `'static` is provided. This is essentially an -automatic transformation turning `&EXPR` into `{ const _PROMOTED = &EXPR; EXPR -}`, but only if `EXPR` qualifies. +automatic transformation turning `&EXPR` into +`{ const _PROMOTED = &EXPR; EXPR}`, but only if `EXPR` qualifies. Note that promotion happens on the MIR, not on surface-level syntax. This is relevant when discussing e.g. handling of panics caused by overflowing @@ -12,9 +12,11 @@ arithmetic. On top of what applies to [consts](const.md), promoteds suffer from the additional issue that *the user did not ask for them to be evaluated at compile-time*. Thus, if CTFE fails but the code would have worked fine at run-time, we broke the user's code for no good reason. +Even if we are sure we found an error in the user's code, we are only allowed to [emit a warning, not a hard error][warn-rfc]. That's why we have to be very conservative with what can and cannot be promoted. -[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1414-rvalue_static_promotion.md +[promotion-rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1414-rvalue_static_promotion.md +[warn-rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1229-compile-time-asserts.md ## Rules From cbd7e7eb76d669bae6a51ba6cf7f5165f63ac798 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 27 Aug 2019 09:28:35 +0200 Subject: [PATCH 04/10] clarify --- static.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/static.md b/static.md index 678ca6a..5714ea7 100644 --- a/static.md +++ b/static.md @@ -1,14 +1,15 @@ # Statics Statics (`static`, `static mut`) are the simplest kind of compile-time evaluated data: -The user explicitly requested them to be evaluated at compile-time, -so evaluation errors from computing the initial value of a static are no concern. -They observably get evaluated *once*, with the result being put at some address known at run-time, -so there are no fundamental restrictions on what statics can do. -The compiler checks that statics are `Sync`, justifying sharing their address across threads. -[Constants](const.md) and [promoteds](promotion.md) are not allowed to read from statics, -so their final value does not have have to be [const-valid](const_safety.md) in any meaningful way -(but as of 2019-08, we do check them for validity anyway, to be conservative). +* The user explicitly requested them to be evaluated at compile-time, + so evaluation errors from computing the initial value of a static are no concern + (in other words, [const safety](const_safety.md) is mostly not an issue). +* They observably get evaluated *once*, with the result being put at some address known at run-time, + so there are no fundamental restrictions on what statics can do. +* The compiler checks that statics are `Sync`, justifying sharing their address across threads. +* [Constants](const.md) and [promoteds](promotion.md) are not allowed to read from statics, + so their final value does not have have to be [const-valid](const_safety.md#const-safety-check-on-values) in any meaningful way. + As of 2019-08, we do check them for validity anyway, to be conservative; and indeed constants could be allowed to read from frozen statics. ## `Drop` From 53d020075d95e21261609373be3e3557cdb310a2 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 27 Aug 2019 09:29:48 +0200 Subject: [PATCH 05/10] link const -> const_safety --- const.md | 1 + 1 file changed, 1 insertion(+) diff --git a/const.md b/const.md index dd73558..645695b 100644 --- a/const.md +++ b/const.md @@ -6,6 +6,7 @@ On top of what applies to [statics](static.md), they are subject to an additiona const CONST: T = EXPR; ``` is supposed to behave as-if `EXPR` was written at every use site of `CONST`. +To make this work, we need to ensure [const safety](const_safety.md). Based on this requirement, we allow other constants and [promoteds](promotion.md) to read from constants. This is why the value of a `const` is subject to validity checks. From 7f8c81cf558e5979a080b7af7befc277431920d8 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 27 Aug 2019 09:33:28 +0200 Subject: [PATCH 06/10] mention rustc_promotable attribute --- promotion.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/promotion.md b/promotion.md index ed8ac21..b2d2e1a 100644 --- a/promotion.md +++ b/promotion.md @@ -72,7 +72,7 @@ everything, so the only possible remaining failure are panics. However, things get more tricky when `const` and `const fn` are involved. -For `const`, based on the const safety check described [here](const_safety.md), +For `const`, based on the const safety check described [here](const_safety.md#const-safety-check-on-values), we can rely on there not being const-unsafe values in the `const`, so we should be able to promote freely. For example: @@ -99,6 +99,9 @@ but to abort compilation of a program that would have compiled fine if we would not have decided to promote. It is the responsibility of `foo` to not fail this way when working with const-safe arguments. +For this reason, only `const fn` that were explicitly marked with the +`#[rustc_promotable]` attribute are subject to promotion. + *Dynamic check.* The Miri engine already dynamically detects const safety violations, but the main point of promoteds is ruling them out statically. From 87c3c871d8db5da96d0f8372c37b7e9ba3a2cbd2 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 27 Aug 2019 18:02:09 +0200 Subject: [PATCH 07/10] we are less strict in const bodies --- promotion.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/promotion.md b/promotion.md index b2d2e1a..75ab593 100644 --- a/promotion.md +++ b/promotion.md @@ -122,7 +122,7 @@ to the user. If such behavior is desired, the user can still use an explicit `st or `const` item and refer to that. *Dynamic check.* The Miri engine could dynamically check this by ensuring that - the result of computing a promoted is a value that does not need dropping. +the result of computing a promoted is a value that does not need dropping. ## `&` in `const` and `static` @@ -135,8 +135,9 @@ const FOO: &'static i32 = { }; ``` -However, since this is in explicit const context, we could be less strict about -promotion in this situation. +However, since this is in explicit const context, we are less strict about +promotion in this situation: all function calls are promoted, not just +`#[rustc_promotable]` functions. Promotion is *not* involved in something like this: From 95acdf25502ae9b291fdfc7bbb83031a124dd52f Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 27 Aug 2019 21:31:42 +0200 Subject: [PATCH 08/10] update status of interning-time const check --- const.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/const.md b/const.md index 645695b..35a1c6a 100644 --- a/const.md +++ b/const.md @@ -55,10 +55,11 @@ at type `&Option>` would be rejected by the naive analysis above, but is actually accepted by the compiler because we know that there is no `UnsafeCell` here that would permit interior mutability. -*Dynamic check.* The Miri engine could check this dynamically by ensuring that -the new data that is interned for a constant is all marked as -immutable. (Constants referring to already existing mutable data are not -inherently problematic.) +*Dynamic check.* The Miri engine enforces this dynamically by ensuring that the +new data that is interned for a constant is all marked as immutable. However, +note the FIXME added [by this PR](https://github.com/rust-lang/rust/pull/63955): +for untyped data in a constant, we currently just *make* it immutable, instead +of checking properly. ### 3. `Sync` From a29f147b5c33acfc0db6b59d1339d85bf21bceab Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Wed, 9 Oct 2019 13:55:10 +0200 Subject: [PATCH 09/10] const_vec_new has been stabilized --- promotion.md | 1 - 1 file changed, 1 deletion(-) diff --git a/promotion.md b/promotion.md index 75ab593..1341834 100644 --- a/promotion.md +++ b/promotion.md @@ -142,7 +142,6 @@ promotion in this situation: all function calls are promoted, not just Promotion is *not* involved in something like this: ```rust -#![feature(const_vec_new)] const EMPTY_BYTES: &Vec = &Vec::new(); const NESTED: &'static Vec = { From 1d2b7b1e03da23d6d3a0b8b7bbd18ffef085193d Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Fri, 11 Oct 2019 23:38:02 +0200 Subject: [PATCH 10/10] clarify promotion-in-consts --- promotion.md | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/promotion.md b/promotion.md index 1341834..5966bb8 100644 --- a/promotion.md +++ b/promotion.md @@ -137,24 +137,38 @@ const FOO: &'static i32 = { However, since this is in explicit const context, we are less strict about promotion in this situation: all function calls are promoted, not just -`#[rustc_promotable]` functions. - -Promotion is *not* involved in something like this: +`#[rustc_promotable]` functions: ```rust -const EMPTY_BYTES: &Vec = &Vec::new(); +const fn bar() -> i32 { 42 } + +const FOO: &'static i32 = { + let x = &bar(); // this gets promoted + x +}; +``` -const NESTED: &'static Vec = { - // This does not work when we have an inner scope: +However, we still do not promote *everything*; e.g., drop-checking still applies: + +```rust +const DROP: &'static Vec = { let x = &Vec::new(); //~ ERROR: temporary value dropped while borrowed x }; ``` -In `EMPTY_BYTES`, the reference obtains the lifetime of the "enclosing scope", -similar to how `let x = &mut x;` creates a reference whose lifetime lasts for -the enclosing scope. This is decided during MIR building already, and does not -involve promotion. +Notice that some code involving `&` *looks* like it relies on promotion but +actually does not: + +```rust +const EMPTY_BYTES: &Vec = &Vec::new(); // Ok without promotion +``` + +As we have seen above, `Vec::new()` does not get promoted. And yet this +compiles. Why that? The reason is that the reference obtains the lifetime of +the "enclosing scope", similar to how `let x = &mut x;` creates a reference +whose lifetime lasts for the enclosing scope. This is decided during MIR +building already, and does not involve promotion. ## Open questions