From cdd007f9715cc68f33279233130317c43b5c07aa Mon Sep 17 00:00:00 2001 From: Roxane Date: Thu, 1 Jul 2021 12:09:05 -0400 Subject: [PATCH 1/6] Provide more information about RFC2229 and migration information --- src/rust-2021/disjoint-capture-in-closures.md | 75 +++++++++++++++++-- 1 file changed, 67 insertions(+), 8 deletions(-) diff --git a/src/rust-2021/disjoint-capture-in-closures.md b/src/rust-2021/disjoint-capture-in-closures.md index 46458c50..cd700dce 100644 --- a/src/rust-2021/disjoint-capture-in-closures.md +++ b/src/rust-2021/disjoint-capture-in-closures.md @@ -3,7 +3,7 @@ ## Summary - `|| a.x + 1` now captures only `a.x` instead of `a`. -- This can subtly change the drop order of things. +- This can subtly change the drop order of things, and auto traits usage with closures. ## Details @@ -29,12 +29,71 @@ let c = || println!("{}", a.y); // Error: Tries to capture all of `a` c(); ``` -Starting in Rust 2021, closures will only capture the fields that they use. +Starting in Rust 2021, closures will only capture the fields that they use, eliminating common borrow check errors. So, the above example will compile fine in Rust 2021. -This new behavior is only activated in the new edition, -since it can change the order in which fields are dropped. -As for all edition changes, an automatic migration is available, -which will update your closures for which this matters. -It can insert `let _ = &a;` inside the closure to force the entire -struct to be captured as before. +Disjoint capture was proposed as part of [RFC 2229](https://github.com/rust-lang/rfcs/blob/master/text/2229-capture-disjoint-fields.md), and we suggest reading the RFC to better understand motivations behind the feature. + +## Changes to semantics because of disjoint capture +`#[feature(capture_disjoint_fields)]` introduces (minor) breaking change to the language. This means that there might be observable changes or valid Rust 2018 code that fails to compile once you move to Rust Edition 2021. You can use `cargo fix` with the `disjoint_capture_migrations` lint to migrate your existing code to Rust 2021. +### Wild Card Patterns +Closures now only capture data that needs to be read, which means the following closures will not capture `x` + +```rust,ignore +let x = 10; +let c = || { + let _ = x; +}; + +let c = || match x { + _ => prinln!("Hello World!") +}; +``` + +### Drop Order +Since only a part of a variable might be captured instead of the entire variable, when different fields or elements (in case of tuple) get drop, the drop order might be affected. + +```rust,ignore +{ + let t = (vec![0], vec![0]); + + { + let c = || { + move(t.0); + }; + } // c and t.0 dropped here +} // t.1 dropped here + +``` + + +### Auto Traits +Structs or tuples that implement (an) auto trait(s) and are passed along in a closure may no longer guarantee that the closure can also implement that/those auto trait(s). + +For instance, a common way to allow passing around raw pointers between threads is to wrap them in a struct and then implement `Send`/`Sync` auto trait for the wrapper. The closure that is passed to `thread::spawn` uses the specific fields within the wrapper but the entire wrapper is captured regardless. Since the wrapper is `Send`/`Sync`, the code is considered safe and therefore compiles successfully. + +With this feature only the specific field mentioned in the closure gets captured, which wasn't originally `Send`/`Sync` defeating the purpose of the wrapper. + + +```rust,ignore +struct Ptr(*mut i32); +unsafe impl Send for Ptr; + + +let mut x = 5; +let px = (&mut x as *mut i32); + +let c = thread::spawn(move || { + *(px.0) += 10; +}); // Closure captured px.0 which is not Send +``` + +## Migrations + +This new behavior is only activated in the new edition, or by specifically enabling it using `#[feature(capture_disjoint_fields)]`, +since it can change the order in which fields are dropped and can impact auto trait usage with closures. + +As for all edition changes, an automatic migration is available. +If you would like to be warned of semantics change that may impact your code, you can [use the lint](https://doc.rust-lang.org/rustc/lints/levels.html) `disjoint_capture_migrations`. The lint is also supported with [cargo fix](https://doc.rust-lang.org/cargo/commands/cargo-fix.html) to automatically migrate your code. + +The migration fix involves adding `let _ = &a;` inside the closure to force the entire variable to be captured as before. \ No newline at end of file From c72d0bcbbc4eebb2764bf3d639a27ca7af57304a Mon Sep 17 00:00:00 2001 From: Roxane Date: Fri, 2 Jul 2021 09:33:37 -0400 Subject: [PATCH 2/6] Apply suggestions Co-authored-by: Ryan Levick Co-authored-by: Eric Huss --- src/rust-2021/disjoint-capture-in-closures.md | 22 +++++++++++-------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/src/rust-2021/disjoint-capture-in-closures.md b/src/rust-2021/disjoint-capture-in-closures.md index cd700dce..b6e06f81 100644 --- a/src/rust-2021/disjoint-capture-in-closures.md +++ b/src/rust-2021/disjoint-capture-in-closures.md @@ -3,7 +3,7 @@ ## Summary - `|| a.x + 1` now captures only `a.x` instead of `a`. -- This can subtly change the drop order of things, and auto traits usage with closures. +- This can subtly change the drop order of things and whether auto traits are applied to closures or not. ## Details @@ -35,7 +35,9 @@ So, the above example will compile fine in Rust 2021. Disjoint capture was proposed as part of [RFC 2229](https://github.com/rust-lang/rfcs/blob/master/text/2229-capture-disjoint-fields.md), and we suggest reading the RFC to better understand motivations behind the feature. ## Changes to semantics because of disjoint capture -`#[feature(capture_disjoint_fields)]` introduces (minor) breaking change to the language. This means that there might be observable changes or valid Rust 2018 code that fails to compile once you move to Rust Edition 2021. You can use `cargo fix` with the `disjoint_capture_migrations` lint to migrate your existing code to Rust 2021. +Disjoint captures introduces a minor breaking change to the language. This means that there might be observable changes or valid Rust 2018 code that fails to compile once you move to Rust Edition 2021. + +When running `cargo fix --edition`, Cargo will update the closures in your code to help you migrate to Rust 2021, as described below in [Migrations](#migrations). ### Wild Card Patterns Closures now only capture data that needs to be read, which means the following closures will not capture `x` @@ -46,7 +48,7 @@ let c = || { }; let c = || match x { - _ => prinln!("Hello World!") + _ => println!("Hello World!") }; ``` @@ -68,11 +70,11 @@ Since only a part of a variable might be captured instead of the entire variable ### Auto Traits -Structs or tuples that implement (an) auto trait(s) and are passed along in a closure may no longer guarantee that the closure can also implement that/those auto trait(s). +Structs or tuples that implement auto traits and are passed along in a closure may no longer guarantee that the closure can also implement those auto traits. For instance, a common way to allow passing around raw pointers between threads is to wrap them in a struct and then implement `Send`/`Sync` auto trait for the wrapper. The closure that is passed to `thread::spawn` uses the specific fields within the wrapper but the entire wrapper is captured regardless. Since the wrapper is `Send`/`Sync`, the code is considered safe and therefore compiles successfully. -With this feature only the specific field mentioned in the closure gets captured, which wasn't originally `Send`/`Sync` defeating the purpose of the wrapper. +With disjoint captures, only the specific field mentioned in the closure gets captured, which wasn't originally `Send`/`Sync` defeating the purpose of the wrapper. ```rust,ignore @@ -90,10 +92,12 @@ let c = thread::spawn(move || { ## Migrations -This new behavior is only activated in the new edition, or by specifically enabling it using `#[feature(capture_disjoint_fields)]`, +This new behavior is only activated starting in the 2021 edition, since it can change the order in which fields are dropped and can impact auto trait usage with closures. -As for all edition changes, an automatic migration is available. -If you would like to be warned of semantics change that may impact your code, you can [use the lint](https://doc.rust-lang.org/rustc/lints/levels.html) `disjoint_capture_migrations`. The lint is also supported with [cargo fix](https://doc.rust-lang.org/cargo/commands/cargo-fix.html) to automatically migrate your code. +When running `cargo fix --edition`, the closures in your code may be updated so that they will retain the old behavior on both the 2018 and 2021 editions. +It does this by enabling the `disjoint_capture_migration` lint which adds statements like `let _ = &a;` inside the closure to force the entire variable to be captured as before. -The migration fix involves adding `let _ = &a;` inside the closure to force the entire variable to be captured as before. \ No newline at end of file +After migrating, it is recommended to inspect the changes and see if they are necessary. +After changing the edition to 2021, you can try to remove the new statements and test that the closure works as expected. +You should review these closures, and ensure that the changes described above will not cause any problems. From fb73551017b13370ee7fb5b1497e6469f1025f21 Mon Sep 17 00:00:00 2001 From: Roxane Date: Fri, 2 Jul 2021 09:55:24 -0400 Subject: [PATCH 3/6] Make the examples testable --- src/rust-2021/disjoint-capture-in-closures.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/src/rust-2021/disjoint-capture-in-closures.md b/src/rust-2021/disjoint-capture-in-closures.md index b6e06f81..b1390f5a 100644 --- a/src/rust-2021/disjoint-capture-in-closures.md +++ b/src/rust-2021/disjoint-capture-in-closures.md @@ -41,7 +41,7 @@ When running `cargo fix --edition`, Cargo will update the closures in your code ### Wild Card Patterns Closures now only capture data that needs to be read, which means the following closures will not capture `x` -```rust,ignore +```rust let x = 10; let c = || { let _ = x; @@ -55,13 +55,14 @@ let c = || match x { ### Drop Order Since only a part of a variable might be captured instead of the entire variable, when different fields or elements (in case of tuple) get drop, the drop order might be affected. -```rust,ignore +```rust +# fn move_value(_: T){} { let t = (vec![0], vec![0]); { let c = || { - move(t.0); + move_value(t.0); // t.0 is moved here }; } // c and t.0 dropped here } // t.1 dropped here @@ -77,16 +78,20 @@ For instance, a common way to allow passing around raw pointers between threads With disjoint captures, only the specific field mentioned in the closure gets captured, which wasn't originally `Send`/`Sync` defeating the purpose of the wrapper. -```rust,ignore +```rust +use std::thread; + struct Ptr(*mut i32); -unsafe impl Send for Ptr; +unsafe impl Send for Ptr {} let mut x = 5; -let px = (&mut x as *mut i32); +let px = Ptr(&mut x as *mut i32); let c = thread::spawn(move || { - *(px.0) += 10; + unsafe { + *(px.0) += 10; + } }); // Closure captured px.0 which is not Send ``` From 6a236ad0ddb52dd856b2b79bf2decdd237b227ed Mon Sep 17 00:00:00 2001 From: Niko Matsakis Date: Tue, 6 Jul 2021 11:46:59 -0400 Subject: [PATCH 4/6] rework text somewhat --- src/rust-2021/disjoint-capture-in-closures.md | 118 ++++++++++++------ 1 file changed, 83 insertions(+), 35 deletions(-) diff --git a/src/rust-2021/disjoint-capture-in-closures.md b/src/rust-2021/disjoint-capture-in-closures.md index b1390f5a..1db9eb34 100644 --- a/src/rust-2021/disjoint-capture-in-closures.md +++ b/src/rust-2021/disjoint-capture-in-closures.md @@ -3,7 +3,8 @@ ## Summary - `|| a.x + 1` now captures only `a.x` instead of `a`. -- This can subtly change the drop order of things and whether auto traits are applied to closures or not. +- This can cause things to be dropped at different times or affect whether closures implement traits like `Send` or `Clone`. + - If possible changes are detected, `cargo fix` will insert statements like `let _ = &a` to force a closure to capture the entire variable. ## Details @@ -11,40 +12,54 @@ automatically capture anything that you refer to from within their body. For example, `|| a + 1` automatically captures a reference to `a` from the surrounding context. -Currently, this applies to whole structs, even when only using one field. +In Rust 2018 and before, closures capture entire variables, even if the closure only uses one field. For example, `|| a.x + 1` captures a reference to `a` and not just `a.x`. -In some situations, this is a problem. -When a field of the struct is already borrowed (mutably) or moved out of, -the other fields can no longer be used in a closure, -since that would capture the whole struct, which is no longer available. +Capturing `a` in its entirety prevents mutation or moves from other fields of `a`, so that code like this does not compile: ```rust,ignore let a = SomeStruct::new(); - drop(a.x); // Move out of one field of the struct - println!("{}", a.y); // Ok: Still use another field of the struct - let c = || println!("{}", a.y); // Error: Tries to capture all of `a` c(); ``` -Starting in Rust 2021, closures will only capture the fields that they use, eliminating common borrow check errors. -So, the above example will compile fine in Rust 2021. +Starting in Rust 2021, closures captures are more precise. Typically they will only capture the fields they use (in some cases, they might capture more than just what they use, see the Rust reference for full details). Therefore, the above example will compile fine in Rust 2021. + +Disjoint capture was proposed as part of [RFC 2229](https://github.com/rust-lang/rfcs/blob/master/text/2229-capture-disjoint-fields.md) and the RFC contains details about the motivation. + +## Migrating to Rust 2021 -Disjoint capture was proposed as part of [RFC 2229](https://github.com/rust-lang/rfcs/blob/master/text/2229-capture-disjoint-fields.md), and we suggest reading the RFC to better understand motivations behind the feature. +Changing the variables captured by a closure can cause programs to change behavior or to stop compiling in two cases: -## Changes to semantics because of disjoint capture -Disjoint captures introduces a minor breaking change to the language. This means that there might be observable changes or valid Rust 2018 code that fails to compile once you move to Rust Edition 2021. +- changes to drop order, or when destructors run ([details](#drop-order)); +- changes to which traits a closure implements ([details](#trait-implementations)). When running `cargo fix --edition`, Cargo will update the closures in your code to help you migrate to Rust 2021, as described below in [Migrations](#migrations). + +Whenever any of the scenarios below are detected, `cargo fix` will insert a "dummy let" into your closure to force it to capture the entire variable: + +```rust +let x = (vec![], vec![]); +let c = move || { + // "Dummy let" that forces `x` to be captured in its entirety + let _ = &x; + + // Otherwise, only `x.0` would be captured here + println!("{:?}", x.0); +}; +``` + +This is a conservative analysis: in many cases, these dummy lets can be safely removed and your program will work fine. + ### Wild Card Patterns -Closures now only capture data that needs to be read, which means the following closures will not capture `x` + +Closures now only capture data that needs to be read, which means the following closures will not capture `x`: ```rust let x = 10; let c = || { - let _ = x; + let _ = x; // no-op }; let c = || match x { @@ -52,8 +67,28 @@ let c = || match x { }; ``` +The `let _ = x` statement here is a no-op, since the `_` pattern completely ignores the right-hand side, and `x` is a reference to a place in memory (in this case, a variable). + +This change by itself (capturing fewer values) doesn't trigger any suggestions, but it may do so in conjunction with the "drop order" change below. + +**Subtle:** There are other similar expressions, such as the "dummy lets" `let _ = &x` that we insert, which are not no-ops. This is because the right-hand side (`&x`) is not a reference to a place in memory, but rather an expression that must first be evaluated (and whose result is then discarded). + ### Drop Order -Since only a part of a variable might be captured instead of the entire variable, when different fields or elements (in case of tuple) get drop, the drop order might be affected. + +When a closure takes ownership of a value from a variable `t`, that value is then dropped when the closure is dropped, and not when the variable `t` goes out of scope: + +```rust +# fn move_value(_: T){} +{ + let t = (vec![0], vec![0]); + + { + let c = || move_value(t); // t is moved here + } // c is dropped, which drops the tuple `t` as well +} // t goes out of scope here +``` + +The above code will run the same in both Rust 2018 and Rust 2021. However, in cases where the closure only takes ownership of _part_ of a variable, there can be differences: ```rust # fn move_value(_: T){} @@ -62,22 +97,47 @@ Since only a part of a variable might be captured instead of the entire variable { let c = || { - move_value(t.0); // t.0 is moved here + // In Rust 2018, captures all of `t`. + // In Rust 2021, captures only `t.0` + move_value(t.0); }; - } // c and t.0 dropped here -} // t.1 dropped here + // In Rust 2018, `c` (and `t`) are both dropped when we + // exit this block. + // + // In Rust 2021, `c` and `t.0` are both dropped when we + // exit this block. + } + +// In Rust 2018, the value from `t` has been moved and is +// not dropped. +// +// In Rust 2021, the value from `t.0` has been moved, but `t.1` +// remains, so it will be dropped here. +} ``` +In most cases, dropping values at different times just affects when memory is freed and is not important. However, some `Drop` impls (aka, destructors) have side-effects, and changing the drop order in those cases can alter the semantics of your program. In such cases, the compiler will suggest inserting a dummy `let` to force the entire variable to be captured. -### Auto Traits -Structs or tuples that implement auto traits and are passed along in a closure may no longer guarantee that the closure can also implement those auto traits. +### Trait implementations + +Closures automatically implement the following traits based on what values they capture: + +- [`Clone`]: if all captured values are [`Clone`]. +- [Auto traits] like [`Send`], [`Sync`], and [`UnwindSafe`]: if all captured values implement the given trait. + +[auto traits]: https://doc.rust-lang.org/nightly/reference/special-types-and-traits.html#auto-traits +[`clone`]: https://doc.rust-lang.org/std/clone/trait.Clone.html +[`send`]: https://doc.rust-lang.org/std/marker/trait.Send.html +[`sync`]: https://doc.rust-lang.org/std/marker/trait.Sync.html +[`unwindsafe`]: https://doc.rust-lang.org/std/marker/trait.UnwindSafe.html + +In Rust 2021, since different values are being captured, this can affect what traits a closure will implement. The migration lints test each closure to see whether it would have implemented a given trait before and whether it still implements it now; if they find that a trait used to be implemented but no longer is, then "dummy lets" are inserted. For instance, a common way to allow passing around raw pointers between threads is to wrap them in a struct and then implement `Send`/`Sync` auto trait for the wrapper. The closure that is passed to `thread::spawn` uses the specific fields within the wrapper but the entire wrapper is captured regardless. Since the wrapper is `Send`/`Sync`, the code is considered safe and therefore compiles successfully. With disjoint captures, only the specific field mentioned in the closure gets captured, which wasn't originally `Send`/`Sync` defeating the purpose of the wrapper. - ```rust use std::thread; @@ -94,15 +154,3 @@ let c = thread::spawn(move || { } }); // Closure captured px.0 which is not Send ``` - -## Migrations - -This new behavior is only activated starting in the 2021 edition, -since it can change the order in which fields are dropped and can impact auto trait usage with closures. - -When running `cargo fix --edition`, the closures in your code may be updated so that they will retain the old behavior on both the 2018 and 2021 editions. -It does this by enabling the `disjoint_capture_migration` lint which adds statements like `let _ = &a;` inside the closure to force the entire variable to be captured as before. - -After migrating, it is recommended to inspect the changes and see if they are necessary. -After changing the edition to 2021, you can try to remove the new statements and test that the closure works as expected. -You should review these closures, and ensure that the changes described above will not cause any problems. From 16d12b46d08341226230514fd6697031c0180ef0 Mon Sep 17 00:00:00 2001 From: Niko Matsakis Date: Tue, 6 Jul 2021 15:46:17 -0400 Subject: [PATCH 5/6] make tests pass --- src/rust-2021/disjoint-capture-in-closures.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rust-2021/disjoint-capture-in-closures.md b/src/rust-2021/disjoint-capture-in-closures.md index 1db9eb34..410040b2 100644 --- a/src/rust-2021/disjoint-capture-in-closures.md +++ b/src/rust-2021/disjoint-capture-in-closures.md @@ -40,7 +40,7 @@ When running `cargo fix --edition`, Cargo will update the closures in your code Whenever any of the scenarios below are detected, `cargo fix` will insert a "dummy let" into your closure to force it to capture the entire variable: ```rust -let x = (vec![], vec![]); +let x = (vec![22], vec![23]); let c = move || { // "Dummy let" that forces `x` to be captured in its entirety let _ = &x; From ce4ffb3fcf96d63cbb6663fa2dee4eb6b763a824 Mon Sep 17 00:00:00 2001 From: Niko Matsakis Date: Tue, 6 Jul 2021 17:09:49 -0400 Subject: [PATCH 6/6] fix broken link --- src/rust-2021/disjoint-capture-in-closures.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/rust-2021/disjoint-capture-in-closures.md b/src/rust-2021/disjoint-capture-in-closures.md index 410040b2..f82b486a 100644 --- a/src/rust-2021/disjoint-capture-in-closures.md +++ b/src/rust-2021/disjoint-capture-in-closures.md @@ -35,8 +35,6 @@ Changing the variables captured by a closure can cause programs to change behavi - changes to drop order, or when destructors run ([details](#drop-order)); - changes to which traits a closure implements ([details](#trait-implementations)). -When running `cargo fix --edition`, Cargo will update the closures in your code to help you migrate to Rust 2021, as described below in [Migrations](#migrations). - Whenever any of the scenarios below are detected, `cargo fix` will insert a "dummy let" into your closure to force it to capture the entire variable: ```rust