From 233d37ff7a8b30f1db9b51673d85f6fa5dee2a7c Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Mon, 19 Aug 2024 10:46:39 -0700 Subject: [PATCH 1/6] 2024: Add rustdoc combined doctests --- src/SUMMARY.md | 1 + src/rust-2024/rustdoc-doctests.md | 76 +++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+) create mode 100644 src/rust-2024/rustdoc-doctests.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index db16540b..652cdb7b 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -54,3 +54,4 @@ - [Never type fallback change](rust-2024/never-type-fallback.md) - [Unsafe `extern` blocks](rust-2024/unsafe-extern.md) - [Unsafe attributes](rust-2024/unsafe-attributes.md) + - [Rustdoc combined tests](rust-2024/rustdoc-doctests.md) diff --git a/src/rust-2024/rustdoc-doctests.md b/src/rust-2024/rustdoc-doctests.md new file mode 100644 index 00000000..1454cfda --- /dev/null +++ b/src/rust-2024/rustdoc-doctests.md @@ -0,0 +1,76 @@ +# Rustdoc combined tests + +🚧 The 2024 Edition has not yet been released and hence this section is still "under construction". +More information may be found in the tracking issue at . + +## Summary + +- [Doctests] are now combined into a single binary which should result in a significant performance improvement. + +## Details + +Prior the the 2024 Edition, rustdoc's "test" mode would compile each code block in your documentation as a separate executable. Although this was relatively simple to implement, it resulted in a significant performance burden when there are a large number of documentation tests. Starting with the 2024 Edition, rustdoc will attempt to combine documentation tests into a single binary, significantly reducing the overhead for compiling doctests. + +```rust +/// Adds two numbers +/// +/// ``` +/// assert_eq!(add(1, 1), 2); +/// ``` +pub fn add(left: u64, right: u64) -> u64 { + left + right +} + +/// Subtracts two numbers +/// +/// ``` +/// assert_eq!(subtract(2, 1), 1); +/// ``` +pub fn subtract(left: u64, right: u64) -> u64 { + left - right +} +``` + +In this example, the two doctests will now be compiled in a single executable. Rustdoc will essentially place each example in a separate function within a single binary. Rustdoc also adds a small `main` function which tells it which test to run. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly. + +This change is only available in the 2024 Edition to avoid potential incompatibilities with existing doctests which may not work in a combined executable. + +[doctests]: ../../rustdoc/write-documentation/documentation-tests.html +[libtest harness]: ../../rustc/tests/index.html + +### Standalone tag + +In some situations it is not possible for rustdoc to combine examples in a single executable. Rustdoc will attempt to automatically detect if this is not possible, for example: + +* Uses the [`compile_fail`][tags] language tag, which indicates that the example fails to compile. +* Uses the [`edition`][tags], which indicates the edition of the example.[^edition-tag] +* Uses global attributes like the [`global_allocator`] attribute, which could potentially interfere with other tests. +* Defines any crate-wide attributes (like `#![deny(unused)]`). +* Defines a macro that uses `$crate`, because the `$crate` path will not work correctly. + +However, rustdoc is not able to automatically determine *all* situations where an example cannot be combined with other examples. In these situations, you can add the `standalone` language tag to indicate that the example should be built as a separate executable. + +```rust +//! ``` +//! let location = std::panic::Location::caller(); +//! assert_eq!(location.line(), 5); +//! ``` +``` + +This example is sensitive to the code structure of how the example is compiled. This won't work with the "combined" approach because the line numbers will shift depending on how the doctests are combined. In these situations, you can add the `standalone` tag to force the example to be built separately just as it did in previous editions. + +```rust +//! ```standalone +//! let location = std::panic::Location::caller(); +//! assert_eq!(location.line(), 5); +//! ``` +``` + +[tags]: ../../rustdoc/write-documentation/documentation-tests.html#attributes +[`global_allocator`]: ../../std/alloc/trait.GlobalAlloc.html + +[^edition-tag]: Note that rustdoc will only combine tests if the entire crate is Edition 2024 or greater. Using `edition2024` tags in older editions will not result in those tests being combined. + +## Migration + +There is no automatic migration to determine which doctests need to be annotated with the `standalone` tag. You will need to update your crate to the 2024 Edition and then run your documentation tests and see if any of them fail. If they do fail, you will need to analyze whether the test can be rewritten to be compatible with the combined approach, or add the `standalone` tag to retain the previous behavior. From 2dc4b859cad15ac4a744608d309cb2a06a4bf8e2 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Mon, 19 Aug 2024 10:58:24 -0700 Subject: [PATCH 2/6] Remove `main` function mention. --- src/rust-2024/rustdoc-doctests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rust-2024/rustdoc-doctests.md b/src/rust-2024/rustdoc-doctests.md index 1454cfda..eaa93d1d 100644 --- a/src/rust-2024/rustdoc-doctests.md +++ b/src/rust-2024/rustdoc-doctests.md @@ -31,7 +31,7 @@ pub fn subtract(left: u64, right: u64) -> u64 { } ``` -In this example, the two doctests will now be compiled in a single executable. Rustdoc will essentially place each example in a separate function within a single binary. Rustdoc also adds a small `main` function which tells it which test to run. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly. +In this example, the two doctests will now be compiled in a single executable. Rustdoc will essentially place each example in a separate function within a single binary. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly. This change is only available in the 2024 Edition to avoid potential incompatibilities with existing doctests which may not work in a combined executable. From 53ac97c4f4c2b1e8da0bec3fc7816e67af52340f Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Mon, 19 Aug 2024 11:01:58 -0700 Subject: [PATCH 3/6] Use an attribute example that actually prevents combining --- src/rust-2024/rustdoc-doctests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rust-2024/rustdoc-doctests.md b/src/rust-2024/rustdoc-doctests.md index eaa93d1d..3e443305 100644 --- a/src/rust-2024/rustdoc-doctests.md +++ b/src/rust-2024/rustdoc-doctests.md @@ -45,7 +45,7 @@ In some situations it is not possible for rustdoc to combine examples in a singl * Uses the [`compile_fail`][tags] language tag, which indicates that the example fails to compile. * Uses the [`edition`][tags], which indicates the edition of the example.[^edition-tag] * Uses global attributes like the [`global_allocator`] attribute, which could potentially interfere with other tests. -* Defines any crate-wide attributes (like `#![deny(unused)]`). +* Defines any crate-wide attributes (like `#![feature(...)]`). * Defines a macro that uses `$crate`, because the `$crate` path will not work correctly. However, rustdoc is not able to automatically determine *all* situations where an example cannot be combined with other examples. In these situations, you can add the `standalone` language tag to indicate that the example should be built as a separate executable. From 91a2bf7de386e4321ac17d9db216290db83f9bed Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Mon, 19 Aug 2024 11:07:21 -0700 Subject: [PATCH 4/6] Add sentence emphasizing that it is unlikely to have issues --- src/rust-2024/rustdoc-doctests.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/rust-2024/rustdoc-doctests.md b/src/rust-2024/rustdoc-doctests.md index 3e443305..8cadc5db 100644 --- a/src/rust-2024/rustdoc-doctests.md +++ b/src/rust-2024/rustdoc-doctests.md @@ -33,7 +33,7 @@ pub fn subtract(left: u64, right: u64) -> u64 { In this example, the two doctests will now be compiled in a single executable. Rustdoc will essentially place each example in a separate function within a single binary. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly. -This change is only available in the 2024 Edition to avoid potential incompatibilities with existing doctests which may not work in a combined executable. +This change is only available in the 2024 Edition to avoid potential incompatibilities with existing doctests which may not work in a combined executable. However, these incompatibilities are expected to be extremely rare. [doctests]: ../../rustdoc/write-documentation/documentation-tests.html [libtest harness]: ../../rustc/tests/index.html @@ -73,4 +73,4 @@ This example is sensitive to the code structure of how the example is compiled. ## Migration -There is no automatic migration to determine which doctests need to be annotated with the `standalone` tag. You will need to update your crate to the 2024 Edition and then run your documentation tests and see if any of them fail. If they do fail, you will need to analyze whether the test can be rewritten to be compatible with the combined approach, or add the `standalone` tag to retain the previous behavior. +There is no automatic migration to determine which doctests need to be annotated with the `standalone` tag. It's very unlikely that doctests will not work correctly when merged. The only way to know is to update your crate to the 2024 Edition and then run your documentation tests and see if any of them fail. If they do fail, you will need to analyze whether the test can be rewritten to be compatible with the combined approach, or add the `standalone` tag to retain the previous behavior. From f5dc745c849cd75fcb9df694f12cb5471b1bf409 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Mon, 19 Aug 2024 11:13:29 -0700 Subject: [PATCH 5/6] Add link to implementation description --- src/rust-2024/rustdoc-doctests.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/rust-2024/rustdoc-doctests.md b/src/rust-2024/rustdoc-doctests.md index 8cadc5db..af35589c 100644 --- a/src/rust-2024/rustdoc-doctests.md +++ b/src/rust-2024/rustdoc-doctests.md @@ -31,13 +31,15 @@ pub fn subtract(left: u64, right: u64) -> u64 { } ``` -In this example, the two doctests will now be compiled in a single executable. Rustdoc will essentially place each example in a separate function within a single binary. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly. +In this example, the two doctests will now be compiled in a single executable. Rustdoc will essentially place each example in a separate function within a single binary. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly.[^implementation] This change is only available in the 2024 Edition to avoid potential incompatibilities with existing doctests which may not work in a combined executable. However, these incompatibilities are expected to be extremely rare. [doctests]: ../../rustdoc/write-documentation/documentation-tests.html [libtest harness]: ../../rustc/tests/index.html +[^implementation]: For more information on the details of how this work, see [Doctests - How were they improved?](https://blog.guillaume-gomez.fr/articles/2024-08-17+Doctests+-+How+were+they+improved%3F). + ### Standalone tag In some situations it is not possible for rustdoc to combine examples in a single executable. Rustdoc will attempt to automatically detect if this is not possible, for example: From d1a6a446f7bbd0d3fadbe4e9bb72c27917ddd83a Mon Sep 17 00:00:00 2001 From: Travis Cross Date: Mon, 19 Aug 2024 23:26:22 +0000 Subject: [PATCH 6/6] Make some editorial changes --- src/rust-2024/rustdoc-doctests.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/src/rust-2024/rustdoc-doctests.md b/src/rust-2024/rustdoc-doctests.md index af35589c..da33268b 100644 --- a/src/rust-2024/rustdoc-doctests.md +++ b/src/rust-2024/rustdoc-doctests.md @@ -9,7 +9,7 @@ More information may be found in the tracking issue at u64 { } ``` -In this example, the two doctests will now be compiled in a single executable. Rustdoc will essentially place each example in a separate function within a single binary. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly.[^implementation] +In this example, the two doctests will now be compiled into a single executable. Rustdoc will essentially place each example in a separate function within a single binary. The tests still run in independent processes as they did before, so any global state (like global statics) should still continue to work correctly.[^implementation] This change is only available in the 2024 Edition to avoid potential incompatibilities with existing doctests which may not work in a combined executable. However, these incompatibilities are expected to be extremely rare. [doctests]: ../../rustdoc/write-documentation/documentation-tests.html [libtest harness]: ../../rustc/tests/index.html -[^implementation]: For more information on the details of how this work, see [Doctests - How were they improved?](https://blog.guillaume-gomez.fr/articles/2024-08-17+Doctests+-+How+were+they+improved%3F). +[^implementation]: For more information on the details of how this work, see ["Doctests - How were they improved?"](https://blog.guillaume-gomez.fr/articles/2024-08-17+Doctests+-+How+were+they+improved%3F). ### Standalone tag -In some situations it is not possible for rustdoc to combine examples in a single executable. Rustdoc will attempt to automatically detect if this is not possible, for example: +In some situations it is not possible for rustdoc to combine examples into a single executable. Rustdoc will attempt to automatically detect if this is not possible. For example, a test will not be combined with others if it: -* Uses the [`compile_fail`][tags] language tag, which indicates that the example fails to compile. -* Uses the [`edition`][tags], which indicates the edition of the example.[^edition-tag] -* Uses global attributes like the [`global_allocator`] attribute, which could potentially interfere with other tests. +* Uses the [`compile_fail`][tags] tag, which indicates that the example should fail to compile. +* Uses an [`edition`][tags] tag, which indicates the edition of the example.[^edition-tag] +* Uses global attributes, like the [`global_allocator`] attribute, which could potentially interfere with other tests. * Defines any crate-wide attributes (like `#![feature(...)]`). * Defines a macro that uses `$crate`, because the `$crate` path will not work correctly. -However, rustdoc is not able to automatically determine *all* situations where an example cannot be combined with other examples. In these situations, you can add the `standalone` language tag to indicate that the example should be built as a separate executable. +However, rustdoc is not able to automatically determine *all* situations where an example cannot be combined with other examples. In these situations, you can add the `standalone` language tag to indicate that the example should be built as a separate executable. For example: ```rust //! ``` @@ -59,7 +59,7 @@ However, rustdoc is not able to automatically determine *all* situations where a //! ``` ``` -This example is sensitive to the code structure of how the example is compiled. This won't work with the "combined" approach because the line numbers will shift depending on how the doctests are combined. In these situations, you can add the `standalone` tag to force the example to be built separately just as it did in previous editions. +This is sensitive to the code structure of how the example is compiled and won't work with the "combined" approach because the line numbers will shift depending on how the doctests are combined. In these situations, you can add the `standalone` tag to force the example to be built separately just as it was in previous editions. E.g.: ```rust //! ```standalone @@ -71,8 +71,8 @@ This example is sensitive to the code structure of how the example is compiled. [tags]: ../../rustdoc/write-documentation/documentation-tests.html#attributes [`global_allocator`]: ../../std/alloc/trait.GlobalAlloc.html -[^edition-tag]: Note that rustdoc will only combine tests if the entire crate is Edition 2024 or greater. Using `edition2024` tags in older editions will not result in those tests being combined. +[^edition-tag]: Note that rustdoc will only combine tests if the entire crate is Edition 2024 or greater. Using the `edition2024` tag in older editions will not result in those tests being combined. ## Migration -There is no automatic migration to determine which doctests need to be annotated with the `standalone` tag. It's very unlikely that doctests will not work correctly when merged. The only way to know is to update your crate to the 2024 Edition and then run your documentation tests and see if any of them fail. If they do fail, you will need to analyze whether the test can be rewritten to be compatible with the combined approach, or add the `standalone` tag to retain the previous behavior. +There is no automatic migration to determine which doctests need to be annotated with the `standalone` tag. It's very unlikely that any given doctest will not work correctly when migrated. We suggest that you update your crate to the 2024 Edition and then run your documentation tests and see if any fail. If one does, you will need to analyze whether it can be rewritten to be compatible with the combined approach, or alternatively, add the `standalone` tag to retain the previous behavior.