Stabilize custom_code_classes_in_docs feature

[GuillaumeGomez]

Fixes #79483.

This feature has been around for quite some time now, I think it's fine to stabilize it now.

Summary

What is the feature about?

In short, this PR changes two things, both related to codeblocks in doc comments in Rust documentation:

• Allow to disable generation of language-* CSS classes with the custom attribute.
• Add your own CSS classes to a code block so that you can use other tools to highlight them.

The custom attribute

Let's start with the new custom attribute: it will disable the generation of the language-* CSS class on the generated HTML code block. For example:

/// ```custom,c
/// int main(void) {
///     return 0;
/// }
/// ```

The generated HTML code block will not have class="language-c" because the custom attribute has been set. The custom attribute becomes especially useful with the other thing added by this feature: adding your own CSS classes.

Adding your own CSS classes

The second part of this feature is to allow users to add CSS classes themselves so that they can then add a JS library which will do it (like highlight.js or prism.js), allowing to support highlighting for other languages than Rust without increasing burden on rustdoc. To disable the automatic language-* CSS class generation, you need to use the custom attribute as well.

This allow users to write the following:

/// Some code block with `{class=language-c}` as the language string.
///
/// ```custom,{class=language-c}
/// int main(void) {
///     return 0;
/// }
/// ```
fn main() {}

This will notably produce the following HTML:

<pre class="language-c">
int main(void) {
    return 0;
}</pre>

Instead of:

<pre class="rust rust-example-rendered">
<span class="ident">int</span> <span class="ident">main</span>(<span class="ident">void</span>) {
    <span class="kw">return</span> <span class="number">0</span>;
}
</pre>

To be noted, we could have written {.language-c} to achieve the same result. . and class= have the same effect.

One last syntax point: content between parens ((like this)) is now considered as comment and is not taken into account at all.

In addition to this, I added an unknown field into LangString (the parsed code block "attribute") because of cases like this:

/// ```custom,class:language-c
/// main;
/// ```
pub fn foo() {}

Without this unknown field, it would generate in the DOM: <pre class="language-class:language-c language-c">, which is quite bad. So instead, it now stores all unknown tags into the unknown field and use the first one as "language". So in this case, since there is no unknown tag, it'll simply generate <pre class="language-c">. I added tests to cover this.

EDIT(camelid): This description is out-of-date. Using custom,class:language-c will generate the output <pre class="language-class:language-c"> as would be expected; it treats class:language-c as just the name of a language (similar to the langstring c or js or what have you) since it does not use the designed class syntax.

Finally, I added a parser for the codeblock attributes to make it much easier to maintain. It'll be pretty easy to extend.

As to why this syntax for adding attributes was picked: it's Pandoc's syntax. Even if it seems clunkier in some cases, it's extensible, and most third-party Markdown renderers are smart enough to ignore Pandoc's brace-delimited attributes (from this comment).

r? @notriddle

[GuillaumeGomez]

Wow, didn't pay attention but that's a lot of removed lines. :D

[GuillaumeGomez]

Ok this is ready for review now.

[notriddle]

@rfcbot fcp merge

[GuillaumeGomez]

Hum, why is the compiler team part of this? Because of the feature in the compiler source maybe?

[compiler-errors]

@rfcbot fcp cancel

Autotagged. I don't think the compiler team needs to participate in this.

[rfcbot]

@compiler-errors proposal cancelled.

[compiler-errors]

Please start a new one @GuillaumeGomez or @notriddle

[GuillaumeGomez]

Oh it's based on PR's tags and not the source code. Noted. Thanks compiler-errors!

@rfcbot fcp merge

[rfcbot]

Team member @GuillaumeGomez has proposed to merge this. The next step is review by the rest of the tagged team members:

• @GuillaumeGomez
• @Manishearth
• @Nemo157
• @aDotInTheVoid
• @camelid
• @jsha
• @notriddle

Concerns:

• syntax complexity resolved by Stabilize custom_code_classes_in_docs feature #124577 (comment)

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[camelid]

Do you mind posting a brief summary of the feature as it currently stands? It seems there was a lot of back-and-forth over things like syntax in the implementation PR.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[GuillaumeGomez]

@bors r=rustdoc

[bors]

📌 Commit 2f6abd1 has been approved by rustdoc

It is now in the queue for this repository.

[bors]

⌛ Testing commit 2f6abd1 with merge 05965ae...

[bors]

☀️ Test successful - checks-actions
Approved by: rustdoc
Pushing 05965ae to master...

[rust-timer]

Finished benchmarking commit (05965ae): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.8%, -0.2%]	9
Improvements ✅ (secondary)	-0.7%	[-0.9%, -0.5%]	2
All ❌✅ (primary)	-0.4%	[-0.8%, -0.2%]	9

Max RSS (memory usage)

Results (primary 1.4%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.4%	[1.4%, 1.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.4%	[1.4%, 1.4%]	1

Cycles

Results (secondary 2.9%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.9%	[2.1%, 4.4%]	10
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 668.037s -> 668.888s (0.13%)
Artifact size: 318.86 MiB -> 318.81 MiB (-0.02%)

[decathorpe]

This PR seems to have caused a regression (or at least a possibly unexpected behaviour change) that affects a large-ish number of crates - previously, code block attributes that were not recognized as "rust code" in some way were ignored and not compiled, but now "invalid" code block attributes raise a new rustdoc warning lint, but are compiled (and run) as Rust code.

For example, the code blocks here use {rust,ignore} instead of rust,ignore - they were previously ignored, but are now attempted to be compiled + run:

https://github.com/servo/rust-cssparser/blob/main/src/lib.rs#L44-L64

I noticed this because the CI we have for Fedora packages started to turn red for 2-3 dozen Rust crates after Rust 1.80 landed a few days ago.

Was this behaviour change intentional? While I agree that it is likely a good idea to warn on "invalid" or unrecognized code block attributes, attempting to compile + run code blocks with unrecognized attributes might not be.