Generate rule docs automatically #615

natikgadzhi · 2023-09-05T05:00:54Z

Summary

First step of Improve dump-configuration with inline comments to explain each rule #606, adds documentation for the list of available rules in Documentation directory.

Changes

Adds a generate-pipeline generator RuleDocumentationGenerator that extracts each rule DocC comment, and writes it as a table into Documentation/RuleDocumentation.md.
In Documentation/Configuration.md, I've added a paragraph on rules configuration that links to the new auto-generated docs file, and also tells users that they can run swift-format dump-configuration to check out the list of available rules.

How to review this

Check out the format of the markdown file in RuleDocumentation.md.
Navigate to Documentation files in the PR repo explorer. Does the markup and content look good?
Does the layout make sense? I started with a table layout, but it wasn't as readable as I would hope (no way to tell lines to vertically align to top, no way to specify column width on GitHub markdown viewer — let's leave this until we can move to DocC?)

TODOs

Rule description markup breaks where the original DocC content had a line break just because of long lines.

Documentation/Configuration.md

Documentation/RuleDocumentation.md

Sources/generate-pipeline/RuleCollector.swift

natikgadzhi · 2023-09-05T05:35:36Z

Sources/generate-pipeline/RuleDocumentationGenerator.swift

+//
+// This source file is part of the Swift.org open source project
+//
+// Copyright (c) 2014 - 2023 Apple Inc. and the Swift project authors


Someone told me to put just 2023 if the file is new.

natikgadzhi · 2023-09-05T05:35:57Z

Sources/generate-pipeline/RuleDocumentationGenerator.swift

+//
+//===----------------------------------------------------------------------===//
+
+import Foundation


Honest question: why do we need Foundation in our generators?

write(into:) takes a FileHandle argument.

So technically we could just write into a file differently, but we just don't aim to support environments where Foundation is not available yet?

I can't think of any platforms that don't have Foundation that I'd want to run generate-pipeline on.

The rest of swift-format also uses Foundation, so I don't know why I'd want to single out generate-pipeline specifically.

- This commit adds a `generate-pipeline` generator `RuleDocumentationGenerator` that extracts each rule DocC comment, and writes it into `Documentation/RuleDocumentation.md`. - In `Documentation/Configuration.md`, I've added a paragraph on rules configuration that links to the new auto-generated docs file, and also tells users that they can run `swift-format dump-configuration` to check out the list of available rules.

natikgadzhi · 2023-09-06T05:09:56Z

Rebased over #614 merge.

@allevato, how does that look to you?

allevato

Thanks, this looks really good! A few suggestions but after that I think this is good to go and the iterate on afterwards.

allevato · 2023-09-06T12:53:21Z

Documentation/RuleDocumentation.md

+[Configuration](Documentation/Configuration.md). All of these rules can be
+applied in the linter, but only some of them can format your source code
+automatically.
+


Could you generate an unordered list here that would act as a table of contents, linking to each of the headers below?

allevato · 2023-09-06T12:54:30Z

Sources/generate-pipeline/RuleDocumentationGenerator.swift

+//
+//===----------------------------------------------------------------------===//
+
+import Foundation


write(into:) takes a FileHandle argument.

allevato · 2023-09-06T12:57:08Z

Sources/generate-pipeline/RuleDocumentationGenerator.swift

+
+  /// Takes the DocC comment of the rule and strip `///` from the beginning of each line.
+  /// Also removes empty lines with under 4 characters.
+  private func ruleDescription(for rule: RuleCollector.DetectedRule) -> String {


We already have more general code for parsing doc comments in DocumentationCommentText. You could use that here instead (it's under @_spi, but that's fine).

generate-pipeline already imports SwiftFormat because it actually pokes at some of the runtime properties of the rules. It's kind of weird, but it means you don't have to worry about introducing that dependency.

- Using `DocumentationCommentText` via `@_spi(Rules)` instead of our own implementation of rule description extracted from trivia. - Added table of contents with all the available rules linked to their respective blocks.

natikgadzhi

Ready for another pass, @allevato!

natikgadzhi · 2023-09-06T22:59:17Z

Documentation/RuleDocumentation.md

+[Configuration](Documentation/Configuration.md). All of these rules can be
+applied in the linter, but only some of them can format your source code
+automatically.
+


natikgadzhi · 2023-09-06T22:59:57Z

Sources/SwiftFormat/Core/DocumentationCommentText.swift

@@ -18,6 +18,7 @@ import SwiftSyntax
 /// structural organization. It automatically handles trimming leading indentation from comments as
 /// well as "ASCII art" in block comments (i.e., leading asterisks on each line).
 @_spi(Testing)
+@_spi(Rules)


Added additional SPI here because using @_spi(Testing) just felt janky.

natikgadzhi · 2023-09-06T23:00:15Z

Sources/generate-pipeline/RuleCollector.swift

@@ -23,6 +23,10 @@ final class RuleCollector {
    /// The type name of the rule.
    let typeName: String

+    /// The description of the rule, extracted from the rule class or struct DocC comment
+    /// with `DocumentationCommentText(extractedFrom:)`
+    let description: String?


Optional because DocumentCommentText() may return nil, but that's fine for us.

natikgadzhi · 2023-09-06T23:00:58Z

Sources/generate-pipeline/RuleDocumentationGenerator.swift

+//
+//===----------------------------------------------------------------------===//
+
+import Foundation


So technically we could just write into a file differently, but we just don't aim to support environments where Foundation is not available yet?

allevato

Thank you for this! I'm really happy with how it turned out and it'll be a big help keeping the rule documentation updated going forward.

If you're looking for something else to do 😁 , it would be nice to get coverage for the rest of the fields in Configuration and the types associated with them. That's a bit more work though, because we're not already collecting the decls today. But basically we'd just want to scan Configuration.swift for all the fields that are public vars, and then do the same for the other types defined in that file when we have config fields that are themselves complex values.

natikgadzhi · 2023-09-07T15:36:00Z

@allevato, thank you for reviewing this and guiding me through it! ❤️

I'm happy to help more. I was curious about trying to convert the Documentation directory to a DocC site first, and maybe even recording that working session / streaming it to get more people comfortable with DocC for their projects. That's likely some time next week.

Do you think it would be okay to convert to DocC first, and then add more documentation coverage in Configuration, as well as in code otherwise?

allevato · 2023-09-07T16:08:13Z

I'm happy to help more. I was curious about trying to convert the Documentation directory to a DocC site first, and maybe even recording that working session / streaming it to get more people comfortable with DocC for their projects. That's likely some time next week.

Do you think it would be okay to convert to DocC first, and then add more documentation coverage in Configuration, as well as in code otherwise?

I think that all sounds fantastic! The public API for swift-format (everything in the API subfolder) is pretty small so getting DocC support up and running should hopefully be fairly straightforward. And I love the idea of recording your work session for the community.

If we can only get Configuration documented in API docs, then that's still a great start, and maybe we could link to that in the user docs. But it would be nice to figure out a way to surface it in the user docs directly too, because I'm thinking it would look better to have the "this is how you configure swift-format" docs look more like the JSON representation than the underlying API representation.

I wonder if, instead of scanning the source, we could consume the symbol graphs that DocC uses to generate that alternate form instead...

I haven't used DocC much myself, so I'm interested to see what you come up with!

…peline/rule-documentation Generate rule docs automatically

natikgadzhi force-pushed the feature/generate-pipeline/rule-documentation branch 2 times, most recently from aa32253 to 8a939da Compare September 5, 2023 05:23

natikgadzhi commented Sep 5, 2023

View reviewed changes

natikgadzhi force-pushed the feature/generate-pipeline/rule-documentation branch from 89390d0 to 8a939da Compare September 5, 2023 17:26

natikgadzhi force-pushed the feature/generate-pipeline/rule-documentation branch from 8a939da to 1556e82 Compare September 6, 2023 05:09

allevato reviewed Sep 6, 2023

View reviewed changes

natikgadzhi added 2 commits September 6, 2023 15:29

Update Sources/generate-pipeline/RuleCollector.swift

e350bde

Improvements to auto-generated rule docs

284ed05

- Using `DocumentationCommentText` via `@_spi(Rules)` instead of our own implementation of rule description extracted from trivia. - Added table of contents with all the available rules linked to their respective blocks.

natikgadzhi requested a review from allevato September 6, 2023 22:58

natikgadzhi commented Sep 6, 2023

View reviewed changes

allevato approved these changes Sep 7, 2023

View reviewed changes

allevato mentioned this pull request Sep 7, 2023

[DO NOT MERGE] allevato's dummy PR for swift-format CI swiftlang/swift-syntax#1086

Open

allevato merged commit f14a625 into swiftlang:main Sep 7, 2023

This was referenced Sep 8, 2023

add documentation for rules #394

Closed

Basic documentation for rules #393

Closed

allevato added a commit to allevato/swift-format that referenced this pull request Sep 14, 2023

Merge pull request swiftlang#615 from natikgadzhi/feature/generate-pi…

f27c0e8

…peline/rule-documentation Generate rule docs automatically

zidhuss mentioned this pull request Jun 1, 2024

Create jsonschema for configuration file #747

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate rule docs automatically #615

Generate rule docs automatically #615

natikgadzhi commented Sep 5, 2023 •

edited

Loading

natikgadzhi Sep 5, 2023

natikgadzhi Sep 5, 2023

allevato Sep 6, 2023

natikgadzhi Sep 6, 2023

allevato Sep 7, 2023

natikgadzhi commented Sep 6, 2023

allevato left a comment

allevato Sep 6, 2023

natikgadzhi Sep 6, 2023

allevato Sep 6, 2023

allevato Sep 6, 2023

natikgadzhi left a comment

natikgadzhi Sep 6, 2023

natikgadzhi Sep 6, 2023

natikgadzhi Sep 6, 2023

natikgadzhi Sep 6, 2023

allevato left a comment

natikgadzhi commented Sep 7, 2023

allevato commented Sep 7, 2023

Generate rule docs automatically #615

Generate rule docs automatically #615

Conversation

natikgadzhi commented Sep 5, 2023 • edited Loading

Summary

Changes

How to review this

TODOs

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

natikgadzhi commented Sep 6, 2023

allevato left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

natikgadzhi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

allevato left a comment

Choose a reason for hiding this comment

natikgadzhi commented Sep 7, 2023

allevato commented Sep 7, 2023

natikgadzhi commented Sep 5, 2023 •

edited

Loading