Using YAML + Markdown format in documentation comments

[ghost]

Using YAML + Markdown format in documentation comments

Summary

Our current style of doumentation comment using XML is too verbose.
We can achieve a cleaner approach using YAML + Markdown similar to OpenAPI specification.

Design

Here is an small example of a YAML based documentation comment.

summary:      A summary of the member.
description:  A description of the member. Markdown syntax may be used for rich text representation.
returns:      A description of the returned value.
parameters:
  - name: a
    description:  A description of the parameter.
  - name: b
    description:  A description of the parameter.

Along with moving from XML to YAML, we may also consider the /** */ comment style instead of using triple slash per line.

The above approach looks cleaner than using the verbose XML start and end tags.
However, with the above approach, we may increase the number of lines than its XML counterpart.
This may not be a problem, as most IDEs keep the doc comments collapsed by default.
Or, we can find a more compact YAML structure.

Using YAML + Markdown can also be a advantage for docfx, which uses YAML as its input.

Below is an detailed example of using all the documentation comment tags into a YAML + Markdown format.

summary:      A summary of the member.
description:  A description of the member. Markdown syntax may be used for rich text representation.
returns:      A description of the returned value.

parameters:
  - name: a
    description:  A description of the parameter.
    throws:
      - ArgumentNullException
      - ArgumentOutOfRangeException
  - name: b
    description:  A description of the parameter.
    throws:
      - ArgumentNullException
      - ArgumentOutOfRangeException

type parameters:
  - name: T
    description:  A description of the type parameter.
  - name: U
    description:  A description of the type parameter.

throws:
  ArgumentNullException:
    description:  A reason of the exception that can be thrown by the member.
  ArgumentOutOfRangeException:
    description:  A reason of the exception that can be thrown by the member.

remarks: |
A long remarks regarding the member.
Markdown syntax may be used for rich text representation.
With code examples like
``cs
var result = DoSomething(a, b);
``

example: |
``cs
var result = DoSomething(a, b);
var tranformed = DoSomethingMore(result);
Console.WriteLine(tranformed);
``

see:
  - OverloadedDoSomething(type, type)
  - OverloadedDoSomething(type, type)
  - MyAnotherClass

Drawbacks

• Older IDE's will not support the newer format. For this, we can generate an XML doc side by side.
• YAML format may also increase the count of lines than its XML counterpart.

[HaloFour]

The C# design team could chase the stylistic preferences for documentation all day long and not satisfy everyone.

In my opinion as the language already has one inline documentation format it doesn't need another one, particularly one where the name means "yet another" and represents another markup language du jour. I particularly don't like that YAML is whitespace sensitive, making it paradigmically different from the language it intends to document. You have to learn different syntax just to manage that whitespace sensitivity, such as when a long value is broken up over multiple lines, which is common in XML documentation. Inline tags, also common, also have an awkward syntax. YAML being a superset of JSON basically means baking two additional languages into the spec.

[yaakov-h]

You say that xmldoc is "too verbose", but then acknowledge that your proposal "may increase the number of lines".

What problem does this actually solve?

[ghost]

XML is no-doubt verbose. But, migrating to stricter YAML standard sometimes increase the count of lines.
For, e.g., in the below code params can be defined in two lines in XML, whereas my proposal YAML requires 3 extra lines.

With XML,

/// <summary>Encrypts an input byte array and returns the encrypted array.</summary>
/// <param name="key">The session key to use for the encryption.</param>
/// <param name="data">The data to encrypt.</param>
/// <returns>A byte array that contains the encrypted data.</returns>
/// <exception cref="ArgumentNullException"><paramref name="data"/> is null.</exception>
/// <exception cref="InvalidOperationException">The session key has expired.</exception>
public byte[] Encrypt(byte[] key, byte[] data) { }

With YAML,

/**
summary: Encrypts an input byte array and returns the encrypted array
returns: A byte array that contains the encrypted data.
parameters:
  - name: key
    description:  The session key to use for the encryption.
  - name: data
    description:  The data to encrypt.
throws:
  ArgumentNullException:
    description:  `data` is null.
  InvalidOperationException:
    description:  The session key has expired.
*/
public byte[] Encrypt(byte[] key, byte[] data) { }

We can find a more compact YAML structure, build a custom C#FM structure, or keep in that way.
But in the long run migrating to YAML + Markdown far outweighs the current XML style.

For, e.g., we can also define the parameter tag in this way. But this loses the advantage of adding additional info to a parameter.

parameters:
  key: The session key to use for the encryption.
  data: The data to encrypt.

Let's take another example to see why XML is not fit for documentating.

With XML,

/// <summary>Encrypts an input byte array and returns the encrypted array</summary>
/// <remarks>
///   <para>
///     The encryption internally uses the ECB mode for encryption.
///     The data that is to be provided must conform to the defined specification.
///   </para>
///   <para>
///     When using the encryption few points needs to be remembered.
///   <para>
///   <list type="bullet">  
///     <item>  
///       <description>The key size must be 256 in size.</description>  
///     </item>
///     <item>
///       <description> The data must be base64 encoded.</description>  
///     </item>
///   </list>
/// </remarks>
public byte[] Encrypt(byte[] key, byte[] data) { }

With YAML + Markdown,

/**
summary:  Encrypts an input byte array and returns the encrypted array.

remarks: |
The encryption internally uses the ECB mode for encryption.
The data that is to be provided must conform to the defined specification.

When using the encryption few points needs to be remembered.
- The key size must be 256 in size.
- The data must be base64 encoded.
*/
public byte[] Encrypt(byte[] key, byte[] data) { }

We can see from the above example, how YAML can benefit over XML.
Someone could try typing the whole comment block in the current version of VS Code.
Also take into consideration the numerous <see/> tag that we use in a good descriptive doc comment.

We can also add few more tags with minimal no. characters to the YAML format like, author, version, contributors, complexity. And Latex!
Moreover, directly writing markdown in code can be very much helpful to the documentation pipeline from code to website generation.

As about stylistic preferences and "yet another", comments play an integral part of coding.
Any changes made to the way we comment directly affects the way we write and read C# code.
As about learing, with our current style, we still have to lookup the Recommended Tags for Documentation Comments many a times just to write remarks and example.
We read comments more directly over code, and lesser through websites. Reading a machine-centric XML doesn't go with beautiful and elegant human-centric C# code.

Regards,
Souvik

[DavidArno]

It strikes me that your examples above don't so much argue for YAML over XML, but argue for markdown support in eg, the remarks section. XML + MD could be a nice compromise:

/// <summary>Encrypts an input byte array and returns the encrypted array</summary>
/// <remarks-md>
/// The encryption internally uses the ECB mode for encryption.
/// The data that is to be provided must conform to the defined specification.
///
/// When using the encryption few points needs to be remembered.
/// - The key size must be 256 in size.
/// - The data must be base64 encoded.
/// </remarks-md>

[MgSam]

I agree that XML comments are horrible and feel like placeholders that somehow became a permanent part of the language. I'd much rather see JavaDoc/JSDoc style comments as they're much less verbose, easy to read and write and being used by many languages shows that they do something right.

[ghost]

Yes, XML + MD is a good idea. I also thought about it originally. But given the design goals of .NET Core and C#, lightweight and clean, lightness should start from the source. And as said earlier comments play an integral part of coding, a rethink from scratch would not be a bad idea.

As for pros and cons of YAML over XML, we all are well aware of the simplicity of YAML, so there is not much to argue. Even if I missed out some facts, I would like to add few more points.

Given XML, the most used mathematical characters, <, >, & is reserved, that we had to use { } for generics. With YAML in place, we have the advantage of defining math freely. To add, the mismatch of end tags, missing out the / in end tag, is also critical, (provided that we are not working in a very smart IDE, that generate tags for us.)

Keeping aside remarks, even if we talk about simple tags, like summary, returns, exception, params, or new to add author, contributors, version, etc., a structure like YAML wins.

It would be bad not to mention cons of YAML, it's difficulty in parsing. But I believe, readability of it will win over cons of it and especially when it comes to C#!

I'd much rather see JavaDoc/JSDoc

JavaDoc is also nice, almost every one knows it, but given that YAML is a subset of JSON, YAML (or JSON) are better candidates for parsing and storing, documentation data rather than define a whole new model.

[sharwell]

💭 This proposal appears to suffer from the same complexity problems I described in dotnet/roslyn#85 (comment) and dotnet/roslyn#85 (comment).

While one could argue that the proposed syntax represents a simplification, it does not simplify things enough to fix the problem. People who currently write improperly-formed comments due to the complexity of XML syntax will not start to write correctly-formed comments simply due to this proposal. In addition, people who do not write comments now will not start to write comments due to this proposal.

[sharwell]

XML + MD could be a nice compromise:

Something like this?

https://twitter.com/samharwell/status/563438274837233666

[erik-kallen]

Strictly speaking, why is doc-comment parsing included in the compiler? Could it not be moved to an external tool/analyzer, and then the community could create other parsers that people could use (if there is interest)?

Edit: I don't mean to move it to an actual analyzer since that is not what they are for, but to some new analyzer-like plugin architecture

[sharwell]

@erik-kallen The driving requirement is tools like Roslyn need to be able to present information to aid in the development process, including the pop-up windows for code completion and quick info (hovering over something). To meet this requirement, at minimum the final output (for C# this is the XML file produced in the output directory) needs to be in a form understood by tools.

[HaloFour]

Potential application of source generators? You use a separate comment syntax which an analyzer/generator picks up and generates replacement members with the correct XML documentation that the compiler will pick up?

public partial class Foo {
    /***
        summary: Does bar
        description: |
            This is a method that does bar.

            It does bar does well.
    */
    public void Bar() { }
}

generates:

public partial class Foo {
    /// <summary>Does bar</summary>
    /// <description>This is a method that does bar.
    /// It does bar very well.</description>
    replace void Bar() {
        original();
    }
}

[whoisj]

I'd love to see markdown as the method of referencing things in code. The current <see cref="..."/> is verbose, ugly, and unnecissary - it also makes placing '<' and '>' characters in the comments needlessly bothersome.

[ghost]

@HaloFour But it will not be a good idea to replace a user written text by a generator in a user written code. What if we have to make a massive change to the comment, then we will just have to write it in MD again or do our changes in XML (that we do right now).

[yaakov-h]

@souvikdc9 the idea of generators is that they are run at compile time. The compiler would run the generator, replace markdown with XML (in memory or a temporary file), then compile the XML. Your code would continue to contain Markdown.

To change the user-written text, just edit the markdown.

[HaloFour]

@souvikdc9

Source generators are a part of the build pipeline and the output source files are not intended to be modified by the developer. If the developer needed to amend the documentation they would do so in YAML and the generator would recreate the XML version with the updates during compilation.

[ghost]

Yup, that's what it should be, an XML created during the build process. Probably something like this:

public partial class Foo {
    /***
        summary: Does bar
        description: |
            This is a method that does bar.

            It does bar does well.
    */
    public void Bar() { }
}

generates:

<doc>
  <members>
    <member name="FooBar">
      <summary>Does bar</summary>
      <description>This is a method that does bar.<br>It does bar very well.</description>
    </member>
  </members>
</doc>

[whoisj]

@souvikdc9 while you're not wrong, I do not believe anyone is discussing a source generator here.

The topic of this issue is a better way to produce assembly meta data.

[erik-kallen]

@sharwell sorry for the late reply, but I understand that a build (and semantic models) need to be able to produce the documentation files. I understand that they probably also need to stay as xml for compatibility reasons. But I still think that it could (theoretically) be possible to specify the doccomment parser in a fashion similar to analyzers, and that parser would be responsible for taking comments in whichever format and produce a conforming xml.

[SpocWeb]

DocFX supports MarkDown which is a gread Boon!