Use Markdown in Documentation Comments to Reference Source Code

[whoisj]

Today we have documentation comments which rely solely on XML, which many (myself included) find ugly, unnecessarily complex, and difficult to read. My ask is specifically to allow for the use of Markdown backtick characters to used as delimiter (book ends) for bits of code which should resolve to real types, values, etc. much in the way <see cref=""/>, <paramref name=""/>, and <see langword=""/> operate today. I would avoid worrying about a method for escaping the backtick character, as it is unlikely that would be necessary.

Example of current XML style:

      /// <summary>
      /// Creates a thread that runs in the virtual address space of another process.
      /// </summary>
      /// <param name="processHandle">
      /// A handle to the process in which the thread is to be created. The handle must have the
      /// <see cref="DesiredAccess.CreateThread"/>, <see cref="DesiredAccess.QueryInformation"/>,
      /// <see cref="DesiredAccess.VmOperation"/>, <see cref="DesiredAccess.VmWrite"/>, and
      /// <see cref="DesiredAccess.VmRead"/> access rights, and may fail without these rights on 
      /// certain platforms.
      /// </param>
      /// <param name="threadAttributes">
      /// A pointer to a <see cref="SecurityAttributes"/> structure that specifies a security
      /// descriptor for the new thread and determines whether child processes can inherit
      /// the returned handle. If <paramref name="threadAttributes"/> is <see langword="null"/>,
      /// the thread gets a default security descriptor and the handle cannot be inherited.
      /// </param>
      /// <param name="stackSize">
      /// The initial size of the stack, in bytes. The system rounds this value to the nearest
      /// page. If this parameter is 0 (zero), the new thread uses the default size for the
      /// executable.
      /// </param>
      /// <param name="threadProc">
      /// A pointer to the application-defined function of type
      /// <see cref="System.Threading.ThreadStart"/> to be executed by the thread and represents
      /// the starting address of the thread in the remote process.
      /// </param>
      /// <param name="parameter">
      /// A pointer to a variable to be passed to the thread function pointed to by
      /// <paramref name="threadProc"/>.
      /// </param>
      /// <param name="flags">The flags that control the creation of the thread.</param>
      /// <param name="threadId">A pointer to a variable that receives the thread
      /// identifier.</param>
      /// <returns>
      /// A <see cref="SafeThreadHandle"/> to the new thread, if successful; otherwise an invalid
      /// <see cref="SafeThreadHandle"/>.
      /// </returns>

Example of Markdown style:

      /// <summary>
      /// Creates a thread that runs in the virtual address space of another process.
      /// </summary>
      /// <param name="processHandle">
      /// A handle to the process in which the thread is to be created. The handle must 
      /// have the `DesiredAccess.CreateThread`, `DesiredAccess.QueryInformation`,
      /// `DesiredAccess.VmOperation`, `DesiredAccess.VmWrite`, and `DesiredAccess.VmRead`
      /// access rights, and may fail without these rights on certain platforms.
      /// </param>
      /// <param name="threadAttributes">
      /// A pointer to a `SecurityAttributes` structure that specifies a security descriptor for
      /// the new thread and determines whether child processes can inherit the returned handle.
      /// If `threadAttributes` is `null`, the thread gets a default security descriptor
      /// and the handle cannot be inherited.
      /// </param>
      /// <param name="stackSize">
      /// The initial size of the stack, in bytes. The system rounds this value to the nearest
      /// page. If this parameter is 0 (zero), the new thread uses the default size for the
      /// executable.
      /// </param>
      /// <param name="threadProc">
      /// A pointer to the application-defined function of type `System.Threading.ThreadStart` to
      /// be executed by the thread and represents the starting address of the thread in the
      /// remote process.
      /// </param>
      /// <param name="parameter">
      /// A pointer to a variable to be passed to the thread function pointed to by `threadProc`.
      /// </param>
      /// <param name="flags">The flags that control the creation of the thread.</param>
      /// <param name="threadId">A pointer to a variable that receives the thread
      /// identifier.</param>
      /// <returns>
      /// A `SafeThreadHandle` to the new thread, if successful; otherwise an invalid
      /// `SafeThreadHandle`.
      /// </returns>

Another suggestion was to allow YAML in place of the XML, which looks even nicer to be honest, but is not really the point of this issue.

      /// Summary:
      ///    Creates a thread that runs in the virtual address space of another process.
      /// Parameters:
      ///    processHandle >
      ///       A handle to the process in which the thread is to be created. The handle must 
      ///       have the `DesiredAccess.CreateThread`, `DesiredAccess.QueryInformation`,
      ///       `DesiredAccess.VmOperation`, `DesiredAccess.VmWrite`, and `DesiredAccess.VmRead`
      ///       access rights, and may fail without these rights on certain platforms.
      ///    threadAttributes >
      ///       A pointer to a `SecurityAttributes` structure that specifies a security
      ///       descriptor for the new thread and determines whether child processes
      ///       can inherit the returned handle. If `threadAttributes` is `null`, the thread gets
      ///       a default security descriptor and the handle cannot be inherited.
      ///    stackSize >
      ///       The initial size of the stack, in bytes. The system rounds this value to the
      ///       nearest page. If this parameter is 0 (zero), the new thread uses the default
      ///       size for the executable.
      ///    threadProc >
      ///       A pointer to the application-defined function of type `System.Threading
      ///       .ThreadStart` to be executed by the thread and represents the starting address
      ///       of the thread in the remote process.
      ///    parameter >
      ///       A pointer to a variable to be passed to the thread function pointed to by
      ///       `threadProc`.
      ///    flags >
      ///       The flags that control the creation of the thread.
      ///    threadId >
      ///       A pointer to a variable that receives the thread identifier.
      /// Returns:
      ///    A `SafeThreadHandle` to the new thread, if successful; otherwise an invalid
      ///    `SafeThreadHandle`.

[sharwell]

One nagging question I've been having: What do Turkish Italian users do in a situation like this? (What do Turkish Italian users do for Markdown on GitHub?)

[whoisj]

@sharwell please explain that question. Is the backtick a commonly used character in Turkish?

[sharwell]

@whoisj My mistake. Turkish keyboards can type ` with Alt+Ş. Italian keyboards are the ones that require NumLock+Alt+Numpad9+Numpad6+Numlock¹.

¹ NumLock use only required on laptops

[MovGP0]

Alt+96 seems to be the easier method to type a backtick then 🙈

[HaloFour]

@whoisj

You're advocating switching to backticks in quite a few scenarios in your example. How do you propose that the compiler resolve those references properly?

[whoisj]

How do you propose that the compiler resolve those references properly?

In theory the compiler know the name scoping of the method or type being decorated. Use the same rules that would apply to the code contained within. How does the compiler ever know what a piece of code is referencing? Name scoping rules. 😏

[whoisj]

@sharwell I hadn't realized that the backtick character was absent from Turkish (or any for that matter) keyboards. Interesting 😔

[sharwell]

How do you propose that the compiler resolve those references properly?

My initial proposal would be:

• For comments not placed in a code block, resolve the comments from a pseudo-context "inside" the element (i.e. parameters resolve, then element name, then containers...)
• For comments preceding a statement which can have child statements, resolve the comments from the beginning of the first child statement
• For comments preceding a standalone statement, resolve the comments from the end of the statement
• For comments at the end of a code block, resolve from the current location

[jnm2]

I would avoid worrying about a method for escaping the backtick character, as it is unlikely that would be necessary.

As I've seen, these can quickly become famous last words. 😆 Luckily you're asking for markdown which specs an escape for the backtick.
The normal ‘try first’ markdown escape, the backslash, works here: \` → `
Within code, you fence with more backticks to allow the backticks in the code to remain untouched:
`` ` `` → `

[HaloFour]

@whoisj

null can be a keyword, a type name and a parameter simultaneously. How do you propose the language resolve that?

[sharwell]

null can be a keyword, a type name and a parameter simultaneously. How do you propose the language resolve that?

Comment	Output
`null`	<see langword="null"/>
`@null`	<paramref name="null"/> or <see cref="...null"/>

[whoisj]

Ew = https://superuser.com/questions/667622/italian-keyboard-entering-the-tilde-and-backtick-characters-without-cha

null can be a keyword, a type name and a parameter simultaneously. How do you propose the language resolve that?

How is different when it's a keyword or a type name? It's still null. 😕 How does Visual Studio treat it? How is it highlighted colored? Does Intellisense treat it differently?

edit: @sharwell points out that C# already has a mechanism for resolving a variable vs keyword collisions: the @ prefix.

[sharwell]

💭 My bigger concern is cases where you want to write "code" that isn't meant to reference anything. For cases where a reference is desired, it's also desired to warn if the resolution fails. But what about cases where you aren't trying to reference anything?

[sharwell]

I've thought about this quite a bit in the past.

How would you identify <summary>/<remarks>?

➡️ First paragraph is summary, subsequent is remarks.

How would you identify <param> elements?

No idea...

How would you identify <exception> elements?

No idea...

How would you identify <seealso> elements?

No idea...

How would you identify <examples> elements?

No idea, but code within them could use the triple-backtick syntax.

💭 Maybe the only reasonable answer is keeping the top-level XML syntax unchanged, and only allowing Markdown inside them?

[las-nsc]

With all of these, it would be nice if we could just put the name of whatever it is in backticks followed by a colon and roslyn can work it out:

• if it's at the start of the line:
  • it's a param for the adjoined method, it's a param,
  • it's the name of a class derived from Exception it's an exception
• if it's in the middle of a sentence it's a seealso

The added benefit of this is back-ticked snippets could be syntax highlighted in the editor too.

for example:

/// This is my awesome method which returns a `bool`
/// ```
/// if (IsItAwesome(42, "The meaning of life"))
///    Console.Write("That's awesome");
/// ```
/// `value`: the awesome value
/// `name` : the awesome name
/// `AwesomeException` : `value` and `name` too awesome to handle
public bool IsItAwesome(int value, string name)
{
    ...
}

becomes:

/// <summary>This is my awesome method which returns a <seealso href="bool"/></summary>
/// <example>
/// if (IsItAwesome(42, "The meaning of life"))
///    Console.Write("That's awesome");
/// </example>
/// <param name="value">the awesome value</param>
/// <param name="name">the awesome name</param>
/// <exception cref="AwesomeException"><c>value</c> and <c>name</c> too awesome to handle</exception>
public bool IsItAwesome(int value, string name)
{
    ...
}

[whoisj]

But what about cases where you aren't trying to reference anything?

We could either use / keep the /// <example> ... </example> model, or consider using GitHub's solution of /// ``` ... ``` .

[jnm2]

First paragraph is summary, subsequent is remarks.

But sometimes I actually want two-para summaries because remarks don't show up in intellisense if there is a summary element.

[joelmartinez]

Interesting conversations here (thanks for the /ht @BillWagner) … with regards to the questions about markdown:

docs.microsoft.com uses MarkDig for working with markdown. This solves the spec question (it is commonmark compliant), and the AST question.

[sharwell]

If that's the case and this whole ask lives in a build task or something then this doesn't belong in the csharplang repo but the roslyn one. I assumed this issue was about the C# language itself accepting markdown in doc comments.

At minimum, my proposal would need two changes to the language specification:

1. A provision saying the interpretation of /// comments in source code is implementation defined for cases where the first non-whitespace character of the comment content is not <.
2. A provision saying the interpretation of /// comments in source code located anywhere other than immediately preceding a code element is implementation defined.

[xoofx]

I'm more in favor of a pass-through mechanism, where the raw comment is converted by an extensibility layer into XML form for producing the XML documentation files. This removes the Markdown compat issues from the compiler layer and opens the door to first-class syntax extensions (implemented in today's XML syntax by custom elements).

While it sounds more practical integration wise, I'm not entirely sure it would be a good idea to parse precisely a Markdown comment fragment with precise CommonMark handling for all markdown elements at Roslyn compilation/intellisense time, as it will require to translate all CommonMark to a valid XML-doc comments...

Just speculating from my experience with CommonMark parsing, but implementation wise, it would be easier to only work on the subset that has to deal with code references and output XML doc files still with markdown embedded comments. Precise CommonMark parsing could be delayed until rendering. You would not need to have an entire CommonMark compliant parser in Roslyn, it could work for example only on inline elements (in CommonMark spec terms), and maybe not working on all syntax elements but just handling the escape embedded code sequence against the Roslyn specific ones (to detect them) could be enough...

[sharwell]

... at Roslyn compilation ... time, as it will require to translate all CommonMark to a valid XML-doc comments...

The intent of compilation time translation to XML documentation is providing the following guarantees:

1. The XML documentation is correct with respect to the author's original intent (XML documentation is the normalized form of documentation for CLR assemblies)
2. The interpretation of Markdown syntax is correct from the author's original perspective at the time of compilation, regardless of any changes to Markdown interpretation which may have occurred since.

... output XML doc files still with markdown embedded comments ...

This binds the Markdown syntax used for authoring documentation comments to the backwards compatibility rules of the C# language. I do not see a way this proposal could ever be successful without removing this coupling (it could move forward, but it would be frozen in time and not allow for extensions/variants/modifications of the Markdown used for documentation). Eventually we'd be back in the same situation we are in today.

Precise CommonMark parsing could be delayed until rendering.

This is an implementation detail, but the goal would certainly be avoiding design-time translation of Markdown to XML except for cases where the result needs to be presented to the user (e.g. the user hovered over a code reference and Quick Info needs to show).

[xoofx]

This binds the Markdown syntax used for authoring documentation comments to the backwards compatibility rules of the C# language.

Not sure to follow this part... Could you elaborate?

This is an implementation detail, but the goal would certainly be avoiding design-time translation of Markdown to XML except for cases where the result needs to be presented to the user (e.g. the user hovered over a code reference and Quick Info needs to show).

It sounds contrary to what you are explaining just before ("translation to XML documentation")... How would that work for all the rest of basic markdown formatting? Would you translate markdown list to xml-doc <list><item>...? Fenced code blocks to XML? Inline backstick codeblocks to <code>? If you use **bold** in a list item, would you render it to <strong>? Common Markdown parsers are typically not allowed to inspect within any <tag>...</tag> so once you start to translate one, you need to translate down the whole fragment...

[CyrusNajmabadi]

That is asking a lot. I think almost nobody has the same requirements as Roslyn, which is why, as far as I can tell, Roslyn uses its own XML parser for documentation comments.

It does/doesn't. Part of Roslyn is that this is also shelled out to a real .Net Xml library to do verification.

I don't think it's reasonable to expect Markdown will be any different.

That's fine. But it definitely raises the costs here substantially. Because Roslyn then needs to now meet this 'spec' (which from above hasn't even been standardized yet).

[CyrusNajmabadi]

If that's the case and this whole ask lives in a build task or something then this doesn't belong in the csharplang repo but the roslyn one. I assumed this issue was about the C# language itself accepting markdown in doc comments.

I woudl very much like this to not be part of the language.

At minimum, my proposal would need two changes to the language specification:

1. A provision saying the interpretation of /// comments in source code is implementation defined for cases where the first non-whitespace character of the comment content is not <.
2. A provision saying the interpretation of /// comments in source code located anywhere other than immediately preceding a code element is implementation defined.

I really like these rules. This helps carve out the language space, while also giving broad leeway in the impl to do the best we can, but know that we can and likely will have to change/tweak things in the future.

[CyrusNajmabadi]

Update the extensibility layer to enable documentation rendering extensions to also provide editor services related to structured content of these comments (completion, tagging, etc.)

If only someone had written such an extensibility subsystem for Roslyn ;-)

On a serious note, i think this would be a fine way to proceed. You can use the VirtualChar system to represent the contents of non <-doccomments, and you could then appropriately load teh write plugins to handle these.

[peteraritchie]

This really needs to be a thing. :)

[mclang]

Found this while looking if C# comments could be done using markdown instead of cumbersome <summary> tags and the like.

I mean I would like to write and IntelliSense to understand something like this:

/// This function splits `vector` parameter using given function.
/// **remarks:**
/// - Parameter `vector` and `func` must be valid
/// - Exceptions must be handled by calling function

Instead of this mess:

/// <summary>This function splits `vector` parameter using given function.</summary>
/// <remarks>
/// <list type="bullet">
/// <item>
///     <description>Parameter `vector` and `func` must be valid</description>
/// </item>
/// <item>
///     <description>Exceptions must be handled by calling function</description>
/// </item>
/// </remarks>

Never gonna happen though I think :/

[bugproof]

Interesting because when I use

/// comment

instead of

/// <summary>
/// comment
/// </summary>

in Rider it seems to work the same way but when generating xml doc, it doesn't put it in <summary> tag.

[schittli]

it's a shame that this git project has to discuss years
and they were not able to find the obvious.

[obx100]

2021 is almost over!!Does any solution about using Markdown or Yaml as comments instead of XML comments?

[aka-nse]

It's a compromise, how about introducing only top-level XML tags?

/// <documentation type="markdown"><![CDATA[
/// Returns the absolute value of a double-precision floating-point number.
/// # Parameters
/// - `value`: `Double`:
///    A number that is greater than or equals to `MinValue`, but less than or equal to `MaxValue`.
/// # Returns
/// `Double`:
/// A double-precision floating-point number, x, such that 0 ≤ x ≤ `MaxValue`.
/// ]]></documentation>
public static double Abs(double value);

Although a little verbose style, type or some other attributes can note compiler to switch XML translator.
Some formats may be supported on compiler-integrated translator, and others (including reStructuredText, doxygen style, and so on) can be delegated to extension.

[MovGP0]

I suggest to not overcomplicate things by thinking about Markdown-replacements for each and every one of the XML documentation elements, since they work fine as they are. Just do those things:

• Allow the omission of the <summary /> element
• Support links to objects (classes, enums, methods, etc.) using Markdown links (ie. [namespace.elementname] instead of <cref name="namespace.elementname" />)
• Support a subset of markdown elements, like code blocks, some basic formating (headers, italic, bold, etc.) and lists (bullet or numbered)

[hydroper]

@sharwell proposal is great: https://gist.github.com/sharwell/ab7a6ccab745c7e0a5b8662104e79735

[leo-costa]

Please make this a thing. XML for documentation is just too noisy. Even Java is contemplating the switch to markdown