Convert the spec to Markdown

[gregsdennis]

As we are no longer publishing with IETF, we don't need to use their tools. Going forward, we'd like to use markdown to author the spec instead of the current XML.

The first tasks are:

• decide markdown flavor and toolset
• convert the existing document

[jdesrosiers]

I have an idea about how hopefully make this process a bit less painful that it sounds. I believe the IETF tooling has the ability to convert from XML style to Markdown style. It's an IETF specific markdown flavor, but it should be easier to convert from one Markdown flavor to another than it would be to convert from XML to Markdown.

[gregsdennis]

We now have two approaches: #1357 (plain Markdown) and #1412 (IETF Kramdown).

I thought it would be wise to enumerate the document features we need/want. Then we can decide which of these approaches best captures that.

Looking at the current (IETF XML) published spec in HTML form, here's what's used:

• Headers (automatic numbering)
• Paragraph styles
  • Normal
  • Code blocks (no syntax highlighting)
• Inline comments (CREFs)
• Links (internal & external)
• Ordered & unordered (both plain and bulleted) lists
• Table of Contents (automatic)

In addition to the above, JSON Path (IETF Kramdown) also has these:

• Text styles
  • italic
  • code (monospace)
• Paragraph styles
  • Code blocks (with captions, no syntax highlighting)
• Tables (with captions)
• Table of Contents (automatic)

In addition to the features our current spec uses, OpenAPI (appears to use simple Markdown) also uses:

• Text styles
  • bold
  • italic
• Paragraph styles
  • Code blocks (with syntax highlighting)
• Tables (no captions)
• Table of Contents (written in the markdown, but rendered as side bar)

In addition to all of the above, do we also want (or could we make use of):

• Text styles
  • underline (not supported directly by markdown, but we can use <u> tags)
  • strikethrough?
• Paragraph styles
  • Block quotes
  • Comments? (see https://chirpy.cotes.page/posts/text-and-typography/#prompts for some examples of this idea)
• Images? (with captions?)

---

Lastly,

• Do we want other publication formats, e.g. PDF, plain text? I think a rudimentary PDF could be generated simply enough from HTML, but the IETF Kramdown supports direct export to these formats.
• The current document (XML) supports automatic section numbering based on the header sequence. The IETF Kramdown also supports this, but while I don't think the basic Markdown does in general, the OpenAPI doc seems to do this automatically somehow.

If we decide to go with basic Markdown, I think we might benefit from "borrowing" some rendering code from OpenAPI's process.

[gregsdennis]

After some time to consider the options, I think I prefer the simple markdown, assuming we can "borrow" the rendering from OpenAPI to extract the TOC and get automatic section numbering.

I'd also like to see support for comments for what we now have as CREFs. It doesn't have to be like the example I've linked to, but I think we need something.

[jdesrosiers]

I agree. I'll work on a proof of concept to ensure we can get a table of contents and automatic section numbering using a markdown engine that supports plugins such as https://remark.js.org/. Assuming that works out (I'm sure it will), I think we're ready to move forward with this step.

[jdesrosiers]

I finished the poc. The markdown I used is here and you can see the rendered result here. (I used water.css for some light default styling. It's not built-in to the rendering.)

Remark appears to be very powerful. There's not much we wouldn't be able to do, but we might have to get our hands dirty if we want to do anything terribly fancy. For example, I had to write a plugin to get numbered headings.

[gregsdennis]

Thanks for putting that together (and quickly). Would it be possible to update the markdown to include all the features we're currently using? Ordered and unordered lists will be a must, and special rendering for comments (CREFs) would be nice.

It's good that we can create plugins, and I really like that the code blocks have syntax highlighting!

[jdesrosiers]

I added a demo of ordered/unordered lists and tables as requested. I also added support for callout boxes, which can be used for comments as well as highlighting experimental sections.

Something missing is a references summary at the end, if that's something we want. We'd have to write a plugin, but I'm fairly confident I could handle that.

[gregsdennis]

That looks great, thanks! A little CSS love (not necessary for this POC), and I think we're good to go!