Consider remark

[pngwn]

The decision to use markdown-it was relatively arbitrary when I put this together initially: the performance was okay and there are a decent array of plugins. I've found it pretty annoying over time, however, and have been investigating alternatives. marked is the obvious candidate due to its focus on performance (performance is a concern for sites with more content) but there don't seem to be as many plugins and I don't want to limit what users can actually do with MDsveX, although it does have a decent API. remarkable is another candidate and, if performance is the priority, might a better option.

In that vein, I began investigating other solutions. remark was my initial preference but I assumed (without any testing whatsoever) that it would be quite a bit slower due to the fact it builds an AST, but my very basic benchmarks have shown this to be a false assumption. It seems to be just as fast as marked in my preliminary tests (these tests include writing to an HTML string). What is more, there are a huge array of plugins available and having access to an AST gives a huge amount of flexibility when it is needed.

remark will almost definitely have higher memory consumption than a simple lexer/parser with no intermediary AST but this seems to be a trade-off that people are willing to make. There is precedent here as MDX uses Remark and no-one seems to mind much, although what is palatable to one community is not necessarily so for another.

I'll look into this over the coming weeks and take a closer look at the impact this change would have on both build times and memory consumption, the plugin ecosystem seems healthy (which matters quite a bit) but I'll do some more research in that regard as well. When I have more concrete data on the tradeoffs between the two I will make a decision.

[swyxio]

oo yes, the plugins are a big draw

[pngwn]

Yeah, that is my main motivation too. Barring any revelations, this will probably happen.

[swyxio]

also todo: enforce (or make very very easy to add) some good defaults like header tags having ID's, and sections having sections:

• https://github.com/remarkjs/remark-autolink-headings
• https://github.com/remarkjs/remark-slug
• https://github.com/jake-low/remark-sectionize

[pngwn]

It should be very simple to install these kinds of plugins, autolink headings kind of makes sense for content sites and so could be made available as an option. I'm not sure about automatic sections though, sounds too opinionated, a user could easily bring that in themselves. rehype-autolink-headings is also an option for heading links and may be preferable since html is rehype's domain.

mdsvex can be used for creating any kind of svelte component, not just 'content'. While I want it to be usable with little configuration, I don't want the defaults to have too many opinions about the output. If you put in some markdown/ svelte and get back a weird mess of section-wrapped code it could be confusing.

[swyxio]

totally. i was stuck thinking about mdx as content rather than mdx as sfc's. i really like that analogy (mdx as a sfc superset)

[pngwn]

This was a biggy. Done in 0.8.

[swyxio]

what a journey!