Support for passing MDAST instead of raw markdown text

[kimmobrunfeldt]

Our documentation is combined from multiple smaller MDX content pieces. To have for example unique slugs, we have a centralized pre-processing for the content which parses the smaller pieces into their own MDASTs and makes sure that the slugger state is shared between these smaller pieces.

This means we would need the ability to pass the MDAST directly to the runtime, instead of passing the raw markdown text as children. Would you be open to merge this feature if I send a PR for it?

The new suggested API of runtime

This would be a backwards compatible change.

With mdast:

<MDX scope={scope} components={components} mdast={myMdast} />

With raw markdown text:

<MDX scope={scope} components={components}>{myMdText}</MDX>

Changes needed

1. Expose createCompiler() from @mdx-js/mdx

This is needed to get direct access to unified API in runtime. We also want to use the DEFAULT_OPTIONS when creating the new compiler, so the function might need refactoring for that part.

The other option would be to expose the DEFAULT_OPTIONS so that runtime gains the knowledge of them, but it sounds a bit more complicated approach to me.

2. Support mdast prop in @mdx-js/runtime

These lines would need to be changed to:

// This is the newly exposed function
const compiler = mdx.createCompiler({
  mdPlugins,
  hastPlugins,
  skipExport: true
})

let jsx
// mdast is a new prop
if (mdast) {
  const tree = compiler().runSync(mdast)
  jsx = compiler.stringify(tree)
} else {
  jsx = compiler.processSync({ contents: children }).contents
}

const { code } = transform(jsx.trim())

[kimmobrunfeldt]

We need this internally, so I started to work on the changes already. You can see the diff here: master...AitoDotAI:master.

I encountered an issue with exports and imports. Normal markdown parser isn't enough, as some JSX may be interpreted as html blocks etc. My first thought that the parameter should actually be MDXAST instead of MDAST, but I checked how @mdx-js/mdx does it, and got it working by also exposing the esSyntax tokenizer which properly extracts the import and export blocks. Can that syntax tree be called as MDXAST yet? It's still missing the transformations done here though: https://github.com/mdx-js/mdx/blob/master/packages/mdx/md-ast-to-mdx-ast.js

[johno]

Thank you very much for the detailed writeup and spiking!

The one thing I'm wondering is if your use case can be handled as a remark plugin that's passed to MDX? If you're operating on the MDAST it sounds like you're using some type of plugin to transform it based on your central slug processing.

I ask because it sounds like we could handle that without modifying core at all (potentially). Let me know what you think!

---

• https://mdxjs.com/plugins
• https://github.com/remarkjs/remark/blob/master/doc/plugins.md

[johno]

Due to lack of activity I'm going to close this for now. Please feel free to open a new issue if your needs can't be addressed with a remark plugin.