Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mdx-loader): the table-of-contents should display toc/headings of imported MDX partials #9684

Merged
merged 35 commits into from
Jan 19, 2024

Conversation

anatolykopyl
Copy link
Contributor

@anatolykopyl anatolykopyl commented Dec 31, 2023

Note: @anatolykopyl started this PR, and then @slorber took it over.

Motivation

Fix #3915

It is now possible to import MDX partials and have headings of the partial appear in the TOC.

We'll consider this as a "new feature" instead of a bug fix, considering it remains an important behavior change, and an improvement over stock MDX. We'll release this in v3.2 instead of v3.1.1


Consider the following:

// _partial.mdx

## Partial heading
// doc.mdx

# Doc

## Doc heading

import Partial from "_partial.mdx"

<Partial/>

This doc now renders a TOC with 2 headings, while previously "Partial heading" didn't appear in the TOC.

In practice, the doc is now compiled as:

// doc.mdx
import Partial, 
  {toc as __tocPartial} from "_partial.mdx"

# Doc

## Doc heading

export const toc = [
  {
    "value": "Doc heading",
    "id": "doc-heading",
    "level": 2
  }, 
  ...__tocPartial,
]

Where __tocPartial contains {"value": "Partial heading","id": "partial-heading","level": 2}

This makes it possible to have headings working for "deeply-nested-partials" where one partial import another.

Notable change: if you write export const toc = ... in your MDX docs, we now never override this value anymore. Previously we did, but only when there were headings. We consider this as a bug fix and not a breaking change.

Test Plan

Unit tests + dogfood

Test links

https://deploy-preview-9684--docusaurus-2.netlify.app/tests/pages/partials-tests

Related issues/PRs

This PR adapts a similar fix from @dimaMachina made in Nextra to Docusaurus: shuding/nextra#2199

@facebook-github-bot facebook-github-bot added the CLA Signed Signed Facebook CLA label Dec 31, 2023
@anatolykopyl anatolykopyl marked this pull request as draft December 31, 2023 00:09
Copy link

netlify bot commented Dec 31, 2023

[V2]

Name Link
🔨 Latest commit 3d43631
🔍 Latest deploy log https://app.netlify.com/sites/docusaurus-2/deploys/65aab464a3d8d3000833330f
😎 Deploy Preview https://deploy-preview-9684--docusaurus-2.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

github-actions bot commented Dec 31, 2023

⚡️ Lighthouse report for the deploy preview of this PR

URL Performance Accessibility Best Practices SEO PWA Report
/ 🟠 83 🟢 98 🟢 100 🟢 100 🟠 89 Report
/docs/installation 🟠 89 🟢 98 🟢 100 🟢 100 🟠 89 Report
/docs/category/getting-started 🟠 76 🟢 100 🟢 100 🟢 90 🟠 89 Report
/blog 🟠 70 🟢 100 🟢 100 🟢 90 🟠 89 Report
/blog/preparing-your-site-for-docusaurus-v3 🟠 63 🟢 97 🟢 100 🟢 100 🟠 89 Report
/blog/tags/release 🟠 69 🟢 100 🟢 100 🟠 80 🟠 89 Report
/blog/tags 🟠 77 🟢 100 🟢 100 🟢 90 🟠 89 Report

@anatolykopyl
Copy link
Contributor Author

I consider this PR done.
@slorber, could you please check it out? Any tips regarding how this can be improved are appreciated! I'd be glad to make changes to this PR if necessary.

@anatolykopyl anatolykopyl marked this pull request as ready for review January 10, 2024 21:37
Copy link
Collaborator

@slorber slorber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like good start thanks 👍

I'm not 100% fan of the implementation yet but overall the solution looks like what we want to adopt.

I'd like to have much more unit tests, testing more complex cases, such as using the same partial twice in a page, or importing the same partial using different names.

That would also be interesting to dogfood with more complex cases, such as a deeply nested tree of partials (having partials importing other partials)

packages/docusaurus-mdx-loader/src/remark/utils/index.ts Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
id: 'thanos',
level: 2
},
...toc0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toc0 is not imported here, is this expected?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we're going down the road of only modifying the AST and not the actual js, so this is expected.

id: child.data!.id!,
level: child.depth,
});
if (child.type === 'mdxjsEsm') {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know the original code was already low-level, but I'd appreciate if we could make it more readable and high-level. I mean the intent of this algorithm should be clear without spending time trying to understand the implementation details. Currently it's a bit hard to see the big picture of it. I would extract things in smaller functions for example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the visitor for each type of node into its own function. This is a bit more readable now, but I can try to do some further reorganisation if needed.

packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
website/_dogfooding/_pages tests/partials-tests.mdx Outdated Show resolved Hide resolved
@slorber slorber changed the title fix: ToC generation when importing markdown fix(mdx-loader): the table-of-contents should display toc/headings of imported MDX partials Jan 12, 2024
@anatolykopyl
Copy link
Contributor Author

Thanks for the detailed feedback! I'll be working on improving this PR in the following weeks.

Copy link
Collaborator

@Josh-Cena Josh-Cena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite happy with this implementation; exactly how I pictured it.

packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
@anatolykopyl
Copy link
Contributor Author

Moved the dogfooding into _docs tests.
Added a deeply nested partial and a partial with props in the heading. Last one is not working, the heading shows the text passed to the prop as expected, but the ToC displays just the text props.propname. Quite interesting, no idea how this can possibly be fixed.

Also since I changed the plugin to not generate any js anymore, only ASTs, the existing unit tests for the ToC plugin became useless, as they checked generated mdx. I changed the unit tests to output ASTs, so the work of the plugin is visible.

Copy link
Contributor

@wooorm wooorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last one is not working, the heading shows the text passed to the prop as expected, but the ToC displays just the text props.propname.

What output do you want, what would be “working”?

The code you use uses toValue: https://github.com/anatolykopyl/docusaurus/blob/1fdd6af8c79e7765ff4d7b4731e605d0d95c1316/packages/docusaurus-mdx-loader/src/remark/utils/index.ts#L76C17-L76C24.
Which calls toString() on an expression ({props.propname}): https://github.com/anatolykopyl/docusaurus/blob/1fdd6af8c79e7765ff4d7b4731e605d0d95c1316/packages/docusaurus-mdx-loader/src/remark/utils/index.ts#L99
Which returns node.value, which is props.propname.

packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
Copy link
Collaborator

@slorber slorber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that looks like it's working nicely.

The {props.propname} is not a problem to consider for now so you can remove that test.

I understand why you changed the test snapshots to MDAST although I don't like it. I'll try to come up with a solution and commit a thing to your PR soon.

packages/docusaurus-mdx-loader/src/remark/toc/utils.ts Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/utils.ts Outdated Show resolved Hide resolved
website/_dogfooding/docs-tests-sidebars.js Outdated Show resolved Hide resolved
yarn.lock Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
packages/docusaurus-mdx-loader/src/remark/toc/index.ts Outdated Show resolved Hide resolved
@slorber slorber added pr: new feature This PR adds a new API or behavior. to backport This PR is planned to be backported to a stable version of Docusaurus labels Jan 18, 2024
@slorber
Copy link
Collaborator

slorber commented Jan 18, 2024

Thanks @anatolykopyl , looks like it's working nicely.

Almost ready, I'm just going to refactor some details and merge asap!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Argos Add this label to run UI visual regression tests. See argos.yml GH action. CLA Signed Signed Facebook CLA pr: new feature This PR adds a new API or behavior. to backport This PR is planned to be backported to a stable version of Docusaurus
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TOC does not work when importing one MDX into another
5 participants