feat: nextjs and mdx compiler

[ovflowd]

Description

This PR does a significant rework on our MDX Parsing and updates Next.js to v14 and MDX to v3.

Motivation

We used a 3rd party package to parse mdx with source strings, aka, next-mdx-remote "maintained" by Hashicorp. But as it became more apparent that we don't need a 3rd party solution and we can use a lightweight solution, I've decided to use the core of MDX itself @mdxjs/mdx without any extra sugar etc.

• This allows us to be always up to date with upstream @mdxjs and all the unified ecosystem (vfile, unified, remark, rehype, etc)
• Allows us to finally use remark-gfm plugin instead of handpicking a few versions and specific plugins and trying to make a live with it
• Removes complexity, simplifies the code, and gives better ownership for our needs.
• Simplifies some of our direct dependencies
• Removed rehype-raw and use plain Markdown. Because Markdown files should not mix HTML. (PLEASE DOUBLE CHECK PAGES ARE WORKING AS EXPECTED (compare with nodejs.org))
• Tried to optimise a few pieces of the build process

The implementation is bare-bones and very simple, and it works fantastically.

Validation

Pages should be rendered as usual without content change...

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
nodejs-org	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 1, 2023 2:10pm

[github-actions]

Lighthouse Results

URL	Performance	Accessibility	Best Practices	SEO	Report
/en	🟢 100	🟢 95	🟢 92	🟢 92	🔗
/en/about	🟢 99	🟢 92	🟢 92	🟢 92	🔗
/en/about/previous-releases	🟢 99	🟢 91	🟢 92	🟢 92	🔗
/en/download	🟢 100	🟢 92	🟢 92	🟢 92	🔗
/en/blog	🟢 98	🟢 92	🟢 92	🟢 92	🔗

[github-actions]

Unit Test Coverage Report

Lines	Statements	Branches	Functions
	95.41% (333/349)	77.21% (61/79)	93.05% (67/72)

Unit Test Report

Tests	Skipped	Failures	Errors	Time
42	0 💤	0 ❌	0 🔥	7.483s ⏱️

[bmuenzenmeyer]

js syntax highlighting is not working

✅ https://nodejs.org/en/learn/manipulating-files/working-with-folders-in-nodejs
❌ https://nodejs-org-git-feat-next-14-lightweight-mdx-compiler-openjs.vercel.app/en/learn/manipulating-files/working-with-file-descriptors-in-nodejs

at times these pages don't render. cannot reliably recreate yet. LH seemed to have the same issue

❌ https://nodejs-org-git-feat-next-14-lightweight-mdx-compiler-openjs.vercel.app/en/about

❌ https://nodejs-org-git-feat-next-14-lightweight-mdx-compiler-openjs.vercel.app/en/blog/vulnerability/openssl-fixes-in-regular-releases-oct2023

[ovflowd]

@bmuenzenmeyer feel free to re-review this ✨

[AugustinMauroy]

SGTM

[canerakdas]

LGTM, I compared the preview with nodejs.org. Also, fixes the markdown formatted tables

Outdated review