Check for broken links

[Dr-Electron]

Is your feature request related to a problem? Please describe.

We need a tool to check for broken links outside of what Docusaurus provides.

Describe the solution you'd like

I think rehype-url-inspector could help and it may even be helpful for #879

Describe alternatives you've considered

None.

[Dr-Electron]

I did some research and it doesn't seem like the above mentioned retype plugin is what we need.
I also think we should prefer remark plugins as that way it is easy(?) to differentiate between internal and external links. And we only need to check externals as internal ones are already checked by Docusaurus itself.

Some useful links/discussions I found:
facebook/docusaurus#6998
https://github.com/remarkjs/remark-lint-no-dead-urls
facebook/docusaurus#3193

[jlvandenhout]

What would you think about the following: I'd consider the option to add remark and rehype plugins to the Docusaurus config a way to alter the actual build. Even though Docusaurus does some link checking during the build, I'd consider checking for broken links part of linting and formatting, before the actual build. So maybe we could utilize the CLI for this, instead of hooking into the Docusaurus build process? Then we have full control of how we report the links, because afaik Docusaurus doesn't do anything with the remark output of this plugin.

[Dr-Electron]

But imo going through the AST (markdown or html) is the ideal solution and as rehype/remark does that it sounded perfect to me. I could also just create an own plugin which logs to console. Seems to be less complicated to me. Implementing in the cli will again make it more complicated (especially for me). We should find a solution. We can then always iterate over it imo.

A pro of doing it with the cli would be that it can easily be used in external docs tho 🤔 .

[jlvandenhout]

Yes. What I mean is we can run this plugin ourselves in the CLI, either from code using the unified framework or remark cli. That way we have full control how we report the broken links. And we could add more of those linting rules.

Otherwise we either need to use the rehype URL inspector you found and implement the URL check logic ourselves or create our own remark plugin that does the same thing as the plugin that checks for dead links, but prints them instead of adding them to the file object. Because adding them to the docs plugin as they currently are, won't help us really.

[Dr-Electron]

I mean adding it to the file object is fine. We just need something like vfile reporter to add it to the wiki log 🤔
As I said. I would prefer having a solution now. But if you can quickly make the cli ready for this, go for it.

[jlvandenhout]

Ah yeah that's of course also an option. Add an additional plugin after the link checker to report the result. Let's do that.

[Dr-Electron]

Ser. I have no clue how tho
I tried and failed miserably 😆

[annie101101046]

Please update the latest status

[jlvandenhout]

Discussed what would be the best option here. As pointed out in one of the linked discussions, it doesn't make sense to check broken links on PRs all the time, because they can break for several reasons, including reasons that are out of scope of the PR where URLs would be checked. So the choice was made to implement it in the Wiki CLI, so URLs can be checked regularly or when needed by running a CLI command. The command and logic to detect all MD(X) documents is implemented. What remains is running Remark with the plugin mentioned above on those documents to detect dead links and report them.