Output directory of versioning - {PLUGIN_INSTANCE_ID}_versioned-docs should be configurable

[maxhr]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Description

I want to organize the project to place the docs Markdown directories outside of Docusaurus root. .

/my-project
    /my-docs-outside
        /main.md
    /docusaurus-website
        /docusaurus.config.js

I configure docs location indocusaurus.config.js:

...
presets: [
    [
      'classic',
      ({
        docs: {
          // ...
          path: '../my-docs-outside', // <<===
        },
    ],
  ],

To tag and create a copy of the current version I run
docusaurus docs:version v2

It creates the versions in Docusaurus root, and I can't configure it otherwise.

/my-project
    /my-docs-outside
        /main.md
    /docusaurus-website
        /docusaurus.config.js
        
        //*** new files: ***
        /versions.json
        /versioned_docs
            /version-v2
                /main.md
        /versioned_sidebars
            /version-v2-sidebars.json

It makes sense to have versions.json in Docusaurus root as it is a configuration file, but the actual Markdowns should at least consider the configured location of the other Markdowns, or be configurable.

Has this been requested on Canny?

No response

Motivation

It's reasonable to assume that all files input and output entires would be configurable.

It makes sense that versions.json and {PLUGIN_INSTANCE_ID}_versions.json to reside in Docusaurus root, as it is configuration, but placing copies of docs (which can be huge) upon every version tagging in root it not acceptable. I might want to have them in a separate repo.

This is solvable with symlinks, but that's just fragile and doesn't always 100% work.

API design

Implementation probably should adjust this getVersionDocsDirPath function:

https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-plugin-content-docs/src/versions/files.ts#L32

Default unconfigured behaviour

Keep it as it is now, create versioned content in Docusaurus root.

Configurable for all versions

The user should have an setting versionedDocsPath.

// docusaurus.config.js
presets: [
    [
      'classic',
      ({
        docs: {
          // ...
          path: '../my-docs-outside',
          versionedDocsPath: '../my-vers-outside'
        },
    ],
  ],

Running docusaurus docs:version v2 should produce the tree below.
versions.json is in Docusaurus root, the versioned copy is under ../my-vers-outside

/my-project
    /docusaurus-website
        /docusaurus.config.js
        
        //*** new file: ***
        /versions.json  
        
    /docs-outside
        /main.md
        
    //*** new files: *** 
    /my-vers-outside
        /versioned_docs
            /version-v2
                /main.md
        /versioned_sidebars
            /version-v2-sidebars.json

Even more configurable for each version separately

Config type VersionConfig should have an additional (optional) property. It should be named path to be consistent with other paths that set the directories' location, but VersionConfig.path is already in use for URL path.
Ideally that current path should be named routePath and path to be used for directory path, but it's a breaking change, so maybe it can be called versionedDocsPath to be clear it overrides the default versionedDocsPath.

Running the tagging script will still output files in directory configured under docs.versionedDocsPath.
Then the user can manually move the files and set versionedDocsPath for specific versions separately.

// docusaurus.config.js
presets: [
    [
      'classic',
      ({
        docs: {
          path: '../my-docs-outside',
          versionedDocsPath: '../my-vers-outside',
          versions: {
            current: {
              // for current version, `versionedDocsPath` is ignored because it's actually `path` from above   
            },
            'v2': {
              versionedDocsPath: '../version2',
              // ^^^ optional path for a specific version
            },
          },
        },
    ],
  ],

Multi intstance configuration

Project structure:

/my-project
    /docusaurus-website
        /docusaurus.config.js
    /docs-outside
        /main.md
    /other-section-docs <<=== a versioned section of the site
        /v-doc.md

docusaurus.config.js:

...
presets: [
    [
      'classic',
      ({
        docs: {
          path: '../docs-outside', 
        },
    ],
    
  ],
plugins: [
    [
      '@docusaurus/plugin-content-docs',
      {
        id: 'my-instance-id',
        path: '../other-section-docs',
        versionedDocsPath: '../my-versions',
      },
    ],
  ],

After running docusaurus docs:version:my-instance-id v2, file tree:

/my-project
    /docusaurus-website
        /docusaurus.config.js
        
        /my-instance-id_versions.json
         
    /docs-outside
        /main.md
    /other-section-docs
        /v-doc.md
        
    /my-versions
        /my-instance-id_versioned_docs
            /version-v2
                /v-doc.md
        /my-instance-id_versioned_sidebars
            /version-v2-sidebars.json

To customize v2 directory the user should move files manually.
Say we make more versions v3 and v4: docusaurus docs:version:my-instance-id v3 && docusaurus docs:version:my-instance-id v4.
After tagging, manually moving files around and configuring the versions, result may look like this:

docusaurus.config.js:

...
presets: [
    [
      'classic',
      ({
        docs: {
          path: '../docs-outside', 
        },
    ],
    
  ],
plugins: [
    [
      '@docusaurus/plugin-content-docs',
      {
        id: 'my-instance-1',
        path: '../other-section-docs',
        versionedDocsPath: '../my-versions',
        versions: {
            // v2 stays in place
            'v3': {
                versionedDocsPath: '../v34-dir
            },
            'v4': {
                versionedDocsPath: '../v34-dir
            }
        }
      },
    ],
  ],

/my-project
    /docusaurus-website
        /docusaurus.config.js
        /my-instance-1_versions.json
         
    /docs-outside
        /main.md
    /other-section-docs
        /v-doc.md
        
    /my-versions
        /my-instance-1_versioned_docs
            /version-v2
                /v-doc.md
        /my-instance-1_versioned_sidebars
            /version-v2-sidebars.json
    /v34-dir
        /my-instance-1_versioned_docs
            /version-v3
                /v-doc.md
            /version-v4
                /v-doc.md
        /my-instance-1_versioned_sidebars
            /version-v3-sidebars.json
            /version-v4-sidebars.json

Have you tried building it?

No response

Self-service

• I'd be willing to contribute this feature to Docusaurus myself.

[Josh-Cena]

Sounds reasonable. Thanks for the detailed writeup! This has been one of the best issue reports in a while 😄

[maxhr]

Cool I'll work on it 👍

[slorber]

Hey @maxhr , sorry for the delay.

To me, it is not necessarily the best idea to put the docs outside the site folder, and can lead to subtle bugs that might be surprising.

MD docs are compiled to React components with MDX, so your content files ARE real components after being compiled. If your docs folder (or any folder containing md content) is outside your site, there's a risk it resolves the React dependency from a different node_modules and you end up with duplicate React versions at runtime.

This is more likely to happen in monorepo setups, see #8091 (comment)

Note that if your content folder does not have any node_module in its parent folders, it should be fine and we'll resolve the react package to website/node_modules/react thanks to Webpack resolve.modules

I'm not against implementing this but please be aware of the limitations of what you want to do.

---

Adding VersionConfig.versionedDocsPath would work but wouldn't allow you to configure the sidebar path on a per-version basis. Is this a problem? Who would use this and why? IMHO having 2 folders (versioned_docs/versioned_sidebars) is to me a legacy problem and I'd rather move later the sidebar file directly inside the docs folder 🤪

Can you explain more of your motivations to move the docs structure outside of the site, and to be able to configure folder with that level of granularity?

[Jersyfi]

I don't know why it is necessary to put it outside but i agree that it is necessary to modify the root path of a docs folder. maxhr described it as versionedDocsPath but i rather would describe it as rootPath. I run in the same issue that i think it looks horrible with the folder structure when using more then 2 docs instances.

With the rootPath the folder structure would look like this:

website
├── multi-docs
│   ├── doc-1
│   │   ├── docs
│   │   │   ├── foo
│   │   │   │   └── bar.md
│   │   │   └── hello.md
│   │   ├── versioned_docs
│   │   │   ├── version-1.1.0
│   │   │   │   ├── foo
│   │   │   │   │   └── bar.md
│   │   │   │   └── hello.md
│   │   │   └── version-1.0.0
│   │   │       ├── foo
│   │   │       │   └── bar.md
│   │   │       └── hello.md
│   │   ├── versioned_sidebars
│   │   │   ├── version-1.1.0-sidebars.json
│   │   │   └── version-1.0.0-sidebars.json
│   │   ├── versions.json
│   │   ├── sidebars.json
│   ├── doc-2
│   │   ├── docs
│   │   │   ├── foo
│   │   │   │   └── bar.md
│   │   │   └── hello.md
│   │   ├── versioned_docs
│   │   │   ├── version-1.1.0
│   │   │   │   ├── foo
│   │   │   │   │   └── bar.md
│   │   │   │   └── hello.md
│   │   │   └── version-1.0.0
│   │   │       ├── foo
│   │   │       │   └── bar.md
│   │   │       └── hello.md
│   │   ├── versioned_sidebars
│   │   │   ├── version-1.1.0-sidebars.json
│   │   │   └── version-1.0.0-sidebars.json
│   │   ├── versions.json
│   │   ├── sidebars.json
├── docusaurus.config.js
└── package.json

[GM1957]

@Jersyfi , I agree with your point. Currently, I am maintaining a single config.js file in each docs folder (such as inside the doc-1 folder, as shown in your folder tree structure). This config.js file includes all plugin details for @docusaurus/plugin-content-docs, such as id, path, sidebarPath, etc. which are then imported into the docusaurus.config.js plugins array. Initially, I believed that running npm run docusaurus docs:version:[plugin-id] [new-version-id] would create the versioned_docs, versioned_sidebars, and versions.json files in the doc-1 folder since I had already specified the path in the plugin config. However, these files are actually being created at the root level. @slorber , can you please provide your input on this scenario? Am I missing something?

[Jersyfi]

@GM1957 Thank you for the information. Your work will also help shrinking the 'docusaurus.config.js' file.

Thinking more about my point I think it would be the best to reorganize the structure in the future and for now just to make the path configurable. As a example at the moment for doc-1 the folders are named doc-1_... too but when the docs are in the same folder this isn't needed anymore. I hope it is understandable what I mean.

When you have something new just let me know and I need to say that you are doing a great job so far with docusaurus.

[GM1957]

Thank you @Jersyfi. Currently is there any way to have versioned_docs, versioned_sidebars and versions.json in a different place instead root level of the project ?

[Jersyfi]

@GM1957 as i could figure it our it is not possible. I looked at the documentation and the code and these files need to be placed in the root level of the project.

[GM1957]

@Jersyfi currently I am trying to achieve a modular structure for my use case because this versions and all will be handled through pipeline, in more of a automated way. so how to move forward with this issue ?

[Jersyfi]

@GM1957 for the v2 version of docusaurus i think it is the best way to add a rootPath value with default: '.'. With this it is possible to group the docs like below. In a future v3 version the structure could be rebuild.

Are you able to add this variable to the code structure? Because i would need some time to understand the code structure in order to implement rootPath.

website
├── multi-docs
│   ├── doc-1
│   │   ├── docs
│   │   │   ├── foo
│   │   │   │   └── bar.md
│   │   │   └── hello.md
│   │   ├── doc-1_versioned_docs
│   │   │   ├── version-1.1.0
│   │   │   │   ├── foo
│   │   │   │   │   └── bar.md
│   │   │   │   └── hello.md
│   │   │   └── version-1.0.0
│   │   │       ├── foo
│   │   │       │   └── bar.md
│   │   │       └── hello.md
│   │   ├── doc-1_versioned_sidebars
│   │   │   ├── version-1.1.0-sidebars.json
│   │   │   └── version-1.0.0-sidebars.json
│   │   ├── doc-1_versions.json
│   │   ├── sidebars.json
│   ├── doc-2
│   │   ├── docs
│   │   │   ├── foo
│   │   │   │   └── bar.md
│   │   │   └── hello.md
│   │   ├── doc-2_versioned_docs
│   │   │   ├── version-1.1.0
│   │   │   │   ├── foo
│   │   │   │   │   └── bar.md
│   │   │   │   └── hello.md
│   │   │   └── version-1.0.0
│   │   │       ├── foo
│   │   │       │   └── bar.md
│   │   │       └── hello.md
│   │   ├── doc-2_versioned_sidebars
│   │   │   ├── version-1.1.0-sidebars.json
│   │   │   └── version-1.0.0-sidebars.json
│   │   ├── doc-2_versions.json
│   │   ├── sidebars.json
├── docusaurus.config.js
└── package.json

[slorber]

rootPath makes sense to me 👍

The way I understand it, you'd like to have something like this:

website/
├─ ios/
│  ├─ docs/
│  ├─ versioned_docs/
│  ├─ versions.json
├─ android/
│  ├─ docs/
│  ├─ versioned_docs/
│  ├─ versions.json

For retrocompatibility reasons it's likely that we'll have to keep the longer folder name versioned_docs_ios instead of just versioned_docs, and similar for versions_ios.json instead of versions.json.

If we want shorter names we need more than just rootPath?
(unless we apply the suffix but only if rootpath: '.' default value 🤷‍♂️ )

---

I don't have time to work on this feature now but if someone want to do a proper RFC/design proposal and submit a PR I'd be happy to have this

[Jersyfi]

@slorber perfekt that you agree with it.

You are right that this needs rootPath and a suffix to change the prefix for versioning and to maintain retrocompatibility. I will take a look if something else is needed and create a PR.

[GM1957]

@slorber correct, but as I can understand this rootPath will again work with docs route. Here are few points which I want to highlight for more customization options.

1. We are already giving path parameter for @docusaurus/plugin-content-docs plugin to specify where our custom doc directory exists.

2. The system defined naming convention for versioned_docs, versioned_sidebars, versions.json are completely okay but the issue is where system is trying to find them. ideally it should be one level before of given path in the path parameter,
   for example:

 website/
├─ ios/
│  ├─ myCustomDocs/
│  ├─ ios-docs_versioned_docs/
│  ├─ ios-docs_versioned_sidebars/
│  ├─ ios-docs_versions.json
│  ├─ sidebar.json

plugin config will look like

[
  "@docusaurus/plugin-content-docs",
  {
    id: "ios-docs",
    path: "./ios/myCustomDocs",
    routeBasePath: "ios-sdk-docs",
    sidebarPath: "./ios/sidebar.json",
  },
];

For this config, system should look for versioned_docs, versioned_sidebars in ios folder, but right now it always looking at website/ .

3. While running npm run docusaurus docs:version:[plugin-id] [new-version-id] its creating all versioned files and folders in root level but it should create according to given path.

I think this will give us more flexibility and customization options, maybe we can take this as a bug.

[slorber]

ideally it should be one level before of given path in the path parameter

I don't know, that feels quite implicit to me and not so flexible. I'd prefer if docs and versioned_docs could live in separate folders: more flexible for many use-cases.

Instead of rootPath it's also possible to have a versionedPath taking a string template or callback, giving you full power to decide where the versioned path should live. Eventually each version could even have a top-level folder for example?

ios-docs/next
ios-docs/v1.0
ios-docs/v2.0
ios-docs/v3.0

[
  "@docusaurus/plugin-content-docs",
  {
    id: "ios",
    path: "./ios-docs/next",
    versionedPath: ({version}) => `./ios/docs/docs/v${version}`,
  },
];

I think the above use-case is a legit docs fs structure so we should allow users to be able to achieve that easily, and even more fancy patterns.

We could even avoid the versionedPath and just provide a callback for path 🤷‍♂️

[Jersyfi]

@slorber yesterday i looked at the code and it is possible to keep the flexebility up running. I will provide a PR and you can take a look at it.

[GM1957]

@slorber makes sense, I think it will be a great enhancement. Thanks @slorber @Jersyfi

[Jersyfi]

@slorber @GM1957 I just wanted to let you know that I have created a PR. Let me know when you have any updates for me. Start your week well.

[Jersyfi]

@slorber then i will keep up the discussion here 👌
Referenced PR comment: #8737 (review)

Firstly, I would like to comment on your comment about versionPath, to better understand what you mean and what you want. Do you want to customise the version file structure? Because in version v2 I thought it would be best to keep the file structure and only give the possibility to change the path of it.

If I understand your code comment correctly, the following configuration will result in the following code. Right? It would be great if you could describe your file structure in detail so that I know what your requirements are.

[
  "@docusaurus/plugin-content-docs",
  {
    id: "ios",
    path: "./ios-docs/v-next",
    versionPath: ({version}) => `./ios-docs/v-${version.toLowerCase().replace(".","-"}`,
  },
];

 website/
├─ ios-docs/
│  ├─ v-next/
│  ├─ v-1-0-0/
│  ├─ // versioned siderbars?
│  ├─ versions.json
│  ├─ sidebar.json

---

With versionPrefix i wanted to give the option to not use the id as a prefix of the folder name. So it would be better to name it something like useVersionPrefix, if your idea requires it at all.

In the following example i describe you what changed with the configurration option. Note that i replaced versionPrefix with useVersionPrefix that is a bit more understandable.

[
  "@docusaurus/plugin-content-docs",
  {
    id: "ios",
    path: "./ios-docs/docs",
    useVersionPrefix: false,
  },
];

 website/
├─ ios-docs/
│  ├─ docs/
│  ├─ versioned_docs/ // instead of `ios_versioned_docs`
│  ├─ versioned_sidebars/ // instead of `ios_versioned_sidebars`
│  ├─ versions.json
│  ├─ sidebar.json

---

I have already noticed the confusion between route path and fs path, but it is understandable once you get it 😂.
But in order to understand what exactly you mean by this point, I need to understand the first comment. Because with the changes I made, it is still possible to use the versions.{version}.path.

[slorber]

Actually forgot that we also need Docusaurus to know where to read/write the sidebar files. In the future, I'd like to get rid of the versioned_sidebars folder and by default put sidebars.js inside the docs folder (versioned or not). That colocation makes more sense for multi-docs plugin instances.

What I don't like with your solution is that:

• {versionPath: "./ios-docs"} => works but users unable to configure the full name for versioned_docs and versioned_sidebars (they can only choose to add/remove the id_ prefix)
• {versionPath: "./ios-docs/versioned_docs"} can configure versioned_docs name but then Docusaurus does not know where to find the sidebar, and versioned_sidebars remains not configurable (and we probably don't want 2 options 🤷‍♂️ )

Considering I'd like to get rid of the versioned_sidebars folder, I'd prefer if we implement something simple and low-level to get started, and the individual config options for each version is the most flexible. It's not the most convenient (a bit low level) but we can add "shortcuts" later to improve the DX.

What I mean:

module.exports = {
  plugins: [
    [
      '@docusaurus/plugin-content-docs',
      {
        path: 'docs', (actually ignored => we configure paths for each version independently)
        versions: {
          current: {
            path: 'ios-docs/v-next',
            sidebarPath: 'ios-docs/v-next/sidebar.js'
          },
          '2.0.0': {
            path: 'ios-docs/v-2.0.0',
            sidebarPath: 'ios-docs/v-2.0.0/sidebar.js'
          },
          '1.0.0': {
            path: 'ios-docs/v-1.0.0',
            sidebarPath: 'ios-docs/v-1.0.0/sidebar.js'
          },
        },
      },
    ],
  ],
};

Does it make sense?

If you don't want to define paths for each version, you can use Node.js code and your own conversion to compute the value for versions

[Jersyfi]

@slorber your new folder structure makes sense. But i think that it is better to define it only once for all versions as you already wrote in an example.

[
  "@docusaurus/plugin-content-docs",
  {
    id: "ios",
    path: "./ios-docs/next",
    versionedPath: ({version}) => `./ios/docs/docs/v${version}`,
  },
];

[GM1957]

Hey @Jersyfi , is there any update regarding the mentioned PR?

[Jersyfi]

@GM1957 I looked at the code, but it's too time-consuming for me to change it the way how @slorber wants. Forgot to close it then.