How to add project with plugins collection?

[mkusz]

Hi,
I'm the author of mkdocs-publisher project that contains multiple plugins. Some of them can be used separately, but some of them can work in cooperation. It's easier to test and maintain, especially when all of them are related to the one goal or personal web page/blog publishing. I want to add this project to this repository, but I'm not sure how to do it "the right way". Adding it as a mkdosc-publisher makes no sense, since there is no proper category for it. The close one is templates, since templates can contain multiple plugins, but it's not a template. The other idea is to add each of the plugins separately, but then I have to link all of them to the same repository, etc. If this is "the way" should I create a separate pull request or do a single one (it's described to do a one per plugin, but since it's a collection, should the same rule apply?). Please advise.

[pawamoy]

Hi!

It seems to fit in the Blogging category. Each project can have multiple plugins attached:

- name: Publisher for MkDocs
  mkdocs_plugin: [pub-meta, pub-blog, pub-obsidian, pub-social, pub-minifier, pub-debugger]
  github_id: mkusz/mkdocs-publisher
  pypi_id: mkdocs-publisher
  labels: [plugin]
  category: blogging

[oprypin]

It's probably also possible to create a few separate entries but specifying the same repository several times. We haven't tested if it actually works but we will surely have a precedent for that sooner or later.
Is there a manually specified "description" field for that case?
In any case it should be 1 pull request

[mkusz]

@pawamoy I only partly agree. Yes blogging is one of the main functionality of the project (initialy, the project was named mkdocs-blog-in), but other plugins like pub-debugger and pub-minifier can be used in total separation to other plugins (also blog by itself can be used in separation). Adding this project only in blogging category, will give a false picture, what actually is inside this project. I know that from one perspective it's a bad decision for this kind of "monorepo" but like I said at the beginning, it also has some advantages.

My own idea was to do something like that:

• pub-blog -> blogging category
• pub-meta -> navigation and page building
• pub-obsidian -> this is kind of problematic, since it's should be in 3 different categories:

1. integration with other tools (if only one is allowed, this should be chosen)
2. links & references
3. charts, images, tables & graphs

• pub-social -> HTML processing and CSS styling
• pub-minifier -> HTML processing and CSS styling
• pub-debugger -> other

The only problem I can see, is that (after @oprypin comment), in this approach, all plugins will contain the same description taken from git repo. Am I correct?

[oprypin]

The only problem I can see, is that (after @oprypin comment), in this approach, all plugins will contain the same description taken from git repo.

Yes but for that it's possible to just set the description: field instead of using the github description.

The actual problems are:

• whether this will actually work (need to just try)

• whether it's a concern that it's basically 6 projects sharing the stars/downloads stats (while still having separate entries) and thus maybe getting an unfairly high rank
  (we'll definitely need to explore some options for this if we ever want to pull this off with the plugins of mkdocs-material that can instead be added as independent plugins)

[oprypin]

pub-obsidian -> this is kind of problematic, since it's should be in 3 different categories:

I don't think we'll allow a plugin to be in several categories, need to pick one

[mkusz]

• whether it's a concern that it's basically 6 projects sharing the stars/downloads stats (while still having separate entries) and thus maybe getting an unfairly high rank
  (we'll definitely need to explore some options for this if we ever want to pull this off with the plugins of mkdocs-material that can instead be added as independent plugins)

Kinda agree on the unfair advantage with the stars, but there is also an another side of the coin: so far all the plugins that are separate entities, don't have to think if they are compatible with any other plugins. This leads to dependency hell and maintenance problems where one project have to wait for potential issue fixes in another project/repo. Having all the plugins inside one project, we can try to minimalize those problems (the same approach can be seen in Material for MkDocs). Also, if we think about the unfair advantage this way, Material for MkDocs has the same problem since this is a theme with a few plugins (search and debug for sure).

I don't think we'll allow a plugin to be in several categories, need to pick one

Probably the other solution would be to create some sort of the meta (or plugin collections) category for projects like this.

I don't fully agree with looking at this as a 6 separate project. It's rather 1 project with 6 plugins that can be used in separation but works best in cooperation (just like Material for MkDocs that is a theme with search plugin that expands theme). Project has some goals and mkdocs-publisher has a goal of delivering plugins that can cooperate (and are cross tested) for easier page/documentation creation (by using Obsidian as an IDE/text editor) and publication.

PS. If you can see this as a single project, there is no problem with unfair adventages ;)

[oprypin]

We mostly agree, but just this part stood out as strange to me:

If you can see this as a single project, there is no problem with unfair adventages ;)

If it's a single project then it can go to a single category 🤔

[mkusz]

If it's a single project then it can go to a single category 🤔

Yes, but which one is the correct one? I don't feel like blogging is the correct one. The only one that now seems the best fit (kinda) is other. Also, because of that, I proposed at the end, a new category (but it also feels strange, to create a new one just for this project and also don't think that there will be any other person crazy enough to do something similar in the near future).

[squidfunk]

Material for MkDocs has the same problem since this is a theme with a few plugins (search and debug for sure).

Material for MkDocs is not a plain theme anymore. It started as a theme 7.5 years ago, but has become a documentation framework built on top of MkDocs. We plan to add the 12 plugins that we bundle which also work with other themes (in fact, all of them) into the separate categories of the catalog as well.

[mkusz]

@squidfunk are you going to separate them into multiple repositories? Asking because (according to above discussion) it's not possible (or at least not desirable) at the current stage of this catalog.

[squidfunk]

I talked to the maintainers of this repository and was told that it is perfectly possible to list the project multiple times in separate categories, only listing the respective plugin that applies to the given category.

[mkusz]

I talked to the maintainers of this repository and was told that it is perfectly possible to list the project multiple times in separate categories, only listing the respective plugin that applies to the given category.

I will follow that path and make a PR with a plugin categorization described in one of my previous comments.

[pawamoy]

If we go this route, I think we should remove the total stars count from the first paragraph in the README, unless best-of-generator deduplicates the info correctly of course. If it doesn't, the 12 Material for MkDocs plugins will blow up the count to something like 200k+ stars instead of 30k 😅

Feel free to send PRs anyway and we'll see how it behaves.

[squidfunk]

Couldn't you deduplicate total stars by grouping by github_id and/or pypi_id?

catalog/projects.yaml

Lines 64 to 70 in b88c86a

	- name: Material for MkDocs
	mkdocs_theme: material
	mkdocs_plugin: [material/search, material/social, material/tags]
	github_id: squidfunk/mkdocs-material
	pypi_id: mkdocs-material
	labels: [theme, plugin]
	category: theming

[pawamoy]

Everything is possible, it's just that we didn't write the code of the best-of-generator, so this would take time.

[mkusz]

There is some code in best-of-list related to stars counting, but didn't have time to follow how it works.

[pawamoy]

Thanks. That lead me to read the docs again, and there are interesting bits:

Property	Description
group	If True, the project will be used as top project for grouping a set of related projects. group_id also needs to be set to the shared group ID.
group_id	Group ID that can be used to group this project to other projects. For every group, there needs to be one project with group set to True.

I don't think groups would apply to Material for MkDocs or Publisher for MkDocs though, since their different plugins are in different categories.

Also it's possible to override the star count of a project with star_count. Maybe setting it to 0 in some places would fix the total star count issue.

[mkusz]

Seems like star_count could be a great workaround. The only thing here would be not to set it manually, but it should be combined with github_id/pypi_id that are supported by this catalog script.

[pawamoy]

The script you mention only checks the correctness/completeness of the metadata, it does not generate the contents of the README. Any custom logic for the Markdown contents generation must happen in the best-of-generator code

EDIT: Maybe you meant to add a warning for duplicated projects that do not set star_count: 0? That's a good idea.

[pawamoy]

By the way, I forgot that @oprypin had forked the generator here: https://github.com/mkdocs/best-of-generator. So at least reviews, merges and releases can happen fast.

[mkusz]

Maybe you meant to add a warning for duplicated projects that do not set star_count: 0? That's a good idea.

Didn't think about this, but it could be sufficient (+ add as a check before merge).

The only thing that I wonder is how it will affect project position inside a category?

[oprypin]

By the way, in the meantime, this plugin hasn't been added to the catalog in any capacity, so we're missing it

[pawamoy]

Thanks for the reminder. I'll unblock some time tomorrow to try working on this.

[mkusz]

I will add my plugins today in the evening. Sorry for the delay.

[pawamoy]

I'm running some experiments. Here's some feedback:

• each plugin must have a different name (otherwise the generator complains)
• each plugin can have a different description
• total stars will not be de-duplicated
• it shoudn't be fair to set stars to 0 for a specific plugins to avoid stars duplication
• it shouldn't be fair either to use the true star count, compared to other projects in the same category
• instead I'd recommend supporting starts_count: /plugins to divide by the number of plugins, and and a note in the output when this was applied¹
• however in the case of Material for MkDocs and maybe others, I suspect we will want to keep listing it in Theming, with all plugins, and the true stars count
• therefore I believe we either get rid of the total stars count, or update the generator to discard stars count when the /plugins or /extensions syntax was used

Footnotes

1. Example: Project has 20K stars, 6 plugins, Project/A should have 20K/6 = 3.3K, Project/B+C should have 20K/6*2 = 6.6K, etc. ↩

[pawamoy]

Groups won't work for this use-case: the main project declared with the group name and group id has a single category, and all projects declaring only the group id will be put under this main group, in the same, single category:

[pawamoy]

Also, other metrics are taken into account when calculating the score: forks count, contributors count, etc., which should all be divided the same way as stars... Might be cumbersome to implement but I'll try. Let me know what you think, there might be better ways I didn't think of. Maybe we could simply get rid of the whole score calculation and simply order projects alphabetically.

[mkusz]

The only reason for trying to have many stars/contributors/etc. is for the project to jump out of the category of project with less than 20 (this is the limit if I remember correctly) stars because when your project/plugin has less than that, it lands in this folded category and because of this it's less visible and harder to find.
I think that when we remove foldable part of the list, everything else doesn't matter. For me the only metric to use/not use a given plugin is if the project is actively maintained. Starts is eventually a metrics that shows if people likes and maybe use given plugin.

[pawamoy]

So your recommendation would be to leave the ranking as it is (without dividing stars and whatever), and just get rid of the "collapsed" feature (never hiding any project)? I'd be fine with that 🙂

[mkusz]

Yes.
Now, when we have collapsed part of the list, as a user I cannot even search for given plugin when it's inside. The only reason for plugin to be inside a collapsed part would be that project is no longer compatible with current MkDocs core and/or not maintained. This should be only for preserving knowledge about it because maybe someone will take the job or create a fork or grab some idea for his own work. But I'm not fully convinced if we should put any project into a collapsed list at all.

[pawamoy]

I removed the total star count, as well as display restrictions (min description, min stars, min rank, homepage), and fixed metadata of a few projects. README updated, ready to accept PRs splitting monorepos across several categories 🙂

[pawamoy]

Hmm there are still 3 occurrences of hidden projects, I'll investigate.
UPDATE: fixed, needed allowed_licenses: ["all"]

[mkusz]

I will work on adding my project to the catalog, so we can see how it goes. Will mention this task in PR message so we will have some trace for future projects.