Hugo taxonomy generator with archetypes and project scaffolding

[rdwatters]

#3772 from @onedrawingperday gave me an idea of how to possibly make Hugo's taxonomy management easier.

Background

• Taxonomies are an incredibly powerful feature of Hugo. According to forum feedback, users are unclear on how to leverage taxonomies for more complex content models. Instead, they think of "taxonomies" as convenient methods for Wordpress-like rendered category/tag pages rather than a powerful feature for managing content metadata; i.e., the un-rendered, one-to-many, one-to-one, and many-to-many relationships among instances of content types (i.e., typically .md files in sections).
• Now that users have the ability to define taxonomies but not render them to their respective taxonomy respective templates via disableKinds, the feature is actually more powerful because it allows arbitrary insertion of related content according to user-defined metrics on multi-content-type rendered pages. (**Note: I'm unsure whether disableKinds means "disable taxonomy page rendering" or "disable taxonomy page rendering and don't afford the taxonomy related methods in other templates as well").
• In order to follow our current approach templating/function, individual terms within a taxonomy need an _index.md for front matter and content and take advantage of powerful methods like .GetPage in a matter consistent with how said methods are used with section/content files.
• Archetypes, especially with their recent upgrades in Hugo functions, make content generation much more powerful/faster/easier than with any other SSG, at least to my knowledge.

Shortcomings

• Creating both new directories and _index.md per taxonomy terms is inconvenient, especially in more common use cases where users use Hugo taxonomies as folksonomies; NB, a "tag," by definition, is a folksonomy.
• Manual generation of the aforementioned directories and _index.md files keep users from leveraging taxonomies the same way they do other facets of Hugo that come with built-in generators (e.g., themes, content files).

Proposed Feature

1. Add archetype support for taxonomies combined with...
2. Added taxonomy directory scaffolding

What this might look like

Assume a user has the following taxonomy defined in config.toml:

[taxonomies]
    member = "members"

.
└── archetypes
    ├── default.md
    ├── events.md
    ├── members.md
    └── posts.md

Inside of members.md we have typical front matter (this is all contrived, but users are going to have all kinds of crazy content, so bear with me):

---
title: "{{ replace .TranslationBaseName "-" " " | title }}"
date: {{ .Date }}
description: ""
reportingunit:  
---

A potential config option. I'm not attached to this exact directive:

generateTaxonomyDirs: true

...with the default set to false so that we don't break every Hugo project in history.

A user could then create their first content/posts/my-first-post.md with the following front matter:

---
title: "My First Post"
date: 2017-08-05T14:09:18-05:00
draft: false
members: [John]
---

Which also adds the corresponding members/john/_index.md file with:

title: "John"
date: 2017-08-05T14:09:18-05:00
description: 

Note that the taxonomy archetype would follow the same pattern of falling back to default.md in this case in the event that no members.md archetype file is provided.

This makes it that much easier to arbitrarily add taxonomy values--it's still up to content authors and content managers to treat Hugo's definition of "taxonomies" in the true sense of controlled vocabularies or folksonomies and add FM/content as needed--while reinforcing the same approach we have for list vs single-content pages.

This is also a more flexible answer, IMNSHO, to the never-ending #3088. Hint: we don't have to think of "authors" as configuration, since it's really content in the first place.

@budparr, I know you share my preference for creating efficient/sane content relationships/models, and you have much more client-facing experience with SSGs than I do. I'd love your thoughts.

cc/ @digitalcraftsman

Note: Apologies for not labeling this as needsfeedback or proposal or whatever, @bep, but I don't have said permissions.

[guayom]

I really like the idea of having archetype support for taxonomies. That would be a very helpful feature.

Now, I've been playing more with taxonomies and I figured that having Hugo create taxonomy terms, and taxonomy pages for you, is really presents great advantages, for instance:

• It helps keeping your /content directory cleaner.
• No problems with Ugly/Pretty urls settings

So, what if the scaffolding generates the files in the data folder instead of the content folder?

The data folder it's also to hold content, just as a page. But the difference is that the metadata for a taxonomy doesn't really belong to a page. In my opinion should be easily retrievable from anywhere.

So... This is how I think that taxonomies could work:

Let's say you want to add metadata to the 'members' taxonomy

1. You create an archetype for taxonomies/members.
2. You create default values with the generator: hugo new taxonomies/members/default.yml it generates the file data/members/default.yml
3. You create a new member: hugo new taxonomies/members/tom.yml it generates the file data/members/tom.yml
4. You access it via .Site.Taxonomies.members.data, just like you would with .Site.Taxonomies.members.Pages

The key part here would be creating the methods or functions that would allow you to get the data through .Site.Taxonomies.members.data

Another nice to have feature, would be this:
If you don't create a data/members/default.yml, throw a warning: 'You don't have default data for your members taxonomy. Please create it with hugo new taxonomies/members/default.yml'

What do you think about that? @rdwatters @onedrawingperday ?

[rdwatters]

No problems with Ugly/Pretty urls settings

Just so I'm clear with the proposal (which is really more a request of the devs to build on an existing feature, so it might all be moot), what I'm talking about here is content management. Ugly/pretty URL settings and permalinks are matters of configuration. I think what you're thinking of is more about how taxonomies render (maybe something similar to #3579 ?).

So, what if the scaffolding generates the files in the data folder instead of the content folder?

I would strongly avoid this approach for multiple reasons. Taxonomies are attributes of content; that's what makes embedded metadata (e.g., yaml/toml) so powerful in that content is entirely composed in one place (e.g., usually a markdown file).

A content file can default to creating a rendered web page (e.g. mysite.com/posts/my-post/), and a taxonomy can default to creating a default web page (e.g., mysite.com/categories/), but really content/*.md is an instance of a content type (in Hugo, an archetype).

What my hope with the above is to help facilitate typical taxonomy usage from

A

key=>values

to

B (Which we already have, but it's not as convenient as it could be)

key=>
    value=>
         value metadata=> 
             value content 
             etc 

Note that B doesn't subvert the documented/existing approach to adding front matter and content (i.e., "content" really means "body copy" in most cases) to taxonomies and taxonomy terms and doesn't create breaking changes.

The data folder, IMNSHO, should be limited to actual serialized data, especially in cases where data is being pulled from outside the project. It could also be used for multiple configuration files (see #3090), but I don't have much of an opinion on that right now, and it's outside the scope of this conversation.

When you access it via .Site.Taxonomies.brands you get the WeightedPages as usual, plus all the data contained in the data file.

If I understand you correctly, you're talking about adding functionality that already exists with .GetPage and use of an _index.md, no?

[guayom]

I would strongly avoid this approach for multiple reasons. Taxonomies are attributes of content; that's what makes embedded metadata (e.g., yaml/toml) so powerful in that content is entirely composed in one place (e.g., usually a markdown file).

Ok, well said. I agree with you.

If I understand you correctly, you're talking about adding functionality that already exists with .GetPage and use of an _index.md, no?

Yes, but that would have been needed in case of putting the metadata in the /data folder

[budparr]

I like both ideas, taxonomy archetype and the scaffolding. Scaffolding would be great for new users in general and would help people utilize more complex front matter types.

[guayom]

@rdwatters at what point do you think that the taxonomy files generator should be triggered?

I think that you could use a terminal command like

hugo generateTaxonomyFiles

Or something like it.

That can search for all your taxonomies, terms and generate all files if they don't exist.

It could have a flag like -rewrite to rewrite any existing files.

[rdwatters]

@guayom you have many good ideas, but keep in mind that I'm the only non-dev on this team, so I'm just looking for feedback on whether this type of feature is worth it. Then I have to make a polite and respectful appeal to the rest of the team if users think it's worth it. And then it's up to the devs as to opening a discussion about how we implement 😄

[guayom]

Sure, sure. I was just wondering what your idea was about triggering, because I don't know how you imagine it. I'm just trying to discuss more on how this could be a useful feature. The problem now is that even when you can scaffold your taxonomy term pages, you have to do it one by one.

If you can generate all files in batch, with one command, that's powerful!

[bep]

I have a few concerns about this approach:

1. It assumes there are a large number of taxonomy terms, which I think is an anti pattern.
2. and assuming you have a large number of terms, a generator should support a rather complex life cycle to be really useful (new, changed, deleted terms), which is hard to get right

I have always thought that we, somehow, would get content in Hugo that would not be in the form of single markdown files, i.e. a product dump from a database etc. Taxonomy content would fit nicely into that thought, the idea being that the content files we have today would be merged with content generated on-the-fly from a set of "page listing definitions".

[guayom]

That would be ideal. It sounds like how middleman creates dynamic pages from yml databases. That is very useful and flexible. If that is on the roadmap... or at least, in the conversation for long term plans, it's great news. And I agree, that would require a completely different approach.

[bep]

If that is on the roadmap...

We have at least one other issue discussing this. So, it may not be on the "road map" (which is outdated and should probably be removed), but it is definitively on my mental road map.

[rdwatters]

It assumes there are a large number of taxonomy terms, which I think is an anti pattern

Depends on your semantics of taxonomy. Even with a small number of terms, blog-centric content models limit taxonomies to just key-value pairs, and "tags" are a folksonomy by definition. Hugo will never be able to control for poor site architecture from users. What I'm suggesting is easier management for a large number of taxonomies, not (only) terms, since the taxonomies demonstrate the relationships and the terms are only instances thereof.

The default false config was a spitball idea to make sure such scaffolding, etc, doesn't break current sites. So maybe this feature is in the same vein as GitInfo? "This works and adds convenience but may slow things down depending on your use case."

Sometimes half the attributes/fields/FM (including taxonomies [controlled vocabularies]) I create in a traditional CMS is based on business needs or general management. Depending on the CMS, I could use these for editorial workflows or just content-model-only attributes for better long-term COPE.

and assuming you have a large number of terms, a generator should support a rather complex life cycle to be really useful (new, changed, deleted terms), which is hard to get right

Agreed, and I don't know how you do that with files rather than a database. I feel like this is a a C request without the RUD. Maybe this is just the first step--or expansion of--@budparr's #3612 As in, "render these two sections and just one of three taxonomies."

I have always thought that we, somehow, would get content in Hugo that would not be in the form of single markdown files

Agreed. Too many files is impractical, and markdown is a syntax and not a strategy. I'm not sure how a project like Hugo can control for this type of user behavior though, either.

I appreciate your time 🙇 @bep

[stale]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[Brahmasmi]

I understand that this proposal is more broader than #3088, because it rightly considers authors as just another content instead of configuration. This may also be more relevant considering that in multilingual setups, there will be one author but with different display names, based on the locale for which the pages are being rendered. As such, I would personally prefer to keep this issue as compared to the #3088.

Again, this is based on my limited understanding, and could be wrong.

Thanks.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.