Option to decorrelate slugs from filesystem

[y-nk]

This proposal is already pretty well documented (as long as a proposal solution) at withastro/astro#6307

Basically, i do think it's a nice feature we had, and generally to have: to be able to format the slugs before serving/building, otherwise said: have a space to be able to map urls to file system programmatically.

I'm trying to write a collection of small snippets of code, i've probably around 200 already. I'd like to keep lean urls such as "example.com/this-cool-snippet" but as the collection grows, the file system organisation becomes quite messy messy. so i thought to add a "001-" prefix to them; but then the url contains that prefix, which i dont want.

One could argue "you could write the slug directly in the frontmatter data of each file", but it doesn't scale well. having a programmatic entry point is, i think, high value.

I think it's a pretty valid usecass to wish to You rearrange filesystem structure to have a 000- prefix, while wishing not to have this spilling in urls (for many reasons, from cosmetic reasons to seo). We could also benefit from this if one wanted to have a collection organized as "yyyy-mm-dd-slug" files while served as a totally different url (with slashes, like wordpress, without having to create a commiting/difficult-to-maintain nested file structure)

Of course this feature would remain optional and non-breaking and the default would still be a 1-1 mapping, it wouldn't affect users not willing to use it.

[bholmesdev]

Thanks for starting a discussion @y-nk! Happy to give a little background on why this feature was cut from our initial 2.0 release.

For those unaware, we proposed a collection-level slug option to programmatically generate slugs based on the entry ID, parsed data, and default slug (code snippet here). There were a few problems with this approach:

First, allowing access to parsed data can be problematic for the dev server and prod builds. Today, all slugs are computed on dev server startup for type inference and performant getEntryBySlug() lookups. By allowing access to data, we would need to parse every entry with your Zod schema before the server starts. This means any schema type errors would halt your server before you've opened your browser, a jarring experienced compared to today's (where schema errors are thrown once you visit a page). This also means getEntryBySlug() cannot be run until all entries are preprocessed. Not impossible, though more complex to maintain. A better solution for programmatic slugs may be just allowing access to the id property. This would address the use case you present here.

The second issue runs deeper: We're unsure if slug is the best abstraction for identifying entries as content collections grow. You may have noticed we have 2 competing identifiers: id and slug. id is based on the file name and cannot be overridden, while slug can be user-specified with the slug frontmatter property (similar to a permalink). This makes sense for the formats content collections supports today, where we assume most entries will be used as pages. But what if we support other data storage formats like JSON and TOML? What does slug mean in this context? Our goal as API designers is to offer the right amount of features to achieve your goals, and leave the rest unexplored when there is uncertainty. We've quickly found that content collections can be used for far more than generating pages, so a better answer for this use case may not be slug at all!

All of this said, I think there is a middle ground to support the use case proposed here. This isn't the first time we've seen the need for programmatic slug generation!

[y-nk]

Thank you for providing this feedback. You took the time to explain it all and I really appreciate it.

I understand the problem with the dev server, I was also thinking this would happen when i tried to patch it back together.

As you proposed, I thought that using the id only was a better compromise ; after all, the 2 competing inputs are slug and id. or, as i saw it: slug is the data to be used, and id is its default fallback value (so they're not really competing in some ways).

It will cut some usecases like using frontmatter data for the slug, but i think it's ok. the usecase i was presenting was not about generating slugs from complex datasets but rather a way to derivate a slug from the file name in order to keep collections tidy in the fs as they grow.

If you want to keep things fast while enabling frontmatter data to be part of this, i would suggest running a separate algorithm for this purpose, one which would only run graymatter on collections without parsing the markdown/mdx content of the file ; it'd still be slower than no parsing at all, that's for sure, but i guess that would remain fairly slow even for a large collection of documents.

happy to help further if necessary

[bholmesdev]

Agreed, I think restricting to a permalink based on the ID makes the most sense. We've discussed making the concept of a slug opt-in as well depending on your use case. Say, adding a type field to your collection to flag pages vs. non-pages:

// just data, will not be used as pages
const authors = defineCollection({
  schema: z.object({...}),
});

// blog collection, will be used as pages
const blog = defineCollection({
  type: 'page',  // slug feature unlocked
  computeSlug: ({ id }) => customSlugify(id),
  schema: z.object({ ... }),
});

Also want to address this:

If you want to keep things fast while enabling frontmatter data to be part of this, i would suggest running a separate algorithm for this purpose

...this is how content collections work today! Data is restricted to a frontmatter call, just running your Zod schema without parsing the Markdown / MDX. The slug frontmatter property is a "reserved" property that skips the Zod schema parsing as well.

[bholmesdev]

Cross-posting issue from @narduin here: withastro/astro#7133 . Agreed that permalinks-as-identifiers is problematic. Creating a new "permalink" concept separate from IDs, specifically for pages, is one option.