Create sitemap / RSS

[Fil]

Like vitepress, build an items array from the list of pages, with a transformItems hook that allows the user to modify the array before passing it to https://www.npmjs.com/package/sitemap (with options).
Ref. https://vitepress.dev/guide/sitemap-generation

It seems sensible to adopt the same approach here.

To build a sitemap, we need to know the full URL (including domain name and subdirectory, ref. #42).

[mbostock]

This is mainly for public projects so moving to future (but it would be nice to have for GA if possible).

[Fil]

I've added RSS to the issue; we could make two issues when the time comes. RSS won't be useful for many projects, but some which regularly add pages will want it and it feels it's table stakes.

[Fil]

#843 removed client-side access to the search index.

It was the closest thing we had to a sitemap.xml/rss list of contents, but now only search.js knows where this file can be found—which is for the better, since it's not supposed to be a public API. But it means a renewed urgency to have a public list of the advertised pages. (By advertised I mean listed in pages or indexed.) As an example use case, I had made a little game on the home page of my pangea website which was using the minisearch.json dataset to display the count of pages and a shuffled random selection of 5 pages to explore.

For the game in question I can replace this with an ad hoc data loader, but it feels like it belongs to this more generic question.

[mbostock]

I think we should transpile import.meta.resolve(…) (in addition to import(…)).

Then you could say fetch(import.meta.resolve("observablehq:minisearch.json")) to load the search index.

Also this would allow you to register additional files to download.

Edit: This is now done in #843.

[Fil]

I wrote a postbuild script to add a simple sitemap to pangea, with the proper headers that reference it.

https://github.com/Fil/pangea/blob/e179717808c66526d5e17a933245f7fe6b91851b/package.json#L7-L8

https://github.com/Fil/pangea/blob/e179717808c66526d5e17a933245f7fe6b91851b/observablehq.config.ts#L101

It would still be nice to have an easier method to push static/public files, but I think we don't have to have native support for RSS or sitemaps (a helper would be nice, but it can all be done in userland); this issue can now be folded under #169.

[Fil]

For more discussion, see #1199 (reply in thread)