Content layer schema references randomly break

[cabaucom376]

Astro Info

Astro                    v5.0.3
Node                     v23.3.0
System                   macOS (arm64)
Package Manager          npm
Output                   static
Adapter                  none
Integrations             @astrojs/solid-js

If this issue only occurs in one browser, which browser is a problem?

No response

Describe the Bug

When editing content files and changing the names of referenced id's a lot occasionally astro will lock up and not allow the site to be built with an error like this:

[InvalidContentEntryDataError] features → core/shards data does not match collection schema.
category: Reference to categories invalid. Entry core does not exist.

The category core has been verified to exist and is defined properly in a yaml file.

./content/categories.yaml:

core:
  sequence: 0
  title: Core
editor:
  sequence: 1
  title: Editor
qol:
  sequence: 3
  title: Quality of Life
connectivity:
  sequence: 4
  title: Connectivity
extensibility:
  sequence: 5
  title: Extensibility
intelligence:
  sequence: 6
  title: Intelligence
platforms:
  sequence: 7
  title: Platforms

./src/content.config.ts:

import { z, defineCollection, reference } from "astro:content";
import { file, glob } from "astro/loaders";

const categories = defineCollection({
  loader: file("content/categories.yaml"),
  schema: z.object({
    sequence: z.number(),
    title: z.string(),
  }),
});

const features = defineCollection({
  loader: glob({ pattern: "**/*.{md, mdx}", base: "content/features" }),
  schema: z.object({
    title: z.string(),
    description: z.string().optional(),
    sequence: z.number(),
    category: reference("categories"),
    status: z.enum([
      "concept",
      "confirmed",
      "upcoming",
      "development",
      "beta",
      "complete",
    ]),
  }),
});

export const collections = {
  categories: categories,
  features: features,
};

./content/features/core/shards.md:

---
title: "Shards"
description: "..."
sequence: 0
category: core
status: "development"
---

This is really hard to reproduce, but it has happened to me twice now in the last 3 days after my V5 migration, once initially while setting up the glob and file loaders, and another while simply modifying the names of categories. When these instances occur, not even reverting the project back to a time it was functional via git allows for it to build. The only solution I have found after many hours of nothing was this:

1. Comment out the problem schema field, ex: category: reference("categories"),
2. Run astro dev, and it should build fine
3. Quit the run
4. Uncomment out the same field
5. Run astro dev again

I feel this is a bug because nothing is changing at all about the schema, yet it can go from not building at all to building perfectly fine.

What's the expected result?

The website should build and not get locked up randomly on reference data updates.

Link to Minimal Reproducible Example

https://stackblitz.com/edit/github-chwz8xrn?file=src%2Fcontent.config.ts

Participation

• I am willing to submit a pull request for this issue.

[FraserThompson]

I started experiencing this too after upgrading to Astro 5. It feels like some sort of race condition, where one collections entries aren't being processed before the other collection references them or something? If I change the reference to an old one, run astro dev, then kill it and change it back to the new one, it works.

[cestef]

Not quite the same bug, but likely related as it also looks like a race condition somewhere:

13:07:32 [glob-loader] Reloaded data from shamir/index.md
Local icons changed, reloading
Local icons changed, reloading
Local icons changed, reloading
The collection "blog" does not exist or is empty. Ensure a collection directory with this name exists.
13:07:38 [WARN] [router] A `getStaticPaths()` route pattern was matched, but no matching static path was found for requested path `/posts/shamir`.

Possible dynamic routes being matched: src/pages/posts/[...slug].astro.
13:07:38 [404] /posts/shamir 2ms
Local icons changed, reloading
13:07:38 [200] /posts/shamir 307ms

When editing a .md file defined in the blog content layer collection:

// ...

export const BlogSchema = z.object({
	title: z.string(),
	tags: z.array(z.string()),
	image: z.string().optional(),
	description: z.string(),
	date: z.date().transform((val) => new Date(val)),
	growth: GrowthSchema.default("seed"),
	related: z.array(reference("blog")).optional(),
});

const blogCollection = defineCollection({
	loader: glob({ pattern: "**/[^_]*.md", base: "./src/data/blog" }),
	schema: BlogSchema,
});

export const collections = {
	blog: blogCollection,
};

The glob loader effectively detects the change and fires a reload on the browser, but the development server seems not be be ready instantly, and I need to refresh it manually a second time to get the actual content.

The reloaded content also sometimes doesn't match the updated one (the old version is still being considered as "current" in the cache or something like that)

After removing the related: ... reference(...) line from my zod schema, it looks like this isn't happening anymore, even though the re-rendering time feels way longer than on astro v4.

[ascorbic]

Could any of the people reporting this confirm whether this is present in the latest version of Astro? We've made a few changes to prevent content collection race conditions so a lot of these sort of issues are now working.

[vorant94]

@ascorbic i can confirm that if I add new entry into let’s say tag collection and at the same time reference it from another collection entry let’s say post and run build I will get invalid reference error

if I first add new tag, then run build, then reference new tag in post and build again everything works

Astro version is 5.1.3

[aluciu]

hi, i just verified on astro v5.1.5 and i get the same errors:

[InvalidContentEntryDataError] articles → article-name data does not match collection schema.
artists.0: Reference to artists could not be resolved: store not available.
This is an Astro bug, so please file an issue at https://github.com/withastro/astro/issues.

[BuckyBuck135]

2 of my projects are running into the same issue. Any time I update something connected to a reference, it breaks the dev server.

• Reloading the window, rebooting dev server or rebooting computer won't help.
• Removing the reference in the collection schema gets rid of the error (but obviously impedes other features)
• Removing the reference in the collection schema, then reloading the window and restarting server fixes the issue and returns to normal behavior. (I'm trying anything that sticks lol)

Astro v5.1.7

const blogCollection = defineCollection({
	loader: glob({ pattern: "**/[^_]*{md,mdx}", base: "./src/data/blog" }),
	schema: ({ image }) =>
		z.object({
			title: z.string(),
			description: z.string(),
			// commenting out the authors reference for now
			// authors: z.array(reference("authors")),
			heroImage: image(),
		}),
});

// authors
const authorsCollection = defineCollection({
	loader: glob({ pattern: "**/[^_]*{md,mdx}", base: "./src/data/authors" }),
	schema: ({ image }) =>
		z.object({
			name: z.string(),
			avatar: image(),
		}),
});