fix: add noindex to server island headers

[sinskiy]

Changes

Sets X-Robots-Tag: noindex to Server Island headers

Fixes #12806

Testing

Added tests that check X-Robots-Tag: noindex of Server Island in both production and development environments

Docs

No changes needed? This just fixes a bug

[changeset-bot]

🦋 Changeset detected

Latest commit: 7667cc2

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[codspeed-hq]

CodSpeed Performance Report

Merging #12827 will not alter performance

_{Comparing sinskiy:main (7667cc2) with main (7fb2184)}

Summary

✅ 4 untouched benchmarks

[matthewp]

Can you try setting it inside of this function instead:

astro/packages/astro/src/core/server-islands/endpoint.ts

Line 95 in ebe2aa9

const page: AstroComponentFactory = async (result) => {

This is the handler for server islands, would be cleaner not to put SI specific code in render-context if we can avoid it.

[sinskiy]

Accidentally updated branch, didn't know what this button does

[sinskiy]

I think we don't need this PR. see review conversation

[ematipico]

@sinskiy I updated the PR by fixing the tests (you didn't update them). Now it passes

[sinskiy]

@ematipico Thanks! But, as I said, x-robots-tag was set to noindex even if I deleted my changes. That's why I didn't push updated tests. I'm confused :/

[matthewp]

@sinskiy I wonder if you were seeing that because your tests were cached or something, making it seem like the header was already there. You could try submitting a PR that removes your change and see if it now fails. If it doesn't that would mean the header is already there.