Build: Fix flakey E2E tests

[shilman]

Describe the bug

As part of #25850 I had to disable a few E2E tests:

• https://github.com/storybookjs/storybook/blob/next/code/e2e-tests/addon-docs.spec.ts#L38
• https://github.com/storybookjs/storybook/blob/next/code/e2e-tests/tags.spec.ts#L7

Before I disabled them I verified by hand that the features are working and it is a race condition in the tests and not a problem with Storybook. We should fix the tests before RC.

To Reproduce

Only one of the race conditions reproduces locally for me in Webkit. The rest can be reproduced by re-enabling the tests and observing CI.

System

No response

Additional context

No response

[kasperpeulen]

Update 20 februari:
I re-enabled them, as we thought they only where flaky in safari, and that we disabled safari all together in playwright.
But we actually still test safari in playwright, so that didn't fix it. 😅

So we should either:

• debug what is going on in safari
• or do disable safari in playwright

[vanessayuenn]

Let's label these with @flaky as discussed at retro last week, for now. And I will leave this open to investigate these flaky tests as a future Empathy task.

[JReinhold]

Some findings on the tags-test:

1. This is not the test being flaky, it's the implementation that is actually bugged in Safari. Building the Storybook and opening it Safari will occasionally show dev-only and docs-only stories in the sidebar that should have been filtered out.
2. The original hypothesis that this was caused by a race condition with setting window.TAGS_OPTIONS is false. Logging that variable just before the it's read shows that it's there even when bug appears.
3. Our best guess right now is that the setFilter function is either called too late or too early in a race with whatever uses the filter. This still doesn't explain why this is only a problem in Safari.

[yannbf]

More findings on tags-test:

1. It's definitely a bug and not a flake.
2. This happens to any project, regardless of sandbox or anything. As long as it's in Safari.
3. Seems like the implementation for the Store component has some inconsistencies in its setState method, where it should be async and it actually isn't (internally).
4. There is a timing issue when the index is fetched and then the internal store state is set, where sometimes the state does contain the filters, sometimes it won't

I have a branch with logs that help understand what's going on: https://github.com/storybookjs/storybook/compare/yann/investigate-safari-flakiness?expand=1

If you run a sandbox on that branch and take a screenshot of the logs, then refresh until the bug happens, and compare, you will see the timing issue.

I couldn't find a solution. Seems like it's a quite delicate thing and potentially there might be other errors related to the store state sync situation.