[Bug]: Docs stories filtered with '!autodocs' behaves weird in Primary, Stories, Controls, etc. components

[JReinhold]

Describe the bug

In 8.1, we introduced the autodocs tag per story. It introduces a slightly inconsistent behavior:

1. The Stories doc block automatically filters out any stories that don't have 'autodocs' - even if it is in a manually written attached MDX. Is that weird? Somewhat fixed in Docs: Fix MDX Stories block tag-filtering behavior #27144
2. The Primary doc block doesn't take the filter into account at all, meaning that if the primary story has '!autodocs' it will still be selected as the primary story. Should it?
3. 👆 Same for Controls and any other doc block supporting primary stories.
4. useOf and resolveOf doesn't do any automatic filtering, causing the issue with 2 and 3 above. We briefly explored filtering out all '!autodocs' stories internally in DocsContext to solve all problems above, but that wasn't the best idea. It would make it impossible to resolve a story with '!autodocs' in a manual MDX, which seems counter intuitive.

I don't know what the right solution is here, or if there even is any, but we should be aware of this.

Additional context

@shilman proposes that all Storybook-provided doc blocks should automatically filter out '!autodocs' stories - even when used manually in MDX/DocsPage, S a way for users to easily re-create the autodocs experience.
But all user-written doc blocks (those that use useOf() or resolveOf()) should not do automatic filtering. Instead we should provide the tools to apply a filter if needed, either by passing a filter: true boolean somewhere, or make the filter function easily accessible.

[tmeasday]

I would have suggested instead we set a react context when rendering the autodocs container that applies that filter to all doc blocks? Something like:

export const Stories = () => {
  const inAutodocs = useContext(InAutodocsContext);
  const { componentStories, projectAnnotations, getStoryContext } = useContext(DocsContext);

  const stories = componentStories({ inAutodocs });
}

Did y'all consider that option?

[JReinhold]

That's interesting @tmeasday , we didn't consider that. It would mean that a custom docs page with parameters.docs.page would also get the filter, right? Maybe that's also what makes the most sense.

[tmeasday]

@JReinhold well that depends on where we put the InAutodocProvider - we could put it either in the DocsContainer or the default DocsPage.

I don't recall in enough detail how you configure things of the top of my head but I suspect there'd be pros/cons to either one in terms of configurability.

[tmeasday]

Possibly if we put it in the container we could actually alter the context which would would avoid the need to alter the individual blocks?

storybook/code/ui/blocks/src/blocks/DocsContainer.tsx

Line 55 in 17f9b72

<DocsContext.Provider value={context}>

We could set context.inAutodocs = true or similar around this line?

[shilman]

Related fix but a slightly different problem: #27143