On-demand story loading is not supported

[benedfit]

The problem

When on-demand story loading is enabled, as per:

// .storybook/main.js

module.exports = {
  ...,
  features: {
    storyStoreV7: true,
  },
};

percy storybook $URL encounters the following error:

$ percy storybook --verbose $STORYBOOK_PREVIEW_URL
[percy] Percy has started!
[percy:storybook:eval] Requesting Storybook: https://deploy-preview-11520--storybook.netlify.app (0ms)
[percy] Stopping percy...
[percy] Build not created
[percy] Error: Storybook object not found on the window. Open Storybook and check the console for errors.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Environment

• Node version: 14.19.1
• @percy/cli version: 1.1.0
• Version of Percy SDK you’re using: @percy/storybook v4.2.0
• If needed, a build or snapshot ID: N/A as Github workflow fails before build is finalized
• OS version: Ubuntu 20.0 via Github workflow
• Type of shell command-line [interface]: CI via Github workflow

Details

We have been using Percy and Storybook together for some time, and were attempting to make use of the on-demand loading feature added in Storybook v6.4. But when we enabled it, as per the example in the docs, percy storybook is no longer able to detect Storybook at the given URL and our Github-based visual regression workflow fails before generating any snapshots.

As a note, our Storybook deployments are password protected, via simple authentication. This has not caused any issues in the past, but to make sure I have tested with authentication disabled, and it causes no differing effects

Debug logs

See above

Code to reproduce issue

Configure the storyStoreV7 feature as per above, run percy storybook, and note the error show in the debug log above

[Robdel12]

Hey @benedfit! Thanks for the info, can you provide a hosted storybook without auth if possible with this? Or a reproduction? The error that's thrown is we can't find this var on the window: https://github.com/percy/percy-storybook/blob/master/src/eval.js#L62 which is how we collect the list of stories. My hope is that var isn't missing with this setting (it's currently the only API they provide to gather a list of stories).

[benedfit]

I can't link to the Netlify one unfortunately, as we have sensitive content. But here is another Storybook that I personally run; https://chipset.newhighsco.re/, that uses the same settings.

It is worth noting that there is also an additional feature called buildStoriesJson that can be enabled that generates a JSON file of all the available stories (e.g. https://chipset.newhighsco.re/stories.json). Could this be something you also check as a fallback?

[badsgahhl]

Bumb. We're currently testing storyStoreV7 as well. This feature is going to be the default in Storybook 7.

https://storybook.js.org/blog/storybook-on-demand-architecture/

This behaviour will be the default in Storybook 7.0 — it’s available in 6.4 via the storyStoreV7 feature flag (see below); the previous single-bundle behaviour is still enabled by default in 6.4 install

Supporting it as early as possible would be great.

[Robdel12]

At the moment this feature is experimental (from their docs):

This feature is experimental, and it has some limitations on what you can and cannot do in your stories files.

We'll add a feature to the SDK to add support for buildStoriesJson, which looks like it might become the default in v7 to make on-demand work.

[Robdel12]

👋🏼 I'd give v4.3.0 a try to see if it works -- we're hopeful it does since it uses their new methods that they also use to extract the stories: https://github.com/percy/percy-storybook/releases/tag/v4.3.0

[benedfit]

@Robdel12 haven't yet been able to confirm if it works as expected, but v4.3.0 is now causing the following config:

percy: {
    widths: [720, 1120],
    skip: true,
    minHeight: 800,
    additionalSnapshots: [
      {
        suffix: ' (dark)',
        queryParams: { globals: 'theme:dark' },
      },
    ],
  }

No longer pass the correct query string parameters so that we can take dark theme snapshots. Has the API changed for configuring this now?

[benedfit]

And we are also seeing assets, such as lazy loaded images, not loading before the snapshots are taken

[benedfit]

I managed to resolve the config issue by adjusting it to:

percy: {
    ...
    // Needed this to prevent all stories staying in the dark theme
    globals: { theme: 'light' },
    additionalSnapshots: [
      {
        suffix: ' (dark)',
        globals: { theme: 'dark' },
      },
    ],
  }

But still currently unable to tell when all the dynamic elements have loaded. The issue is that we load our SVGs asynchronously, so we are unable to use waitForSelector as there may be n number of SVGs per snapshot, and configuring this by handle seems brittle. Would it be possible to expand the waitFor... functionality to allow a function to be called? We currently use a loading placeholder elements for the SVGs, so if we could check the DOM to make sure that none of those elements exist before taking the screenshot it would resolve the issue e.g. waitFor(() => !document.querySelector([data-cy="svgLoading"]));

[benedfit]

I configured enableJavaScript: true to resolve the issue with the dynamic elements not loading in v4.3.0

[benedfit]

I can confirm that storyStoreV7 works with v4.3.0

[na-ji]

With enableJavaScript, right?