Release 0.4.0

README.md

@@ -11,7 +11,7 @@ Read the announcement: [Interaction Testing with Storybook](https://storybook.js
 - [CLI Options](#cli-options)
 - [Configuration](#configuration)
 - [Running against a deployed Storybook](#running-against-a-deployed-storybook)
-  - [Stories.json mode](#storiesjson-mode)
+  - [Index.json mode](#indexjson-mode)
 - [Running in CI](#running-in-ci)
   - [1. Running against deployed Storybooks on Github Actions deployment](#1-running-against-deployed-storybooks-on-github-actions-deployment)
   - [2. Running against locally built Storybooks in CI](#2-running-against-locally-built-storybooks-in-ci)
@@ -116,10 +116,11 @@ Usage: test-storybook [options]
 | Options                         | Description                                                                                                                      |
 | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
 | `--help`                        | Output usage information <br/>`test-storybook --help`                                                                            |
-| `-s`, `--stories-json`          | Run in stories json mode. Automatically detected (requires a compatible Storybook) <br/>`test-storybook --stories-json`          |
-| `--no-stories-json`             | Disables stories json mode <br/>`test-storybook --no-stories-json`                                                               |
+| `-i`, `--index-json`            | Run in index json mode. Automatically detected (requires a compatible Storybook) <br/>`test-storybook --index-json`              |
+| `--no-index-json`               | Disables index json mode <br/>`test-storybook --no-index-json`                                                                   |
 | `-c`, `--config-dir [dir-name]` | Directory where to load Storybook configurations from <br/>`test-storybook -c .storybook`                                        |
-| `--watch`                       | Run in watch mode <br/>`test-storybook --watch`                                                                                  |
+| `--watch`                       | Watch files for changes and rerun tests related to changed files.<br/>`test-storybook --watch`                                   |
+| `--watchAll`                    | Watch files for changes and rerun all tests when something changes.<br/>`test-storybook --watchAll`                              |
 | `--coverage`                    | Indicates that test coverage information should be collected and reported in the output <br/>`test-storybook --coverage`         |
 | `--url`                         | Define the URL to run tests in. Useful for custom Storybook URLs <br/>`test-storybook --url http://the-storybook-url-here.com`   |
 | `--browsers`                    | Define browsers to run tests in. One or multiple of: chromium, firefox, webkit <br/>`test-storybook --browsers firefox chromium` |
@@ -132,7 +133,7 @@ Usage: test-storybook [options]
 
 ## Configuration
 
-The test runner is based on [Jest](https://jestjs.io/) and will accept the [CLI options](https://jestjs.io/docs/cli) that Jest does, like `--watch`, `--maxWorkers`, etc.
+The test runner is based on [Jest](https://jestjs.io/) and will accept the [CLI options](https://jestjs.io/docs/cli) that Jest does, like `--watch`, `--watchAll`, `--maxWorkers`, etc.
 
 The test runner works out of the box, but if you want better control over its configuration, you can run `test-storybook --eject` to create a local `test-runner-jest.config.js` file in the root folder of your project, which will be used by the test runner.
 
@@ -153,23 +154,38 @@ Or by using the `--url` flag:
 yarn test-storybook --url https://the-storybook-url-here.com
 ```
 
-### Stories.json mode
+### Index.json mode
 
-By default, the test runner transforms your story files into tests. It also supports a secondary "stories.json mode" which runs directly against your Storybook's `stories.json`, a static index of all the stories.
+By default, the test runner transforms your story files into tests. It also supports a secondary "index.json mode" which runs directly against your Storybook's index data, which dependending on your Storybook version is located in a `stories.json` or `index.json`, a static index of all the stories.
 
-This is particularly useful for running against a deployed storybook because `stories.json` is guaranteed to be in sync with the Storybook you are testing. In the default, story file-based mode, your local story files may be out of sync--or you might not even have access to the source code. Furthermore, it is not possible to run the test-runner directly against `.mdx` stories, and stories.json mode must be used.
+This is particularly useful for running against a deployed storybook because `index.json` is guaranteed to be in sync with the Storybook you are testing. In the default, story file-based mode, your local story files may be out of sync – or you might not even have access to the source code. Furthermore, it is not possible to run the test-runner directly against `.mdx` stories, and `index.json` mode must be used.
 
-To run in stories.json mode, first make sure your Storybook has a v3 `stories.json` file. You can navigate to:
+<!-- TODO: switch details to 6.4 once Storybook 7.0 becomes default -->
+
+<details>
+<summary>If you're using Storybook 7.0</summary>
+
+To run in `index.json` mode, first make sure your Storybook has a v4 `index.json` file. You can find it when navigating to:
 
 ```
-https://your-storybook-url-here.com/stories.json
+https://your-storybook-url-here.com/index.json
 ```
 
-It should be a JSON file and the first key should be `"v": 3` followed by a key called `"stories"` containing a map of story IDs to JSON objects.
+It should be a JSON file and the first key should be `"v": 4` followed by a key called `"entries"` containing a map of story IDs to JSON objects.
+
+In Storybok 7.0, `index.json` is enabled by default, unless you are using the `storiesOf()` syntax, in which case it is not supported.
 
-If your Storybook does not have a `stories.json` file, you can generate one provided:
+</details>
+
+On Storybook 6.4 and 6.5, to run in `index.json` mode, first make sure your Storybook has a file called `stories.json` that has `"v": 3`, available at:
 
-- You are running SB6.4 or above
+```
+https://your-storybook-url-here.com/stories.json
+```
+
+If your Storybook does not have a `stories.json` file, you can generate one, provided:
+
+- You are running Storybook 6.4 or above
 - You are not using `storiesOf` stories
 
 To enable `stories.json` in your Storybook, set the `buildStoriesJson` feature flag in `.storybook/main.js`:
@@ -180,21 +196,21 @@ module.exports = {
 };
 ```
 
-Once you have a valid `stories.json` file, your Storybook will be compatible with the "stories.json mode".
+Once you have a valid `stories.json` file, your Storybook will be compatible with the "index.json mode".
 
-By default, the test runner will detect whether your Storybook URL is local or remote, and if it is remote, it will run in "stories.json mode" automatically. To disable it, you can pass the `--no-stories-json` flag:
+By default, the test runner will detect whether your Storybook URL is local or remote, and if it is remote, it will run in "index.json mode" automatically. To disable it, you can pass the `--no-index-json` flag:
 
 ```bash
-yarn test-storybook --no-stories-json
+yarn test-storybook --no-index-json
 ```
 
-If you are running tests against a local Storybook but for some reason want to run in "stories.json mode", you can pass the `--stories-json` flag:
+If you are running tests against a local Storybook but for some reason want to run in "index.json mode", you can pass the `--index-json` flag:
 
 ```bash
-yarn test-storybook --stories-json
+yarn test-storybook --index-json
 ```
 
-> **NOTE:** stories.json mode is not compatible with watch mode.
+> **NOTE:** index.json mode is not compatible with watch mode.
 
 ## Running in CI
 
@@ -378,6 +394,20 @@ module.exports = {
 };
 ```
 
+When running with `--stories-json`, tests get generated in a temporary folder and snapshots get stored alongside. You will need to `--eject` and configure a custom [`snapshotResolver`](https://jestjs.io/docs/configuration#snapshotresolver-string) to store them elsewhere, e.g. in your working directory:
+
+```js
+const path = require('path');
+
+module.exports = {
+  resolveSnapshotPath: (testPath, snapshotExtension) =>
+    path.join(process.cwd(), '__snapshots__', path.basename(testPath) + snapshotExtension),
+  resolveTestPath: (snapshotFilePath, snapshotExtension) =>
+    path.join(process.env.TEST_ROOT, path.basename(snapshotFilePath, snapshotExtension)),
+  testPathForConsistencyCheck: path.join(process.env.TEST_ROOT, 'example.test.js'),
+};
+```
+
 ### Image snapshot recipe
 
 Here's a slightly different recipe for image snapshot testing:

bin/test-storybook.js

@@ -29,11 +29,11 @@ const log = (message) => console.log(`[test-storybook] ${message}`);
 const error = (message) => console.error(`[test-storybook] ${message}`);
 
 // Clean up tmp files globally in case of control-c
-let storiesJsonTmpDir;
+let indexTmpDir;
 const cleanup = () => {
-  if (storiesJsonTmpDir) {
-    log(`Cleaning up ${storiesJsonTmpDir}`);
-    fs.rmSync(storiesJsonTmpDir, { recursive: true, force: true });
+  if (indexTmpDir) {
+    log(`Cleaning up ${indexTmpDir}`);
+    fs.rmSync(indexTmpDir, { recursive: true, force: true });
   }
 };
 
@@ -67,7 +67,7 @@ async function reportCoverage() {
   // --check-coverage if we want to break if coverage reaches certain threshold
   // .nycrc will be respected for thresholds etc. https://www.npmjs.com/package/nyc#coverage-thresholds
   execSync(`npx nyc report --reporter=text -t ${coverageFolder} --report-dir ${coverageFolder}`, { stdio: 'inherit' })
-} 
+}
 
 const onProcessEnd = () => {
   cleanup();
@@ -87,6 +87,9 @@ function sanitizeURL(url) {
   // remove iframe.html if present
   finalURL = finalURL.replace(/iframe.html\s*$/, '');
 
+  // remove index.html if present
+  finalURL = finalURL.replace(/index.html\s*$/, '');
+
   // add forward slash at the end if not there
   if (finalURL.slice(-1) !== '/') {
     finalURL = finalURL + '/';
@@ -97,19 +100,21 @@ function sanitizeURL(url) {
 
 const checkForIncompatibilities = () => {
   try {
-    const jestVersion = require('jest/package.json').version
+    const jestVersion = require('jest/package.json').version;
     if (semver.gte(jestVersion, '28.0.0')) {
       error(dedent`We detected that your project is using Jest 28.0.0 or higher, which is currently incompatible with the test runner.
 
       You can find more info at: https://github.com/storybookjs/test-runner#errors-with-jest-28
-      `)
-      process.exit(1)
+      `);
+      process.exit(1);
     }
   } catch (err) {
-    error('We detected that Jest is not installed in your project. Please install it and run test-storybook again.')
-    process.exit(1)
+    error(
+      'We detected that Jest is not installed in your project. Please install Jest@27 and run test-storybook again.'
+    );
+    process.exit(1);
   }
-}
+};
 
 async function executeJestPlaywright(args) {
   const jest = require('jest');
@@ -142,25 +147,56 @@ async function checkStorybook(url) {
   }
 }
 
-async function fetchStoriesJson(url) {
+async function getIndexJson(url) {
+  const indexJsonUrl = new URL('index.json', url).toString();
   const storiesJsonUrl = new URL('stories.json', url).toString();
+
+  const [indexRes, storiesRes] = await Promise.all([
+    fetch(indexJsonUrl),
+    fetch(storiesJsonUrl)
+  ]);
+
+  if (indexRes.ok) {
+    try {
+      const json = await indexRes.text();
+      return JSON.parse(json);
+    } catch (err) { }
+  }
+
+  if(storiesRes.ok) {
+    try {
+      const json = await storiesRes.text();
+      return JSON.parse(json);
+    } catch (err) { }
+  }
+
+  throw new Error(dedent`
+    Failed to fetch index data from the project.
+
+    Make sure that either of these URLs are available with valid data in your Storybook:
+    ${
+      // TODO: switch order once index.json becomes more common than stories.json
+      storiesJsonUrl
+    }
+    ${indexJsonUrl}
+
+    More info: https://github.com/storybookjs/test-runner/blob/main/README.md#indexjson-mode
+  `);
+}
+
+async function getIndexTempDir(url) {
   let tmpDir;
   try {
-    const res = await fetch(storiesJsonUrl);
-    const json = await res.text();
-    const titleIdToTest = transformPlaywrightJson(json);
+    const indexJson = await getIndexJson(url)
+    const titleIdToTest = transformPlaywrightJson(indexJson);
 
     tmpDir = tempy.directory();
     Object.entries(titleIdToTest).forEach(([titleId, test]) => {
       const tmpFile = path.join(tmpDir, `${titleId}.test.js`);
       fs.writeFileSync(tmpFile, test);
     });
   } catch (err) {
-    console.error(`Failed to fetch stories.json from ${storiesJsonUrl}`);
-    console.error(
-      'More info: https://github.com/storybookjs/test-runner/blob/main/README.md#storiesjson-mode\n'
-    );
-    console.error(err);
+    error(err);
     process.exit(1);
   }
   return tmpDir;
@@ -195,10 +231,12 @@ const main = async () => {
   }
 
   // set this flag to skip reporting coverage in watch mode
-  isWatchMode = jestOptions.watch;
+  isWatchMode = jestOptions.watch || jestOptions.watchAll;
+
+  const rawTargetURL = process.env.TARGET_URL || runnerOptions.url || 'http://localhost:6006';
+  await checkStorybook(rawTargetURL);
 
-  const targetURL = sanitizeURL(process.env.TARGET_URL || runnerOptions.url);
-  await checkStorybook(targetURL);
+  const targetURL = sanitizeURL(rawTargetURL)
 
   process.env.TARGET_URL = targetURL;
 
@@ -217,16 +255,16 @@ const main = async () => {
   const { hostname } = new URL(targetURL);
 
   const isLocalStorybookIp = await isLocalhostIp(hostname, true);
-  const shouldRunStoriesJson = runnerOptions.storiesJson !== false && !isLocalStorybookIp;
-  if (shouldRunStoriesJson) {
+  const shouldRunIndexJson = runnerOptions.indexJson !== false && !isLocalStorybookIp;
+  if (shouldRunIndexJson) {
     log(
-      'Detected a remote Storybook URL, running in stories json mode. To disable this, run the command again with --no-stories-json'
+      'Detected a remote Storybook URL, running in index json mode. To disable this, run the command again with --no-index-json\n'
     );
   }
 
-  if (runnerOptions.storiesJson || shouldRunStoriesJson) {
-    storiesJsonTmpDir = await fetchStoriesJson(targetURL);
-    process.env.TEST_ROOT = storiesJsonTmpDir;
+  if (runnerOptions.indexJson || shouldRunIndexJson) {
+    indexTmpDir = await getIndexTempDir(targetURL);
+    process.env.TEST_ROOT = indexTmpDir;
     process.env.TEST_MATCH = '**/*.test.js';
   }
 

package.json

@@ -53,6 +53,7 @@
     "test-storybook": "./bin/test-storybook.js"
   },
   "devDependencies": {
+    "@auto-it/released": "^10.37.1",
     "@babel/cli": "^7.12.1",
     "@babel/core": "^7.12.3",
     "@babel/preset-env": "^7.12.1",
@@ -118,5 +119,11 @@
     "@storybook/csf-tools": "^6.5.0",
     "@storybook/store": "^6.5.0",
     "jest": "^26.6.3 || ^27.0.0"
+  },
+  "auto": {
+    "plugins": [
+      "npm",
+      "released"
+    ]
   }
 }