Spec for Channel events to trigger test run

[ghengeveld]

This document outlines the specifications for triggering test runs using the Storybook internal API. The test runs can be triggered for individual stories, story files, or groups of stories. Additionally, it covers the cancellation of test runs and watch mode functionality.

Scenarios

1. Trigger all tests in the test module
2. Trigger a specific test for a component/story/group in the sidebar

Run tests

Request Payloads

Running Tests

To run tests for a single story, a story file or a group of story the following event is emitted:

emit(`TESTING_MODULE_RUN_REQUEST`, {
  providerId: 'Addon_Test',
  payload: [
    {
      stories: [
        {
          id: 'button--primary',
          name: 'Primary',
        },
        {
          id: 'button--secondary',
          name: 'Secondary',
        },
      ],
      importPath: 'path/to/button.stories.ts',
      componentPath: 'path/to/button.ts',
    },
    {
      stories: [
        {
          id: 'header--login',
          name: 'LoggedIn',
        },
        {
          id: 'header--logout',
          name: 'LoggedOut',
        },
      ],
      importPath: 'path/to/header.stories.ts',
      componentPath: 'path/to/header.ts',
    },
  ],
} as TestingModuleRunStoryGroupRequest);

Running All Tests Without Filters

To run all tests without any filters, emit the following event:

emit(`TESTING_MODULE_RUN_ALL_REQUEST`, {
  providerId: 'Addon_Test'
} as TestingModuleRunAllRequest);

Response Payloads

Handling Test Run Responses

To handle responses for test runs, listen to the following events. The payload is a batch response containing information about the progress of the test execution.

channel.on(`TESTING_MODULE_RUN_PROGRESS_RESPONSE`, (result: TestingModuleRunProgressResponsePayload) => {});

Cancellation

Triggering a Cancellation Request

To trigger a cancellation request for each registered test provider, emit the following event:

emit(`TESTING_MODULE_CANCEL_TEST_RUN_REQUEST`, {
  providerId: 'Addon_Test',
} as TestingModuleCancelTestRunRequest);

Handling Cancellation Responses

To handle responses for cancellation requests, listen to the following event:

channel.on(
  `TESTING_MODULE_CANCEL_TEST_RUN_RESPONSE`,
  ({ status, message }: TestingModuleCancelTestRunResponse) => {}
);

Watch Mode

Triggering Watch Mode

The watch mode is currently only triggered globally. Clicking on watch mode should not trigger test runs. To trigger watch mode, emit the following event:

emit('TESTING_MODULE_WATCH_MODE_REQUEST', {
  providerId: "Addon_Test"
} as TestingModuleWatchModeRequest);

Types

Provider

type ProviderId = string;

TestingModuleRunRequest

type TestingModuleRunStoryGroupRequest = {
  providerId: ProviderId;
  payload: {
    stories: {
      id: string;
      name: string;
    }[];
    importPath: string;
    componentPath: string;
  }[];
};

TestingModuleRunAllRequest

type TestingModuleRunAllRequest = {
  providerId: ProviderId;
};

TestingModuleRunProgressResponsePayload

type TestingModuleRunProgressResponsePayload = {
  providerId: ProviderId;
  payload: {
    numTotalTests: number;
    numPassedTests: number;
    numFailedTests: number;
    numPendingTests: number;
    progress: number;
    startTime: number;
    success: boolean;
    testResults: TestingModuleRunResult[];
  };
  status: 'success' | 'pending'
} | {
  providerId: ProviderId;
  error: {
    name: string;
    message: string;
    stack?: string;
  }
  status: 'failed';
}

TestingModuleRunResult

type TestingModuleRunResult = {
  results: TestingModuleRunAssertionResult[];
  startTime: number;
  endTime: number;
  status: 'passed' | 'failed';
  message?: string;
};

TestingModuleRunAssertionResult

type TestingModuleRunAssertionResult = {
  status: 'passed' | 'pending';
  duration: number;
  storyId: string;
} | {
  status: 'failed';
  duration: number;
  failureMessages: string[];
  storyId: string;
};

Status

type Status = 'success' | 'failed' | 'pending';

TestingModuleCancelTestRunRequest

type TestingModuleCancelTestRunRequest = {
  providerId: ProviderId;
};

TestingModuleCancelTestRunResponse

type TestingModuleCancelTestRunResponse = {
  status: 'success';
} | {
  status: 'failed';
  message: string;
}

TestingModuleWatchModeRequest

type TestingModuleWatchModeRequest = {
  providerId: ProviderId;
};