diff --git a/docs/src/ci-intro-js.md b/docs/src/ci-intro-js.md new file mode 100644 index 0000000000000..9958177b3630b --- /dev/null +++ b/docs/src/ci-intro-js.md @@ -0,0 +1,115 @@ +--- +id: ci-intro +title: "CI Github Actions" +--- + +When installing Playwright you are given the option to add a [GitHub Actions](https://docs.github.com/en/actions). This creates a `playwright.yml` file inside a `.github/workflows` folder containing everything you need so that your tests run on each push and pull request into the main/master branch. + +**What you will learn:** + +- [How to use GitHub Actions to run your tests](#github-actions) +- [How to create a repo and push to GitHub](#create-a-repo-and-push-to-github) +- [How to open the workflows](#opening-the-workflows) +- [How to view the test logs](#viewing-test-logs) +- [How to download the report from GitHub](#downloading-the-playwright-report) +- [How to view the report](#viewing-the-playwright-report) +- [How to view the trace](#viewing-the-trace) + +## GitHub Actions + +Tests will run on push or pull request on branches main/master. The [workflow](https://docs.github.com/en/actions/using-workflows/about-workflows) will install all dependencies, install Playwright and then run the tests. It will also create the HTML report. + +```yaml +name: Playwright Tests +on: + push: + branches: [main, master] + pull_request: + branches: [main, master] +jobs: + test: + timeout-minutes: 60 + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - uses: actions/setup-node@v2 + with: + node-version: "14.x" + - name: Install dependencies + run: npm ci + - name: Install Playwright Browsers + run: npx playwright install --with-deps + - name: Run Playwright tests + run: npx playwright test + - uses: actions/upload-artifact@v2 + if: always() + with: + name: playwright-report + path: playwright-report/ + retention-days: 30 +``` + +### Create a Repo and Push to GitHub + +[Create a repo on GitHub](https://docs.github.com/en/get-started/quickstart/create-a-repo) and create a new repository or push an existing repository. Follow the instructions on GitHub and don't forget to [initialize a git repository](https://github.com/git-guides/git-init) using the `git init` command so you can [add](https://github.com/git-guides/git-add), [commit](https://github.com/git-guides/git-commit) and [push](https://github.com/git-guides/git-push) your code. + +Create a Repo and Push to GitHub + +### Opening the Workflows + +Click on the **Actions** tab to see the workflows. Here you will see if your tests have passed or failed. + +Opening the Workflows + +On Pull Requests you can also click on the **Details** link in the [PR status check](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks). + +pr status checked + +### Viewing Test Logs + +Clicking on the workflow run will show you the all the actions that GitHub performed and clicking on **Run Playwright tests** will show the error messages, what was expected and what was received as well as the call log. + +Viewing Test Logs + + + +## HTML Report + +The HTML Report shows you a full report of your tests. You can filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. + +### Downloading the HTML Report + +In the Artifacts section click on the **playwright-report** to download your report in the format of a zip file. + +Downloading the HTML Report + +### Viewing the HTML Report + +Locally opening the report will not work as expected as you need a web server in order for everything to work correctly. First, extract the zip, preferably in a folder that already has Playwright installed. Using the command line change into the directory where the report is and use `npx playwright show-report` followed by the name of the extracted folder. This will serve up the report and enable you to view it in your browser. + + +```bash +npx playwright show-report name-of-my-extracted-playwright-report +``` + +Viewing the HTML Report + +To learn more about reports check out our detailed guide on [HTML Reporter](/test-reporters.md#html-reporter) + +### Viewing the Trace + +Once you have served the report using `npx playwright show-report`, click on the trace icon next to the test's file name as seen in the image above. You can then view the trace of your tests and inspect each action to try to find out why the tests are failing. + + +Viewing the Trace + +To learn more about traces check out our detailed guide on [Trace Viewer](/trace-viewer.md). + +To learn more about running tests on CI check out our detailed guide on [Continuous Integration](/ci.md). + + +## What's Next + +- [Learn how to use Web First Assertions](/test-assertions.md) +- [Learn how to use Selectors](/selectors.md) +- [Learn how to use Locators](/locators.md) diff --git a/docs/src/codegen.md b/docs/src/codegen.md index 2cb9a4ac3cb1d..d3d372f33b368 100644 --- a/docs/src/codegen.md +++ b/docs/src/codegen.md @@ -288,4 +288,4 @@ await page.PauseAsync(); ## What's Next -- [See a trace of your tests](./trace-viewer.md) +- [See a trace of your tests](./trace-viewer-intro.md) diff --git a/docs/src/getting-started-vscode-js.md b/docs/src/getting-started-vscode-js.md index 3a5adf57192bf..b33fb4158f311 100644 --- a/docs/src/getting-started-vscode-js.md +++ b/docs/src/getting-started-vscode-js.md @@ -93,4 +93,4 @@ As you interact with the page Codegen will generate the test for you in the newl - [Write tests using web first assertions, page fixtures and locators](./writing-tests.md) - [See test reports](./running-tests.md#test-reports) -- [See a trace of your tests](./trace-viewer.md) +- [See a trace of your tests](./trace-viewer-intro.md) diff --git a/docs/src/html-reporter-js.md b/docs/src/html-reporter-js.md deleted file mode 100644 index 5ec3e54f884f9..0000000000000 --- a/docs/src/html-reporter-js.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -id: html-reporter -title: "HTML Reporter" ---- - -The HTML Reporter shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed. - -```bash -npx playwright show-report -``` - -image - -You can click on each test to get more info on the test and explore the errors section which will show you what errors there are and on what line as well as suggestions on how to fix the errors. Each step of the test can be expanded and looked into in detail. - -image \ No newline at end of file diff --git a/docs/src/intro-csharp.md b/docs/src/intro-csharp.md index 0295fa98b8101..bc0258cfe40d6 100644 --- a/docs/src/intro-csharp.md +++ b/docs/src/intro-csharp.md @@ -194,5 +194,5 @@ See our doc on [Test Runners](./test-runners.md) to learn more about running tes - [Run single tests, multiple tests, headed mode](./running-tests.md) - [Learn more about the NUnit and MSTest base classes](./test-runners.md) - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) +- [See a trace of your tests](./trace-viewer-intro.md) - [Using Playwright as library](./library.md) diff --git a/docs/src/intro-js.md b/docs/src/intro-js.md index 8c5554ddbb44b..bf6bb1422d378 100644 --- a/docs/src/intro-js.md +++ b/docs/src/intro-js.md @@ -82,7 +82,7 @@ See our doc on [Running Tests](./running-tests.md) to learn more about running t ## HTML Test Reports -Once your test has finished running a [HTML Reporter](./html-reporter.md) will have been created which shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. You can click on each test and explore the test's errors as well as each step of the test. By default, the HTML report is opened automatically if some of the tests failed. +Once your test has finished running a [HTML Reporter](./test-reporters.md#html-reporter) will have been created which shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. You can click on each test and explore the test's errors as well as each step of the test. By default, the HTML report is opened automatically if some of the tests failed. ```bash npx playwright show-report @@ -96,4 +96,4 @@ npx playwright show-report - [Write tests using web first assertions, page fixtures and locators](./writing-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md) - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) \ No newline at end of file +- [See a trace of your tests](./trace-viewer-intro.md) \ No newline at end of file diff --git a/docs/src/intro-python.md b/docs/src/intro-python.md index 0aac99652d22b..dbf3c81df8253 100644 --- a/docs/src/intro-python.md +++ b/docs/src/intro-python.md @@ -64,4 +64,4 @@ See our doc on [Running Tests](./running-tests.md) to learn more about running t - [Write tests using web first assertions, page fixtures and locators](./writing-tests.md) - [Run single tests, multiple tests, headed mode](./running-tests.md) - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) +- [See a trace of your tests](./trace-viewer-intro.md) diff --git a/docs/src/running-tests-csharp.md b/docs/src/running-tests-csharp.md index 7b7cd500d9c82..a76ece265e9e3 100644 --- a/docs/src/running-tests-csharp.md +++ b/docs/src/running-tests-csharp.md @@ -93,4 +93,4 @@ Check out our [debugging guide](./debug.md) to learn more about the [Playwright ## What's Next - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) +- [See a trace of your tests](./trace-viewer-intro.md) diff --git a/docs/src/running-tests-js.md b/docs/src/running-tests-js.md index 532293a06a931..739bffee717a8 100644 --- a/docs/src/running-tests-js.md +++ b/docs/src/running-tests-js.md @@ -90,7 +90,7 @@ Check out our [debugging guide](./debug.md) to learn more about the [Playwright ## Test Reports -The [HTML Reporter](./html-reporter.md) shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed. +The [HTML Reporter](././test-reporters.md#html-reporter) shows you a full report of your tests allowing you to filter the report by browsers, passed tests, failed tests, skipped tests and flaky tests. By default, the HTML report is opened automatically if some of the tests failed. ```bash npx playwright show-report @@ -105,4 +105,4 @@ You can click on each test and explore the tests errors as well as each step of ## What's Next - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) +- [See a trace of your tests](./trace-viewer-intro.md) diff --git a/docs/src/running-tests-python.md b/docs/src/running-tests-python.md index d53be9a927d52..e250baa94f724 100644 --- a/docs/src/running-tests-python.md +++ b/docs/src/running-tests-python.md @@ -83,4 +83,4 @@ Check out our [debugging guide](./debug.md) to learn more about the [Playwright ## What's Next - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) \ No newline at end of file +- [See a trace of your tests](./trace-viewer-intro.md) \ No newline at end of file diff --git a/docs/src/trace-viewer-intro-csharp-java-python.md b/docs/src/trace-viewer-intro-csharp-java-python.md new file mode 100644 index 0000000000000..409a565083450 --- /dev/null +++ b/docs/src/trace-viewer-intro-csharp-java-python.md @@ -0,0 +1,120 @@ +--- +id: trace-viewer-intro +title: "Trace Viewer" +--- + +Playwright Trace Viewer is a GUI tool that lets you explore recorded Playwright traces of your tests meaning you can go back and forward though each action of your test and visually see what was happening during each action. + +**You will learn** + +- How to record a trace +- How to open the HTML report +- How to open the trace viewer + +## Recording a trace + +Traces can be recorded using the [`property: BrowserContext.tracing`] API as follows: + +```java +Browser browser = browserType.launch(); +BrowserContext context = browser.newContext(); + +// Start tracing before creating / navigating a page. +context.tracing().start(new Tracing.StartOptions() + .setScreenshots(true) + .setSnapshots(true) + .setSources(true)); + +Page page = context.newPage(); +page.navigate("https://playwright.dev"); + +// Stop tracing and export it into a zip archive. +context.tracing().stop(new Tracing.StopOptions() + .setPath(Paths.get("trace.zip"))); +``` + +```python async +browser = await chromium.launch() +context = await browser.new_context() + +# Start tracing before creating / navigating a page. +await context.tracing.start(screenshots=True, snapshots=True, sources=True) + +await page.goto("https://playwright.dev") + +# Stop tracing and export it into a zip archive. +await context.tracing.stop(path = "trace.zip") +``` + +```python sync +browser = chromium.launch() +context = browser.new_context() + +# Start tracing before creating / navigating a page. +context.tracing.start(screenshots=True, snapshots=True, sources=True) + +page.goto("https://playwright.dev") + +# Stop tracing and export it into a zip archive. +context.tracing.stop(path = "trace.zip") +``` + +```csharp +await using var browser = playwright.Chromium.LaunchAsync(); +await using var context = await browser.NewContextAsync(); + +// Start tracing before creating / navigating a page. +await context.Tracing.StartAsync(new() +{ + Screenshots = true, + Snapshots = true, + Sources = true +}); + +var page = context.NewPageAsync(); +await page.GotoAsync("https://playwright.dev"); + +// Stop tracing and export it into a zip archive. +await context.Tracing.StopAsync(new() +{ + Path = "trace.zip" +}); +``` + +This will record the trace and place it into the file named `trace.zip`. + + +## Opening the trace + +You can open the saved trace using Playwright CLI or in your browser on [`trace.playwright.dev`](https://trace.playwright.dev). + +```bash js +npx playwright show-trace trace.zip +``` + +```bash java +mvn exec:java -e -Dexec.mainClass=com.microsoft.playwright.CLI -Dexec.args="show-trace trace.zip" +``` + +```bash python +playwright show-trace trace.zip +``` + +```bash csharp +pwsh bin\Debug\netX\playwright.ps1 show-trace trace.zip +``` + + +## Viewing the trace + +View traces of your test by clicking through each action or hovering using the timeline and see the state of the page before and after the action. Inspect the log, source and network during each step of the test. The trace viewer creates a DOM snapshot so you can fully interact with it, open devtools etc. + + + +image + + + +To learn more check out our detailed guide on [Trace Viewer](/trace-viewer.md). + + diff --git a/docs/src/trace-viewer-intro-js.md b/docs/src/trace-viewer-intro-js.md new file mode 100644 index 0000000000000..8f04e7082aa8c --- /dev/null +++ b/docs/src/trace-viewer-intro-js.md @@ -0,0 +1,89 @@ +--- +id: trace-viewer-intro +title: "Trace Viewer" +--- + +Playwright Trace Viewer is a GUI tool that lets you explore recorded Playwright traces of your tests meaning you can go back and forward though each action of your test and visually see what was happening during each action. + +**You will learn** + +- [How to record a trace](/trace-viewer-intro.md#recording-a-trace) +- [How to open the HTML report](/trace-viewer-intro.md#opening-the-html-report) +- [How to open and view the trace](/trace-viewer-intro.md#viewing-the-trace) + + +## Recording a Trace + +By default the [playwright.config](/test-configuration.md#record-test-trace) file will contain the configuration needed to create a `trace.zip` file for each test. Traces are setup to run `on-first-retry` meaning they will be run on the first retry of a failed test. Also `retires` are set to 2 when running on CI and 0 locally. This means the traces will be recorded on the first retry of a failed test but not on the first run and not on the second retry. + +```js tab=js-js +// @ts-check + +/** @type {import('@playwright/test').PlaywrightTestConfig} */ +const config = { + retries: process.env.CI ? 2 : 0, // set to 2 when running on CI + ... + use: { + trace: 'on-first-retry', // record traces on first retry of each test + }, +}; + +module.exports = config; +``` + +```js tab=js-ts +import type { PlaywrightTestConfig } from '@playwright/test'; +const config: PlaywrightTestConfig = { + retries: process.env.CI ? 2 : 0, // set to 2 when running on CI + ... + use: { + trace: 'on-first-retry', // record traces on first retry of each test + }, +}; +export default config; +``` + +To learn more about available options to record a trace check out our detailed guide on [Trace Viewer](/trace-viewer.md). + +Traces are normally run in a Continuous Integration(CI) environment as locally you can use [debugging](/debug.md) methods to debug tests. However should you want to run traces locally you can force tracing to be on with `--trace on`. + +```bash +npx playwright test --trace on +``` + +:::note +The `trace-on` flag was introduced in Playwright v1.25. Check your `package.json` to sure you have at least this version of Playwright installed. +::: + +## Opening the HTML Report + +If you have a failed test then tests will run a total of 3 times. On the first retry the trace will be recorded. After the second retry the tests will stop running and a HTML report is available to view. + +```bash +npx playwright show-report +``` + +In the HTML report click on the trace icon to directly open the trace file. + +image + +You can also click on the test file and then click the 'Retry #1' tab which will show you a traces section in your html report. Here you can open the trace by clicking on the screenshot. + +image + + +To learn more about reporters check out our detailed guide on reporters including the [HTML Reporter](/test-reporters.md#html-reporter). + +## Viewing the Trace + +View traces of your test by clicking through each action or hovering using the timeline and see the state of the page before and after the action. Inspect the log, source and network during each step of the test. The trace viewer creates a DOM snapshot so you can fully interact with it, open devtools etc. + + + +image + + + +To learn more about traces check out our detailed guide on [Trace Viewer](/trace-viewer.md). + + diff --git a/docs/src/trace-viewer.md b/docs/src/trace-viewer.md index c6ff5d36c496b..0f3bc3f270f5f 100644 --- a/docs/src/trace-viewer.md +++ b/docs/src/trace-viewer.md @@ -52,13 +52,14 @@ await page.goto('https://playwright.dev'); await context.tracing.stop({ path: 'trace.zip' }); ``` -You can also use `trace: 'retain-on-failure'` if you do not enable retries but still want traces for failed tests. - Available options to record a trace: +- `'on-first-retry'` - Record a trace only when retrying a test for the first time. - `'off'` - Do not record a trace. -- `'on'` - Record a trace for each test. +- `'on'` - Record a trace for each test. (not recommended as it's performance heavy) - `'retain-on-failure'` - Record a trace for each test, but remove it from successful test runs. -- `'on-first-retry'` - Record a trace only when retrying a test for the first time. + + +You can also use `trace: 'retain-on-failure'` if you do not enable retries but still want traces for failed tests. If you are not using Playwright Test, use the [`property: BrowserContext.tracing`] API instead. diff --git a/docs/src/writing-tests-csharp.md b/docs/src/writing-tests-csharp.md index 4d147d3e11307..b3e3aed8c2175 100644 --- a/docs/src/writing-tests-csharp.md +++ b/docs/src/writing-tests-csharp.md @@ -237,4 +237,4 @@ public class UnitTest1 : PageTest - [Run single tests, multiple tests, headed mode](./running-tests.md) - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) \ No newline at end of file +- [See a trace of your tests](./trace-viewer-intro.md) \ No newline at end of file diff --git a/docs/src/writing-tests-js.md b/docs/src/writing-tests-js.md index 1046b7e2ad77e..6ed96a4667685 100644 --- a/docs/src/writing-tests-js.md +++ b/docs/src/writing-tests-js.md @@ -146,4 +146,4 @@ test.describe("navigation", () => { - [Run single tests, multiple tests, headed mode](./running-tests.md) - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) \ No newline at end of file +- [See a trace of your tests](./trace-viewer-intro.md) \ No newline at end of file diff --git a/docs/src/writing-tests-python.md b/docs/src/writing-tests-python.md index 6e1ab0904f91b..1fc0deb1375fc 100644 --- a/docs/src/writing-tests-python.md +++ b/docs/src/writing-tests-python.md @@ -104,4 +104,4 @@ def test_main_navigation(page: Page): - [Run single tests, multiple tests, headed mode](./running-tests.md) - [Generate tests with Codegen](./codegen.md) -- [See a trace of your tests](./trace-viewer.md) \ No newline at end of file +- [See a trace of your tests](./trace-viewer-intro.md) \ No newline at end of file