Skip to content

Latest commit

 

History

History
362 lines (226 loc) · 11.1 KB

README.md

File metadata and controls

362 lines (226 loc) · 11.1 KB
Raw
:octocat: An open-source, on-premise, self-hosted alternative to Cypress dashboard 🌲

Table of contents

Features

  • run cypress tests in parallel without any limitation
  • upload failure screenshots to S3 bucket
  • browse test results, failures and screenshots
  • run in light mode w/o persistency or with MongoDB storage attached
  • on-premise self-hosted cypress dashboard by design - use your own infrastructure, own your data

Setup

  1. Point Cypress to your service - set https://sorry-cypress-demo-director.herokuapp.com/ as api_url of <cypress-root>/packages/server/config/app.yml
  2. Run multiple instances of cypress run --parallel --record --key xxx --ci-build-id <buildId>
  3. See the the tests running in parallel 🚀

Running Cypress test in parallel demo

Demo & Example

Visit https://sorry-cypress-demo.herokuapp.com/ and see the alpha version of the web dashboard in action.

This demo is a free heroku instance, it takes a minute to spin it up when you first navigate.

You can reconfigure Cypress to use api_url: "https://sorry-cypress-demo-director.herokuapp.com/", run your tests and see the results appear in the dashboard.

Also consider the example with detailed example of parallelization.

The results of tests from the example app: Web dashboard prototype

On-premise installation instructions

Docker images

Each package has a Dockerfile - use it to build your own images.

Pre-built Docker images are available at https://hub.docker.com/u/agoldis.

Docker image tag corresponds to the git tag, while latest points to the master git branch.

Heroku

Click the button below to deploy the basic, in-memory configuration of director to Heroku:

Deploy

If you need help deploying statefull version of the services, please file an issue!

More to come...

Documentation

Reconfiguring Cypress

Find cypress installation path

$ DEBUG=cypress:* cypress version

...
# here it is
cypress:cli Reading binary package.json from: /Users/agoldis/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/package.json +0ms
...

In my case it is: /Users/agoldis/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/

Change the default dashboard URL

$ cat /Users/agoldis/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/config/app.yml

...
# Replace this with a URL of the alternative dashboard
production:
  # api_url: "https://api.cypress.io/"
  api_url: "http://localhost:1234/"
...

Project structure

The repository consists of 3 packages - you can deploy them on your own infrastructure:

director service

The director service is responsible for:

  • paralellization and coordination of test runs
  • saving tests results
  • saving failed tests screenshots

When you launch Cypress on a CI environment with multiple machines, each machine first contacts the dashboard to get the next test to run.

The dashboard coordinates the requests from differents machines and assigns tests to each.

That is what director service does 👆

Starting the service

cd packages/director

npm install
npm run build
npm run start

# ...

Initializing "in-memory" execution driver...
Initializing "dummy" screenshots driver...
Listening on 1234...

By default, the service will start on port 1234 with in-memory execution driver and dummy snapshots driver.

That is what running on https://sorry-cypress.herokuapp.com - it is a stateless execution, that just parallelizes tests, but does not persist test results and does not uploads screenshots of failed tests.

Configuration

The service uses dotenv package - to change the default configuration, create .env file in service's root:

$ pwd
/Users/agoldis/sorry-cypress/packages/director

$ cat .env

PORT=1234

# DASHBOARD_URL is what Cypress client shows as a "Run URL"
DASHBOARD_URL="https://sorry-cypress.herokuapp.com"

# Read more about execution drivers below
EXECUTION_DRIVER="../execution/in-memory"

# Read more about screenshot drivers below
SCREENSHOTS_DRIVER="../screenshots/dummy.driver"

Drivers

The director uses "drivers" that define different aspects of its functionality.

Execution driver

...is what drives the execution flow.

There're 2 "execution drivers" implemented:

Stateless

Keeps the state of runs in-memory. That means that restarting the service wipes everything.

That's the simplest and most naive implementation.

If you just want to run the tests in parallel and not worry about storing test results.

MongoDB persisted

The state - test runs and results - are persisted in MongoDB, thus, can be queried and displayed in a dashboard.

To enable this driver, set the configuration of .env file:

EXECUTION_DRIVER="../execution/mongo/driver"
MONGODB_URI="monodgb://your-DB-URI"
MONGODB_DATABASE="your-DB-name"

With MongoDB driver you can use the other services - api and dashboard to see the results of your runs.

Snapshots driver

...is what allows you to save the snapshots of failed tests.

It provides the client (Cypress runner) a URL for uploading the screenshots.

Dummy

Is the default driver and it does nothing - snapshots won't be saved.

You can set it explicity in .env:

SCREENSHOTS_DRIVER="../screenshots/dummy.driver"
S3 Driver

The driver generates upload URLs for S3 bucket. To enable it, set in .env:

SCREENSHOTS_DRIVER="../screenshots/s3.driver"
S3_BUCKET="your_bucket_name"

Please make sure that AWS credentials with proper access to invoke s3.getSignedUrl are available in the environment.

api service

...is a simple GraphQL service, that allows to query the data persisted by MongoDB.

Create a .env file and define MongoDB connection details:

MONGODB_URI='mongodb://mongo:27017'
MONGODB_DATABASE='sorry-cypress'

dashboard service

...is a web dashboard implemented in ReactJS. It is in alpha stage and still very naive - you can explore test details, failures and see screenshots.

In production mode you will need to provide environment variable GRAPHQL_SCHEMA_URL - graphql client will use the URL to download the schema.

E.g. in .env file:

GRAPHQL_SCHEMA_URL=https://sorry-cypress-demo-api.herokuapp.com

You can explore currently available features at https://sorry-cypress-demo.herokuapp.com/.

Development

The project uses yarn workspaces, bootstrap everything by running yarn in the root directory.

Run each package in development mode: yarn dev.

It is recommended to use docker-compose to run the backend services (director and api) and to run the dashboard on host machine.

Using docker-compose for backend services

The project uses docker-compose to conviniently run backend services in dockerized containers.

Run docker-compose build from the project's root directory Run docker-compose up to start the services.

The latter command will create 3 services:

  • MongoDB instance on port 27017
  • director service on port 1234
  • api service on 4000

You can change the configuration using the environment variables defined in docker-compose.yml file.

Behind the scenes

  1. Each machine sends the same initial request with:
  • specs lists
  • machine hardware details
  • git commit details
  • --ci-build-id and other CLI parameters

  1. The director creates or fetches an existing run, based on the parameters and responds with a randomly generated machineId and the allocated runId

  2. Each cypress client requests a next available task for the runId which was returned previously

  3. The service looks at the list of specs and returns next available test

Original Cypress dashboard implements different "smart" strategies for picking the next test

  1. When there're no more available tests for a run, the service sends an "empty" response - client reports that it is finished

The official guide on Cypress parallelization.

FAQ

Why?

I was upset because:

  • dashboard crashes and blocks my tests
  • parallelization stops working after paid plan has reached its limit

Is it legal?

Yes, Cypress is an open source software.

Is it production-ready?

The director service - yes. I have been using https://sorry-cypress.herokuapp.com/ to run 500+ parallelized tests, each with ±90 spec files and 200+ tests.

The other services are still very naive.

What Cypress clients does it support?

Tested with Cypress 3.4.1 and 3.2.0

License

MIT