Skip to content

Commit

Permalink
feat: support test required tags (#14)
Browse files Browse the repository at this point in the history
## Required tags

Sometimes you might want to run a test or a suite of tests _only_ if a specific tag or tags are present. For example, you might have a test that cleans the data. This test is meant to run nightly, not on every test run. Thus you can set a `required` tag:

```js
it('cleans up the data', { requiredTags: '@nightly' }, () => {...})
```

When you run the tests now, this test will be skipped, as if it were `it.skip`. It will only run if you use the tag `@nightly`, for example: `npx cypress run --env grepTags=@nightly`.
  • Loading branch information
bahmutov authored Jan 23, 2023
1 parent 488fcd6 commit 77a2d28
Show file tree
Hide file tree
Showing 13 changed files with 264 additions and 70 deletions.
23 changes: 17 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,18 +54,29 @@ jobs:
--config specPattern=cypress/e2e/effective-tags/*.js \
--expect-exactly expects/effective-tags/spec-a-smoke.json

test-only-on-tags:
test-required-tags:
runs-on: ubuntu-20.04
steps:
- name: Checkout 🛎
uses: actions/checkout@v3

- name: Run single tag 🧪
- name: Install Cypress 🧪
uses: cypress-io/github-action@v5
with:
command: npx cypress-expect run \
--project tests/only-tags \
--expect-exactly ./tests/only-tags/expect.json
runTests: false

- name: running the tests without the required tag
run: |
npx cypress-expect run \
--project tests/required-tags \
--expect-exactly ./tests/required-tags/expect.json
- name: running the tests WITH the required tag
run: |
npx cypress-expect run \
--project tests/required-tags \
--env grepTags=@special \
--expect-exactly ./tests/required-tags/expect-with-required-tag.json
release:
needs:
Expand All @@ -74,7 +85,7 @@ jobs:
test-one-tag,
test-one-spec-filter,
test-effective-tags-AND-filter,
test-only-on-tags,
test-required-tags,
]
runs-on: ubuntu-20.04
if: github.ref == 'refs/heads/main'
Expand Down
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ Table of Contents
- [Omit filtered tests (grepOmitFiltered)](#omit-filtered-tests-grepomitfiltered)
- [Disable grep](#disable-grep)
- [Burn (repeat) tests](#burn-repeat-tests)
- [Required tags](#required-tags)
- [TypeScript support](#typescript-support)
- [General advice](#general-advice)
- [DevTools console](#devtools-console)
Expand Down Expand Up @@ -410,6 +411,16 @@ You can pass the number of times to run the tests via environment name `burn` or

If you do not specify the "grep" or "grep tags" option, the "burn" will repeat _every_ test.

## Required tags

Sometimes you might want to run a test or a suite of tests _only_ if a specific tag or tags are present. For example, you might have a test that cleans the data. This test is meant to run nightly, not on every test run. Thus you can set a `required` tag:

```js
it('cleans up the data', { requiredTags: '@nightly' }, () => {...})
```

When you run the tests now, this test will be skipped, as if it were `it.skip`. It will only run if you use the tag `@nightly`, for example: `npx cypress run --env grepTags=@nightly`.

## TypeScript support

Because the Cypress test config object type definition does not have the `tags` property we are using above, the TypeScript linter will show an error. Just add an ignore comment above the test:
Expand Down
120 changes: 114 additions & 6 deletions cypress/e2e/unit.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/// <reference types="cypress" />
// @ts-check

import {
parseGrep,
Expand All @@ -7,6 +8,7 @@ import {
parseTagsGrep,
shouldTestRun,
shouldTestRunTags,
shouldTestRunRequiredTags,
shouldTestRunTitle,
} from '../../src/utils'

Expand Down Expand Up @@ -174,7 +176,10 @@ describe('utils', () => {
const parsed = parseTagsGrep('--@tag1,--@tag2')

expect(parsed).to.deep.equal([
[{ tag: '@tag1', invert: true }, { tag: '@tag2', invert: true }],
[
{ tag: '@tag1', invert: true },
{ tag: '@tag2', invert: true },
],
])
})
})
Expand Down Expand Up @@ -325,6 +330,109 @@ describe('utils', () => {
})
})

context('shouldTestRunRequiredTags', () => {
const shouldIt = (used, requiredTags, expected) => {
const parsedTags = parseTagsGrep(used)

expect(
shouldTestRunRequiredTags(parsedTags, requiredTags),
`"${used}" against only "${requiredTags}"`,
).to.equal(expected)
}

it('tags is included', () => {
shouldIt('smoke', ['smoke'], true)
shouldIt('nice smoke', ['smoke'], true)
shouldIt('all different tags and smoke', ['smoke'], true)
})

it('two tags are both listed', () => {
shouldIt('two one', ['one', 'two'], true)
})

it('has no only tags', () => {
shouldIt('nice smoke', [], true)
})

it('tag is not listed', () => {
shouldIt('nope', ['smoke'], false)
shouldIt('all different tags', ['smoke'], false)
})

it('one of two needed flags', () => {
// the test needs both "one" and "two" to run
shouldIt('smoke one', ['one', 'two'], false)
shouldIt('smoke two', ['one', 'two'], false)
})
})

context('combination of tags and required tags', () => {
const checkName = (grep, grepTags) => {
const parsed = parseGrep(grep, grepTags)

expect(parsed).to.be.an('object')

return (testName, testTags = [], requiredTags = []) => {
expect(testName, 'test title').to.be.a('string')
expect(testTags, 'test tags').to.be.an('array')

return shouldTestRun(parsed, testName, testTags, false, requiredTags)
}
}

it('simple tags', () => {
// command line grep tags
const t = checkName(null, 'tag1')

// test information (title, tags, requiredTags)
expect(t('my test', ['tag1'])).to.be.true
expect(t('my test', ['tag2'])).to.be.false
})

it('tags plus requiredTags prevent run', () => {
// command line grep tags
const t = checkName(null, 'tag1')

// test information (title, tags, requiredTags)
expect(t('my test', ['tag1'])).to.be.true
// if the test specified "requiredTags" to require only1
expect(t('my test', ['tag1'], ['only1'])).to.be.false
})

it('tags plus requiredTags allow run', () => {
// command line grep tags
const t = checkName(null, 'tag1 only1')

expect(t('my test', ['tag1'])).to.be.true
// the test tagged tag1 and requiring only1 tag
// will run when the user passes "only1"
expect(t('my test', ['tag1'], ['only1'])).to.be.true
})

it('empty tags plus requiredTags allow run', () => {
// command line grep tags
const t = checkName(null, 'only1')

expect(t('my test', ['tag1'])).to.be.false
expect(t('my test', ['tag1'], [])).to.be.false
// the test tagged tag1 and requiring only1 tag
// will run when the user passes "only1"
expect(t('my test', [], ['only1'])).to.be.true
expect(t('my test', ['tag1'], ['only1'])).to.be.true
})

it('several only tags', () => {
// command line grep tags
const t = checkName(null, 'only1 only2')

expect(t('my test', ['tag1'])).to.be.false
expect(t('my test', ['tag1'], [])).to.be.false
expect(t('my test', [], ['only1'])).to.be.true
expect(t('my test', [], ['only2'])).to.be.true
expect(t('my test', ['tag1'], ['only2', 'only1'])).to.be.true
})
})

context('shouldTestRun', () => {
// a little utility function to parse the given grep string
// and apply the first argument in shouldTestRun
Expand Down Expand Up @@ -379,7 +487,7 @@ describe('utils', () => {
expect(t('has only @tag1 in the name', ['@tag1'])).to.be.true
expect(t('has only @tag2 in the name', ['@tag2'])).to.be.true
expect(t('has @tag1 and @tag2 in the name', ['@tag1', '@tag2'])).to.be
.true
.true
})

it('OR with AND option', () => {
Expand All @@ -389,7 +497,7 @@ describe('utils', () => {
expect(t('has only @tag1 in the name', ['@tag1'])).to.be.true
expect(t('has only @tag2 in the name', ['@tag2'])).to.be.false
expect(t('has only @tag2 in the name and also @tag3', ['@tag2', '@tag3']))
.to.be.true
.to.be.true

expect(
t('has @tag1 and @tag2 and @tag3 in the name', [
Expand All @@ -404,7 +512,7 @@ describe('utils', () => {
const t = checkName('-name;-hey;number')

expect(t('number should only be matches without a n-a-m-e')).to.be.true
expect(t('number can\'t be name')).to.be.false
expect(t("number can't be name")).to.be.false
expect(t('The man needs a name')).to.be.false
expect(t('number hey name')).to.be.false
expect(t('numbers hey name')).to.be.false
Expand All @@ -415,8 +523,8 @@ describe('utils', () => {
it('Only inverted strings', () => {
const t = checkName('-name;-hey')

expect(t('I\'m matched')).to.be.true
expect(t('hey! I\'m not')).to.be.false
expect(t("I'm matched")).to.be.true
expect(t("hey! I'm not")).to.be.false
expect(t('My name is weird')).to.be.false
})
})
Expand Down
14 changes: 7 additions & 7 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"dependencies": {
"cypress-plugin-config": "^1.2.0",
"debug": "^4.3.2",
"find-test-names": "^1.23.0",
"find-test-names": "1.25.0",
"globby": "^11.0.4"
},
"devDependencies": {
Expand Down
14 changes: 14 additions & 0 deletions src/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,15 @@ declare namespace Cypress {
* describe('block with config tag', { tags: '@smoke' }, () => {})
* @example multiple tags
* describe('block with config tag', { tags: ['@smoke', '@slow'] }, () => {})
* @see https://github.com/bahmutov/cy-grep
*/
tags?: string | string[]
/**
* Provide a tag or a list of tags that is required for this suite to run.
* @example describe('mobile tests', { requiredTags: '@mobile' }, () => {})
* @see https://github.com/bahmutov/cy-grep
*/
requiredTags?: string | string[]
}

// specify additional properties in the TestConfig object
Expand All @@ -21,8 +28,15 @@ declare namespace Cypress {
* it('logs in', { tags: '@smoke' }, () => { ... })
* @example multiple tags
* it('works', { tags: ['@smoke', '@slow'] }, () => { ... })
* @see https://github.com/bahmutov/cy-grep
*/
tags?: string | string[]
/**
* Provide a tag or a list of tags that is required for this test to run.
* @example it('cleans the data', { requiredTags: '@nightly' }, () => {})
* @see https://github.com/bahmutov/cy-grep
*/
requiredTags?: string | string[]
}

interface Cypress {
Expand Down
Loading

0 comments on commit 77a2d28

Please sign in to comment.