Add test.todo #6996

mattphillips · 2018-09-18T20:04:34Z

Summary

Adds test.todo and it.todo similar to ava's todo. I'm opening this up early for feedback from @SimenB @thymikee @rickhanlonii @aaronabramov 😄

Motivation: see comment by @SimenB

Closes #6430 #6602 and addresses #6256

This change re-introduces the behaviour of planning a test, similar to Jest@22 test('do something')

Test plan

unit tests ✅
e2e tests ✅

Given this test suite:

it('passes', () => {
  expect(10).toBe(10);
});

it('fails', () => {
  expect(10).toBe(101);
});

it.skip('skips', () => {
  expect(10).toBe(101);
});

it.todo('todo');

The new output will look like:

Todo

Add docs
Add typings

arcdev1 · 2018-09-18T20:17:30Z

If this gets merged, it would be nice to add a configuration option that allows for treating missing callbacks as "todos" so that we support the legacy behaviour for those who want/need it. That would be a great way to fully address: #6602 #6430 and #6256 in my opinion.

thymikee · 2018-09-18T20:22:42Z

This is lovely!

pk-nb · 2018-09-18T20:29:27Z

Thanks for doing this! I love the new state in the output, regardless of the function option. If this is merged, it would be good to add this to the jest-codemods so that pending specs in jasmine/mocha get converted.

thymikee

So nice

thymikee · 2018-09-18T20:27:45Z

e2e/__tests__/__snapshots__/test-todo.test.js.snap

+"FAIL __tests__/todo_non_string.test.js
+  ● Test suite failed to run
+
+    Invalid first argument: () => {}. Todo must be called with a string.


This might look weird, when the fn is not a oneliner.

True, perhaps we should just remove this as per @rickhanlonii's comments below. I'm pretty flexible with how this API looks. Maybe just validation of a minimum one arg with the first being a string? :)

I think just a simple "todo must be called with just a string", or something similar, is better. No need to stringify the implementation as long as our stack trace point points back to the declaration

thymikee · 2018-09-18T20:28:19Z

e2e/__tests__/__snapshots__/test-todo.test.js.snap

+        |    ^
+      9 |
+
+      at packages/jest-jasmine2/build/jasmine/Env.js:480:15


can we strip this from stack trace? cc @SimenB

@SimenB please teach me this magic (again), I've tried but not been able to remove it :(

thymikee · 2018-09-18T20:37:33Z

packages/jest-jasmine2/src/jasmine/jasmine_light.js

@@ -88,6 +88,10 @@ exports.interface = function(jasmine: Jasmine, env: any) {
      return env.xit.apply(env, arguments);
    },

+    tit(description: string) {


haha this was my first attempt at getting it to work - I thought I removed it 🤣

rickhanlonii

This is awesome

What do you think about allowing and ignoring the function argument?

I'm thinking of two use cases:

Stubbing out future tests to finish later
Making it easy to "disable" tests with a todo to come back and fix them

SimenB · 2018-09-18T21:34:10Z

@skovhus for the codemod ❤️

SimenB · 2018-09-18T21:37:20Z

If this gets merged, it would be nice to add a configuration option that allows for treating missing callbacks as "todos" so that we support the legacy behaviour for those who want/need it.

I don't think it's worth it. That's behaviour from almost 6 months ago, and I think a codemod fulfills the same purpose better.

I'm thinking of two use cases:

Aren't those covered by .skip?

rickhanlonii · 2018-09-18T21:40:33Z

Aren't those covered by .skip?

Does that depend on the use case? What's really the difference between .skip and .todo?

arcdev1 · 2018-09-18T22:20:29Z

What's really the difference between .skip and .todo?

todo implies there is work to be done.
skip implies we want to ignore the test for now.

Generally we don't want to ignore work to be done.

I think the first use case; stubbing for later is a valid todo. The second one sounds like skip to me.

SimenB · 2018-09-19T10:18:47Z

packages/jest-circus/src/index.js

@@ -121,6 +121,26 @@ test.only = (testName: TestName, fn: TestFn, timeout?: number) => {
  });
 };

+test.todo = (testName: TestName, ...rest: Array<mixed>) => {
+  if (rest.length > 0 || typeof testName !== 'string') {
+    throw new Error('Todo must be called with only a description.');


const e = new Error('Todo must be called with only a description.'); if (Error.captureStackTrace) { Error.captureStackTrace(e, test.todo); } throw e;

ish 🙂

❤️amazing!

I tried something similar to this but wasn't passing the correct call site

SimenB · 2018-09-19T10:22:00Z

We could ignore the implementation and have a lint rule for it. I do like that it throws though, so you don't forget it by accident

SimenB · 2018-09-19T11:13:11Z

Also missing changelog 😱

codecov-io · 2018-09-19T16:42:13Z

Codecov Report

Merging #6996 into master will decrease coverage by 0.21%.
The diff coverage is 15.51%.

@@            Coverage Diff             @@
##           master    #6996      +/-   ##
==========================================
- Coverage    66.9%   66.68%   -0.22%     
==========================================
  Files         250      250              
  Lines       10436    10483      +47     
  Branches        3        3              
==========================================
+ Hits         6982     6991       +9     
- Misses       3453     3491      +38     
  Partials        1        1

Impacted Files	Coverage Δ
packages/jest-cli/src/constants.js	`100% <ø> (ø)`	⬆️
packages/jest-jasmine2/src/jasmine/Env.js	`0% <0%> (ø)`	⬆️
packages/jest-jasmine2/src/index.js	`0% <0%> (ø)`	⬆️
.../src/legacy_code_todo_rewrite/jest_adapter_init.js	`0% <0%> (ø)`	⬆️
packages/jest-runner/src/run_test.js	`2.98% <0%> (ø)`	⬆️
...ackages/jest-cli/src/reporters/verbose_reporter.js	`20.37% <0%> (-2.08%)`	⬇️
packages/jest-jasmine2/src/jasmine/Spec.js	`0% <0%> (ø)`	⬆️
packages/jest-circus/src/run.js	`0% <0%> (ø)`	⬆️
packages/jest-circus/src/event_handler.js	`12% <0%> (-0.33%)`	⬇️
packages/jest-cli/src/testResultHelpers.js	`9.09% <0%> (-0.29%)`	⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2af1c62...eb2289c. Read the comment docs.

SimenB

Please update https://github.com/facebook/jest/blob/master/docs/GlobalAPI.md

SimenB · 2018-09-19T16:43:01Z

CHANGELOG.md

@@ -2,6 +2,8 @@

 ### Features

+- `[jest-jasmine2/jest-circus/jest-cli]` Add test.todo ([#6996](https://github.com/facebook/jest/pull/6996))
+


remove this newline

SimenB · 2018-09-19T16:43:56Z

e2e/__tests__/test-todo.test.js

+  expect(result.status).toBe(1);
+  const output = extractSummary(result.stderr)
+    .rest.split('\n')
+    .map(line => line.trimRight())


Is this needed? If yes, please extract to a helper function instead of copying it in every test

mattphillips · 2018-09-19T16:58:54Z

ping @SimenB just updated with your comments :)

SimenB · 2018-09-21T08:55:31Z

@cpojer @aaronabramov @mjesun this is new API, so I'd like your approval before merging 🙂

palmerj3 · 2018-09-21T13:10:58Z

I like seeing todo's aggregated separately from skipped tests for reporters. Nice work!

SimenB · 2018-09-24T08:21:13Z

@mattphillips could you update the error we throw when it or test is missing an implementation to suggest using this new method?

SimenB · 2018-09-24T08:23:18Z

packages/jest-cli/src/constants.js

@@ -15,6 +15,7 @@ export const ICONS = {
  failed: isWindows ? '\u00D7' : '\u2715',
  pending: '\u25CB',
  success: isWindows ? '\u221A' : '\u2713',
+  todo: '\u270E',


Can you check how this looks on windows?

@SimenB I don't have a windows machine, does anyone else have one?

http://modern.ie

wldcordeiro · 2018-09-25T01:45:28Z

Adding a --todo flag that made a little table of the todos like --coverage does would be icing on the cake! I love this! 🎉 😄

SimenB · 2018-09-25T06:40:17Z

Once we land this you can do that with a custom reporter 🙂

rickhanlonii · 2018-09-25T14:08:07Z

jest-reporter-todo would be dope

SimenB · 2018-09-29T08:10:25Z

@mattphillips ping 🙂 We can iterate on the name after landing if we want

SimenB · 2018-09-29T12:07:35Z

github-actions · 2021-05-12T08:08:04Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

mattphillips added 3 commits September 18, 2018 20:40

Add test.todo to jest-jasmine

fdd340b

Add test.todo to jest-circus

3cbe1c0

Add end to end tests

392c1d0

facebook-github-bot added the cla signed label Sep 18, 2018

mattphillips mentioned this pull request Sep 18, 2018

Re-allowed single string arguments for pending tests #6602

Closed

thymikee approved these changes Sep 18, 2018

View reviewed changes

rickhanlonii approved these changes Sep 18, 2018

View reviewed changes

Remove unused interface

ec08b6f

mattphillips added 2 commits September 18, 2018 22:56

Group todo tests into one log per suite

663ed55

Remove stringifying of invalid arg

a720cde

SimenB reviewed Sep 19, 2018

View reviewed changes

skovhus mentioned this pull request Sep 19, 2018

Support todo skovhus/jest-codemods#132

Closed

Add capture stack trace to invalid args error

1a17252

SimenB requested changes Sep 19, 2018

View reviewed changes

SimenB reviewed Sep 19, 2018

View reviewed changes

mattphillips added 2 commits September 19, 2018 17:47

Add changelog entry

54fac6d

Remove unnessecary snapshot normalisation

8891c36

mattphillips force-pushed the test-todo branch from eb2289c to 8891c36 Compare September 19, 2018 16:47

Add docs

0e3fb84

SimenB approved these changes Sep 19, 2018

View reviewed changes

SimenB mentioned this pull request Sep 20, 2018

Excluding second argument should not throw, but instead mark the test as skipped #7013

Closed

SimenB reviewed Sep 24, 2018

View reviewed changes

palmerj3 mentioned this pull request Sep 27, 2018

Add support for tests without a callback function #7054

Closed

Add use test.todo suggestion to tests without callback function

7325d16

SimenB merged commit 49b2380 into jestjs:master Sep 29, 2018

SimenB mentioned this pull request Dec 6, 2018

Jest should mark test as 'skipped' if empty function is provided as second argument #2584

Closed

SimenB mentioned this pull request Feb 2, 2019

Reports "NaN total" jest-community/jest-runner-eslint#64

Closed

samgladstone mentioned this pull request May 29, 2020

test.todo should allow and ignore implementation function #10100

Closed

github-actions bot locked as resolved and limited conversation to collaborators May 12, 2021

		@@ -2,6 +2,8 @@

		### Features

		- `[jest-jasmine2/jest-circus/jest-cli]` Add test.todo ([#6996](https://github.com/facebook/jest/pull/6996))

Add test.todo #6996

Add test.todo #6996

Conversation

mattphillips commented Sep 18, 2018 • edited by SimenB Loading

Summary

Test plan

Todo

arcdev1 commented Sep 18, 2018

thymikee commented Sep 18, 2018

pk-nb commented Sep 18, 2018 • edited Loading

thymikee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rickhanlonii left a comment

Choose a reason for hiding this comment

SimenB commented Sep 18, 2018 • edited Loading

SimenB commented Sep 18, 2018 • edited Loading

rickhanlonii commented Sep 18, 2018

arcdev1 commented Sep 18, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimenB commented Sep 19, 2018

SimenB commented Sep 19, 2018

codecov-io commented Sep 19, 2018

Codecov Report

SimenB left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattphillips commented Sep 19, 2018

SimenB commented Sep 21, 2018

palmerj3 commented Sep 21, 2018

SimenB commented Sep 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wldcordeiro commented Sep 25, 2018

SimenB commented Sep 25, 2018

rickhanlonii commented Sep 25, 2018

SimenB commented Sep 29, 2018

SimenB commented Sep 29, 2018

github-actions bot commented May 12, 2021

mattphillips commented Sep 18, 2018 •

edited by SimenB

Loading

pk-nb commented Sep 18, 2018 •

edited

Loading

SimenB commented Sep 18, 2018 •

edited

Loading

SimenB commented Sep 18, 2018 •

edited

Loading

arcdev1 commented Sep 18, 2018 •

edited

Loading