Upgrade to Jest 21.0.2 #3088

willdurand · 2017-09-09T08:39:31Z

This PR adds Jest 21+ compatibility.

Since Jest 21.0+, Node unhandled rejection warnings are caught and converted into errors (see: jestjs/jest#4016). One of our test cases rejects a Promise to test error handling on the server, but Jest now detects that the promise is not handled correctly and fails the test case, also causing subsequent tests to fail.

This behavior is not sound as redux-saga should handle promise rejections, and also our callApi implementation does that too. There are many try/catch everywhere so I decided to fix the test case with a mutation of the promise, letting Node think the rejection is handled (see: http://thecodebarbarian.com/unhandled-promise-rejections-in-node.js.html). This does not affect the original behavior, only silent the warning, allowing the test case to pass.


UPDATE: Added the SinonJS way to reject promises... see: https://github.com/sinonjs/sinon/pull/1211.

codecov-io · 2017-09-09T08:48:00Z

Codecov Report

Merging #3088 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    mozilla/addons-frontend#3088   +/-   ##
=======================================
  Coverage   95.17%   95.17%           
=======================================
  Files         156      156           
  Lines        2839     2839           
  Branches      552      552           
=======================================
  Hits         2702     2702           
  Misses        118      118           
  Partials       19       19

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9e8128...24f9c03. Read the comment docs.

tofumatt

Works for me!

tofumatt · 2017-09-09T14:04:54Z

tests/unit/core/server/testServer.js

+      // unhandled rejection warning that Jest 21.0+ converts into an error. We
+      // do want to throw exceptions on the server. See also:
+      // http://thecodebarbarian.com/unhandled-promise-rejections-in-node.js.html
+      error.catch(() => {});


This feels pretty naughty, like something that should be labelled with HACK: haha.

But the reasoning seems fine to me. I honestly forget why we want to throw on the server though, maybe you could add a comment explaining why?

tofumatt · 2017-09-09T14:05:29Z

tests/unit/core/server/testServer.js

@@ -180,6 +195,12 @@ describe(__filename, () => {
        .end();

      expect(response.statusCode).toEqual(500);
+      mockUserApi.verify();
+
+      // This ensures the error caught is the expected one, that is the one


"caught is the expected one, that is the one" should be simplified to "caught is the expected one, from"

willdurand · 2017-09-11T17:27:02Z

@kumar303 what do you think of this?

kumar303

Thanks for figuring this out. I'm sure the new Jest will have some nice features but if this patch slows you down then we can always revisit it later.

The fact that we have an unhandled promise rejection on the server does raise a bit of a flag for me. As far as I can tell, that's buried in Express but I'm not sure. Since we do have an error handler in the server code I can't think of a reason why we should be leaking an unhandled promise rejection. Perhaps I'm not fully understanding the problem though.

kumar303 · 2017-09-11T20:17:50Z

tests/unit/core/server/testServer.js

    });

    it('returns a 500 error page when retrieving the user profile fails', async () => {
+      const errorMessage = 'example of an API error';
+      const error = Promise.reject(new Error(errorMessage));


Ugh, this is a confusing workaround.

I think it would be helpful to make a test helper for this that looks like:

const errorResult = rejectedPromise(new Error(errorMessage));

This would allow us to put the comments in the helper function itself.

We use Promise.reject() all over the test suite so I don't understand why only this test needed updating. Is it something to do with how this promise runs in a separate process? I.E. the test server promise.

A helper works for me.

I think the problem comes from the server-side rendering indeed. We boot a real Express app and I think node or jest loses track of the promise and thinks it is not handled (that is what I understand from this blog post: http://thecodebarbarian.com/unhandled-promise-rejections-in-node.js.html, especially the section about to sinon and stubs).

I have attached .catch() in many different places and tried a few other try/catch combinations. I am not able to catch this "unhandled rejection warning".

Not sure if we can do anything else, because it seems we have to hook into process.on('unhandledRejection', () => {}). That is also what Jest does since version 21 actually. If I comment the Jest code related to that, no more issue... I tried to catch the error in the sagas, but that was not working either.

I don't think this is a perfect solution though. One more thing, I used node 8 too and could not generate unhandled promise warnings using the app for real, but the test suite was failing like on Travis.

kumar303 · 2017-09-11T20:18:56Z

tests/unit/core/server/testServer.js

+      // This ensures the error caught is the expected one, that is the one
+      // from `userProfile` API call.
+      const { loadState } = store.getState().reduxAsyncConnect;
+      expect(loadState.ServerBase.error.sagaStack).toContain(errorMessage);


is this assertion necessary? I would rather we didn't rely on reduxAsyncConnect because it will be removed in https://github.com/mozilla/addons-frontend/issues/2356

I could not come with something better to be 100% sure the error caught came from the promise rejection, not something else (example: a check in the reducer called right after callApi() saying the profile argument was not provided).

I did that because I found a few fixes to make the test suite all green but that was not correct. For instance, it would return a 500 due to an error related to the reducer rather than the saga.

This is a high-level integration test so I don't think it's crucial that we make an assertion about the exact error that triggered the 500. In other words, we won't rely on this specific test to tell us that fetching a user profile fully works as expected. We have better unit tests for that.

Then, I can get rid of the promise "hack" I guess by not rejecting the promise but resolving it with empty/null data. The reducer will throw an error and this error will be caught, correctly.

Well, we will need to set up a rejected promise in a server test sooner or later so setting a precedent for how to do it would be nice. I prefer tests that set up explicit errors rather than trying to create the error by side effect.

If I understand correctly the problem (and remember what I have done last week), it is not because of the use of Promise.reject(). This, I am pretty sure, works and is handled somewhere in the code. I tend to think this side-effect is a false-positive and would be hard to reproduce in the app.

kumar303 · 2017-09-11T20:23:36Z

tests/unit/core/server/testServer.js


      expect(response.statusCode).toEqual(200);
      expect(api.token).toEqual(token);
      expect(user.id).toEqual(42);
      expect(user.username).toEqual('babar');
      mockUserApi.verify();
+
+      const { loadState } = reduxAsyncConnectState;
+      expect(loadState).toEqual({});


I guess this is a safety measure since you're relying on it in the next test? I'd rather not rely on this state because it will be removed in https://github.com/mozilla/addons-frontend/issues/2356

This can be removed here but the check is quite useful to be sure the rendering was successful.

muffinresearch · 2017-09-14T08:18:26Z

Unless there's a really strong cause for concern, I would suggest that we go ahead with this, the patch is very concise. If there's something that we want to investigate later then we should file an issue to come back to, but otherwise move forward.

@kumar303 does that work for you?

muffinresearch · 2017-09-19T10:25:51Z

@kumar303 any thoughts ^

kumar303

I'll fix this up in #3178

Wow, I didn't know this turned into such a simple patch \o/

I guess the uncaught exception was a red herring? I'm not seeing it locally anymore.

kumar303 · 2017-09-19T15:36:17Z

tests/unit/core/server/testServer.js

@@ -170,7 +170,7 @@ describe(__filename, () => {
      mockUserApi
        .expects('userProfile')
        .once()
-        .returns(Promise.reject(new Error('example of an API error')));
+        .rejects('example of an API error');


This should be rejecting with an error instance, not a string

muffinresearch · 2017-09-19T16:21:18Z

Thanks @kumar303

This was referenced Sep 9, 2017

Update jest to the latest version 🚀 #3067

Closed

chore(package): update jest to version 21.0.1 #3074

Closed

chore(package): update babel-jest to version 21.0.2 #3084

Closed

chore(package): update jest to version 21.0.2 #3085

Closed

tofumatt approved these changes Sep 9, 2017

View reviewed changes

kumar303 suggested changes Sep 11, 2017

View reviewed changes

Upgrade to Jest 21.0.2

24f9c03

willdurand assigned willdurand and unassigned willdurand Sep 13, 2017

willdurand mentioned this pull request Sep 15, 2017

Update jest to the latest version 🚀 #3132

Closed

kumar303 mentioned this pull request Sep 19, 2017

Upgrade Jest to 21.0.2 #3178

Merged

kumar303 suggested changes Sep 19, 2017

View reviewed changes

kumar303 closed this in #3178 Sep 19, 2017

willdurand deleted the jest-21 branch September 26, 2017 09:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to Jest 21.0.2 #3088

Upgrade to Jest 21.0.2 #3088

willdurand commented Sep 9, 2017 •

edited

Loading

codecov-io commented Sep 9, 2017 •

edited

Loading

tofumatt left a comment

tofumatt Sep 9, 2017

tofumatt Sep 9, 2017

muffinresearch Sep 11, 2017

willdurand commented Sep 11, 2017

kumar303 left a comment

kumar303 Sep 11, 2017

willdurand Sep 11, 2017

kumar303 Sep 11, 2017

willdurand Sep 11, 2017

kumar303 Sep 11, 2017

willdurand Sep 11, 2017

kumar303 Sep 11, 2017

willdurand Sep 11, 2017 •

edited

Loading

kumar303 Sep 11, 2017

willdurand Sep 11, 2017

muffinresearch commented Sep 14, 2017

muffinresearch commented Sep 19, 2017

kumar303 left a comment

kumar303 Sep 19, 2017

willdurand Sep 19, 2017

muffinresearch commented Sep 19, 2017

Upgrade to Jest 21.0.2 #3088

Upgrade to Jest 21.0.2 #3088

Conversation

willdurand commented Sep 9, 2017 • edited Loading

codecov-io commented Sep 9, 2017 • edited Loading

Codecov Report

tofumatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willdurand commented Sep 11, 2017

kumar303 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willdurand Sep 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

muffinresearch commented Sep 14, 2017

muffinresearch commented Sep 19, 2017

kumar303 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

muffinresearch commented Sep 19, 2017

willdurand commented Sep 9, 2017 •

edited

Loading

codecov-io commented Sep 9, 2017 •

edited

Loading

willdurand Sep 11, 2017 •

edited

Loading