beforeEach not running in order when using --runInBand after migrating from 20.0.4 to 22.0.4

[bilby91]

Do you want to request a feature or report a bug?
Not sure if it's a bug or something in the API change since 20.0.8 regarding setup and teardown.

What is the current behavior?
We are migrating our test suite for Suttna from 20.0.4 to the latest jest version (22.0.4). Our bot is a server side node js application using typescript that depends on a SQL database for running the suite. On jest 20.0.4 we were using setupTestFrameworkScriptFile and setting up a beforeAll block to run migrations and a beforeEach block to clean up the database between tests. Another important aspect is that we use runInBand flag to run the tests sequentially.

After migrating to 22.0.4, a lot of tests started failing. After some investigation we realized that the problem was that the beforeEach block that cleaned the database was being called after another test started running. This behaviour generates inconsistent data in the database obviously.

The setupTestFrameworkScriptFile should be working in the same fashion as before or something changed ? I noticed that a new testEnvironment option is available to provide a custom class that will setup the environment. This should work with transformers ? I tried it but couldn't make it work with typescript.

I added the following logging information in the cleanup script:

beforeEach(async () => {
  console.info("BEFORE CLEAN")
  await cleanDatabase(X, Y)
  console.info("AFTER CLEAN")
})

Output on 20.0.4:

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

Output on 22.0.4:

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:29
    AFTER CLEAN

  console.info __tests__/spec_helper.ts:27
    BEFORE CLEAN

What is the expected behavior?
Get the same behaviour we had before :), not really sure if is something that we need to change, I'm guessing is something wrong on our end.

Please provide your exact Jest configuration and mention your Jest, node,
yarn/npm version and operating system.

Jest: 22.0.4
Node: 8.9.1
Yarn: 1.3.2

[bilby91]

Also tried adding --testEnvironment=node but I get the same execution order issue.

[SimenB]

I cannot reproduce this, could you create a repo I can clone and see the failure?

[bilby91]

Will try to reproduce it in an independent repository!

[bilby91]

@SimenB Got a repo with the failure.

https://github.com/bilby91/jest-22.0.4-before-each-order-issue

The problem is with the throw.

[bilby91]

@SimenB I have narrowed the problem in my test suite. If I comment an it statement that throws an error, all the other tests are passing and the order is respected. The problem starts after this it block is called.

it("throws a CustomError", () => {
  expect(service.serviceMethodName(object)).rejects.toThrowError(CustomError)
})

export class MyService {
  public async serviceMethodName(object: Object): Promise<IInterface> {
    if (object.shouldRise()) {
      throw new CustomError()
    }
    // More code
  }
}

Tried to make it as similar as possible.

[bilby91]

If I change expect(service.serviceMethodName(object)).rejects.toThrowError(CustomError) to expect(service.serviceMethodName(object)).rejects.toThrowError() everything works as expected.

My base class for errors and a custom one:

export class BaseError extends Error {
  public metadata: any

  constructor(message?: string) {
    super(message)

    Object.setPrototypeOf(this, new.target.prototype)
  }
}

export class CustomError extends BaseError {
  public get name() {
    return "CustomError"
  }
}

[SimenB]

When you use expect().rejects or expect().resolves you need to return or await the expectation, otherwise the test doesn't wait for your error. Changing that seems to make your example work correctly. Can you test that?

diff --git i/__tests__/index.spec.ts w/__tests__/index.spec.ts
index bcf03a5..69f75c2 100644
--- i/__tests__/index.spec.ts
+++ w/__tests__/index.spec.ts
@@ -10,30 +10,30 @@ describe("TestService", () => {
 
   describe("when output is ServiceOutput.Failure", () => {
     it("throws an error", () => {
-      expect(service.method1(2)).rejects.toThrowError(CustomError)
+      return expect(service.method1(2)).rejects.toThrowError(CustomError)
     })
 
     describe("when output is ServiceOutput.Failure", () => {
       it("throws an error", () => {
-        expect(service.method1(2)).rejects.toThrowError(CustomError)
+        return expect(service.method1(2)).rejects.toThrowError(CustomError)
       })
     })
 
     describe("when output is ServiceOutput.Failure", () => {
       it("throws an error", () => {
-        expect(service.method1(2)).rejects.toThrowError(CustomError)
+        return expect(service.method1(2)).rejects.toThrowError(CustomError)
       })
     })
 
     describe("when output is ServiceOutput.Failure", () => {
       it("throws an error", () => {
-        expect(service.method1(2)).rejects.toThrowError(CustomError)
+        return expect(service.method1(2)).rejects.toThrowError(CustomError)
       })
     })
 
     describe("when output is ServiceOutput.Failure", () => {
       it("throws an error", () => {
-        expect(service.method1(2)).rejects.toThrowError(CustomError)
+        return expect(service.method1(2)).rejects.toThrowError(CustomError)
       })
     })
 
@@ -41,38 +41,38 @@ describe("TestService", () => {
 
   describe("when output is ServiceOutput.Failure", () => {
     it("throws an error", () => {
-      expect(service.method1(2)).rejects.toThrowError(CustomError)
+      return expect(service.method1(2)).rejects.toThrowError(CustomError)
     })
   })
 
   describe("when output is ServiceOutput.Failure", () => {
     it("throws an error", () => {
-      expect(service.method1(2)).rejects.toThrowError(CustomError)
+      return expect(service.method1(2)).rejects.toThrowError(CustomError)
     })
   })
 
   describe("when output is ServiceOutput.Failure", () => {
     it("throws an error", () => {
-      expect(service.method1(2)).rejects.toThrowError(CustomError)
+      return expect(service.method1(2)).rejects.toThrowError(CustomError)
     })
   })
 
 
   describe("when output is ServiceOutput.Success", () => {
     it("throws an error", () => {
-      expect(service.method1(1)).resolves.toBe(true)
+      return expect(service.method1(1)).resolves.toBe(true)
     })
   })
 
   describe("when output is ServiceOutput.Success", () => {
     it("throws an error", () => {
-      expect(service.method1(1)).resolves.toBe(true)
+      return expect(service.method1(1)).resolves.toBe(true)
     })
   })
 
   describe("when output is ServiceOutput.Success", () => {
     it("throws an error", () => {
-      expect(service.method1(1)).resolves.toBe(true)
+      return expect(service.method1(1)).resolves.toBe(true)
     })
   })
 })

See http://facebook.github.io/jest/docs/en/asynchronous.html#resolves-rejects

[bilby91]

Thanks, my problem was that the promise was not returned. The wired thing was that in previous versions of jest this was passing and now no. I guess that is an improvement!

[bilby91]

@SimenB Thanks! Make sense!

[ambessh]

beforeEach won't work if you are using --runInBand for resolving the EADDRINUSE instead while closing the server await that because it returns a promise.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.