Multiple files, multiple beforeEach, not all run

[nWidart]

Hello,

Current behavior:

I have 2 test files, both have their beforeEach with some commands that stub xhr requests.
However, only one of the 2 beforeEach sections gets run. I do see it in the "Routes" section, but with 0 hits.

When I run both files in isolation, they work as expected.

Example command:

Cypress.Commands.add('mockGetActivationsData', (fixture = 'getActivations.json') => {
  cy.server();
  cy.route('GET', '**/**/business-units/*/activations', `fixture:activations/${fixture}`);
});

Desired behavior:

Each beforeEach section should allow commands to be fired to intercept xhr requests.

Steps to reproduce:

Create 2 test files. Each of them have a beforeEach section that call a command, that stubs a XHR request.

It works fine when I remove my command calls from every beforeEach section and duplicate them across every test.

Versions

• cypress 2.1.0
• chrome 66

Thanks

[jennifer-shehane]

When you run your tests with "run all" or using cypress run currently, we basically merge all of the test files together into one "big file". This will change soon.

I'm not exactly clear as to what is happening since there is no code provided for each spec file. Are you wrapping the beforeEach in each file within a describe or context block? I would suggest doing that so that each file has it's appropriate beforeEach defined for the context of the file when they get merged into 1 file.

spec_file1.js

describe('File 1 tests', () => {
  beforeEach(() => {
    // commands here
  })
})

spec_file2.js

describe('File 2 tests', () => {
  beforeEach(() => {
    // commands here
  })
})
``

[nWidart]

I have indeed 1 global before each in each file, and before each in describes.

I don't see what code could clarify this as it's straightforward but:

The global one:

beforeEach(() => {
  cy.mockAllUnhandledHttpCalls();
  cy.mockGetAuthenticationData();
  cy.mockGetBusinessUnitData();
  cy.mockGetPermissionsData();
  cy.mockGetNavigationData();

  cy.mockGetActivationPartnersData();
});

In describe:

describe('something', () => {
  beforeEach(() => {
    cy.visit('/activations');
    cy.mockGetActivationPartnersData();
    cy.mockGetActivationsData();
  });
  it('should display a list of activations', () => {
    cy.get('.activation').should('have.length', 2);
  });

[brian-mann]

I am 100% sure there is not a bug in the way this works - but there is likely a misunderstanding of how it works.

Hooks come from mocha and they definitely work as advertised. they are added in the order they appear. Having multiple root level beforeEach or before hooks will all work together. I would put a console.log or debugger statement in each and I'm very confident they will indeed work and fire.

For us to look at this you'll need to put together a reproducible repo with two test files and the code that you're describing. Just add something like console.log(...) to show that they are indeed hitting.

[nWidart]

I have only one root level beforeEach per file. The issue is apparently when merging multiple files together.

For me it's a bug, since it's not merging correctly.

[jennifer-shehane]

If you want to run a beforeEach globally shared and run before every file, I suggest you put it in a support file as outline here.

I'd have to investigate more on whether this is an issue - but this is a suggestion for now.

[nWidart]

Thank you.
The issue, however, is more than about the global beforeEach in those 2 files. I could indeed extract those out, even though I need custom global before each in each file, which makes this not possible.

Every before each the in the second test didn't get touched, making the test fail. The workaround is to copy the code in the multiple beforeEach blocks in every test, which works but provides a lot of code duplication.

[jennifer-shehane]

Can you just wrap a larger describe within each file to contain the global beforeEach as described in my original comment?

file1.js

describe('file 1 something', () => {
  beforeEach(() => {
    cy.mockAllUnhandledHttpCalls();
    cy.mockGetAuthenticationData();
    cy.mockGetBusinessUnitData();
    cy.mockGetPermissionsData();
    cy.mockGetNavigationData();

    cy.mockGetActivationPartnersData();
  });

  describe('something', () => {
    beforeEach(() => {
      cy.visit('/activations');
      cy.mockGetActivationPartnersData();
      cy.mockGetActivationsData();
    });
    it('should display a list of activations', () => {
      cy.get('.activation').should('have.length', 2);
    });
  });
});

file2.js

describe('file 2 something', () => {
  beforeEach(() => {
    cy.mockAllUnhandledHttpCalls();
    cy.mockGetAuthenticationData();
    cy.mockGetBusinessUnitData();
    cy.mockGetPermissionsData();
    cy.mockGetNavigationData();

    cy.mockGetActivationPartnersData();
  });

  describe('something', () => {
    beforeEach(() => {
      cy.visit('/activations');
      cy.mockGetActivationPartnersData();
      cy.mockGetActivationsData();
    });
    it('should display a list of activations', () => {
      cy.get('.activation').should('have.length', 2);
    });
  });
});

[nWidart]

I have beforeEach sections in multiple describe "blocks", which don't get called, in the second file, only when run together with the first file also containing describes, beforeEachs, etc.

When running each test file independently, no issues.

What you link in file 2 for instance, is what isn't working, hence why I mentioned I had to duplicate that code in every test.

To give you an idea (this duplication goes on for another 3-4 tests):

    cy.mockGetActivationsData('emptyArray.json');
    cy.mockGetActivationPartnersData();

Were in the beforeEach before, and is working when running the file independently, but not when running the suite.

It seems to be something related to commands, or maybe the mock http calls, since the beforeEach contains the cy.visit() which works.

[jennifer-shehane]

Ok, so, I should have taken a closer look to begin with at what the code was doing . You made the assertion that the beforeEach was not working - and I jumped into that assumption, but I believe the issue is how the order of these commands are structured.

You will always want to set up cy.server() and cy.route() before calling a cy.visit(). The route definition is saying - "listen for these xhr requests", so if they are not set up before the visit...the xhr requests could have already gone out by the time you tell them to listen.

I think the reason these routes are not being mocked is because you are defining them after the cy.visit() here:

  beforeEach(() => {
    cy.visit('/activations');
    cy.mockGetActivationPartnersData();
    cy.mockGetActivationsData();
  });

This should be:

  beforeEach(() => {
    cy.mockGetActivationPartnersData();
    cy.mockGetActivationsData();
    cy.visit('/activations');
  });

I'm actually a bit confused how your last example could be working reliably on every run.

If this doesn't help, I think we need to take a step back and have you define what exactly about this is "not working" in the runner exactly. They're not being stubbed? What does the screenshot of the test runner show?

[nWidart]

I was indeed mocking the xhr calls before cy.visit() this is what is not working with running multiple files.

Adding them in the test methods themselves they work on every run, at least for now :D But I do agree it's strange and not logical.

Here's is the diff for example (part of it), that I had to do to make it work with multiple test files.
The line at the top was from the global beforeEach which also had to be moved in each and every test method.

I should have been more clear sorry, by not working I ment the xhr requests aren't mocked, picked up by the command that contains acy.server&cy.route().
This was mentioned in the first post,

Desired behavior:

Each beforeEach section should allow commands to be fired to intercept xhr requests.
And the current behavior section.

[jennifer-shehane]

phew, I didn't think beforeEach could be this busted since we use it extensively every day for our own tests. 😅

Let us know if you have any other issues - or ask in our community chat https://gitter.im/cypress-io/cypress

[nWidart]

Hm so no resolution?

[nWidart]

I'm not sure why this was closed... @jennifer-shehane ?

Here's a screenshot of the dashboard with the failing tests.

The routes section correctly shows the xhr requests to mock (even if duplicated), but they're just not getting hit.

expected '' to have a length of 2 but got 0

Means the mock xhr resource didn't get hit, and thus makes this fail.

[jennifer-shehane]

The cy.get('.activation-partner') is only going to wait 4 seconds for that DOM element to be there before failing, so if your requests don't all respond within 4 seconds it will fail.

You generally want to set up cy.wait() to wait for each route to respond that is required for your application's view. This will not only provide an extra layer of tests for your routes, but also make the tests less brittle (instead of arbitrary 4 seconds, it will wait for all routes specifically) https://on.cypress.io/wait

[nWidart]

Indeed, but the request never responds because it's not hit, ie the problem. :D So no matter how long I wait, it'll never hit.

Again, this works fine when running the single test file on its own. It fails when running multiple files.
Also when duplicating the command that stubs XHR, it works too.

[jennifer-shehane]

Yeah, we'll need a reproducible example of this behavior - the beforeEach not working as described.

I'm unable to figure out why an example like below would not work with the information provided.

Cypress.Commands.add('mockGetActivationsData', (fixture = 'getActivations.json') => {
  cy.server();
  cy.route('GET', '**/**/business-units/*/activations', `fixture:activations/${fixture}`).as("getActivations");
});

describe('something', () => {
  beforeEach(() => {
    cy.mockGetActivationsData();
    cy.visit('/activations');
  });

    it('should display a list of activations', () => {
      cy.wait("@getActivations")
      cy.get('.activation').should('have.length', 2);
    });
});

[nWidart]

I tried with .wait:

Where:

global beforeEach contains cy.mockGetActivationPartnersData();, which is:

Cypress.Commands.add('mockGetActivationPartnersData', (fixture = 'getActivationPartners.json') => {
  cy.server();
  cy
    .route('GET', '**/**/activation-partners', `fixture:activations/${fixture}`)
    .as('getActivationPartners');
});

And :cy.mockGetActivationsData();, is inside a before each of a describe block.

Which is

Cypress.Commands.add('mockGetActivationsData', (fixture = 'getActivations.json') => {
  cy.server();
  cy
    .route('GET', '**/**/program-manager-service/business-units/*/activations', `fixture:activations/${fixture}`)
    .as("getActivations");
});

And finally the test:

  it('should display a list of activations', () => {
    cy.wait('@getActivations');
    cy.wait('@getActivationPartners');
    cy.get('.activation').should('have.length', 2);
  });

The error message said it timed out waiting for the request, which I can see in the network tab.

And again, when I run this test on its own, it works fine (with or without the cy.wait() technique).

[jennifer-shehane]

Here's a simplified version of the pieces you are explaining from our example app. This example is passing. Maybe you could have this reflect what you are doing to get this to fail for me? I'm just unable to recreate this as being a beforeEach bug and honestly can't look into it any further without a reproducible example.

spec1.js

beforeEach(() => {
  cy.server()
  cy.route('GET', 'comments/*', {
    'body': 'laudantium enim quasi est quidem magnam voluptate ipsam eos↵tempora quo necessitatibus↵dolor quam autem quasi↵reiciendis et nam sapiente accusantium',
    'email': 'Eliseo@gardner.biz',
    'id': 1,
    'name': 'id labore ex et quam laborum',
    'postId': 1,
  }).as('getComment')
})

context('Network Requests', () => {
  beforeEach(() => {
    cy.server()
    cy.route('POST', '/comments', 'foo').as('postComment')
    cy.visit('https://example.cypress.io/commands/network-requests')
  })

  it('cy.route() - route responses to matching requests', () => {
    let message = 'whoa, this comment does not exist'

    cy.get('.network-btn').click()
    cy.wait('@getComment').its('status').should('eq', 200)

    cy.get('.network-post').click()
    cy.wait('@postComment')
  })
})

spec2.js

beforeEach(() => {
  cy.server()
  cy.route('POST', '/comments', 'foo').as('postComment')
})

context('Network Requests', () => {
  beforeEach(() => {
    cy.server()
    cy.route('GET', 'comments/*', {
      'body': 'laudantium enim quasi est quidem magnam voluptate ipsam eos↵tempora quo necessitatibus↵dolor quam autem quasi↵reiciendis et nam sapiente accusantium',
      'email': 'Eliseo@gardner.biz',
      'id': 1,
      'name': 'id labore ex et quam laborum',
      'postId': 1 }).as('getComment')
    cy.visit('https://example.cypress.io/commands/network-requests')
  })

  it('cy.route() - route responses to matching requests', () => {
    let message = 'whoa, this comment does not exist'

    cy.get('.network-btn').click()
    cy.wait('@getComment').its('status').should('eq', 200)

    cy.get('.network-post').click()
    cy.wait('@postComment')
  })
})

[jennifer-shehane]

Unfortunately we'll have to close this issue if no reproducible example is provided. Can anyone provide a way to reproduce this?

[zeralight]

I am having a similar issue where the root describe isn't running with nesting describes
In some test, I delay a network request to test a loading state.

describe("...", () => {
  beforeEach(() => cy.setCookie(...));
  describe("...", () => {
    beforeEach(() => cy.route2("/foo", { delayMs: 2000 }).as("requestFoo"));

    it("...", () => {
      /* cy.setCookie() running and cy.route2() running */
      /* trigger action to request Foo */
      /* run asserts: OK */
      // cy.wait('@requestFoo')
    });
  });

  describe("...", () => {
   beforeEach(() => something());
    it("...", () => {
      /* something() running */
     /* cy.setCookie() not running */
    });
  });
});

When I uncomment the line // cy.wait('@requestFoo'), the issue disappears.
Cypress: 5.3.0

[jennifer-shehane]

@zeralight Can you provide an example we can run completely? There's a lot of things not included in your example that may be affecting the way it runs. You can edit our tests against our example.cypress.io website, which has some routing. Repo to fork: https://github.com/cypress-io/cypress-example-kitchensink

And opening a new issue would probably be best than this older issue.