Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't work if test name contains $ #17

Closed
jods4 opened this issue Oct 10, 2018 · 8 comments
Closed

Doesn't work if test name contains $ #17

jods4 opened this issue Oct 10, 2018 · 8 comments

Comments

@jods4
Copy link

jods4 commented Oct 10, 2018

This snippet:

describe("Two", () => {
  it("should equal $data", () => {
    let $data = 2;
    expect($data).toBe(2);
  });
});

Does not update the UI (test states stays a gray square).

If I rename the test should equal data it works.

My guess: a regexp is built without encoding and $ means end-of-string -> no match.

@hbenl
Copy link
Owner

hbenl commented Oct 12, 2018

I couldn't reproduce this, everything works for me no matter how I name the test:
screenshot_2018-10-12_11-24-04

Note that there is one difference in my tests: I used chai and hence had to write expect($data).to.equal(2);. I don't think this should make a difference in this case, but anyway: what assertion library are you using?

@jods4
Copy link
Author

jods4 commented Oct 12, 2018

That's strange. I'm gonna try to debug this and tell you exactly where it breaks for me.

I'm using expect.js (now under jest governance) but this shouldn't matter, right?
The variable name doesn't matter in my case, only the test name.

When I change it("should equal $data") to it("should equal data") the test lights up in the explorer.

@hbenl
Copy link
Owner

hbenl commented Oct 12, 2018

I'm using expect.js (now under jest governance) but this shouldn't matter, right?

Right. And I just checked, it still works for me when I use expect.js.
When you say "the test lights up in the explorer" you mean that the state icon changes when you run the test, right?

@jods4
Copy link
Author

jods4 commented Oct 12, 2018

Yes.

  • The test is discovered and always there.
  • It goes to running (blue).
  • Test completes but test icon goes back to grayed out (square), as if it didn't capture a result.

When I change the name, test becomes green as on your screen.

@hbenl
Copy link
Owner

hbenl commented Oct 12, 2018

You could try setting "mochaExplorer.nodePath": null in your configuration to see if that changes anything. If not, you could set "mochaExplorer.logpanel": true and copy the "Mocha Explorer Log" output channel here for both cases ($data and data).

@jods4
Copy link
Author

jods4 commented Oct 12, 2018

Apologies, the flaw was in my own code.
I wrote a terribly hacky test-runner that runs mocha tests in headless browsers instead of node and integrated it with your test explorer.
When serializing the grep option, a string wasn't properly encoded (slashes need escaping).

@jods4 jods4 closed this as completed Oct 12, 2018
@hbenl
Copy link
Owner

hbenl commented Oct 14, 2018

I wrote a terribly hacky test-runner that runs mocha tests in headless browsers instead of node and integrated it with your test explorer.

Wow, this is the second time I hear from a user who wants to run his Mocha tests in a different environment using the Test Explorer (see also #13).
I want to try to offer some tools that make setting up such a scenario a bit less hacky and easier to do.
Could you describe how you did the integration between your test runner and the Test Explorer?

@jods4
Copy link
Author

jods4 commented Oct 14, 2018

Sure.

Foreword: I'm gonna glance over lots of stuff. I had lots of headaches putting all this stuff together. Ideas are basic but realization was tricky.

Background: I'm writing a UI framework. Obviously you want to test this in browsers, not in node.
Some people use JSDOM but I find that it lacks too many API + it's not a real substitute for browsers. Easy to let slip a non-implemented feature in browser X or a discrepancy in browser Y.
Previously I used Karma for the task but I grew tired of its weight and the lack of tooling (like VS Code Test Explorer).

Core idea: I'm just spawning a tiny express web server that serves the test-runner page and my code (lib + tests). I added on-the-fly TS compilation to the mix because I'm using it.

So at this point I can just node serve-tests and I have a running server. Then I can navigate to localhost and my tests run in any browser. I can debug them and the test report is displayed on the page.

Mocha can run in browser so there's nothing really groundbreaking here. Since my tests modify the DOM I isolated them in an iframe distinct from the reporter, that's maybe the first tricky bit.

Going headless: Then I just added a headless mode. After starting the server, I spawn a headless browser that runs the tests.

Best option here for portability is probably to use webdriver, but because I'm a cheapo I just start Chrome with --repl so that it stays open waiting for commands.

Second trick: I created a reporter that posts events back to my server. The server then uses regular reporters to display the test run in CLI. This is not too hard but tests objects don't serialize well (circular references, methods and more), so I had to hand pick what I want to transfer over the wire and reproduce it on the server-side. I did it so that most built-in reporters work: dot, spec , min, landing and progress all work.

When the test run is done (end event arrived at server-side) I kill the whole process.

Integrating VS Test Explorer: That's the most hacky part. Up to here things were relatively clean-ish.
To be fair, a better design would be to write a specific test explorer, but again my lazyness had the best of me.

Mocha Test Explorer lets you specify the path to Mocha in its options. I directed it to my own patched Mocha. I'm basically re-exporting Mocha with a modified run prototype, which basically does the same as headless CLI above. I just use your reporter that consumes events to update the UI instead of the CLI ones.

The really hacky part is discovering tests. Once again, you guessed it, I'm a sloth. The right thing to do is to do that remotely in a browser but that's a lot of work. So instead I'm just letting you do your thing inside Node.

The problem with that is that a lot of code loaded by my tests doesn't run. To workaround that part (here's the worst hack):

  • I'm ensuring that my spec files have a strict policy of doing nothing when tests are not run. Setup must be entirely inside a before.
  • I'm blanking out my source modules by hacking into Node. So everything that my tests imports is just module.exports = {}. This is because one or two of my source modules have side-effects that can't be performed in Node. It works since specs that import them do nothing, see previous bullet.
    Bonus: this keeps discovering tests fast.

And voilà. I'm very happy with the result.
I can run my Mocha tests interactively in a browser, do quick runs on the CLI and see/run them without leaving VS and it's quite fast. :)

Thanks for the Test Explorer!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants