istanbul cover --report html tests/tests.js doesn’t create any HTML files

[mathiasbynens]

I’m trying the same command I’m using for other projects to generate the HTML coverage reports:

istanbul cover --report html tests/tests.js

The files coverage.json, prettify.css, prettify.js are generated in the ./coverage directory (as expected), but it seems no HTML files are being created.

I’m using the latest available version of Istanbul. Any ideas?

[gotwarlost]

does the coverage.json file have content in it?

[mathiasbynens]

Yes, in fact, it looks just fine. Seems to contain all the required data. Here’s the coverage.json file: https://gist.github.com/mathiasbynens/e084459d7b0637fa92d5

Any hints on how to debug this are welcome! I’ve been scratching my head over this one.

[mathiasbynens]

Here’s the coverage.json file: https://gist.github.com/mathiasbynens/e084459d7b0637fa92d5

[gotwarlost]

I cannot figure out why this is happening. One thing you might try is to run istanbul report on the saved coverage file and see if that generates HTML correctly.

There have been no changes to this part of the code for some time now. Could you also try a previous version of istanbul on this project and the latest version on your existing projects that work to see if this is a regression of some kind?

[mathiasbynens]

There have been no changes to this part of the code for some time now. Could you also try a previous version of istanbul on this project and the latest version on your existing projects that work to see if this is a regression of some kind?

The latest version of Istanbul works just fine for all my other projects, using the exact same command. I will try downgrading Istanbul and running the command again, and will report back.

FWIW, the repository for the project where this issue occurs can be found here: https://github.com/mathiasbynens/regenerate For me, the issue is reproducible on a fresh clone of the repository (clone it, then npm install, then istanbul cover --report html --verbose --dir coverage tests/tests.js, then be patient — the tests take about a minute). You can try to reproduce the issue here.

[mathiasbynens]

Hah! I tried your suggestion to run istanbul report --root coverage --format html after generating the coverage.json file, and it worked!

I’m happy to use this workaround, but still I wonder what’s causing this bug. Let me know if I can assist in tracking it down.

[gotwarlost]

That's weird - I see the problem as well. No idea what's going on yet.

The only thing that looks very much out of the ordinary is the line execution counts - you have lines in that code that are being executed >100 million times as part of the tests (line 179). Which is why I suspect the test runtime skyrockets from 2 seconds to 30 seconds with coverage turned on.

[gotwarlost]

I did futz around with this for 20 minutes or so and got to the shallow reason as to why this is happening. I still don't know the deep reason.

The HTML reporter is throwing when generating reports - the root cause is a throw from the handlebars code. You can wrap a try-catch around the HTML reporter code and see it for yourself.

Since the HTML reporting works correctly as a post-procesing step in a different process, I believe there is something in your code or your test framework that is hacking the prototype of a global object (my bet is on the Function object) and causing it to behave differently which in turns causes a perfectly innocuous handlebars template to throw.

[mathiasbynens]

In my library’s code, I don’t extend any natives.

In the tests, I extend Object.prototype as follows, to see if my library can handle it:

// Extend `Object.prototype` to see if Regenerate can handle it.
// 0xD834 is the high surrogate code point for U+1D306 (among others).
Object.prototype[0xD834] = true;

I just tested commenting this line out, and then Istanbul generates the HTML report just fine!

Any chance we can get the templating part to work with extended natives? Seems like adding a hasOwnProperty check should be enough to fix this.

[mathiasbynens]

The issue has been fixed in Handlebars. We just have to wait until Handlebars v1.0.12 gets released.

[gotwarlost]

cool. Thanks for following up!

[mathiasbynens]

…and Handlebars v1.0.12 aka the official v1.0.0 release is out: https://github.com/wycats/handlebars.js/blob/master/release-notes.md#v1012--100---may-31-2013 Updating to it should fix the problem.

[gotwarlost]

Closing this, since fix is in handlebars.