Coffeescript compilation failure results in 0 specs and success exit code #112
Comments
|
I've no idea how to solve this. Let me explain my findings so far: When Rails is started in development environment and either an implementation file or a spec file has a CoffeeScript compilation error, then a compilation error is raised (with ExecJS it's an Not optimal, but at least it's failing correctly. Now when you start the Rails server in the test environment, Sprockets will not fail, instead the resulting JavaScript file that usually contains all the compiled, minified CoffeeScript files that are combined into a single JS file is just missing. That means that Jasminerice is serving the test runner and runs it, but since no tests are found it does not fail. I do not see a way to detect a compilation error other than monkey patching Sprockets, which is something I don't want to do. My conclusion is that people should develop in the development environment and only use the Thor or Rake integration in the test environment after they have verified that it runs fine in development environment. I always use guard-coffeescript with the |
|
We never see any details errors like your example. Even that would give a clue that something was wrong. We typically run guard, with guard-jasmine and growl notifications. We're hacking away at specs and code-under-test. In this scenario, which seems to be the primary use case for a guard plugin, guard-jasmine swallows errors and reports 0 specs (pass), to both growl and in the shell running guard. |
|
Did you read my comment or just looked at the error message? Guard::Jasmine does not swallow any errors, it outputs the result of the Jasmine spec runner. If there are no specs (because for example a compilation error), then it will pass, because no tests have failed. If you wan't to see nice error messages, then configure Sprockets to raise errors and not to fail silently. The Guard::Jasmine PhantomJS runner waits for the spec results and if no results are returned within the configured timeout, then it simply returns the innerText of the body element, which is then outputted to the console as error message. |
|
Woah there. I'm trying to figure out how to improve the out-of-the-box experience of using your gem. I'm a fan and have blogged about it as being part of our standard toolkit. I'm not blaming guard-jasmine, just trying to find the best fix for it. Out of the box, there is no indicator that something went wrong when coffeescript compilation fails. For me and the folks over here, it's been a surprise and the source of confusion. We migrated from jasmine-headless-webkit, which reports asset compilation problems out of the box.
How do you configure sprockets to raise errors? I tried I'll try guard-coffeescript, but at first blush it sounds like a band aid since something else is already compiling assets and the output isn't being used for anything. I'd love to find an alternate solution. Thanks! |
|
I'm very sorry, my comment was not meant to be offensive in any way and I've noticed your support in the past and really appreciate it. Bare with me, I'm a non native speaker and perhaps the words I chose aren't always the best. I just do not understand the problem: We're talking about isolated JavaScript tests, so the Rails backend should not be touched and the only reason for having it run is to serve the assets and reuse the Sprockets manifest to avoid duplication (thus no need for e.g. a separated database). This means to me that we should let guard-jasmine start the server in
All of the above is not true in The command line tool and the Rake task are meant to be run on a CI server (or locally to reproduce errors or staging), where we want to have a production like environment and so we want to have our assets compiled, minified and concatenated. This is why the default environment is So what do you suggest? How should guard-jasmine handle the case, when Sprockets doesn't throw an exception when a compilation error happens and instead just skip the file in the resulting html? |
|
My apologies for jumping to the wrong conclusion there. Water under the bridge. I agree with everything you said re: test and development modes. And I see that guard-jasmine, when run from guard, is in development mode. I'm still seeing "0 specs" when there's a problem compiling assets. To be clear, are you suggesting that when running jasmine-guard from guard (in development mode), I should see some sort of error, rather than "0 specs"? Thanks! |
Absolutely! You do not see it? Wow, now I understand finally why we talk at cross purposes. I just did a check so see if it was a setting I changed in the past and forget about it or if this is Rails standard behavior. Create a new Rails app: otherland ~
» rails new testapp
create
create README.rdoc
create Rakefile
create config.ru
create .gitignore
create Gemfile
create app
create app/assets/images/rails.png
create app/assets/javascripts/application.js
create app/assets/stylesheets/application.css
create app/controllers/application_controller.rb
create app/helpers/application_helper.rb
create app/views/layouts/application.html.erb
create app/mailers/.gitkeep
create app/models/.gitkeep
create config
create config/routes.rb
create config/application.rb
create config/environment.rb
create config/environments
create config/environments/development.rb
create config/environments/production.rb
create config/environments/test.rb
create config/initializers
create config/initializers/backtrace_silencers.rb
create config/initializers/inflections.rb
create config/initializers/mime_types.rb
create config/initializers/secret_token.rb
create config/initializers/session_store.rb
create config/initializers/wrap_parameters.rb
create config/locales
create config/locales/en.yml
create config/boot.rb
create config/database.yml
create db
create db/seeds.rb
create doc
create doc/README_FOR_APP
create lib
create lib/tasks
create lib/tasks/.gitkeep
create lib/assets
create lib/assets/.gitkeep
create log
create log/.gitkeep
create public
create public/404.html
create public/422.html
create public/500.html
create public/favicon.ico
create public/index.html
create public/robots.txt
create script
create script/rails
create test/fixtures
create test/fixtures/.gitkeep
create test/functional
create test/functional/.gitkeep
create test/integration
create test/integration/.gitkeep
create test/unit
create test/unit/.gitkeep
create test/performance/browsing_test.rb
create test/test_helper.rb
create tmp/cache
create tmp/cache/assets
create vendor/assets/javascripts
create vendor/assets/javascripts/.gitkeep
create vendor/assets/stylesheets
create vendor/assets/stylesheets/.gitkeep
create vendor/plugins
create vendor/plugins/.gitkeep
run bundle install
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.6.1)
Using activesupport (3.2.12)
Using builder (3.0.4)
Using activemodel (3.2.12)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.5)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.12)
Using mime-types (1.21)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.12)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.12)
Using activeresource (3.2.12)
Using bundler (1.2.4)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.7)
Using rdoc (3.12.1)
Using thor (0.17.0)
Using railties (3.2.12)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.1)
Using rails (3.2.12)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
otherland ~
» cd testappAdd a bad CoffeeScript file that breaks: otherland ~/testapp
» echo "for k,v of window" > app/assets/javascripts/test.js.coffeeStart the server: otherland ~/testapp
» rails s -d
=> Booting WEBrick
=> Rails 3.2.12 application starting in development on http://0.0.0.0:3000Request the asset: otherland ~/testapp
» curl http://localhost:3000/assets/test.js
throw Error("ExecJS::ProgramError: Error: Parse error on line 1: Unexpected 'TERMINATOR'\n (in /Users/michi/testapp/app/assets/javascripts/test.js.coffee)")%
otherland ~/testapp
» Bam, an exception occurred when requesting the asset. So let's check Jasminerice next. Install Jasminerice: otherland ~/testapp
» echo "gem 'jasminerice'" >> Gemfile
otherland ~/testapp
» bundle
Resolving dependencies...
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.6.1)
Using activesupport (3.2.12)
Using builder (3.0.4)
Using activemodel (3.2.12)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.5)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.12)
Using mime-types (1.21)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.12)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.12)
Using activeresource (3.2.12)
Using bundler (1.2.4)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.7)
Using rdoc (3.12.1)
Using thor (0.17.0)
Using railties (3.2.12)
Using coffee-rails (3.2.2)
Using haml (4.0.0)
Using jasminerice (0.0.10)
Using jquery-rails (2.2.1)
Using rails (3.2.12)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.Generate the default Jasminerice files: otherland ~/testapp
» rails g jasminerice:install
create spec/javascripts/spec.js.coffee
create spec/javascripts/foo_spec.js.coffee
create spec/javascripts/spec.css
create spec/javascripts/fixtures/baz.html.hamlRemove unneeded spec: otherland ~/testapp
» rm spec/javascripts/foo_spec.js.coffeeInclude our bad CoffeeScript file: Restart Rails server: otherland ~/testapp
» kill -9 `pgrep ruby`
otherland ~/testapp
» rails s -d
=> Booting WEBrick
=> Rails 3.2.12 application starting in development on http://0.0.0.0:3000Get the Jasmine runner: otherland ~/testapp
» curl http://localhost:3000/jasmine
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>Action Controller: Exception caught</title>
<style>
<-- removed for the sake of brevity -->
</style>
</head>
<body>
<h1>
ExecJS::ProgramError in
Jasminerice/spec#index
</h1>
<p>
Showing <i>/Users/michi/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/jasminerice-0.0.10/app/views/jasminerice/spec/index.html.haml</i> where line <b>#7</b> raised:
<pre><code>Error: Parse error on line 1: Unexpected 'TERMINATOR'
(in /Users/michi/testapp/app/assets/javascripts/test.js.coffee)</code></pre>
</p>
<p>Extracted source (around line <b>#7</b>):
<pre><code>4: = stylesheet_link_tag "jasmine"
5: = stylesheet_link_tag "spec"
6: = javascript_include_tag "jasminerice"
7: = javascript_include_tag @specenv
8: = csrf_meta_tags
9: %body
</code></pre></p>
<p></p>
<p><code>Rails.root: /Users/michi/testapp</code></p>
<div id="traces">
<-- removed for the sake of brevity -->
</div>
<h2 style="margin-top: 30px">Request</h2>
<p><b>Parameters</b>: <pre>None</pre></p>
<p><a href="#" onclick="document.getElementById('session_dump').style.display='block'; return false;">Show session dump</a></p>
<div id="session_dump" style="display:none"><pre></pre></div>
<p><a href="#" onclick="document.getElementById('env_dump').style.display='block'; return false;">Show env dump</a></p>
<div id="env_dump" style="display:none"><pre>GATEWAY_INTERFACE: "CGI/1.1"
HTTP_ACCEPT: "*/*"
REMOTE_ADDR: "127.0.0.1"
REMOTE_HOST: "localhost"
SERVER_NAME: "localhost"
SERVER_PROTOCOL: "HTTP/1.1"</pre></div>
<h2 style="margin-top: 30px">Response</h2>
<p><b>Headers</b>: <pre>None</pre></p>
</body>
</html>Fine, the exception is still available in the Jasmine spec. So let's check guard-jasmine. Install guard-jasmine: otherland ~/testapp
» echo "gem 'guard-jasmine'" >> Gemfile
otherland ~/testapp
» echo "gem 'rb-fsevent'" >> Gemfile
otherland ~/testapp
» bundle
Resolving dependencies...
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.6.1)
Using activesupport (3.2.12)
Using builder (3.0.4)
Using activemodel (3.2.12)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.5)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.12)
Using mime-types (1.21)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.12)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.12)
Using activeresource (3.2.12)
Using bundler (1.2.4)
Using ffi (1.3.1)
Using childprocess (0.3.8)
Using coderay (1.0.8)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.7)
Using rdoc (3.12.1)
Using thor (0.17.0)
Using railties (3.2.12)
Using coffee-rails (3.2.2)
Using listen (0.7.2)
Using lumberjack (1.0.2)
Using method_source (0.8.1)
Using slop (3.4.3)
Using pry (0.9.12)
Using terminal-table (1.4.5)
Using guard (1.6.2)
Using guard-jasmine (1.12.2)
Using haml (4.0.0)
Using jasminerice (0.0.10)
Using jquery-rails (2.2.1)
Using rails (3.2.12)
Using rb-fsevent (0.9.3)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.Initialize Guard: otherland ~/testapp
» bundle exec guard init jasmine
21:33:49 - INFO - Writing new Guardfile to /Users/michi/testapp/Guardfile
21:33:49 - INFO - jasmine guard added to Guardfile, feel free to edit itAnd start it: otherland ~/testapp
» bundle exec guard 1.9.3-p327
21:36:36 - INFO - Guard uses TerminalTitle to send notifications.
21:36:37 - INFO - Guard::Jasmine starts webrick spec server on port 64477 in development environment (coverage off).
21:36:39 - INFO - Waiting for Jasmine test runner at http://localhost:64477/jasmine
21:36:39 - ERROR - Jasmine test runner fails with response code 500
21:36:39 - ERROR - <!DOCTYPE html>
> [#49A8F9C47DFB] <html lang="en">
> [#49A8F9C47DFB] <head>
> [#49A8F9C47DFB] <meta charset="utf-8" />
> [#49A8F9C47DFB] <title>Action Controller: Exception caught</title>
> [#49A8F9C47DFB] <style>
> [#49A8F9C47DFB] body { background-color: #fff; color: #333; }
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] body, p, ol, ul, td {
> [#49A8F9C47DFB] font-family: helvetica, verdana, arial, sans-serif;
> [#49A8F9C47DFB] font-size: 13px;
> [#49A8F9C47DFB] line-height: 18px;
> [#49A8F9C47DFB] }
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] pre {
> [#49A8F9C47DFB] background-color: #eee;
> [#49A8F9C47DFB] padding: 10px;
> [#49A8F9C47DFB] font-size: 11px;
> [#49A8F9C47DFB] white-space: pre-wrap;
> [#49A8F9C47DFB] }
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] a { color: #000; }
> [#49A8F9C47DFB] a:visited { color: #666; }
> [#49A8F9C47DFB] a:hover { color: #fff; background-color:#000; }
> [#49A8F9C47DFB] </style>
> [#49A8F9C47DFB] </head>
> [#49A8F9C47DFB] <body>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <h1>
> [#49A8F9C47DFB] ExecJS::ProgramError in
> [#49A8F9C47DFB] Jasminerice/spec#index
> [#49A8F9C47DFB] </h1>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <p>
> [#49A8F9C47DFB] Showing <i>/Users/michi/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/jasminerice-0.0.10/app/views/jasminerice/spec/index.html.haml</i> where line <b>#7</b> raised:
> [#49A8F9C47DFB] <pre><code>Error: Parse error on line 1: Unexpected 'TERMINATOR'
> [#49A8F9C47DFB] (in /Users/michi/testapp/app/assets/javascripts/test.js.coffee)</code></pre>
> [#49A8F9C47DFB] </p>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <p>Extracted source (around line <b>#7</b>):
> [#49A8F9C47DFB] <pre><code>4: = stylesheet_link_tag "jasmine"
> [#49A8F9C47DFB] 5: = stylesheet_link_tag "spec"
> [#49A8F9C47DFB] 6: = javascript_include_tag "jasminerice"
> [#49A8F9C47DFB] 7: = javascript_include_tag @specenv
> [#49A8F9C47DFB] 8: = csrf_meta_tags
> [#49A8F9C47DFB] 9: %body
> [#49A8F9C47DFB] </code></pre></p>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <p></p>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <p><code>Rails.root: /Users/michi/testapp</code></p>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <div id="traces">
[REMOVED]
> [#49A8F9C47DFB] </div>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB]
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <h2 style="margin-top: 30px">Request</h2>
> [#49A8F9C47DFB] <p><b>Parameters</b>: <pre>None</pre></p>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <p><a href="#" onclick="document.getElementById('session_dump').style.display='block'; return false;">Show session dump</a></p>
> [#49A8F9C47DFB] <div id="session_dump" style="display:none"><pre></pre></div>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <p><a href="#" onclick="document.getElementById('env_dump').style.display='block'; return false;">Show env dump</a></p>
> [#49A8F9C47DFB] <div id="env_dump" style="display:none"><pre>GATEWAY_INTERFACE: "CGI/1.1"
> [#49A8F9C47DFB] HTTP_ACCEPT: "*/*"
> [#49A8F9C47DFB] REMOTE_ADDR: "127.0.0.1"
> [#49A8F9C47DFB] REMOTE_HOST: "localhost"
> [#49A8F9C47DFB] SERVER_NAME: "localhost"
> [#49A8F9C47DFB] SERVER_PROTOCOL: "HTTP/1.1"</pre></div>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] <h2 style="margin-top: 30px">Response</h2>
> [#49A8F9C47DFB] <p><b>Headers</b>: <pre>None</pre></p>
> [#49A8F9C47DFB]
> [#49A8F9C47DFB]
> [#49A8F9C47DFB]
> [#49A8F9C47DFB] </body>
> [#49A8F9C47DFB] </html>
> [#49A8F9C47DFB]
21:36:39 - INFO - Guard is now watching at '/Users/michi/testapp'
[1] guard(main)> Still fine, the exception is shown in the console (ok, the stack trace is completely unnecessary, but that's another topic). Now let's try to run a spec that passes because it has no failing expectation: [1] guard(main)> .touch spec/javascripts/test_spec.js.coffee
21:39:55 - INFO - Run Jasmine suite spec/javascripts/test_spec.js.coffee
21:39:55 - INFO - Run Jasmine suite at http://localhost:64477/jasmine
21:40:56 - ERROR - An error occurred: Timeout waiting for the Jasmine test results!
> [#]
> [#] ExecJS::ProgramError in Jasminerice/spec#index
> [#]
> [#] Showing /Users/michi/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/jasminerice-0.0.10/app/views/jasminerice/spec/index.html.haml where line #7 raised:
> [#]
> [#] Error: Parse error on line 1: Unexpected 'TERMINATOR'
> [#] (in /Users/michi/testapp/app/assets/javascripts/test.js.coffee)
> [#] Extracted source (around line #7):
> [#]
> [#] 4: = stylesheet_link_tag "jasmine"
> [#] 5: = stylesheet_link_tag "spec"
> [#] 6: = javascript_include_tag "jasminerice"
> [#] 7: = javascript_include_tag @specenv
> [#] 8: = csrf_meta_tags
> [#] 9: %body
> [#] Rails.root: /Users/michi/testapp
> [#]
> [#] Application Trace | Framework Trace | Full Trace
> [#] Request
> [#]
> [#] Parameters:
> [#]
> [#] None
> [#] Show session dump
> [#]
> [#] Show env dump
> [#]
> [#] Response
> [#]
> [#] Headers:
> [#]
> [#] None
> [#]
[1] guard(main)> Great, I see the exception. The bad thing is that it takes 60 until the error appears, that's not good. To summarize:
I cannot decrease the timeout, because then I get more issues reported here when people run guard-jasmine against an app with an initial asset compilation time over that timeout 🌴 Perhaps just check if the jasmine/jasminerice assets have been declared in a script tag. |
The Rails error page is too large to print to the console, just show a message that more diagnosis information is available at the runner url, which was printed to the console one line above.
This adds a check to see if Jasmine is available by looking at `window.jasmine`. If so, the normal waiting for the spec results happens until the given timeout is reached. If Jasmine is missing, then it's very likely that an exception is show by the Rails error page (which of course doesn't include Jasmine). In that case fail the specs with the body text as error message, so that diagnosis directly in the console is possible.
|
1.13.0 is out of the door that addresses the two identified issues. Let me know how it goes... |
|
This is great. Can't wait to try it in our apps. |
|
@netzpirat Thanks for working ont this so quickly and sorry I haven't had time to test. I will tomorrow. |
When there's a coffeescript syntax error in a spec, guard-jasmine runs and reports 0 specs with a success exit code. This is confusing during development and on a build server looks like a successful build (false positive).
The rake task, guard, and guard-jasmine all do the same thing: 0 specs, PASS.
Visiting http://0.0.0.0:3000/jasmine produces an empty page. This gives you the sense that's something wrong, but it's still not very clear.
Ideally, any error in asset compilation would bubble up and cause a failure when running js specs.
Is there something guard-jasmine can do to highlight the problem or do you think it's the responsibility of something else?
The text was updated successfully, but these errors were encountered: