Error "TypeError: Cannot read property 'suite' of undefined" when using multiple frameworks

[caffed]

Hi there. Just want to first thank you for putting the time in to write this.

I am getting this error when I include ['jasmine', 'benchmark'] as frameworks for Karma:

ERROR [karma]: [TypeError: Cannot read property 'suite' of undefined]
TypeError: Cannot read property 'suite' of undefined
  at [object Object].specSuccess (/Users/user/projects/app/node_modules/karma-benchmark-reporter/index.js:35:33)
  at [object Object].onSpecComplete (/Users/user/projects/app/node_modules/karma/lib/reporters/base.js:83:12)
  at [object Object].<anonymous> (/Users/user/projects/app/node_modules/karma/lib/events.js:15:22)
  at [object Object].emit (events.js:98:17)
  at [object Object].onResult (/Users/user/projects/app/node_modules/karma/lib/browser.js:219:13)
  at Socket.<anonymous> (/Users/user/projects/app/node_modules/karma/lib/events.js:15:22)
  at Socket.emit [as $emit] (events.js:117:20)
  at SocketNamespace.handlePacket (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/namespace.js:335:22)
  at Manager.onClientMessage (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/manager.js:488:38)
  at WebSocket.Transport.onMessage (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transport.js:387:20)
  at Parser.<anonymous> (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transports/websocket/default.js:36:10)
  at Parser.emit (events.js:95:17)
  at Parser.parse (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transports/websocket/default.js:343:12)
  at Parser.parse (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transports/websocket/default.js:346:19)
  at Parser.parse (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transports/websocket/default.js:346:19)
  at Parser.parse (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transports/websocket/default.js:346:19)
  at Parser.add (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transports/websocket/default.js:315:8)
  at Socket.<anonymous> (/Users/user/projects/app/node_modules/karma/node_modules/socket.io/lib/transports/websocket/default.js:169:17)
  at Socket.emit (events.js:117:20)
  at Socket.<anonymous> (_stream_readable.js:765:14)
  at Socket.emit (events.js:92:17)
  at emitReadable_ (_stream_readable.js:427:10)
  at emitReadable (_stream_readable.js:423:5)
  at readableAddChunk (_stream_readable.js:166:9)
  at Socket.Readable.push (_stream_readable.js:128:10)
  at TCP.onread (net.js:529:21)

I logged the result being passed.
code karma-benchmark-reporter/index.js:33-36:

  this.specSuccess = function(browser, result) {
    var browser = browser.name;
    var suite = result.benchmark.suite; // fails here
    var name = result.benchmark.name;

result:

{ description: 'is defined',
  id: 'spec0',
  log: [],
  skipped: false,
  success: true,
  suite: [ 'App' ],
  time: 2 }

It looks like result.benchmark does not exist.
Is this expected behavior?

I have just created a separate karma.conf.js file for BenchmarkJS in leu of this error.

[gadicc]

+1

@caffed, did you ever get this working with a single karma.conf.js? I'm also using jspm as a framework, for module loading, so this work around won't help me unfortunately :(

[gadicc]

Oops, that's our bad. Karma supports multiple frameworks but not multiple testing frameworks [1] :> Using jspm + benchmark in a separate config works just fine.

I guess this can be closed, maybe a note in the README would be good (I'm sure @JamieMason mason would welcome a PR on that).

[1] https://groups.google.com/forum/#!topic/karma-users/SRNwt6HwhBs

[mitermayer]

same problem

[JamieMason]

Thanks @mitermayer, @gadicc, @caffed. Sorry for such a delay in replying.

It's been a long while since I've worked on karma-benchmark, but once I've finished what I'm doing on shrinkpack and ImageOptim-CLI I plan to pick this back up again in a couple of months.

[JamieMason]

FYI 0.6.0 has been released.

If I understand correctly, and also noticing @gadicc's #7 (comment) – I wouldn't mix performance benchmarks and unit tests in the same job as the latter would likely affect your benchmarks and possibly make them inaccurate.

I will close based on this but please reopen if I've misunderstood.