Reject the underlying promise in the Module API when Cypress fails to run the tests

[brian-mann]

Currently if Cypress errors out and does not run the test (which can happen due to misconfiguration, wrong arguments, failing to record, etc) it does not preserve the error - nor does it make it accessible via the module API.

Currently when using the module API and an error occurs, we have fallback code in the CLI here: https://github.com/cypress-io/cypress/blob/master/cli%2Flib%2Fcypress.js#L31 which resolves with a pretty useless object.

Instead, we should actually reject the underlying promise and rehydrate the error instance that caused Cypress to abandon running the tests.

Here's an issue for reference: #4021

[jennifer-shehane]

Wanted to add the example of what a workaround for this looks like today.

Some idea of kind of how the code may look as originally posted by @brian-mann here: #4021 (comment)

const cypress = require('cypress')

const didRunError = (results) => {
  return results.message && results.message.includes('Could not find Cypress test run results')
}

// run using our module API
cypress.run()
.then((results = {}) => {
  if (didRunError(results)) {
    // do your fallback behavior here
  }
})

[Nomeasmo]

I think the above workaround does not

Instead, we should actually reject the underlying promise and rehydrate the error instance that caused Cypress to abandon running the tests.

[jennifer-shehane]

The Module API does now have an error accessible when it fails to run. See https://on.cypress.io/module-api#Handling-errors