feat: add console logs on test failure

[yannbf]

Issue: #155

For now, the feature is always enabled. It only provides the logs when there are failures.

Here's a code example:

export const Logs = (args) => {
  console.log("here's an object", { hello: "world" })
  console.warn("The API is deprecated!")
  console.error("Some error happened.")

  return (
    <button type="button" onClick={() => args.onSubmit('clicked')}>
      Click
    </button>
  )
};
Logs.play = async ({ args, canvasElement }) => {
  console.log("log from play function")
  await userEvent.click(within(canvasElement).getByRole('buttons')); // <-- will fail
  await expect(args.onSubmit).toHaveBeenCalledWith(expect.stringMatching(/([A-Z])\w+/gi));
};

And here's the output:

Here's how it looks like on a more realistic scenario:

📦 Published PR as canary version: 0.6.0--canary.157.66af119.0

✨ Test out this PR locally via:

npm install @storybook/test-runner@0.6.0--canary.157.66af119.0
# or 
yarn add @storybook/test-runner@0.6.0--canary.157.66af119.0

Release notes

Features

feat: add console logs on test failure
This version adds console logs to the output of failed tests, in order to as provide as much information as possible to help you diagnose your issues.

Version

Published prerelease version: v0.6.0-next.1

Changelog

Release Notes

feat: add console logs on test failure (#157)

Features

feat: add console logs on test failure
This version adds console logs to the output of failed tests, in order to as provide as much information as possible to help you diagnose your issues.

feat: jest 28 support (#154)

Features

This release updates jest-playwright to version 2.0.0 which adds support for Jest 28. In order to maintain backwards compatibility with Jest 27, you might have to take a few steps in case you are installing the test runner for the first time, or if you don't keep package locks in your project.

You can find more info at https://github.com/storybookjs/test-runner#jest-27-support

---

🚀 Enhancement

• feat: add console logs on test failure #157 (@yannbf)
• feat: jest 28 support #154 (@andykenward @yannbf)

Authors: 2

• Andy Kenward (@andykenward)
• Yann Braga (@yannbf)

[codecov-commenter]

Codecov Report

Merging #157 (51fa138) into next (91cf672) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             next     #157   +/-   ##
=======================================
  Coverage   77.52%   77.52%           
=======================================
  Files          11       11           
  Lines         178      178           
  Branches       38       38           
=======================================
  Hits          138      138           
  Misses         40       40           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 91cf672...51fa138. Read the comment docs.

[IanVS]

Hi, I tested this out and overall I'm very 👍 on the change. I like that it happens automatically for failing tests, but passing tests do not add logs to create noise.

I did notice two things, neither of which I see as dealbreakers.

1. The originalFn.name doesn't seem to work in the production version, instead, I see:

    Browser logs:

    : this is a log
    : this is a warn
    : this is a error

2. Console groups are not printed correctly. For instance, here's what I see from mirage in one of my stories:

But in the test output I get:

Browser logs:

    : [object Object]
    : null
    : [object Object]
    : [object Object]
    : null
    : [object Object]

I played around a bit, and this is what I came up with to fix both issues:

        const spyOnConsole = (originalFn, name) => {
          return function() {
            const [txt, ...args] = arguments;
            logs.push(name + ": " + ((typeof txt === 'object') ? JSON.stringify(txt, null, 2) : txt ) + (args.length > 0 ? " " + JSON.stringify(args) : ""));
            originalFn.apply(console, arguments);
          }
        }
        console.log = spyOnConsole(console.log, 'log');
        console.warn = spyOnConsole(console.warn, 'warn');
        console.error = spyOnConsole(console.error, 'error');

Maybe there's a better way, but this at least gives me all the logs, which can definitely be useful I think when debugging a failed test.

[yannbf]

Thanks for the ideas @IanVS! the logging is much improved now thanks to your feedback <3

[IanVS]

I tested out the canary in my app and it looks great!

[github-actions]

🚀 PR was released in v0.6.0 🚀