Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract message from the test error and print this instead of truncating the whole error object #4465

Closed
4 tasks done
ben-eb opened this issue Nov 10, 2023 · 2 comments · Fixed by #4467
Closed
4 tasks done
Labels
enhancement New feature or request p2-nice-to-have Not breaking anything but nice to have (priority)

Comments

@ben-eb
Copy link

ben-eb commented Nov 10, 2023

Clear and concise description of the problem

Hello 👋

Many thanks for your work on vitest so far. I would like to request that errors printed by vitest are handled in a similar way to Jest, where the message is printed in red and the rest of the object omitted.

When certain debug flags are provided to the tests I would like to throw a descriptive error message in case they fail validation; and then depending on the result of the debug flag I can change how my internal logic behaves. This is to enable things like using a mock server vs calling the network whilst using the same test definitions.

In order to get this behaviour I am applying the following workaround:

import { describe, expect, it } from 'vitest';

async function debug() {
    const trueValues = ['y', 'yes', 't', 'true', '1'];
    const falseValues = ['n', 'no', 'f', 'false', '0'];

    const allValues = [...trueValues, ...falseValues];

    const { DEBUG } = process.env;

    if (!DEBUG) {
        return true;
    }

    if (!allValues.includes(DEBUG)) {
        throw new Error(
            `Expected process.env.DEBUG to be one of ${allValues.join(
                ', ',
            )} but got "${DEBUG}" instead`,
        );
    }

    return trueValues.includes(DEBUG);
}

describe('debug', () => {
    it('does not truncate the error message using extra boilerplate', async () => {
        try {
            const result = await debug();
            expect(result).toEqual(true);
        } catch (error) {
            expect.fail(error.message);
        }
    });

    it('truncates the error message otherwise', async () => {
        await expect(debug()).resolves.toEqual(true);
    });
});

Run this with DEBUG=never vitest run in order to print the following:

 ❯ src/truncate.test.ts (2)
   ❯ debug (2)
     × does not truncate the error message using extra boilerplate
     × truncates the error message otherwise

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

 FAIL  src/truncate.test.ts > debug > does not truncate the error message using extra boilerplate
AssertionError: Expected process.env.DEBUG to be one of y, yes, t, true, 1, n, no, f, false, 0 but got "never" instead
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/2]⎯

 FAIL  src/truncate.test.ts > debug > truncates the error message otherwise
AssertionError: promise rejected "Error: Expected process.env.DEBUG to be o…" instead of resolving
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/2]⎯

 Test Files  1 failed (1)
      Tests  2 failed (2)

Suggested solution

Contrast this to Jest; when we run the second test we get the following output:

  debug
    ✕ truncates the error message otherwise (3 ms)

  ● debug › truncates the error message otherwise

    expect(received).resolves.toEqual()

    Received promise rejected instead of resolved
    Rejected to value: [Error: Expected process.env.DEBUG to be one of y, yes, t, true, 1, n, no, f, false, 0 but got "never" instead]

      35 |
      36 |     it('truncates the error message otherwise', async () => {
    > 37 |         await expect(debug()).resolves.toEqual(true);
         |               ^
      38 |     });
      39 | });
      40 |

      at expect (node_modules/expect/build/index.js:113:15)
      at Object.<anonymous> (src/truncate.test.ts:37:15)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total

Alternative

I am using this workaround:

try {
    const result = await debug();
    expect(result).toEqual(true);
} catch (error) {
    expect.fail(error.message);
}

This works but it isn't as nice. 😅

Additional context

I came across another workaround:

import {defineConfig} from "vitest/config";

export default defineConfig({
    test: {
        chaiConfig: {
            truncateThreshold: 120
        }
    }
});

When using this I get the following:

 FAIL  src/truncate.test.ts > debug > truncates the error message otherwise
AssertionError: promise rejected "Error: Expected process.env.DEBUG to be one of y, yes, t, true, 1, n, no, f, false, 0 but got "never" instead" instead of resolving

The problem I have with this is that it sets the threshold for all serialized objects, so if I want to increase the limit (for other purposes) I will get a lot of additional output for error objects too.

Validations

@sheremet-va
Copy link
Member

Really funny coincidence, but I had the exact same idea yesterday. PR should be ready soon.

@sheremet-va sheremet-va added enhancement New feature or request p2-nice-to-have Not breaking anything but nice to have (priority) labels Nov 10, 2023
@sheremet-va
Copy link
Member

My current idea is to use cause property. It would looks something like this:

Screenshot 2023-11-10 at 14 01 35

If promise has extra properties, they will also be displayed

@github-actions github-actions bot locked and limited conversation to collaborators Nov 25, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request p2-nice-to-have Not breaking anything but nice to have (priority)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants