[Feature]: Custom AsymmetricMatchers

[amberUCP]

🚀 Feature Request

The current custom matcher extend function does not seem to allow custom asymmetric matchers.

E.g. fixtures.ts

import { expect as baseExpect } from '@playwright/test';
export { test } from '@playwright/test';

export const expect =  baseExpect.extend({
  async toBeNumberOrNull(received) {
    const assertionName = `toBeNullOrType`;
    let pass: boolean;
    let matcherResult: any;
    const expected = 'Null or Number';
    let isNumber = false;
    let isNull = false;

    try {
      await baseExpect(received).toEqual(baseExpect.any(Number));
      
      isNumber = true;
    } catch (e: any) {
      matcherResult = e.matcherResult;
      isNumber = false;
    }

    try {
      await baseExpect(received).toEqual(null);
      isNull = true;
    } catch (e: any) {
      matcherResult = e.matcherResult;
      isNull = false;
    }
    
    pass = isNull || isNumber;

    const message = pass
      ? () => this.utils.matcherHint(assertionName, undefined, undefined, { isNot: this.isNot }) +
          '\n\n' +
          `Expected: ${this.isNot ? 'not' : ''}${this.utils.printExpected(expected)}\n` +
          (matcherResult ? `Received: ${this.utils.printReceived(matcherResult.actual)}` : '')
      : () =>  this.utils.matcherHint(assertionName, undefined, undefined, { isNot: this.isNot }) +
          '\n\n' +
          `Expected: ${this.utils.printExpected(expected)}\n` +
          (matcherResult ? `Received: ${this.utils.printReceived(matcherResult.actual)}` : '');

    return {
      message,
      pass,
      name: assertionName,
      expected,
      actual: matcherResult?.actual,
    };
  }
});

This works when writing an symmetric assertion such as:

  test('Test matcher', async () => {
   await expect(1).toBeNumberOrNull();
  });

This does not work when writing an asymmetric assertion such as:

  test('Gets order', async ({ request }) => {
    const response = await request.get('/order');

    expect(response.status()).toBe(200);

    const response = await response.json();

    expect(response.data).toMatchObject({
      id: expect.any(String),
      name: expect.any(String),
      item: {
        id: expect.any(Number),
        price: expect.any(Number),
        discount: expect.toBeNumberOrNull(), // this custom asymmetric matcher is not currently supported
      },
    });
  });

Example

No response

Motivation

This will allow custom matchers to be used asymmetrically, e.g. where expected values could be a few different values such as the example above.

[muhqu]

Technically asymmetric matchers are already supported, it's just that their type definitions are not correctly retained. You can work around the issue by explicitly adding type casts on your export const expect.

Your example:

import { expect as baseExpect } from '@playwright/test'

type ValueOf<T> = T[keyof T]

// Playwright does not export the type of the return value of the custom matcher
type MatcherReturnType = Exclude<ReturnType<ValueOf<Parameters<typeof baseExpect.extend>[0]>>, Promise<unknown>>

const toBeNumberOrNull = (received: unknown): MatcherReturnType => {
  const pass = typeof received === 'number' || received === null
  const message = pass ? () => `Expected '${received}' not to be a number or null` : () => `Expected ${received} to be a number or null`
  return { pass, message, actual: received }
}

const customMatchers = {
  /**
   * Check if the received value is a `Number` or `null`.
   */
  toBeNumberOrNull,

  // for better readability when using as asymmetric matcher
  numberOrNull: toBeNumberOrNull,
}

const customExpect = baseExpect.extend(customMatchers)

type OmitFirstArg<F> = F extends (x: any, ...args: infer P) => infer R ? (...args: P) => R : never
type AsymmetricMatcher = Record<string, unknown>
type CustomMatchers = {
  [K in keyof typeof customMatchers]: K extends string
    ? OmitFirstArg<(...args: Parameters<(typeof customMatchers)[K]>) => AsymmetricMatcher>
    : never
}

// Playwright expect.extend() misses the type info of custom asymmetric matchers,
// so we need to explicitly cast it here...
export const expect = customExpect as typeof customExpect & CustomMatchers

Usage:

expect(42).toBeNumberOrNull() // OK
expect(null).toBeNumberOrNull() // OK
expect({ foo: 42, bar: null }).toEqual(expect.objectContaining({
    foo: expect.numberOrNull(),
})) // OK

[muhqu]

The above workaround works until playwright 1.47.1. In latest main it does no longer work due to the recently merged #32366 which changes how expect.extend(…) is implemented. It's now fixed again in main via #32795 . 🎉

However, I've created #32740 which should make the above workaround unnecessary.

[muhqu]

@dgozman as you mentioned on #32740 you…

…are not sure we'd like to keep this API for asymmetric matchers going forward.

Can you give an example why this API for asymmetric matchers is not desirable? I like to better understand your concerns. IMHO, extensibility is one of the great features of playwright.

[pavelfeldman]

Can you give an example why this API for asymmetric matchers is not desirable? I like to better understand your concerns. IMHO, extensibility is one of the great features of playwright.

Could you give us a couple of use cases where they would make most sense to you? We'd like to make sure there are no reasonable alternatives that would not involve asymmetric matchers.