Support np.testing.assert_array_max_ulp

[mrahtz]

When working with bfloat16, given how variable the precision is, it would be super useful to be able to make assertions based on the number of ULPs between numbers rather than traditional measures of tolerance like atol/rtol.

However, np.testing.assert_array_max_ulp does not currently work with the bfloat16 type from ml_dtypes:

>>> a = np.array(0.0, dtype=ml_dtypes.bfloat16)
>>> np.testing.assert_array_max_ulp(a, a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "numpy/testing/_private/utils.py", line 1638, in assert_array_max_ulp
    ret = nulp_diff(a, b, dtype)
  File "numpy/testing/_private/utils.py", line 1686, in nulp_diff
    t = np.common_type(x, y)
  File "<__array_function__ internals>", line 180, in common_type
  File "numpy/lib/type_check.py", line 730, in common_type
    raise TypeError("can't get common type for non-numeric array")
TypeError: can't get common type for non-numeric array

Given how ml_dtypes integrates with NumPy/the existence of NumPy's own float16 dtype, I'm not sure whether this is possible - but I'll open this issue on the off chance that it might be :)

Thanks!

[jakevdp]

Thanks for the request – it looks like numpy would have to provide some sort of hook in order for us to do this: the supported dtypes are hard-coded here: https://github.com/numpy/numpy/blob/073d1f39f6ed2735a7fca029a34648c258a80550/numpy/lib/type_check.py#L669

Similar to ml_dtypes.finfo, we could provide a version of this and related functions that would support the dtype extensions here, or we could advocate for numpy to make its code more extensible.