Unittest tolerance handling improvements

[DickJC123]

Description

This PR moves to consolidate and standardize the way our data comparison routines in test_utils.py handle tolerances. Over time, the number of these routines has grown and the approaches have diverged. For example, two of the most frequently used routines are:

def assert_almost_equal(a, b, rtol=None, atol=None, …):
    rtol = get_rtol(rtol)
    atol = get_atol(atol)
    ….

and

def check_consistency(sym, …,  tol=None, … ):
    if tol is None:
        tol = {np.dtype(np.float16): 1e-1,
               np.dtype(np.float32): 1e-3,
               np.dtype(np.float64): 1e-5,
               …}
…

As show, assert_almost_equal() offers separate arguments for specifying relative and absolute tolerances (by default rtol, atol = 1e-5, 1e-20), but these do not vary with dtype. In contrast, check_consistency() only has one tol argument, which it uses for both atol and rtol, but the tol default is dtype-dependent.

This PR unifies these approaches by having both check_consistency() and assert_almost_equal() support rtol and atol, drawing upon the same set of dtype-dependent defaults. The goal here is for the testing framework to make sensible tolerance choices for our unittest data comparisons when possible, rather than burden each test developer to do so. In effect, this PR addresses the current codebase TODO in test_utils.py:

def get_rtol(rtol=None):
    """Get default numerical threshold for regression test."""
    # _TODO: get from env variable, different threshold might
    # be needed for different device and dtype
    return 1e-5 if rtol is None else rtol

While I believe setting tolerances via env var is not appropriate, test tolerances should be a function of the dtype and device (i.e. context). A first application for this concept is in support for the newly announced A100 GPU with its TensorFloat-32 (TF32) mode of computation. The A100 will by default round the mantissa of float32 GEMM and Convolution inputs to a float16 precision. Therefore, although operator i/o’s might be float32, the “effective dtype” for the purposes of tolerance selection is float16. By consolidating the tolerance handling logic, this PR can now easily incorporate an effective_dtype(dat) routine based on context, so that unittests can run 32-bit models and seamlessly apply the appropriate tolerances for each context, be it A100 or non-A100.

This PR was developed on an in-house CI system that runs on many GPU architectures, including the A100. As part of the PR development, some unittests were modified so that now all tests pass on the A100. These tests had typically explicitly hard-coded a tolerance appropriate for a float32 test. The modification for those tests involved dropping the fixed tolerance, allowing this PR’s adaptive context- and dtype-dependent tolerance selection logic to take over.

Thanks to @Kh4L for supplying the GEMM-flag adaptations for TF32 that are part of this PR.
Thanks also to @drivanov for his prior efforts to adapt assert_almost_equal() to work directly on ndarrays, thereby enabling context-dependent default tolerance selection.

@ptrendx @eric-haibin-lin @marcoabreu

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

• [X ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
• [ X] Changes are complete (i.e. I finished coding on this PR)
• [ X] All changes have test coverage:
• Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
• Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
• Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
• [ X] Code is well-documented:
• For user-facing API changes, API doc string has been updated.
• For new C++ functions in header files, their functionalities and arguments are documented.
• For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
• Check the API doc at https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
• [X ] To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

• Feature1, tests, (and when applicable, API doc)
• Feature2, tests, (and when applicable, API doc)

Comments

• If this change is a backward incompatible change, why must this change be made.
• Interesting edge cases to note here

[xidulu]

LGTM regarding the changes to test_gluon_probability

[szha]

@DickJC123 thanks for the change and for fixing the issues you found along the way. Remember to mark the issues this PR resolve in the description.

[szha]

Thanks for the fixes, @DickJC123. They are really helpful. I did a code review and examined the analysis included in each of them as well as the specific fixes.

[DickJC123]

Thanks @szha! You probably saw that I've been struggling to get a passing CI- running into and fixing many issues unrelated to my PR along the way.