CTest follow-up: expand range of tolerances used for checksum benchmarks

[EZoni]

Expand on what is described in #5205, expose more tolerances:

• single-precision tolerance
• double-precision loose tolerance (for hardware portability)
• double-precision strict tolerance (for same-machine benchmarks)

[EZoni]

Copied from #5205 (comment) because it is relevant here as well:

To generalize the analysis scripts, we could change the APIs of the tolerance argument to the checksum API, and pass a dictionary (double and single as keys) for the tolerances. Or have two arguments, one double tolerance one float (with a dictionary, we keep it open to have other types later, but I do not think this is timely). Inside the checksum API, we could do a test for WarpX config used or type of a data file entry read to determine if something was a SP or DP run.

[EZoni]

@ax3l

I guess I would go for multiple arguments, say

• rtol_single_portable (default: ...)
• rtol_double_portable (default: ...)
• rtol_double_machine_precision (default ...)

because I don't see a good way to handle defaults if we pass a dictionary.

For example, with multiple arguments, we could do something like

checksumAPI.evaluate_checksum(testname, filename, rtol_double_portable=1e-3)

whenever a test needs to pass a larger "double-portable" tolerance for whatever reason, without having to worry about the defaults for rtol_single_portable and rtol_double_machine_precision, if they don't need adjustments.

On the other end, how would we do this if we have to pass a dictionary? Something like

checksumAPI.evaluate_checksum(testname, filename, tol_dict={"single_portable": None, "double_portable": 1e-3, "double_machine_precision": None})

and then having, inside checksumAPI, the logic that uses defaults if None is passed, maybe?

I feel like the first option could be easier to use.