Basis.toAxisAngle return NaN as angle

[fabriceci]

Godot version

4.0.dev #b3df275

System information

MacOS 11.6

Issue description

When the rotation angle is very small, the angle returned is NaN.

Steps to reproduce

In c++:

Basis bugNan(1.00000024, 0 , 0.000100001693, 0, 1, 0, -0.000100009143, 0, 1.00000024);
bugNan.get_axis_angle(axis, angle);
print_line(angle) // NaN

Minimal reproduction project

No response

[fire]

@TokageItLab did the quaternion version of this.

[TokageItLab]

Yeah I recall that I did some changes such as to return an initial Quaternion Quaternion(0, 0, 0, 1) when AxisAngle is very small in Quaternion::exp() and add angle checking Quaternion::get_axis() in #57675. I guess that some math checking is lacking in Basis.

[fabriceci]

I found the issue in Basis, I will open a PR. The arcos() function receive a value greater than one, so it returns NaN, I clamped the value to avoid the issue. I was wondering if we could put a debug code to find this kind of problem for arcos/arcsin.