math: fix big.int div, gcd and bitAnd edge-cases

[tiehuis]

Fixes #10932.

Disclaimer: The divmod change was fairly quick and I've yet to go through the algorithm again fully with the change in mind. The gcd/bitAnd change is fairly trivial.

[tiehuis]

To elaborate on the issues:

• the gcd issue occurred when the gcd result exceeds a single limb and is reduced entirely in the main loop and not the single-limb processing at the end
• the bitAnd was an incorrect assert which was being triggered when the limb_count(a) > limb_count(b) where a > b
• the divmod issue occurs when a intermediate step results in a working value x which is normalized to 0, it's been a while since I reviewed this algorithm so no further comment yet