-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decimal to double conversion loses precision #72125
Comments
Tagging subscribers to this area: @dotnet/area-system-numerics Issue DetailsDescriptionThe current algorithm for casting a decimal to a double loses precision. Reproduction Steps
Expected behaviorThe above three prints should print the same value. Actual behaviorThere is a precision loss when casting from decimal to double. I included two methods of constructing the decimal to demonstrate that the behavior is specifically tied to the decimal -> double conversion, and not the construction of the decimal. Regression?No response Known WorkaroundsNo response ConfigurationNo response Other informationNo response
|
At present we mimic behavior of runtime/src/libraries/System.Private.CoreLib/src/System/Decimal.DecCalc.cs Lines 1870 to 1882 in 7f9ed8f
|
Yes, I believe that's the actual conversion code. @tannergooding and I were brainstorming some ways to improve its precision. We think a method in which we split the decimal into a fractional and integral part using Truncate, translate each mantissa to a double, process the fractional part, and then combine the two parts could be the correct solution. |
Right. For values above 2^53, we already have a valid (and efficient) Since we have 96-bits of significand precision in There are other ways to do this as well and the "correct" (but slow) way is |
Description
The current algorithm for casting a decimal to a double loses precision.
Reproduction Steps
Example
Expected behavior
The above three prints should print the same value.
Actual behavior
There is a precision loss when casting from decimal to double. I included two methods of constructing the decimal to demonstrate that the behavior is specifically tied to the decimal -> double conversion, and not the construction of the decimal.
The text was updated successfully, but these errors were encountered: