Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: add getPrecision method. #58

Open
bendavies opened this issue May 19, 2021 · 11 comments
Open

RFC: add getPrecision method. #58

bendavies opened this issue May 19, 2021 · 11 comments

Comments

@bendavies
Copy link

Hi,

Similar to getScale, would you be on favor of adding a method getPrecision?
I want to perform app side validation on BigDecimal before passing to Postgres numeric fields, and getPrecision would make that easier.

Thanks!

@BenMorel
Copy link
Member

Hi, what would be the difference between getScale() and getPrecision()?

@bendavies
Copy link
Author

hi @BenMorel

The PostgreSQL manual will say it better than me:

We use the following terms below: The precision of a numeric is the total count of significant digits in the whole number, that is, the number of digits to both sides of the decimal point. The scale of a numeric is the count of decimal digits in the fractional part, to the right of the decimal point. So the number 23.5141 has a precision of 6 and a scale of 4. Integers can be considered to have a scale of zero.

@bendavies
Copy link
Author

@BenMorel any interest? thanks!

@alexviar
Copy link

alexviar commented Jul 30, 2022

@BenMorel

Working with precision is generally better than working with scale

Check this example:

(0.03/360)*8*30 = 0.02

Woking with scale of 10:
0.03/360 = 0.0000833333(3)
0.0000833333*8=0.0006666664(0)
0.0006666664*30=0.0199999920(0)

❌ 0.0199999920 == 0.02

Working with precision of 10:
0.03/360 = 0.00008333333333(3)
0.00008333333333*8=0.0006666666666(40)
0.0006666666666*30=0.01999999999(80)=0.02

✔0.02==0.02

@bendavies
Copy link
Author

@alexviar i'm not sure what your example shows but it's not precision of 10

Working with precision of 10:
0.03/360 = 0.00008333333333(3)

0.00008333333333(3) has as precision of 16

@alexviar
Copy link

alexviar commented Aug 3, 2022

@alexviar i'm not sure what your example shows but it's not precision of 10

Working with precision of 10:
0.03/360 = 0.00008333333333(3)

0.00008333333333(3) has as precision of 16

It has a scale of 14 but a precision of 10. The precision refers to how many significant digits it has, not how many decimal places it has. https://www.logicbig.com/quick-info/programming/precision-and-scale.html

For example Big.js uses scale whereas Decimal.js uses precision

@alexviar
Copy link

alexviar commented Aug 3, 2022

@alexviar i'm not sure what your example shows but it's not precision of 10

Working with precision of 10:
0.03/360 = 0.00008333333333(3)

0.00008333333333(3) has as precision of 16

I did put in parentheses digits that was rounded, and underlined digits are the Periodo of decimal

@bendavies
Copy link
Author

ah right thanks, i forgot that leading zeros are not significant.
but also for storage purposes (postgres), 0 <= scale <= precision so, It has a scale of 14 but a precision of 10. doesn't work, in practise.

anyway, the point of this of was to simple add a getPrecision() method, similar to getScale, nothing mnore

@BenMorel
Copy link
Member

BenMorel commented Aug 4, 2022

Hi @bendavies, sorry for the late reply. I'm not against adding a getPrecision() method if it is useful to you, so please feel free to open a PR along with a few tests, in particular to clarify what happens for leading zeros.

@alexviar I'm not sure what the decimal period has to do with the precision? Anyway, you make me think that this would be a great addition to BigRational: a way to output 10/3 as 3.3(3) for example. Would you want to work on this?

@BenMorel
Copy link
Member

BenMorel commented Aug 4, 2022

I opened issue #74 to track the BigRational as decimal number with period separately.

@alexviar
Copy link

alexviar commented Aug 6, 2022

@alexviar I'm not sure what the decimal period has to do with the precision?

I think I didn't explain myself well. I did put rounded decimals (and decimal period) in parentheses for illustrative purposes only. Precision is given by working with n significant digits instead of n decimal place:

Operation Rounded to 10 significant digits Rounded to 10 decimal places
0.03 0.03000000000 0.0300000000
360 360.0000000 360.0000000000
8 8.000000000 8.0000000000
30 30.00000000 8.0000000000
0.03/360 0.00008333333333 0.0000833333
(0.03/360)*8 0.0006666666666 0.0006666666
(0.03/360)*8*30 0.02000000000 0.0200000000

Now, if we take the rounded values to do subsequent operations, we have:

Operation Rounded to 10 significant digits Rounded to 10 decimal places
0.03 0.03000000000 0.0300000000
360 360.0000000 360.0000000000
8 8.000000000 8.0000000000
30 30.00000000 8.0000000000
0.03/360 0.00008333333333 0.0000833333
(0.03/360)*8 0.0006666666666 0.0006666664
(0.03/360)*8*30 0.02000000000 0.0199999920

Currently it can be circunvent by using BigRational, but this workaround doesn't work with irrational opreations like square root for example (sqrt(2)*sqrt(2) Never mind, I had read an article with an example like this, but I can not found it)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants