Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RoundToPrecision function rounds to the incorrect number of decimal places #2915

Closed
hugohills-regnosys opened this issue May 7, 2024 · 1 comment · Fixed by #2916 or #2917
Closed

Comments

@hugohills-regnosys
Copy link
Contributor

hugohills-regnosys commented May 7, 2024

Function cdm.base.math.RoundToPrecision rounds to the incorrect number of decimal places.

func RoundToPrecision: <"Round a rate to the supplied precision, using the supplied rounding direction">
    inputs:
        value number (1..1) <"The original (unrounded) number.">
        precision int (1..1) <"The number of decimal digits of precision.">
        roundingMode RoundingDirectionEnum (1..1) <"The method of rounding (up/down/nearest).">
    output:
        roundedValue number (1..1) <"The value to the desired precision">

The following examples show the function behaviour:

RoundToPrecision(1023.123456789, 5, RoundingDirectionEnum -> NEAREST)

  • should return 1023.12346
  • existing function throws ArithmeticException

RoundToPrecision(1023.123456789, 5, RoundingDirectionEnum -> UP)

  • should return 1023.12346
  • existing function throws ArithmeticException

RoundToPrecision(1023.123456789, 5, RoundingDirectionEnum -> DOWN)

  • should return 1023.12345
  • existing function throws ArithmeticException

RoundToPrecision(1023.123456789, 0, RoundingDirectionEnum -> NEAREST)

  • should return 1023
  • existing function returns 1023.0

RoundToPrecision(1023.1, 7, RoundingDirectionEnum -> NEAREST)

  • should return 1023.1000000
  • existing function returns 1023.10000000)
@winitall01
Copy link
Contributor

Hello Hugo,
This is great, however, can we also check that we do not add any trailing 0's in the end if they do not already exists.

RoundToPrecision(1023, 5, RoundingDirectionEnum -> NEAREST)
should return 1023
existing function returns 1023.00000

RoundToPrecision(9999999999999999999999999, 4, RoundingDirectionEnum -> NEAREST) -
should return 9999999999999999999999999
existing function returns 9999999999999999999999999.0000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants