Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Could dec(expression) replace all number literals in the expression with decimals? #764

Closed
christianp opened this issue Nov 10, 2020 · 0 comments
Closed

Could dec(expression) replace all number literals in the expression with decimals? #764

christianp opened this issue Nov 10, 2020 · 0 comments
Labels
Needs thinking about

Comments

@christianp
Copy link
Member

christianp commented Nov 10, 2020 •
edited
Loading

At the moment, dec(expression) evaluates expression, then converts it to a decimal. This means that precision is lost before the conversion, when it looks like it shouldn't be.

If dec was a lazy operator, it could rewrite all the number literals in the expression to decimals, forcing calculations to happen in decimals instead of numbers, potentially saving a lot of precision.
christianp added a commit that referenced this issue Aug 9, 2022
@christianp
decimal() rewrites number literals in its argument to decimals
0fb5a7c 
fixes #764

I think this is safe to do because a decimal will cast down to a number
if necessary, so this just makes operations which can happen in decimals
more accurate.
@christianp christianp mentioned this issue Jan 18, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Needs thinking about
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@christianp