Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function to_long() returns unexpected value when input is float with decimals #99

Open
abitmore opened this issue Sep 3, 2022 · 1 comment

Comments

@abitmore
Copy link
Member

abitmore commented Sep 3, 2022

As mentioned in bitshares/bitshares-ui#3545 (comment), the function to_long() in SerializerValidation.js may return unexpected result if the input is a float with decimals (E.G. 12.999999999999998). Code:

to_long(value, field_name = "", unsigned = false) {
if (this.is_empty(value)) {
return value;
}
if (Long.isLong(value)) {
return value;
}
this.no_overflow64(value, field_name, unsigned);
if (typeof value === "number") {
value = "" + value;
}
return Long.fromString(value, unsigned);
},

> parseFloat("0.00013")*100000
12.999999999999998
> Long.fromString(""+parseFloat("0.00013")*100000)
Long { low: 1150981118, high: 30, unsigned: false }
> Long.fromString(""+parseFloat("0.00013")*100000).toString()
129999999998

To fix this, IMHO, as a library, we shouldn't change user input, but we should require the input to be valid.

@christophersanborn
Copy link
Member

christophersanborn commented Sep 4, 2022

For a little extra context, it looks like Long (from the bytebuffer library), will correctly handle the first five digits after the decimal point, whereafter it seems to just append digits.

> var Long = require("bytebuffer").Long;
undefined
> Long.fromString("12.12345")
Long { low: 12, high: 0, unsigned: false }
> Long.fromString("12.123456")
Long { low: 126, high: 0, unsigned: false }
> Long.fromString("12.1234567")
Long { low: 1267, high: 0, unsigned: false }
> 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants