Support decimal type from typespec

[MaryGao]

https://github.com/Azure/azure-rest-api-specs-pr/pull/13119#discussion_r1376998182

In typespec, we have decimal and decimal128. And in health insight we have decimal type used.

  @doc("Numerical value (with implicit precision)")
  value?: decimal;

https://microsoft.github.io/typespec/language-basics/built-in-types#numeric-types

TypeSpec definition for decimal:

/**
 * A decimal number with any length and precision. This represent any `decimal` value possible.
 * It is commonly represented as `BigDecimal` in some languages.
 */
scalar decimal extends numeric;

/**
 * A 128-bit decimal number.
 */
scalar decimal128 extends decimal;

We need to decide on how to support this type in both RLC and Modular.

[xirzec]

How is this being represented over the wire? Presumably they are not sending it as a JSON number since that would potentially lose precision. If it's a string my preference is to keep it as a string and let the consumer use whatever userland library they prefer to handle the number manipulation.

[MaryGao]

@xirzec @bterlson Let me explain this with more details. I'm not sure whether we should adopt any third party libs or we have better choices to support decimal.

Background

The JSON standard does not specify any length and precision limitation for number type – a number is just defined syntactically as a sequence of digits, optionally followed by a dot and some more digits etc. And RFC7159 mentions to follow the IEEE 754-2008 binary64 (double precision) numbers [IEEE754]. It's JSON serialization and deserialization tool to control the supported precision.

This specification allows implementations to set limits on the range and precision of numbers accepted.

When supporting decimal type(which is a decimal number with any length and precision) in TypeSpec we have two challenges.

No native support in JS

• We only have BigInt supported (definition link)
• We have the decimal proposal however it is only in Stage 1 of the TC39 process

No native support in JSON.parse and JSON.stringify

Even for BigInt JSON.stringify would throw exceptions.

const res = BigInt("9223372036854775807");
console.log(JSON.stringify(res));
// ============= Here is the output =============
/**
[ERR]: "Executed JavaScript Failed:" 
[ERR]: Do not know how to serialize a BigInt
 */

And for general cases we may lose the precision or lead to wrong numbers.

const json = '{ "foo" : 0.1111111111111111222222222222222111111111111, "bar": 9223372036854775807, "baz": 123 }';
const r = JSON.parse(json);

console.log('JSON.parse(input).foo : ', r.foo.toString());
console.log('JSON.parse(input).bar : ', r.bar.toString());
console.log('JSON.parse(input).baz : ', r.baz.toString());
// ============= Here is the output =============
/**
[LOG]: "JSON.parse(input).foo : ",  "0.11111111111111112",  "number" 
[LOG]: "JSON.parse(input).bar : ",  "9223372036854776000",  "number" 
[LOG]: "JSON.parse(input).baz : ",  "123",  "number"  
 */

[bterlson]

Regarding how it's sent on the wire, we decided to use actual numbers by default. Looking at the general use cases, most often we're talking about currency amounts that fit well inside a double's range and precision. The string just added confusion. This does mean that if a service that cares strongly about preserving decimal's precision, it should use @encode to encode it as a string.

My opinion is that, for RLC, it's just whatever is on the wire, so number by default, but maybe string if @encode is used.

For modular, I don't think it's worth using a library like decimal.js or similar. Again, going back to common use cases, we're usually talking about numbers within the safe range. Problems due to rounding may crop up when users need to use those numbers in calculations, but I think this is far less common, and anyway it's fairly trivial for a user to pick their favorite decimal lib and convert a number or string to the decimal prior to doing any calculations. So I think my recommendation there is also to just use numbers or strings depending on @encode.

[MaryGao]

Using @encode as string could be our recommandation. But not sure it could be a general one considering below reasons:

• Java/.Net/Python have the native support for decimal
• Service team may follow an external specification and we have no choice to encode them as string e.g health insight follows the FHIR standard quantity model and discussion here.

I agree that for calculations, users could pick their own lib then converting to a number or string for us(aka client SDK) and within the safe range it should be safe also.

However I am worrying about that the support has potential risks but it is a silent process. Not sure we need to involve more communications with service team to clarify this because eventually they will take the cost of the risk.

On the other hand how to educate end users - the contact with server side is based on decimal so you may pick your own decimal lib for any calculations. And we expect 0.2 + 0.1 = 0.3 not 0.30000000000000004. Maybe more comments in generated code?

[MaryGao]

pls ignore the wrong linked to lro epic :)

[bterlson]

To be explicit, there are two risks related to precision loss:

1. Precision loss from conversion from the wire in languages like JavaScript that have insufficient precision in the only data type available to represent numbers.
2. Precision loss from customer code using mathematical operations on floats.

For 1, the damage is limited to JavaScript as I believe most other languages have the ability to either dynamically choose an appropriate data type or specify the data type when parsing number literals from JSON. For JS in particular, the risk of 1 seems very low considering I have not seen a use case in Azure yet where numbers would fall outside the range of a double, which isn't particularly surprising given that the main usecase is currency amounts, and currency amounts above 2^53-1 or outside the precision of floats for typical currency values (~15 decimal places) seems very unlikely. However, services should be aware of these limitations and use @encode properly, documenting this somewhere for service teams seems good.

For 2, again the damage is mostly limited to JavaScript, since other languages can presumably vend a decimal appropriately regardless of how the decimal is placed on the wire. For JavaScript in particular, the risk seems low considering that the use cases I examined didn't involve round trip of currency values that are calculated by the client. However, it would probably be a good idea to document fields with the decimal type with some standard verbiage explaining that it's a decimal value and to take care when doing math on the value to avoid precision loss.