JSON support for BigInt in ES6

ES6 has recently been upgraded to support a native BigInt type. Currently there is no explicit support for using BigInt with the ES6 JSON object. This document contains a proposal for extending the ES6 platform to support BigInt both according to the JSON standard for numeric data, as well as existing practices relying on JSON strings. Since JSON do not distinguish between different numbers (aka weakly typed), the described deserialization schemes all presume that a JSON consumer honors an in advance known "contract" including serialization method used by the producer.

Related issue: tc39/proposal-bigint#162

Also see summary of changes.

1 Default Mode

The current ES6 implementation throws an exception if you try to serialize a BigInt using JSON.stringify(). This specification recommends keeping this behavior for numerous reasons including:

• Diverging views on what the "right" serialization solution is
• Changing default serialization to use JSON Number would give unexpected/unwanted results
• Widely deployed systems relying on custom BigInt serialization (base64/hex), also including current IETF & W3C standards defining JSON structures holding BigInt objects
• TC39's dismissal of the serialization scheme used for Date
• Availability of a BigInt.prototype.toJSON() option which greatly simplifies customized serialization

2 RFC Mode

RFC mode denotes the number serialization scheme specified by the JSON RFC.

2.1 The JSONNumber Primitive

This proposal builds on the introduction of a new primitive type called JSONNumber, intended for serialization and deserialization of numeric data which cannot be represented by the ES6 Number type. JSONNumber is a thin wrapper holding a string in proper JSON Number notation. The typeof operator recognizes JSONNumber as "jsonnumber".

To enable the new functionality JSON.stringify() must be updated to recognize JSONNumber as a valid data type which always serializes verbatim as a string but without quotes.

In the deserializing mode JSONNumber can also be used for verifying that a number actually has expected syntax (in the current JSON.parse() implementation there is no possibility distinguishing between 10 or 10.0).

JSONNumber is intended to be usable "as is" with a future BigNum type, including when only supplied as a "polyfill".

2.1.1 Interface

Method	Comment
JSONNumber(string)	Constructor. The string argument MUST be in proper JSON Number notation
JSONNumber.prototype.toString()	Returns string value
JSONNumber.prototype.isInteger()	Returns true for integer syntax (=string contains no decimal point or exponent)
JSONNumber.prototype.isPositive()	Returns true for positive numbers
JSONNumber.prototype.isNumber()	Returns true if a converted string would be 100% compatible with an ES6 Number with respect to precision and range

Question: Since JSONNumber does not seem to have any use except for operations related to the ES6 JSONobject, would it be possible (and useful) making JSONNumber a member like JSON.JSONNumber?

2.1 RFC Mode Serialization

The following code shows how RFC mode BigInt serialization can be added to JSON.stringify.

BigInt.prototype.toJSON = function() { 
  return JSONNumber(this.toString()); 
}
 
JSON.stringify({big: 555555555555555555555555555555n, small:55});

Expected result: '{"big":555555555555555555555555555555,"small":55}'

NOTE: RFC mode support requires JSON.stringify() to be upgraded to accept JSONNumber as a serializable object.

2.2 RFC Mode Deserialization

Deserialization of BigInt cannot be automated like serialization; the selection between different number types usually depends on conventions between JSON consumers and producers. The selections are either managed through the JSON.parse() reviver option or are performed after parsing has completed.

NOTE: RFC mode deserialization requires a new optional flag to JSON.parse(). When this flag is set to true, JSON Number elements must only be parsed for correctness with respect to syntax, while the parsed string itself is returned in a JSONNumber for application level processing.

2.2.1 Property Based Deserialization Selection

Below is an example of a scheme having a single property holding a BigInt:

JSON.parse('{"big":55,"small":55}', 
  (k,v) => typeof v === 'jsonnumber' ? k == 'big' ? BigInt(v.toString()) : Number(v.toString()) : v,
  true   // New flag to make all numbers be returned as JSONNumber
);

Expected result: {big: 55n, small: 55}

2.2.2 Value Based Deserialization Selection

Below is an example where the actual value of an object is used for type selection:

JSON.parse('{"big":555555555555555555555555555555,"small":55}', 
  (k,v) => typeof v === 'jsonnumber' ? v.isNumber() ? Number(v.toString()) : BigInt(v.toString()) : v,
  true   // New flag to make all numbers be returned as JSONNumber
);

Expected result: {big: 555555555555555555555555555555n, small: 55}

2.2.3 Syntax Checking Deserialization

Below is an example of a syntax checker using JSONNumber:

JSON.parse('{"int1":55,"int2":10.0}', 
  (k,v) => {
     if (typeof v === 'jsonnumber') {
       if (!v.isInteger()) {
         throw new Error('Not integer: ' + k);
       }
       return Number(v.toString());
     }
     return v;
  },
  true   // New flag to make all numbers be returned as JSONNumber
);

Expected result: An error message containing the name int2;

3 Quoted String Mode

Although not the method suggested by the JSON RFC, there are quite few systems relying on BigInt objects being represented as JSON Strings. Unfortunately this practice comes in many flavors making a standard solution out of reach, or at least not particularly useful. However, there is no real problem to solve either since the ES6 JSON API as it stands can cope with any variant.

3.1 Quoted String Mode Serialization

Here follows a few examples on how to deal with quoted string serialization for BigInt.

3.1.1 Serialization Using Decimal Digits

BigInt.prototype.toJSON = function() { 
  return this.toString(); 
}
 
JSON.stringify({big: 555555555555555555555555555555n, small:55});

Expected result: '{"big":"555555555555555555555555555555","small":55}'

3.1.2 Serialization Using Base64Url Encoded Data

// Browser specific solution
BigInt.prototype.toJSON = function() {
  function hex2bin(c) {
    return c - (c < 58 ? 48 : 87); 
  }
  let value = this.valueOf();
  let sign = false;
  if (value < 0) {
    value = -value;
    sign = true;
  }
  let hex = value.toString(16);
  if (hex.length & 1) hex = '0' + hex;
  let binary = new Uint8Array(hex.length / 2);
  let j = binary.length;
  let q = hex.length;
  let carry = 1;
  while(q > 0) {
     let byte = hex2bin(hex.charCodeAt(--q)) + (hex2bin(hex.charCodeAt(--q)) << 4);
     if (sign) {
       byte = ~byte + carry;
       if (byte > 255) {
         carry = 1;
       } else {
         carry = 0;
       }
     }
     binary[--j] = byte;
  }
  if (sign ^ (binary[0] > 127)) {
    let binp1 = new Uint8Array(binary.length + 1);
    binp1[0] = sign ? 255 : 0;
    for (let i = 0; i < binary.length; i++) {
      binp1[i + 1] = binary[i];
    }
    binary = binp1;
  }
  let text = '';
  for (let i = 0; i < binary.length; i++) {
    text += String.fromCharCode(binary[i]);
  }
  return window.btoa(text).replace(/\+/g,'-').replace(/\//g,'_').replace(/=/g,'');
}

JSON.stringify({big: 555555555555555555555555555555n, small:55});

Expected result: '{"big":"BwMYyOV8edmCI4444w","small":55}'

NOTE: This code is lengthy, complex and potentially incorrect. There should be a BigInt method returning a byte array in two-complement format like in Java: https://docs.oracle.com/javase/8/docs/api/java/math/BigInteger.html#toByteArray--

3.2 Quoted String Mode Deserialization

Since the is no generally accepted method for adding type information to data embedded in strings, the selection of encoding method is effectively left to the developers of the actual JSON ecosystem. The selections are either managed through the JSON.parse() reviver option or are performed after parsing has completed.

Here follows a few examples on how to deal with quoted string deserialization for BigInt.

3.2.1 Deserialization Using Decimal Digits

JSON.parse('{"big":"55","small":55}', 
  (k,v) => k == 'big' ? BigInt(v) : v
);

Expected result: {big: 55n, small: 55}

3.2.2 Deserialization Using Base64Url Encoded Data

// Browser specific solution
function base64Url2BigInt(b64) {
  let charbin = window.atob(b64.replace(/\-/g,'+').replace(/_/g,'/').replace(/=/g,'?'));
  let binary = new Uint8Array(charbin.length);
  for (let i = 0; i < binary.length; i++) {
    binary[i] = charbin.charCodeAt(i);
  }
  let sign = false;
  if (binary[0] > 127) {
    sign = true;
    let carry = 1;
    let q = binary.length; 
    while (q > 0) {
      let byte = ~binary[--q] + carry;
      binary[q] = byte;
      if (byte > 255) {
        carry = 1;
      } else {
        carry = 0;
      }
    }
  }
  let value = BigInt(0n);
  for (let i = 0; i < binary.length; i++) {
    value *= 256n;
    value += BigInt(binary[i]);
  }
  return sign ? -value : value;
}

JSON.parse('{"big":"BwMYyOV8edmCI4444w","small":55}', 
  (k,v) => k == 'big' ? base64Url2BigInt(v) : v
);

Expected result: {big: 555555555555555555555555555555n, small: 55}

NOTE: This code is lengthy, complex and potentially incorrect. There should be a method for creating a BigInt value from a byte array in two-complement format like in Java: https://docs.oracle.com/javase/8/docs/api/java/math/BigInteger.html#BigInteger-byte:A-

Summary Of Changes

• Adding a JSONNumber primitive type
• Enhancing JSON.stringify() to always accept JSONNumber for serialization
• Adding an optional flag to JSON.parse() requiring the parsing process to return JSONNumber instead of Number
• Optionally improving BigInt for dealing with two complement serialization formats

Aknowledgements

This specification was influenced by input from many persons including T.J. Crowder, J Decker, Rob Ede, Daniel Ehrenberg, Kevin Gibbons, Richard Gibson, Jordan Harband, Ranando King, Jakob Kummerow, Isiah Meadows, Claude Pache, Claude Petit, Michael Theriot, Michał Wadas and Kai Zhu.

Current Version

0.1