jdata.js - JData Annotation Encoder/Decoder for JavaScript and NodeJS

JData specification is a language-independent data annotation standard that enables convenient storage and exchange of complex data structures, such as strongly-typed binary N-D arrays, complex-valued and sparse arrays, tables and graphs, using lightweight JSON name:value pairs. This allows applications to exchange complex data records between programming languages, such as between MATLAB, Python and JavaScript. Both the JData specification and this library are developed under the NeuroJSON project (http://neurojson.org) funded by the U.S. National Institute of Health (NIH) grant # U24-NS124027.

Using the JData/JSON format, a strongly-typed uint8 integer 3D array of size 3x2x4

data=[
    [[1,7,13,19],[4,10,16,22]],
    [[2,8,14,20],[5,11,17,23]],
    [[3,9,15,21],[6,12,18,24]]
]

can be encoded using JData/JSON [annotated ND-array] format as

data={
    _ArrayType_: "uint8",
    _ArraySize_: [3,2,4],
    _ArrayData_: [1,7,13,19,4,10,16,22,2,8,14,20,5,11,17,23,3,9,15,21,6,12,18,24]
 }

JData specification also permits data-level compression via _ArrayZip*_ tags, for example, the same uint8 3-D array can be stored after zip compression (and then Base64-encoded) as

data={
    _ArrayType_:"uint8",
    _ArraySize_:[3,2,4],
    _ArrayZipType_:"zlib",
    _ArrayZipSize_:[1,24],
    _ArrayZipData_:"eJxjZOcVZuESEGPi4BNh5RYUZ+bkF2XjEZIAAA1CAS0="
}

With the help of additional lightweight annotation tags, JData format permits scientific applications to exchange a wide variety of complex data structures, including complex-valued arrays, sparse arrays, maps, tables, graphs etc, effortlessly via JSON and binary-JSON serialized files or streams. Because JSON/binary JSON parsers are widely available, this not only makes data human-readable and self-documenting, but also easily extensible and inteoperable between applications.

This lightweight jdata.js module provides JData-annotation encoding and decoding functionalities for JavaScript/web applications as well as NodeJS applications. It can automatically recognize the JData annotation tags, such as _ArrayType_ etc, and convert the encoded data to the native JavaScript/Node data structures, such as TypedArray, ArrayBuffer, NumJS NDarray, and BigInt etc. It also performs in the reverse direction, i.e. encode native JavaScript data structures into JSON/JData encoded forms that are easy to be stored in data files, databases and shared between programming environments. Compression and decompression using zlib, gzip and lzma algorithms are currently supported in this library.

Check out the Github repo for the source code. Visit module site for API docs and examples. Extra information available in wiki.

Installation

To use the jdata.js module in Node.js applications, you must first install the below dependencies

npm install jda

This will automatically install the jdata.js package as well as its dependencies: atob, btoa, pako, numjs.

To use jdata.js in a JavaScript application in a web browser, you must include the below dependencies

<script src="https://cdnjs.cloudflare.com/ajax/libs/numjs/0.16.0/numjs.min.js