Use of int64, uint64, float64 in the HTTP API

[achingbrain]

Checklist

• This is a bug report, not a question. Ask questions on discuss.ipfs.io.
• I have searched on the issue tracker for my bug.
• I am running the latest go-ipfs version or have an issue updating.

Installation method

built from source

Version

No response

Config

No response

Description

The HTTP API docs for /api/v0/stats/bw say RateIn and RateOut are float64 and TotalIn and TotalOut are int64.

They are returned as numbers from the API:

$ curl -X POST http://127.0.0.1:5002/api/v0/stats/bw
{"TotalIn":382097,"TotalOut":219659,"RateIn":5594.9794181522075,"RateOut":74.06595710119922}

If we need this amount of precision, these should be returned as stringified numbers instead, otherwise some implementations (eg. JSON.parse in js) will not be able to parse them, since their native number type does not have enough precision to do so:

JSON.parse('{"TotalIn": 107654321001234567800}')
// {TotalIn: 107654321001234560000}

If they are returned as strings they can be converted into BigInts by client libraries and the correct values will be retained.

This applies to the following endpoints:

• /api/v0/add
• /api/v0/bitswap/ledger
• /api/v0/bitswap/stat
• /api/v0/dag/stat
• /api/v0/file/ls
• /api/v0/files/ls
• /api/v0/files/stat
• /api/v0/filestore/ls
• /api/v0/filestore/verify
• /api/v0/ls
• /api/v0/object/get
• /api/v0/object/links
• /api/v0/object/new
• /api/v0/object/patch/add-link
• /api/v0/object/patch/append-data
• /api/v0/object/patch/rm-link
• /api/v0/object/patch/set-data
• /api/v0/object/put
• /api/v0/repo/stat
• /api/v0/stats/bw
• /api/v0/stats/bitswap
• /api/v0/stats/repo
• /api/v0/tar/add

[rpodgorny]

please be sure to at least keep the native (non-string) types for parsers which are not dumb... ;-)

[radek@ruprt ~]$ python3
Python 3.9.6 (default, Jun 30 2021, 10:22:16) 
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> json.loads('{"TotalIn": 107654321001234567800}')
{'TotalIn': 107654321001234567800}

[aschmahmann]

@rvagg @achingbrain can dag-json help us out here if we allow outputting data as dag-json instead of just a json serializer?

[rvagg]

Funny you should mention it, because yes it can:

> dagJSON.decode(new TextEncoder().encode('{"TotalIn": 107654321001234567800}'))
{ TotalIn: 107654321001234567800n }

As of the new version of @ipld/dag-json yesterday ipld/js-dag-json#46. For cases involving large messages or high-throughput messages then it'd be worth benchmarking to see if there's any cost.

But in terms of usability, I think @achingbrain's point here might be in general user consumption of these end-points rather than just our own js-ipfs-http-client's consumption. Pick your poison:

1. Give users numbers they can't accurately parse (maybe accuracy is not so important in the high ranges?)
2. Telling them use our custom library to parse JSON (it would give you native CIDs and also bytes if we're using it on the other end too, which is neat)
3. Telling them that their numbers are coming out as strings and they need to BigInt(str) or parseFloat(str) them.