Track ES BigInt work

[inexorabletash]

As raised over in tc39/proposal-bigint#123

When BigInt support lands in ECMAScript they won't be usable out of the box as keys in Indexed DB.

[inexorabletash]

If we add new key types we should toss boolean in as well. #76

[littledan]

Serialization of BigInts is proposed in this HTML PR. This would mean you can use BigInts and BigInt wrappers as IndexedDB values. Any concerns with this proposal?

For keys, I like @inexorabletash 's suggestions in tc39/proposal-bigint#123 . To make it into a concrete straw-person--how does this plan sound?

• Permit BigInt primitive keys in IndexedDB
• Don't associate BigInt and Number keys with each other, even if they are mathematically equal, and always preserve the original type
• Make BigInts all come after any Number in sort order, and before other types which follow Number.

Is there any particular need to make autoIncrement tie in with BigInts? Seems like the current semantics are pretty closely tied to Number, so this could be rather complex to work out. Is anyone running into issues with overflow of these counters?

[inexorabletash]

Serialization of BigInts... Any concerns with this proposal?

None. Should just work.

how does this plan sound?

@aliams ?

Is there any particular need to make autoIncrement tie in with BigInts?

I don't think so. "53 bits should be enough for anyone" - here they are specific to a database instance on someone's machine, not 64 bit identifiers floating around across systems.

[aliams]

I just gave this a read and I think it all makes sense. It's possible that there could be an impact for some underlying databases by increasing the index size, but I suppose there are always way to deal with that.

[littledan]

Wow, thanks for the extremely fast turnaround for a technical review from the editors! Impressive.

Make BigInts all come after any Number in sort order, and before other types which follow Number.

Not sure how unambiguous this sentence is; I was imagining that BigInt would compare as less than Number. (Seems like there's nothing less than Number, so not sure what I meant by the second part.) However, I saw a note in the specification that made this sound like a bad idea:

As a result of the above rules, negative infinity is the lowest possible value for a key.

Seems like it might be good to preserve this property, in case someone is depending on it, and instead have all BigInts compare as greater than all Numbers.

[aliams]

Sounds good to me!

[inexorabletash]

TPAC 2019 Web Apps Indexed DB triage notes:

@littledan - have you heard any requests for this recently?

[fosskers]

Hello, I found this issue and PR #231 after a mysterious failure upon trying to use unsigned 64-bit integers as IndexedDB keys. Our entire application is WASM built from a higher-level language, where such types are common. Originally I had read in the documentation that "numbers could be used as IndexedDB keys", but experimentally it seems like that specifically refers to a JS Number, namely a 64-bit float. Naturally a 64-bit float cannot represent all the values a 64-bit int can, so this puts us in a bit of a pickle.

My guess is that my unsigned int is represented lower down as a BigInt upon passing an FFI barrier, hence I'm given a "invalid key" error in my console. If I manually cast down to a 64-bit float before writing, it works (but I've lost some bits of accuracy).

Right, surely you don't actually need all 64 of those bits.

Yes, you'd be correct, but the unsigned int is the result of a Standard Library hashing scheme that always yields a 64-bit value. Since that hash is already the thing guaranteeing uniqueness, the idea of shaving off bits makes me nervous.

Would it be possible to revisit this work of supporting BigInts as keys?

[fosskers]

If I manually cast down to a 64-bit float before writing, it works (but I've lost some bits of accuracy).

Actually, it seems as if all the bits are still there even after a conversion to a float; it's just the representation to the human eye that has changed. So a quick conversion before reading/writing might be sufficient for my purposes.

[davidcallanan]

It's a disaster that we cannot use BigInt's as keys, however, we can use a left-padded string instead.

This should maintain numerical ordering when using IDBKeyRange etc.

const NUM_DIGITS = 20; // increase if larger numbers are needed
const key_str = `${key_bigint}`.padStart(NUM_DIGITS, "0");

I'm not sure what the performance implications are with using strings here, but it's better than the alternative of not being able to use IndexedDB at all.

Conversion back to BigInt is very easy:

const key_bigint = BigInt(key_str);

The only problem with this approach is that you must impose a limit on how big the numbers can be, controlled by NUM_DIGITS above. However, this is ok for the majority of use-cases.