-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Replace node Buffers with Uint8Arrays #3220
Comments
We could also adjust the examples. Eg.: https://github.com/ipfs/js-ipfs/tree/master/examples/ipfs-101 From To or at least make a note about the usage within Node/Browser. |
@Weedshaker when this effort bubbles up to a PR for this repo, yes, the documentation will be updated. |
Some notes on transitioning:
|
It's worth noting that |
BufferList has “shallowSlice” which is roughly what subarray does, just returning a new BufferList instead of Uint8array. But I would imagine that what you often want from BufferList is still .slice() because you probably need a complete binary value.
…-Mikeal
________________________________
From: Alex Potsides <notifications@github.com>
Sent: Tuesday, August 11, 2020 7:53:04 AM
To: ipfs/js-ipfs <js-ipfs@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Subject: Re: [ipfs/js-ipfs] Replace node Buffers with Uint8Arrays (#3220)
It's worth noting that .subarray cannot be used with BufferList<https://www.npmjs.com/package/bl>s, which occasionally complicates matters.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#3220 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAAAEQ6T3ERS552OHDBPVPLSAFLNBANCNFSM4P2DZHJQ>.
|
Is |
The push here is really to have our ecosystem modules accept For converting |
This shipped in js-ipfs@0.50.0! We did it! |
https://github.com/ipfs/js-datastore-idb seems to have been missed |
@HexaField until ipfs/js-stores#198 is resolved, that module shouldn't be used anywhere in the stack, instead we should be using datastore-level - are you seeing problems? |
Switch to using Uint8Arrays instead of node Buffers. This is something the js-ipfs folks apparently have gone through, too: ipfs/js-ipfs#3220
* Use ipfs-core instead of ipfs for a lighter dependency * Make yarn node --version be 15.x in nix-shell * Implement the complete crypto DI for node * Fix cbor encode/decode errors on node Switch to using Uint8Arrays instead of node Buffers. This is something the js-ipfs folks apparently have gone through, too: ipfs/js-ipfs#3220 * ESLint fixes * Fix eslint errors in index.ts * Revert "Fix cbor encode/decode errors on node" This reverts commit dbc3b3d. * Fix type * Test cbor en/decoding * Implement helper for totally in-memory ipfs * Add integration test & fix filesystem running in node * Remove logging * Use types added to the workaround repo * Fix CBOR decoding errors of encrypted data in node * Switch back to borc * Update jest & puppeteer & base58-universal * Add jest puppeteer types * Make it possible to run multiple ipfs in-memory If I didn't disable swarm addresses, then multiple instances of ipfs would request socket access and block each other. * Actually switch back from cborg to borc * Remove unneeded Buffer.from call * Make jest properly exit after all tests * wtfnode * Revert "wtfnode" This reverts commit 5cbfd03. * Force jest to exit, even if ipfs holds resources * Revert "Make jest properly exit after all tests" This reverts commit eab6bc8.
In order to treat browsers as first-class citizens, we should not use modules from node core unless we can absolutely guarantee that the code we are writing will not run in a browser.
The next generation of multiformats & IPLD modules will also be
Uint8Array
s all the way down so embracing more modern formats will mean the upgrade path easier to follow when they are ready for use.Modern JavaScript runtimes have
TypedArray
s such asUint8Array
backed byArrayBuffer
s, as well asDataView
s that wrap anArrayBuffer
and allow you to do bit-twiddly operations like writing little endian 32 bit floats at arbitrary offsets, etc.Node's
Buffer
module pre-dates all of this but since node v3 it extends theUint8Array
class, so we should be able to treatBuffer
s asUint8Array
s internally in our stack, though some dependencies may requireBuffer
s as input until they too can be refactored.PRs to modules
IPFS
IPLD
libp2p
Multiformats
Other
The text was updated successfully, but these errors were encountered: