This repository has been archived by the owner on Feb 12, 2024. It is now read-only.
Awesome Endeavour: Async Iterators #1670
Assignees
Labels
awesome endeavour
exp/wizard
Extensive knowledge (implications, ramifications) required
help wanted
Seeking public contribution on this issue
status/in-progress
In progress
Comments
alanshaw
added
exp/wizard
|
This is a "Everyone is Welcome" effort and a excellent opportunity for everyone to get their feet wet on the codebase, improve documentation, testing and refresh the code. Hoping that we can count with help from the IPFS GUI, IPFS in Web Browsers, Community WG and Dynamic Data & Capabilities. This will make our JS APIs better for all the modern/new JS developers! @ipfs/javascript-team Unite ✨👋🏽⚡️ |
This was referenced Oct 26, 2018
|
this is awesome and makes me so happy to see 🎉 |
|
add me to multihashing-async, ipfsd-ctl, ipfs-multipart |
|
@hugomrdias you got it! :) |
2 tasks
alanshaw
added a commit
to alanshaw/js-libp2p-crypto-secp256k1
that referenced
this issue
Oct 28, 2018
This PR changes this module to remove callbacks and use async/await. The API is unchanged aside from the obvious removal of the `callback` parameter. depends on `multihashing-async` PR TODO refs ipfs/js-ipfs#1670 License: MIT Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
1 task
|
Updated the table with my assignements |
|
@alanshaw I'm taking a stab at libp2p/js-peer-id#87 😄 Update: I added myself to |
|
Thoughts and views on https://github.com/libp2p/interface-peer-discovery/issues/2 please! |
|
@alanshaw I was thinking at taking a look at
|
This was referenced Oct 30, 2018
We (@vmx and I) aren't too happy with Perhaps we should accelerate the timeline for this refactor rather than trying to update the current interface? |
|
Yeah, that would be awesome. @mikeal @vmx as you can see by my previous comment, I already created a PR on |
Sounds great, but I do want to make sure I'm prioritizing things in the right order (there's a lot on my plate from being out on vacation and now I'm sick). Is this refactor already blocking downstream refactors? |
This was referenced Nov 22, 2019
alanshaw
added a commit
that referenced
this issue
Dec 9, 2019
This PR allows ipfsx to be used by calling `IPFS.create(options)` with `{ EXPERIMENTAL: { ipfsx: true } }` options.
It adds a single API method `add` that returns an iterator that yields objects of the form `{ cid, path, size }`. The iterator is decorated with a `first` and `last` function so users can conveniently `await` on the first or last item to be yielded as per the [proposal here](https://github.com/ipfs-shipyard/ipfsx/blob/master/API.md#add).
In order to boot up a new ipfsx node I refactored the boot procedure to enable the following:
1. **Remove the big stateful blob "`self`" - components are passed just the dependencies they need to operate.** Right now it is opaque as to which components require which parts of an IPFS node without inspecting the entirety of the component's code. This change makes it easier to look at a component and know what aspects of the IPFS stack it uses and consequently allows us to understand which APIs should be available at which points of the node's lifecycle. It makes the code easier to understand, more maintainable and easier to mock dependencies for unit tests.
1. **Restrict APIs to appropriate lifecycle stage(s).** This PR introduces an `ApiManager` that allows us to update the API that is exposed at any given point. It allows us to (for example) disallow `ipfs.add` before the node is initialized or access `libp2p` before the node is started. The lifecycle methods `init`, `start` and `stop` each define which API methods are available after they have run avoiding having to put boilerplate in every method to check if it can be called when the node is in a particular state. See #1438
1. **Safer and more flexible API usage.** The `ApiManager` allows us to temporarily change APIs to stop `init` from being called again while it is already running and has the facility to rollback to the previous API state if an operation fails. It also enables piggybacking so we don't attempt 2 or more concurrent start/stop calls at once. See #1061 #2257
1. **Enable config changes at runtime.** Having an API that can be updated during a node's lifecycle will enable this feature in the future.
**FEEDBACK REQUIRED**: The changes I've made here are a little...racy. They have a bunch of benefits, as I've outlined above but the `ApiManager` is implemented as a `Proxy`, allowing us to swap out the underlying API at will. How do y'all feel about that? Is there a better way or got a suggestion?
resolves #1438
resolves #1061
resolves #2257
refs #2509
refs #1670
License: MIT
Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
alanshaw
added a commit
that referenced
this issue
Dec 10, 2019
This PR allows ipfsx to be used by calling `IPFS.create(options)` with `{ EXPERIMENTAL: { ipfsx: true } }` options.
It adds a single API method `add` that returns an iterator that yields objects of the form `{ cid, path, size }`. The iterator is decorated with a `first` and `last` function so users can conveniently `await` on the first or last item to be yielded as per the [proposal here](https://github.com/ipfs-shipyard/ipfsx/blob/master/API.md#add).
In order to boot up a new ipfsx node I refactored the boot procedure to enable the following:
1. **Remove the big stateful blob "`self`" - components are passed just the dependencies they need to operate.** Right now it is opaque as to which components require which parts of an IPFS node without inspecting the entirety of the component's code. This change makes it easier to look at a component and know what aspects of the IPFS stack it uses and consequently allows us to understand which APIs should be available at which points of the node's lifecycle. It makes the code easier to understand, more maintainable and easier to mock dependencies for unit tests.
1. **Restrict APIs to appropriate lifecycle stage(s).** This PR introduces an `ApiManager` that allows us to update the API that is exposed at any given point. It allows us to (for example) disallow `ipfs.add` before the node is initialized or access `libp2p` before the node is started. The lifecycle methods `init`, `start` and `stop` each define which API methods are available after they have run avoiding having to put boilerplate in every method to check if it can be called when the node is in a particular state. See #1438
1. **Safer and more flexible API usage.** The `ApiManager` allows us to temporarily change APIs to stop `init` from being called again while it is already running and has the facility to rollback to the previous API state if an operation fails. It also enables piggybacking so we don't attempt 2 or more concurrent start/stop calls at once. See #1061 #2257
1. **Enable config changes at runtime.** Having an API that can be updated during a node's lifecycle will enable this feature in the future.
**FEEDBACK REQUIRED**: The changes I've made here are a little...racy. They have a bunch of benefits, as I've outlined above but the `ApiManager` is implemented as a `Proxy`, allowing us to swap out the underlying API at will. How do y'all feel about that? Is there a better way or got a suggestion?
resolves #1438
resolves #1061
resolves #2257
refs #2509
refs #1670
License: MIT
Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
alanshaw
added
status/in-progress
alanshaw
added a commit
that referenced
this issue
Jan 23, 2020
A roundup of the following PRs: * closes #2658 * closes #2660 * closes #2661 * closes #2668 * closes #2674 * closes #2676 * closes #2680 * test fixes and other fix ups --- To allow us to pass the interface tests, the timeout option is now supported in the `object.get` and `refs` APIs. It doesn't actually cancel the operation all the way down the stack, but allows the method call to return when the timeout is reached. https://github.com/ipfs/js-ipfs/pull/2683/files#diff-47300e7ecd8989b6246221de88fc9a3cR170 --- Supersedes #2724 --- resolves #1438 resolves #1061 resolves #2257 resolves #2509 resolves #1670 refs ipfs-inactive/interface-js-ipfs-core#394 BREAKING CHANGE: `IPFS.createNode` removed BREAKING CHANGE: IPFS is not a class that can be instantiated - use `IPFS.create`. An IPFS node instance is not an event emitter. BREAKING CHANGE: Instance `.ready` property removed. Please use `IPFS.create` instead. BREAKING CHANGE: Callbacks are no longer supported on any API methods. Please use a utility such as [`callbackify`](https://www.npmjs.com/package/callbackify) on API methods that return Promises to emulate previous behaviour. BREAKING CHANGE: `PeerId` and `PeerInfo` classes are no longer statically exported from `ipfs-http-client` since they are no longer used internally. BREAKING CHANGE: `pin.add` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property. BREAKING CHANGE: `pin.ls` now returns an async iterable. BREAKING CHANGE: `pin.ls` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property. BREAKING CHANGE: `pin.rm` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property. BREAKING CHANGE: `add` now returns an async iterable. BREAKING CHANGE: `add` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property. BREAKING CHANGE: `addReadableStream`, `addPullStream` have been removed. BREAKING CHANGE: `ls` now returns an async iterable. BREAKING CHANGE: `ls` results now contain a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `hash` property. BREAKING CHANGE: `files.readPullStream` and `files.readReadableStream` have been removed. BREAKING CHANGE: `files.read` now returns an async iterable. BREAKING CHANGE: `files.lsPullStream` and `files.lsReadableStream` have been removed. BREAKING CHANGE: `files.ls` now returns an async iterable. BREAKING CHANGE: `files.ls` results now contain a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `hash` property. BREAKING CHANGE: `files.ls` no longer takes a `long` option (in core) - you will receive all data by default. BREAKING CHANGE: `files.stat` result now contains a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `hash` property. BREAKING CHANGE: `get` now returns an async iterable. The `content` property value for objects yielded from the iterator is now an async iterable that yields [`BufferList`](https://github.com/rvagg/bl) objects. BREAKING CHANGE: `stats.bw` now returns an async iterable. BREAKING CHANGE: `addFromStream` has been removed. Use `add` instead. BREAKING CHANGE: `addFromFs` has been removed. Please use the exported `globSource` utility and pass the result to `add`. See the [glob source documentation](https://github.com/ipfs/js-ipfs-http-client#glob-source) for more details and an example. BREAKING CHANGE: `addFromURL` has been removed. Please use the exported `urlSource` utility and pass the result to `add`. See the [URL source documentation](https://github.com/ipfs/js-ipfs-http-client#url-source) for more details and an example. BREAKING CHANGE: `name.resolve` now returns an async iterable. It yields increasingly more accurate resolved values as they are discovered until the best value is selected from the quorum of 16. The "best" resolved value is the last item yielded from the iterator. If you are interested only in this best value you could use `it-last` to extract it like so: ```js const last = require('it-last') await last(ipfs.name.resolve('/ipns/QmHash')) ``` BREAKING CHANGE: `block.rm` now returns an async iterable. BREAKING CHANGE: `block.rm` now yields objects of `{ cid: CID, error: Error }`. BREAKING CHANGE: `dht.findProvs`, `dht.provide`, `dht.put` and `dht.query` now all return an async iterable. BREAKING CHANGE: `dht.findPeer`, `dht.findProvs`, `dht.provide`, `dht.put` and `dht.query` now yield/return an object `{ id: CID, addrs: Multiaddr[] }` instead of a `PeerInfo` instance(s). BREAKING CHANGE: `refs` and `refs.local` now return an async iterable. BREAKING CHANGE: `object.data` now returns an async iterable that yields `Buffer` objects. BREAKING CHANGE: `ping` now returns an async iterable. BREAKING CHANGE: `repo.gc` now returns an async iterable. BREAKING CHANGE: `swarm.peers` now returns an array of objects with a `peer` property that is a `CID`, instead of a `PeerId` instance. BREAKING CHANGE: `swarm.addrs` now returns an array of objects `{ id: CID, addrs: Multiaddr[] }` instead of `PeerInfo` instances. BREAKING CHANGE: `block.stat` result now contains a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `key` property. BREAKING CHANGE: `bitswap.wantlist` now returns an array of [CID](https://github.com/multiformats/js-cid) instances. BREAKING CHANGE: `bitswap.stat` result has changed - `wantlist` and `peers` values are now an array of [CID](https://github.com/multiformats/js-cid) instances. BREAKING CHANGE: the `init` option passed to the IPFS constructor will now not take any initialization steps if it is set to `false`. Previously, the repo would be initialized if it already existed. This is no longer the case. If you wish to initialize a node but only if the repo exists, pass `init: { allowNew: false }` to the constructor. BREAKING CHANGE: removed `file ls` command from the CLI and HTTP API. BREAKING CHANGE: Delegated peer and content routing modules are no longer included as part of core (but are still available if starting a js-ipfs daemon from the command line). If you wish to use delegated routing and are creating your node _programmatically_ in Node.js or the browser you must `npm install libp2p-delegated-content-routing` and/or `npm install libp2p-delegated-peer-routing` and provide configured instances of them in [`options.libp2p`](https://github.com/ipfs/js-ipfs#optionslibp2p). See the module repos for further instructions: - https://github.com/libp2p/js-libp2p-delegated-content-routing - https://github.com/libp2p/js-libp2p-delegated-peer-routing
alanshaw
added a commit
to ipfs-inactive/js-ipfs-http-client
that referenced
this issue
Jan 23, 2020
TLDR;
* Remove Node.js streams and pull streams
* Remove callbacks
* Remove `peer-info` and `peer-id`
---
Now that internals are all async/await/iterables and `fetch` the next step is to bubble up that goodness to the core API, removing the multiple stream APIs and removing callback support.
I'm also proposing removing `peer-info` and `peer-id`, since these drastically increase the bundle size by pulling in `libp2p-crypto`, for which 99% of it's capability is unused. In place of `peer-id` we return a `CID`, which can easily be converted to a `PeerId` instance via:
```js
const peerId = PeerId.createFromCID(peerCid)
```
In place of `peer-info` we return an object `{ id: CID, addrs: Multiaddr[] }`, which can easily be converted to a `PeerInfo` like:
```js
const peerInfo = new PeerInfo(PeerId.createFromCID(info.id))
info.addrs.forEach(addr => peerInfo.multiaddrs.add(addr))
```
refs ipfs/js-ipfs#1670
refs ipfs/js-ipfs#2611
refs ipfs-inactive/interface-js-ipfs-core#394
TODO:
* [x] Refactor local tests
* [x] Remove `addFromFs` and `addFromUrl` and export `globSource` and `urlSource` instead
* [x] Refactor `interface-ipfs-core` tests
* [x] Document new APIs in `interface-ipfs-core`
* [x] Update README with API changes
* [x] [Document upgrade path from Node.js/pull streams to async iterables](https://gist.github.com/alanshaw/04b2ddc35a6fff25c040c011ac6acf26)
Depends on:
* [x] ipfs/js-ipfs-utils#15
* [x] ipfs-inactive/interface-js-ipfs-core#567
BREAKING CHANGE: Callbacks are no longer supported on any API methods. Please use a utility such as [`callbackify`](https://www.npmjs.com/package/callbackify) on API methods that return Promises to emulate previous behaviour.
BREAKING CHANGE: `PeerId` and `PeerInfo` classes are no longer statically exported from `ipfs-http-client` since they are no longer used internally.
BREAKING CHANGE: `pin.add` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property.
BREAKING CHANGE: `pin.ls` now returns an async iterable.
BREAKING CHANGE: `pin.ls` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property.
BREAKING CHANGE: `pin.rm` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property.
BREAKING CHANGE: `add` now returns an async iterable.
BREAKING CHANGE: `add` results now contain a `cid` property (a [CID instance](https://github.com/multiformats/js-cid)) instead of a string `hash` property.
BREAKING CHANGE: `addReadableStream`, `addPullStream` have been removed.
BREAKING CHANGE: `ls` now returns an async iterable.
BREAKING CHANGE: `ls` results now contain a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `hash` property.
BREAKING CHANGE: `files.ls` now returns an async iterable.
BREAKING CHANGE: `files.readPullStream` and `files.readReadableStream` have been removed.
BREAKING CHANGE: `files.read` now returns an async iterable.
BREAKING CHANGE: `files.lsPullStream` and `files.lsReadableStream` have been removed.
BREAKING CHANGE: `files.ls` now returns an async iterable.
BREAKING CHANGE: `files.ls` results now contain a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `hash` property.
BREAKING CHANGE: `files.ls` no longer takes a `long` option (in core) - you will receive all data by default.
BREAKING CHANGE: `files.stat` result now contains a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `hash` property.
BREAKING CHANGE: `get` now returns an async iterable. The `content` property value for objects yielded from the iterator is now an async iterable that yields [`BufferList`](https://github.com/rvagg/bl) objects.
BREAKING CHANGE: `stats.bw` now returns an async iterable.
BREAKING CHANGE: `addFromStream` has been removed. Use `add` instead.
BREAKING CHANGE: `isIPFS` is no longer exported from the client, please `npm i is-ipfs` or include the CDN script tag `<script src="https://unpkg.com/is-ipfs/dist/index.min.js"></script>` to use this utility in your applications.
BREAKING CHANGE: `addFromFs` has been removed. Please use the exported `globSource` utility and pass the result to `add`. See the [glob source documentation](https://github.com/ipfs/js-ipfs-http-client#glob-source) for more details and an example.
BREAKING CHANGE: `addFromURL` has been removed. Please use the exported `urlSource` utility and pass the result to `add`. See the [URL source documentation](https://github.com/ipfs/js-ipfs-http-client#url-source) for more details and an example.
BREAKING CHANGE: `name.resolve` now returns an async iterable. It yields increasingly more accurate resolved values as they are discovered until the best value is selected from the quorum of 16. The "best" resolved value is the last item yielded from the iterator. If you are interested only in this best value you could use `it-last` to extract it like so:
```js
const last = require('it-last')
await last(ipfs.name.resolve('/ipns/QmHash'))
```
BREAKING CHANGE: `block.rm` now returns an async iterable.
BREAKING CHANGE: `block.rm` now yields objects of `{ cid: CID, error: Error }`.
BREAKING CHANGE: `dht.findProvs`, `dht.provide`, `dht.put` and `dht.query` now all return an async iterable.
BREAKING CHANGE: `dht.findPeer`, `dht.findProvs`, `dht.provide`, `dht.put` and `dht.query` now yield/return an object `{ id: CID, addrs: Multiaddr[] }` instead of a `PeerInfo` instance(s).
BREAKING CHANGE: `refs` and `refs.local` now return an async iterable.
BREAKING CHANGE: `object.data` now returns an async iterable that yields `Buffer` objects.
BREAKING CHANGE: `ping` now returns an async iterable.
BREAKING CHANGE: `repo.gc` now returns an async iterable.
BREAKING CHANGE: `swarm.peers` now returns an array of objects with a `peer` property that is a `CID`, instead of a `PeerId` instance.
BREAKING CHANGE: `swarm.addrs` now returns an array of objects `{ id: CID, addrs: Multiaddr[] }` instead of `PeerInfo` instances.
BREAKING CHANGE: `block.stat` result now contains a `cid` property (whose value is a [CID instance](https://github.com/multiformats/js-cid)) instead of a `key` property.
BREAKING CHANGE: `bitswap.wantlist` now returns an array of [CID](https://github.com/multiformats/js-cid) instances.
BREAKING CHANGE: `bitswap.stat` result has changed - `wantlist` and `peers` values are now an array of [CID](https://github.com/multiformats/js-cid) instances.
|
Thank you EVERYONE who contributed to this. It has been the biggest journey I've ever taken - you've all been with me the whole way and you're all incredible. There's still some bits to finish up but expect to see all this goodness in js-ipfs 0.41 soon. I'll also write a blog post and try to note down some of the many wins this work has made and enabled for the future. Once again ❤️ to you all. |
This was referenced Feb 5, 2020
Closed
achingbrain
pushed a commit
that referenced
this issue
Jun 22, 2020
On `js-ipfs@0.41`, we removed the `websocket-star` module from the libp2p default configuration for browser nodes in `js-ipfs`. We removed it because this module was not refactored during #1670, since the libp2p goal is to [sunset the star protocols](libp2p/js-libp2p#385) and we preferred to use the time refactoring `websocket-star` into improving the browser support. In the refactor, the `webrtc-star` module was refactored so that browser users could discover other peers in the network. In addition, `circuit-relay` may be used for establishing connections between browser nodes using `websockets`. However, this migration has not been smooth for `js-ipfs` users. Users previously using `websocket-star` swarm addresses, did not have enough visibility of this change, since the removal of `websocket-star` was transparent for them and on the default `js-ipfs` release and not technically a programmable breaking change. With the above in mind, once `js-ipfs` users updated they started getting errors from `js-libp2p` in scenarios where they were providing only `websocket-star` swarm addresses. When `js-libp2p` receives listening multiaddrs, it throws an error if it cannot use any to listen on the configured transports, which was the case here (For instance #2779). In `js-libp2p`, we added an option to tolerate these errors [libp2p/js-libp2p#643](libp2p/js-libp2p#643), which was also mentioned for some other cases earlier. After syncing about this issue, we decided to temporarily throw an error when `js-ipfs` receives `websocket-star` multiaddrs. This aims to help users who still need to migrate to identify the problem faster.
github-actions bot
pushed a commit
to ipfs-examples/js-ipfs-custom-libp2p
that referenced
this issue
Jun 14, 2021
Since I was going through these anyway to ensure they work with [0.37](ipfs/js-ipfs#2192) I've updated the examples to use the new [`IPFS.create` constructor](https://github.com/ipfs/js-ipfs#ipfs-constructor) and switched to using the promised API so that we onboard new users to using promises and minimise the disruption caused when ipfs/js-ipfs#1670 bubbles up to here. License: MIT Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
oliveriosousa
pushed a commit
to ipfs-examples/js-ipfs-examples
that referenced
this issue
Jul 26, 2021
…#2068) This is part of the Awesome Endeavour: Async Iterators: ipfs/js-ipfs#1670 Depends on * [x] ipld/js-ipld-dag-pb#137 * [x] ipfs-inactive/interface-js-ipfs-core#473 * [x] ipfs-inactive/js-ipfs-http-client#1010 * [x] ipfs/js-ipfs-http-response#25 resolves #1995 BREAKING CHANGE: The default string encoding for version 1 CIDs has changed to `base32`. IPLD formats have been updated to the latest versions. IPLD nodes returned by `ipfs.dag` and `ipfs.object` commands have significant breaking changes. If you are using these commands in your application you are likely to encounter the following changes to `dag-pb` nodes (the default node type that IPFS creates): * `DAGNode` properties have been renamed as follows: * `data` => `Data` * `links` => `Links` * `size` => `size` (Note: no change) * `DAGLink` properties have been renamed as follows: * `cid` => `Hash` * `name` => `Name` * `size` => `Tsize` See CHANGELOGs for each IPLD format for it's respective changes, you can read more about the [`dag-pb` changes in the CHANGELOG](https://github.com/ipld/js-ipld-dag-pb/blob/master) License: MIT Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
oliveriosousa
pushed a commit
to ipfs-examples/js-ipfs-examples
that referenced
this issue
Jul 26, 2021
Since I was going through these anyway to ensure they work with [0.37](ipfs/js-ipfs#2192) I've updated the examples to use the new [`IPFS.create` constructor](https://github.com/ipfs/js-ipfs#ipfs-constructor) and switched to using the promised API so that we onboard new users to using promises and minimise the disruption caused when ipfs/js-ipfs#1670 bubbles up to here. License: MIT Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
and others
JS IPFS supports two types of stream at the API level, but uses pull streams for internals. If I was working on js-ipfs at the time I'd have made the same decision. Since then, async/await became part of the JS language and the majority of JavaScript runtimes now support async/await, async iterators and for/await/of (i.e. no need to transpile). These tools give us the power to stream data without needing to rely on a library.
Just because there are new language features available doesn't mean we should switch to using them. It's a significant upheaval to change the core interface spec and its implementations (js-ipfs, js-ipfs-api etc.) without good reason.
That said, it has become apparent that there are a growing number of good reasons to do this:
asyncmodule.npm installtime - fewer dependencies to install.addPullStream,addReadableStream.interface-ipfs-corecompatible interface becomes a whole lot easier, no dual promise/callback API and no multiple stream implementation variations of the same function. It would also reduce the number of tests in theinterface-ipfs-coretest suite for the same reasons.awaitstack traces better than promise stack tracesThe rough plan is:
This will require significant discussion and coordination from the JS teams. We'll need to reach agreement on the best API to expose for each module and manage releases carefully.
Below is a table documenting the multiformats, libp2p, ipld and ipfs modules that will likely need work. I suspect that some of these modules can be removed as they do not expose an async API. Likewise there's probably modules that got missed. If you notice either way then please edit the table or comment below.
If you'd like to own this enhancement task for a module then please comment below (or add yourself to the table if you know what you are doing). Please open a PR against the module asap (does not have to be anywhere near complete!) so we can add it here also and track progress.
Core
Multiformats
libp2p
libp2p/js-libp2p#266
IPLD
Please read #1670 (comment) before contributing.
IPFS
Dependents
These modules use IPFS and fall under the ipfs/ipfs-shipyard umbrella so should also be updated.
Current progress:
The text was updated successfully, but these errors were encountered: