feat!: upgrade to esm libp2p/ipfs #462

achingbrain · 2022-05-09T10:29:24Z

Updates deps and adds types to tests.

BREAKING CHANGE: needs ESM release of ipfs/libp2p

Updates deps and adds types to tests. BREAKING CHANGE: needs ESM release of ipfs/libp2p

BigLep · 2022-05-27T14:07:33Z

CI won't pass until new js-ipfs ships.

SgtPooki · 2022-06-08T21:51:11Z

test/cid-version-agnostic.js

-      go1.api.swarm.connect(go0.api.peerId.addresses[0]),
-      js0.api.swarm.connect(go0.api.peerId.addresses[0]),
-      go0.api.swarm.connect(js0.api.peerId.addresses[0])
+      js0.api.swarm.connect(js1.peer.addresses[0]),


Some type failures here. not sure if they're expected.

SgtPooki · 2022-06-08T21:52:21Z

test/circuit.js

@@ -5,11 +5,16 @@ import allV1 from './circuit/v1/all.js'
 import allV2 from './circuit/v2/all.js'


You should add the ts-check pragma to the top of files if you're using jsdoc typings until you move this to typescript.

test/circuit/v1/all.js

test/exchange-files.js

SgtPooki · 2022-06-08T22:07:39Z

test/exchange-files.js

@@ -94,13 +101,15 @@ const timeout = isCi ? 15 * min : 10 * min
 describe('exchange files', function () {
  this.timeout(timeout)

+  /** @type {Record<string, ('go' | 'js')[]>} */


This seems to be an incorrect type for what you're trying to accomplish. lmk if you want to allow a 3-tuple.

type arr = ('go' | 'js')[] const array: arr = ['go', 'js', 'go'] // False Positive const array1: arr = ['go', 'go'] // True positive const array2: arr = ['go', 'js'] // True positive const array3: arr = ['js', 'go'] // True positive const array4: arr = ['js', 'js'] // True positive type goOrJs = 'go' | 'js' type arrFix = [goOrJs, goOrJs] const arrayFixed: arrFix = ['go', 'js', 'go'] // True Negative const arrayFixedException1: arrFix = ['go', 'go'] // True positive const arrayFixedException2: arrFix = ['go', 'js'] // True positive const arrayFixedException3: arrFix = ['js', 'go'] // True positive const arrayFixedException4: arrFix = ['js', 'js'] // True positive

Playground Link: Provided

SgtPooki · 2022-06-08T22:20:38Z

test/pin.js

@@ -50,6 +60,11 @@ describe('pin', function () {
    return daemon
  }

+  /**
+   * @template T


You should restrict the types of your template. @template {Type} T

SgtPooki · 2022-06-08T22:24:34Z

test/utils/daemon-factory.js

+    // @ts-expect-error env var could be undefined
    ipfsHttpModule = await import(process.env.IPFS_JS_HTTP_MODULE)
  } catch {
    ipfsHttpModule = await import('ipfs-http-client')


Elsewhere you use await import(process.env.IPFS_JS_HTTP_MODULE || 'ipfs-http-client') should we create a wrapper for the module import and use that everywhere?

SgtPooki · 2022-06-08T22:24:43Z

test/utils/daemon-factory.js

+    // @ts-expect-error env var could be undefined
    ipfsModule = await import(process.env.IPFS_JS_MODULE)
  } catch {
    ipfsModule = await import('ipfs')


SgtPooki · 2022-06-08T22:25:25Z

test/utils/relayd.js

+// @ts-expect-error no types
 import goenv from 'go-platform'


create global.d.ts with declare module 'go-platform'

SgtPooki · 2022-06-08T22:25:48Z

test/utils/relayd.js

@@ -28,7 +37,7 @@ export async function getRelayV (version, factory) {
      id = text.split('I am')[1].split('\n')[0].trim()
    }
  }
-  const config = JSON.parse(fs.readFileSync(configPath))
+  const config = JSON.parse(fs.readFileSync(configPath).toString())


try/catch around this?

SgtPooki

LGTM (the changes since last review)

SgtPooki · 2022-09-13T18:21:14Z

tried running code from this locally since I'm on mac and that CI test is failing and I get a different error:

  1) circuit
       v1
         js-rv1-js
           connect:
     HTTPError: All promises were rejected
      at Object.errorHandler [as handleError] (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/ipfs-http-client/src/lib/core.js:103:15)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)
      at Client.fetch (node_modules/ipfs-utils/src/http.js:149:9)
      at Object.connect (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/ipfs-http-client/src/swarm/connect.js:14:17)
      at Object.connect (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/test/utils/circuit.js:283:3)



Command failed with exit code 1: mocha test/node.{js,cjs,mjs} test/**/*.spec.{js,cjs,mjs} dist/test/node.{js,cjs,mjs} dist/test/**/*.spec.{js,cjs,mjs} --ui bdd --require source-map-support/register --timeout=60000 --bail

SgtPooki · 2022-09-13T18:33:57Z

here is the output from DEBUG='ipfs*,libp2p*' GOLOG_LOG_LEVEL='debug,core/server=debug' npm run test -- -g 'js-rv1-js' -d &> js-rv1-js.log: js-rv1-js.log

top of the stack trace:

  ipfsd-ctl:daemon:stderr 2022-09-13T18:31:53.261Z libp2p:upgrader:error error multiplexing outbound stream Error: protocol selection failed
  ipfsd-ctl:daemon:stderr     at Module.select (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/@libp2p/multistream-select/dist/src/select.js:43:19)
  ipfsd-ctl:daemon:stderr     at processTicksAndRejections (node:internal/process/task_queues:96:5)
  ipfsd-ctl:daemon:stderr     at async EventTarget._multiplexOutbound (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/libp2p/dist/src/upgrader.js:483:42)
  ipfsd-ctl:daemon:stderr     at async EventTarget.upgradeOutbound (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/libp2p/dist/src/upgrader.js:206:37)
  ipfsd-ctl:daemon:stderr     at async WebSockets.dial (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/@libp2p/websockets/dist/src/index.js:28:22)
  ipfsd-ctl:daemon:stderr     at async EventTarget.dial (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/libp2p/dist/src/transport-manager.js:78:20)
  ipfsd-ctl:daemon:stderr     at async DialRequest.dialAction (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/libp2p/dist/src/connection-manager/dialer/index.js:179:20)
  ipfsd-ctl:daemon:stderr     at async file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/libp2p/dist/src/connection-manager/dialer/dial-request.js:67:28
  ipfsd-ctl:daemon:stderr     at async Promise.any (index 0)
  ipfsd-ctl:daemon:stderr     at async DialRequest.run (file:///Users/sgtpooki/code/work/protocol.ai/ipfs/interop/node_modules/libp2p/dist/src/connection-manager/dialer/dial-request.js:53:20) {
  ipfsd-ctl:daemon:stderr   code: 'ERR_UNSUPPORTED_PROTOCOL'
  ipfsd-ctl:daemon:stderr }

achingbrain · 2022-09-14T17:23:34Z

tried running code from this locally since I'm on mac and that CI test is failing and I get a different error:

That's very strange - I'm also on a Mac and I don't see any failures at all, nor on Windows running under Parallels, only on CI. *shakes fist at sky*.

I didn't have any time to look at this more today, I'll try again tomorrow.

SgtPooki · 2022-09-15T19:06:51Z

tried running code from this locally since I'm on mac and that CI test is failing and I get a different error:

That's very strange - I'm also on a Mac and I don't see any failures at all, nor on Windows running under Parallels, only on CI. shakes fist at sky.

I didn't have any time to look at this more today, I'll try again tomorrow.

Are you on M1 mac? (sw_vers && arch)

╰─ ✔ ❯ gh-issue-info
ProductName:    macOS
ProductVersion: 12.5.1
BuildVersion:   21G83
arm64
nodejs          16.17.0         /Users/sgtpooki/code/work/protocol.ai/ipfs/interop/.tool-versions

Also, which node version did you use? GH action is using 16.17.0

achingbrain · 2022-09-16T06:52:04Z

Yes, M1 Max and node 16.17.0. The only difference I can see is that the CI machine will be resource constrained and my laptop significantly less so, so there must be a difference in timing somewhere.

achingbrain · 2022-09-16T10:51:00Z

ipfsd-ctl:daemon:stderr 2022-09-13T18:31:53.261Z libp2p:upgrader:error error multiplexing outbound stream Error: protocol selection failed

I just saw this locally running the tests on a linux VM. Removing the downloaded libp2p-relay-daemon binary and old *.identity files from /scripts in the repo "fixed" it, now all the tests are back to passing.

achingbrain · 2022-09-16T17:26:42Z

achingbrain · 2022-09-16T17:29:59Z

Updates all dependencies and adds types to tests. BREAKING CHANGE: uses ESM release of ipfs/libp2p

BigLep · 2022-09-16T17:32:34Z

Wow @achingbrain ! You're a hero. Thanks a lot!

Updates all dependencies and adds types to tests. BREAKING CHANGE: uses ESM release of ipfs/libp2p

SgtPooki · 2022-09-16T18:59:33Z

@achingbrain you rock!

## [9.0.0](v8.0.11...v9.0.0) (2022-09-17) ### ⚠ BREAKING CHANGES * uses ESM release of ipfs/libp2p ### Features * upgrade to esm libp2p/ipfs ([#462](#462)) ([b006606](b006606)) ### Bug Fixes * remove randomness from get directory tests ([#497](#497)) ([56f3370](56f3370))

achingbrain added 23 commits May 9, 2022 11:28

feat: upgrade to esm libp2p/ipfs

46b7554

Updates deps and adds types to tests. BREAKING CHANGE: needs ESM release of ipfs/libp2p

chore: upgrade ipfsd-ctl

4314291

fix: update files

99e5750

fix: deps

f28aaeb

chore: update test

a316b01

chore: run browser interop tests

b6191c7

chore: update files tests

72248b7

chore: refactor ipns pubsub tests to remove asserting on failure

2ec9f4e

chore: only run one dht test temporarily

347fb03

chore: run more tests

d328428

chore: run more tests

fd977dd

chore: longer delay

a14d853

chore: delay after creating network?

c8970fb

chore: revert

316abaf

chore: fewer tests

0487037

chore: wip

afe6577

chore: check routing tables before test

6c68ded

chore: run more

4177511

chore: check routing tables before tests

40a51b2

chore: shorten timeout

6954e3a

chore: wait for all nodes to be connected

2552897

chore: update test

104df78

fix: skip flaky test

59ec9f1

BigLep requested a review from SgtPooki May 27, 2022 14:07

BigLep marked this pull request as draft May 27, 2022 14:08

SgtPooki suggested changes Jun 8, 2022

View reviewed changes

achingbrain added 2 commits June 22, 2022 11:04

chore: fix ws filters

e4b86d3

chore: correct from type

9ef44ef

SgtPooki reviewed Jun 27, 2022

View reviewed changes

achingbrain added 2 commits September 13, 2022 16:46

chore: run all test

b393ee9

chore: enable other platforms

c7067a8

achingbrain added 7 commits September 16, 2022 16:56

chore: fix up exchange files on windows to not use filesystem

f693dd0

chore: restore depth

985f042

chore: decrease timeout

623fc48

chore: linting

11f5a90

chore: timeout is already set

84cc965

chore: reject with expection failure

5e8d78b

chore: timeout publisher and subscriber individually

4251b6d

achingbrain merged commit 939e6d6 into master Sep 16, 2022

achingbrain deleted the feat/upgrade-to-esm-libp2p branch September 16, 2022 17:30

achingbrain added a commit that referenced this pull request Sep 16, 2022

feat!: upgrade to esm libp2p/ipfs (#462)

29d29e1

Updates all dependencies and adds types to tests. BREAKING CHANGE: uses ESM release of ipfs/libp2p

achingbrain added a commit that referenced this pull request Sep 16, 2022

feat!: upgrade to esm libp2p/ipfs (#462)

e2cb915

Updates all dependencies and adds types to tests. BREAKING CHANGE: uses ESM release of ipfs/libp2p

achingbrain added a commit that referenced this pull request Sep 16, 2022

feat!: upgrade to esm libp2p/ipfs (#462)

b006606

Updates all dependencies and adds types to tests. BREAKING CHANGE: uses ESM release of ipfs/libp2p

This was referenced Sep 19, 2022

fix!: require V2 signatures ipfs/js-ipns#180

Merged

Create js-kubo-rpc-client library ipfs/kubo#9125

Closed

achingbrain restored the feat/upgrade-to-esm-libp2p branch October 27, 2022 16:04

achingbrain deleted the feat/upgrade-to-esm-libp2p branch October 27, 2022 16:26

achingbrain restored the feat/upgrade-to-esm-libp2p branch October 27, 2022 16:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: upgrade to esm libp2p/ipfs #462

feat!: upgrade to esm libp2p/ipfs #462

achingbrain commented May 9, 2022

BigLep commented May 27, 2022 •

edited

Loading

SgtPooki Jun 8, 2022

SgtPooki Jun 8, 2022

SgtPooki Jun 8, 2022

SgtPooki Jun 8, 2022

SgtPooki Jun 8, 2022

SgtPooki Jun 8, 2022

SgtPooki Jun 8, 2022

SgtPooki Jun 8, 2022

SgtPooki left a comment •

edited

Loading

SgtPooki commented Sep 13, 2022

SgtPooki commented Sep 13, 2022

achingbrain commented Sep 14, 2022 •

edited

Loading

SgtPooki commented Sep 15, 2022

achingbrain commented Sep 16, 2022

achingbrain commented Sep 16, 2022

achingbrain commented Sep 16, 2022

achingbrain commented Sep 16, 2022

BigLep commented Sep 16, 2022

SgtPooki commented Sep 16, 2022

		@@ -5,11 +5,16 @@ import allV1 from './circuit/v1/all.js'
		import allV2 from './circuit/v2/all.js'

feat!: upgrade to esm libp2p/ipfs #462

feat!: upgrade to esm libp2p/ipfs #462

Conversation

achingbrain commented May 9, 2022

BigLep commented May 27, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SgtPooki left a comment • edited Loading

Choose a reason for hiding this comment

SgtPooki commented Sep 13, 2022

SgtPooki commented Sep 13, 2022

achingbrain commented Sep 14, 2022 • edited Loading

SgtPooki commented Sep 15, 2022

achingbrain commented Sep 16, 2022

achingbrain commented Sep 16, 2022

achingbrain commented Sep 16, 2022

achingbrain commented Sep 16, 2022

BigLep commented Sep 16, 2022

SgtPooki commented Sep 16, 2022

BigLep commented May 27, 2022 •

edited

Loading

SgtPooki left a comment •

edited

Loading

achingbrain commented Sep 14, 2022 •

edited

Loading