feat!: use Helia's blockBroker interface #406

SgtPooki · 2023-11-29T01:10:14Z

Update Helia and libp2p dependencies
- also updated others. basically ncu '/ipfs|libp2p|ipld|multi/'
Use Helia blockBroker interface for block retrieval
~~Use kubo-rpc-client as a blockBroker~~
- Use Kubo as a trustless gateway
- Remove dependency on kubo-rpc-client (and many older ipfs dependencies)
Remove all places where kuboClient was passed through since that is no longer needed.
Update hashers passed to helia so it can support parsing of multiple content types
Disable libp2p autoDialer because we only want to fetch on demand.

These updates also put a big dent in dependencies holding back ipfs-webui ipfs/ipfs-webui#1965

whizzzkid

wow 🤩

src/bundles/helia.ts

src/lib/get-raw-block.ts

src/lib/init-helia.ts

SgtPooki

self review and explainers

SgtPooki · 2023-11-29T18:43:10Z

dev/devPage.jsx

+    const { doInitHelia } = props
    useEffect(() => {
-      props.doInitHelia()
-    }, [props])
+      doInitHelia()
+    }, [doInitHelia])


this fixes an issue where doInitHelia was being called on every page load.

SgtPooki · 2023-11-29T18:44:26Z

src/components/object-info/ObjectInfo.jsx

+/**
+ * replace bigint or other non-JSON-serializable values with appropriate values for the react-inspector
+ * Note that data for some blocks (e.g. bafyreicnokmhmrnlp2wjhyk2haep4tqxiptwfrp2rrs7rzq7uk766chqvq) currently do not
+ * look like NormalizedDagNode['data']
+ *
+ * @param {import('../../types').NormalizedDagNode['data']} data
+ */
+const getObjectInspectorData = (data) => {
+  if (data == null) return data
+  if (data.blockSizes != null) {
+    data.blockSizes = data.blockSizes.map(Number)
+  }
+  return data
+}


needed since the unixfs update uses bigint now.

SgtPooki · 2023-11-29T18:44:32Z

src/components/object-info/ObjectInfo.jsx

+/**
+ * @param {object} props
+ * @param {import('react-i18next').TFunction} props.t
+ * @param {boolean} props.tReady
+ * @param {string} props.className
+ * @param {string} props.type
+ * @param {string} props.cid
+ * @param {string} props.localPath
+ * @param {bigint} props.size
+ * @param {import('../../types').NormalizedDagNode['data']} props.data
+ * @param {object[]} props.links
+ * @param {string} props.format
+ * @param {Function} props.onLinkClick
+ * @param {string} props.gatewayUrl
+ * @param {string} props.publicGatewayUrl
+ */
 const ObjectInfo = ({ t, tReady, className, type, cid, localPath, size, data, links, format, onLinkClick, gatewayUrl, publicGatewayUrl, ...props }) => {


SgtPooki · 2023-11-29T18:44:43Z

src/components/object-info/ObjectInfo.jsx

 const ObjectInfo = ({ t, tReady, className, type, cid, localPath, size, data, links, format, onLinkClick, gatewayUrl, publicGatewayUrl, ...props }) => {
+  if (!tReady) return null


this is always true in testing, but may not always be in prod.

SgtPooki · 2023-11-29T18:45:00Z

src/components/object-info/ObjectInfo.jsx

@@ -130,7 +162,7 @@ const ObjectInfo = ({ t, tReady, className, type, cid, localPath, size, data, li
          <div className='dt dt--fixed pt2'>
            {/* eslint-disable-next-line jsx-a11y/label-has-associated-control */}
            <label className='dtc silver tracked ttu f7' style={{ width: 48 }}>Size</label>
-            <div className='dtc truncate charcoal monospace'>{humansize(size)}</div>
+            <div className='dtc truncate charcoal monospace'>{humansize(Number(size))}</div>


bigint conversion

SgtPooki · 2023-11-29T18:45:58Z

src/lib/get-codec-for-cid.ts

 interface CodecWrapper<DecodedType = any> {
-  decode: (bytes: Uint8Array) => DecodedType
-  resolve: (path: string, bytes: Uint8Array) => Promise<ResolveType<DecodedType>>
+  decode(bytes: Uint8Array): DecodedType
+  resolve(path: string, bytes: Uint8Array): Promise<ResolveType<DecodedType>>
 }


eslint-config-ipfs auto lint updates

SgtPooki · 2023-11-29T18:46:13Z

src/lib/get-raw-block.ts

    const timeoutId = setTimeout(() => { abortController.abort('Request timed out') }, timeout)
-    const rawBlock = await Promise.any([kuboClient.block.get(cid), getBlockFromAnyGateway(cid, abortController.signal), helia.blockstore.get(cid, { signal: abortController.signal })])
-    abortController.abort('Content obtained') // abort any other requests.
+    const rawBlock = await helia.blockstore.get(cid, { signal: abortController.signal })


this is so much nicer =D

SgtPooki · 2023-11-29T18:46:21Z

src/lib/helpers.ts

@@ -11,7 +11,7 @@ export interface DigestFn {
  (data: Uint8Array): Promise<MultihashDigest<number>>
 }

-export async function addDagNodeToHelia <T> (helia: Helia, codec: { encode: (n: T) => Uint8Array, code: number }, node: T, digestFn?: DigestFn): Promise<CID> {
+export async function addDagNodeToHelia <T> (helia: Helia, codec: { encode(n: T): Uint8Array, code: number }, node: T, digestFn?: DigestFn): Promise<CID> {


eslint-config-ipfs auto lint updates

SgtPooki · 2023-11-29T18:47:12Z

src/lib/init-helia.ts

@@ -52,11 +51,18 @@ export default async function initHelia (kuboClient: ReturnType<typeof createKub
    ],
    services: {
      identify: identifyService(),
-      autoNAT: autoNATService()
+      autoNAT: autoNATService(),
+      delegatedRouting: () => createDelegatedRoutingV1HttpApiClient('https://delegated-ipfs.dev')


we use delegated routingv1httpApi instead of local kubo

SgtPooki · 2023-11-29T18:48:27Z

vitest.config.js

+export default defineConfig((configEnv) => mergeConfig(
+  viteConfig(configEnv),
+  defineConfig({
+    test: {
+      globals: true,
+      environment: 'jsdom',
+      setupFiles: './test/unit/setup.js',
+      include: [
+        'src/**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'
+      ],
+      deps: {
+        inline: [
+          'ipld-explorer-components'
+        ]
+      }
    }
-  }
-}))
+  })
+))


clearing out package-lock.json and refreshing it caused an error here.. vitest must've updated their api (they're still pre v1)

## [5.0.0](v4.0.3...v5.0.0) (2023-11-30) ### ⚠ BREAKING CHANGES * update of many ipfs/ipld/libp2p/helia deps that may break consumers. API of ipld-explorer-components has not changed. ### Features * use Helia's blockBroker interface ([#406](#406)) ([c964a76](c964a76)) * use ipfs-css colors and add dag-jose example ([#408](#408)) ([4e96491](4e96491)) ### Trivial Changes * bump protobufjs from 6.11.3 to 6.11.4 ([#393](#393)) ([f8db274](f8db274)) * **ci:** remove circleci config ([#388](#388)) ([13698af](13698af)) * pull new translations ([#390](#390)) ([2caac4a](2caac4a)) * pull new translations ([#396](#396)) ([73799f8](73799f8)) * pull new translations ([#405](#405)) ([5805bd8](5805bd8))

SgtPooki added 2 commits November 28, 2023 16:29

deps: update helia

6f29b4d

feat: use new Helia blockBroker interface

73cfbe6

SgtPooki requested a review from a team as a code owner November 29, 2023 01:10

SgtPooki requested a review from lidel November 29, 2023 01:10

SgtPooki added 3 commits November 28, 2023 17:16

deps: update 'ipld|multi' deps

c2fa4ad

deps: update 'ipfs' deps

dd05dcd

chore: fix lint after eslint-config-ipfs update

57fe725

SgtPooki changed the title ~~feat: use Helia's blockBroker interface~~ feat!: use Helia's blockBroker interface Nov 29, 2023

SgtPooki mentioned this pull request Nov 29, 2023

feat: support ipns/<name> & dnsLink resolution #400

Open

feat!: remove kubo-rpc-client entirely

e18328b

whizzzkid approved these changes Nov 29, 2023

View reviewed changes

src/bundles/helia.ts Show resolved Hide resolved

src/lib/get-raw-block.ts Show resolved Hide resolved

src/lib/init-helia.ts Outdated Show resolved Hide resolved

SgtPooki added 4 commits November 29, 2023 09:48

chore: apiOpts -> kuboGatewayOptions

3b5f578

fix: types and errors

5a035a1

fix: bigint use in ObjectInfo.jsx

d05498b

fix: tests and vitest config

252c6fb

SgtPooki commented Nov 29, 2023

View reviewed changes

SgtPooki merged commit c964a76 into master Nov 29, 2023
16 checks passed

SgtPooki deleted the feat/use-blockBroker-interface branch November 29, 2023 18:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: use Helia's blockBroker interface #406

feat!: use Helia's blockBroker interface #406

SgtPooki commented Nov 29, 2023 •

edited

Loading

whizzzkid left a comment

SgtPooki left a comment

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

SgtPooki Nov 29, 2023

		const ObjectInfo = ({ t, tReady, className, type, cid, localPath, size, data, links, format, onLinkClick, gatewayUrl, publicGatewayUrl, ...props }) => {
		if (!tReady) return null

feat!: use Helia's blockBroker interface #406

feat!: use Helia's blockBroker interface #406

Conversation

SgtPooki commented Nov 29, 2023 • edited Loading

whizzzkid left a comment

Choose a reason for hiding this comment

SgtPooki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SgtPooki commented Nov 29, 2023 •

edited

Loading