feat: use ipld-explorer-components@4.0.0 #119

SgtPooki · 2023-05-25T19:10:43Z

Note that this code is based on changes included in ipfs/ipld-explorer-components#360 and must wait to be merged until we can publish a new ipld-explorer-components lib once that PR is merged.

check out the preview: https://bafybeif65hxysjwygngoahx7v6a2ouzsfi4y24u7ovr4vd5ya4jhq7ngjy.on.fleek.co/

Check out the demo:

explore.ipld.io-2023-05-25-update.mp4

fixes #117

~~Note that fleek build is failing due to being set to node16,~~ RESOLVED

~~but we can't update fleek settings unless 5bb40dd is merged to master.~~ RESOLVED see fix: npm install works with node@18.16.0 #120

following steps from https://www.robinwieruch.de/vite-create-react-app/

SgtPooki · 2023-05-25T20:16:58Z

package.json

-    "ipld-git": "^0.2.3",
-    "ipld-raw": "^2.0.1",
-    "ipld-zcash": "^0.4.1",
+    "ipld-explorer-components": "file:../../ipfs/ipld-explorer-components/ipld-explorer-components-3.0.3.tgz",


will be updated once ipfs/ipld-explorer-components#360 is merged

SgtPooki · 2023-05-25T20:17:07Z

.env

@@ -1 +1 @@
-REACT_APP_VERSION=$npm_package_version
+VITE_VERSION=$npm_package_version


part of vite migration

SgtPooki · 2023-05-25T20:17:16Z

index.html

+    <link rel="manifest" href="/manifest.json">
+    <link rel="shortcut icon" href="/favicon.ico">


part of vite migration

SgtPooki · 2023-05-25T20:17:19Z

index.html

@@ -36,5 +27,6 @@
      To begin the development, run `npm start` or `yarn start`.
      To create a production bundle, use `npm run build` or `yarn build`.
    -->
+    <script type="module" src="/src/index.jsx"></script>


part of vite migration

SgtPooki · 2023-05-25T20:17:43Z

package.json

+    "serve": "vite preview",
+    "test": "run-s test:unit test:e2e",
+    "test:unit": "vitest run --environment=node",
+    "test:unit-playwright": "playwright test -c playwright-ct.config.js",


not currently used, but available to run unit tests via playwright if we want to normalize on tests.

SgtPooki · 2023-05-25T20:19:22Z

src/App.jsx

@@ -11,7 +11,7 @@ export class App extends Component {
    doUpdateUrl: PropTypes.func.isRequired,
    queryObject: PropTypes.object.isRequired,
    registerServiceWorker: PropTypes.func,
-    route: PropTypes.oneOfType([PropTypes.func, PropTypes.element]).isRequired
+    route: PropTypes.oneOfType([PropTypes.func, PropTypes.element, PropTypes.elementType]).isRequired


fixes an in-app error regarding prop types mismatch.

SgtPooki · 2023-05-25T20:20:21Z

test/e2e/smoke.test.js

-  // const page = await browser.newPage()
-  const page = (await browser.pages())[0]
+// it('should resolve a dag-pb cid with a path', async () => {
+//   // Save 3s per run by using the initial page rather than creating a new one!


todo: need to migrate this test code to playwright.

SgtPooki · 2023-05-26T18:23:41Z

interesting issue found when doing js-ipfs deprecation: ipfs/js-ipfs#1416

…fbe9327740b

lidel

Tested latest version from https://bafybeiepdrogejw7eji6qdbq3hpd3ewuuhecyvf4nyofvhntsbyb3w7tqa.on.fleek.co

@SgtPooki looks ok for available content, but trying to fetch something new timeouts bit too early? Lmk if I got things right.

I tried to add a PDF and explore its CID (bafybeibms6xfhs4kx3tun7dg6bjbj5obca7yj43cl5jl3pc2n7muw6ipfm) and the gateway request for the root block was stuck in "Pending":

and after a while that request got cancelled:

So two problems:

Why is request for single block in Pending? It should fhit gateway asap, before anything else. Are WebTransport requests still suffocating the request queue browser allocates to the page?
Why it cancels (timeouts?) so fast? As long gateway request is in progress or pending, it should not timeout.

SgtPooki · 2023-07-04T00:47:49Z

Why is request for single block in Pending? It should [hit] gateway asap, before anything else. Are WebTransport requests still suffocating the request queue browser allocates to the page?

Helia should not be making any requests except for to the delegatedClient via kubo-rpc-client: https://github.com/ipfs/ipld-explorer-components/blob/ca19310defb84611f55c46b1efd399217ba4a8c6/src/lib/init-helia.ts#L28-L63

Why it cancels (timeouts?) so fast? As long gateway request is in progress or pending, it should not timeout.

Mostly to keep the site responsive and gateway-friendly. I've currently got the timeout set to 30 seconds. I've heard significant feedback from users that explore.ipld.io would spin and spin and they never knew if their content was going to load. I think a much better approach is to fail early, and have the users refresh/retry if they want.

Were you able to fetch your content from http://ipfs.io/ipfs/bafybeibms6xfhs4kx3tun7dg6bjbj5obca7yj43cl5jl3pc2n7muw6ipfm directly? Maybe you removed the content, but I just attempted to access it and it failed after

We shouldn't have users guessing whether a request is going to succeed or not. Also, for failed gateway requests, the gateway keeps the connection open for up to 2 minutes, and sends back almost 1K of data:

If we cancel requests that are pending for 30 seconds, we can free up the gateway to do other work, and the user can re-request things if they want. If there is a different timeout value you think would be better or a better way to handle this without significant feature work, then please let me know.

Drawbacks

The code is currently set to try the gateways in serial which means if ipfs.io takes more than timeout ms, dweb.link is never attempted. If ipfs.io takes less than timeout ms, then dweb.link is never attempted. These could be made parallel since we removed the non-delegated Helia fetch attempts.

Other things to think about

Higher default timeout - Gives users the expectation that something will resolve, when in all likelihood, if it hasn't resolved in 30 seconds, it probably won't. In my experience improving ipld-explorer-components, 30 seconds was a pretty nice spot.
Better error message - I looked into displaying a better error message previously, but the way the network requests are hidden so deep in ipld-explorer-components, it was a larger task than I wanted to fit into feat!: remove all old deps ipfs/ipld-explorer-components#360.
- It would be great to be able to show more details about failed fetch attempts ("failed to load via ipfs.io: Gateway Timeout", "failed to load via dweb.link: Gateway Timeout", "failed to load via local node: request blocked", etc..) and also where we were able to successfully fetch the request from
Configurable timeouts - Allow users to specify whether they want the request to timeout (i'm just curiously browsing), or whether they're willing to wait for long enough to load it (I need to actually explore this content for some reason)
Stagger parallel gateway requests - We should not broadcast to all gateways at the same time, but we could delay the requests to subsequent gateways by something liketimeout/2 for example.
Separate timeouts + staggered requests - We could have a global-get-raw-block-timeout and stagger requests to gateways by ( global-get-raw-block-timeout / N+1 gateways tested)

lidel

Thank you @SgtPooki, failing fast at 30s mark makes sense as a trade-off,
the problem is bad UX on timeout – user adds data to their local node, then tries to inspect it, timeouts, app looks broken:

One way to fix this without rewriting the entire thing is to inspect error before printing the red box and if it is timeout/promise cancellation, replace it with friendly "Failed to fetch content in 30s, refresh page to retry or try different CID."? It would be good enough to unblock this PR.

Since ipld-explorer-components@4.0.0 is released, would it be ok to merge this as-is, and fill issues for any remaining work ("Drawbacks", "Other things to think about"), and do it in future PRs (if we have time).

Would be shame if this PR got stuck in review forever: It is already better than the old version! :)

SgtPooki · 2023-07-07T17:35:27Z

Failed to fetch content in 30s, refresh page to retry or try different CID

I see your approval, but I will work on that error message prior to merging.

SgtPooki · 2023-07-12T02:15:05Z

@lidel PR out to address this in ipld-explorer-components: ipfs/ipld-explorer-components#378

SgtPooki · 2023-07-12T17:23:43Z

new preview url, https://bafybeif65hxysjwygngoahx7v6a2ouzsfi4y24u7ovr4vd5ya4jhq7ngjy.on.fleek.co/, updated in description.

tested new error message at https://bafybeif65hxysjwygngoahx7v6a2ouzsfi4y24u7ovr4vd5ya4jhq7ngjy.on.fleek.co/#/explore/bafybeibms6xfhs4kx3tun7dg6bjbj5obca7yj43cl5jl3pc2n7muw6ipfm

SgtPooki · 2023-07-12T17:29:33Z

Changes deployed on explore.ipld.io.

https://explore.ipld.io/#/explore/bafybeibms6xfhs4kx3tun7dg6bjbj5obca7yj43cl5jl3pc2n7muw6ipfm shows the error message appropriately.

SgtPooki added 18 commits May 24, 2023 18:10

fix: npm install works with node@18.16.0

5bb40dd

chore: uninstall react-scripts, install vite

0b9e0a4

following steps from https://www.robinwieruch.de/vite-create-react-app/

chore: jsx files should use .jsx extension

994525c

chore: create vite.config.js and mv index.html

5ec6090

chore: update index.html for vite

4486d2f

chore: fix npm start

2c121a4

chore: use import.meta.env instead of process.env

e5b2886

chore: use VITE env vars

0e986c9

fix: runtime errors

a78458c

fix: global not being defined properly

82fd023

fix: Buffer is not defined

c1c3f63

fix: polyfills and aliases

4955d7f

fix: App propTypes for route

2a37375

fix: use ipld-explorer-components from remove-js-ipfs branch

72d08b5

fix: unit tests

140a91f

fix: add e2e and unit tests

93e4246

fix: production build works

f6bb05a

fix: build scripts

68981de

SgtPooki mentioned this pull request May 25, 2023

feat!: remove all old deps ipfs/ipld-explorer-components#360

Merged

12 tasks

Merge branch 'master' into use-vite

cf212c6

SgtPooki commented May 25, 2023

View reviewed changes

tmp: add ipld-explorer-components tgz to repo

3029941

SgtPooki changed the title ~~feat: Update to latest ipld-explorer-components~~ [WIP] feat: Update to latest ipld-explorer-components May 25, 2023

SgtPooki linked an issue May 26, 2023 that may be closed by this pull request

Remove Ethereum example? #74

Closed

SgtPooki added 5 commits May 31, 2023 14:48

chore: use explorer-components tgz from 2254cedb1cbd50a8e8100eaf7a7a4…

22e96f3

…fbe9327740b

update package-lock.json

faba246

chore: update ipld-explorer-components

43425d6

fix: use doInitHelia instead of doInitIpfs

0aec90e

chore: update ipld-explorer-components and vite

d474933

SgtPooki added 2 commits June 7, 2023 14:26

chore: update ipld-explorer-components

fab1a5a

chore: use ipld-explorer-components@4.0.0

93b2f5b

SgtPooki changed the title ~~[WIP] feat: Update to latest ipld-explorer-components~~ feat: use ipld-explorer-components@4.0.0 Jun 8, 2023

SgtPooki self-assigned this Jun 8, 2023

SgtPooki marked this pull request as ready for review June 8, 2023 22:50

SgtPooki requested review from lidel, hacdias and rvagg June 9, 2023 17:16

lidel requested changes Jun 28, 2023

View reviewed changes

SgtPooki requested a review from lidel July 4, 2023 17:51

lidel approved these changes Jul 7, 2023

View reviewed changes

SgtPooki mentioned this pull request Jul 12, 2023

fix: fetch timeout has better error UX ipfs/ipld-explorer-components#378

Merged

SgtPooki mentioned this pull request Jul 12, 2023

bug: all gateways should be attempted ipfs/ipld-explorer-components#379

Closed

fix: use error message on fetch timeout

c11fa04

SgtPooki requested a review from a team as a code owner July 12, 2023 17:00

SgtPooki merged commit 65f1fef into master Jul 12, 2023

SgtPooki deleted the use-vite branch July 12, 2023 17:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: use ipld-explorer-components@4.0.0 #119

feat: use ipld-explorer-components@4.0.0 #119

SgtPooki commented May 25, 2023 •

edited

Loading

SgtPooki May 25, 2023

SgtPooki May 25, 2023

SgtPooki May 25, 2023

SgtPooki May 25, 2023

SgtPooki May 25, 2023

SgtPooki May 25, 2023

SgtPooki May 25, 2023

SgtPooki commented May 26, 2023 •

edited

Loading

lidel left a comment •

edited

Loading

SgtPooki commented Jul 4, 2023 •

edited

Loading

lidel left a comment •

edited

Loading

SgtPooki commented Jul 7, 2023

SgtPooki commented Jul 12, 2023

SgtPooki commented Jul 12, 2023

SgtPooki commented Jul 12, 2023

		@@ -1 +1 @@
		REACT_APP_VERSION=$npm_package_version
		VITE_VERSION=$npm_package_version

		<link rel="manifest" href="/manifest.json">
		<link rel="shortcut icon" href="/favicon.ico">

feat: use ipld-explorer-components@4.0.0 #119

feat: use ipld-explorer-components@4.0.0 #119

Conversation

SgtPooki commented May 25, 2023 • edited Loading

SgtPooki May 25, 2023

Choose a reason for hiding this comment

SgtPooki May 25, 2023

Choose a reason for hiding this comment

SgtPooki May 25, 2023

Choose a reason for hiding this comment

SgtPooki May 25, 2023

Choose a reason for hiding this comment

SgtPooki May 25, 2023

Choose a reason for hiding this comment

SgtPooki May 25, 2023

Choose a reason for hiding this comment

SgtPooki May 25, 2023

Choose a reason for hiding this comment

SgtPooki commented May 26, 2023 • edited Loading

lidel left a comment • edited Loading

Choose a reason for hiding this comment

SgtPooki commented Jul 4, 2023 • edited Loading

Drawbacks

Other things to think about

lidel left a comment • edited Loading

Choose a reason for hiding this comment

SgtPooki commented Jul 7, 2023

SgtPooki commented Jul 12, 2023

SgtPooki commented Jul 12, 2023

SgtPooki commented Jul 12, 2023

SgtPooki commented May 25, 2023 •

edited

Loading

SgtPooki commented May 26, 2023 •

edited

Loading

lidel left a comment •

edited

Loading

SgtPooki commented Jul 4, 2023 •

edited

Loading

lidel left a comment •

edited

Loading