feat: improved Origin detection via img tag

[lidel]

This PR:

• normalizes gateways.json list to use canonical path-based notation

  • This follows our docs at https://docs.ipfs.io/how-to/address-ipfs-on-web/#automatic-leverage-the-gateway-in-go-ipfs

• changes the way we detect Origin support by doing a robust check

  • We now are doing real network-based test: we try to fetch 1x1 png image, so the test will work even when CORS is not set up, or when original test done via JS request is blocked by browser/extensions like Privacy Badger.
  • This also fixes Online check for about ~6 gateways which block loading .js (and always looked offline before)

• adds padlock to gateways that have Origin isolation, as a soft incentive to provide it:

  

• Distributes the number of checks page does at the same time in the background

  • I wanted to avoid full rewrite, so I did the bare minimum to make the initial load of the page more responsive on slow devices

[lidel]

I've reached out to @LinusCDE, and we are working on a fix, but for now its better to remove it to reduce confusion.
We will add it back when it supports proper subdomain isolation and interop with path-gateways

[LinusCDE]

Good idea. Something came up, so it's good to remove it for now. I'll try to address it tomorrow.
Would a redirect from https://cosmos-ink.net/ipfs/<cid> and https://ipfs.cosmos-ink.net/ipfs/<cid> to https://<cid>.ipfscosmos-ink.net/ to the trick?

What about the IPNS? Same for that?

[lidel]

Yes, redirects will work (ideally, redirect returned by go-ipfs, because that gives you CID conversion for free)

For ipns you could use https://<libp2p-key>.ipns.cosmos-ink.net/

[LinusCDE]

@lidel I've now added a rule that should fix that.

if ($request_uri ~ ^/(?<prefix>ipfs|ipns)/(?<full_path>(?<cid_1>.*?)(?=/)(?<cid_1_path>/?.*)|(?<cid_2>.*))$) {
    # Either $cid_1 or $cid_2 will be empty. When $cid_1 is used, $cid_1_path is not empty as well
    return 301 "https://$cid_1$cid_2.$prefix.cosmos-ink.net$cid_1_path";
  }

This nginx rule is applied to ipfs.cosmos-ink.net, ipns.cosmos-ink.net, cosmos-ink.net.
Basicially for any path like https://(ipfs.)cosmos-ink.net/ipfs/<cid>/<otherstuff> I 301 it to https://<cid>.ipfs.cosmos-ink.net/<otherstuff>. Same goes for ipns.

I also added your PublicGateway suggestion from your second mail.

ideally, redirect returned by go-ipfs, because that gives you CID conversion for free

Running it on my Pi (cosmos-ink.net main domain) would be probably more trouble at this point, because I would need to rebuild the docker-image for armv7 or aarch64 support. But for any conversion that isn't https://(ipfs|ipns).cosmos-ink.net/(ipfs/ipns)/..., go-ipfs can feel free to convert that.

For the ipfs/ipns subdomains I also added two entries to prevent using that redirection rule when the cid is already in the subdomain (to allow https://<cid>.ipfs.cosmos-ink.net/ipfs/... in case ipfs web apps have a directory called "ipfs").

Could you check whether it now works as expected? I'll probably then create a PR to re-add the entry.

[lidel]

It works fine for CIDv1, but redirect alone is not enough if someone passes CIDv0.

To illustrate, if you open https://cosmos-ink.net/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR it should redirect to https://bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi.ipfs.cosmos-ink.net/
(go-ipfs takes care of CID conversion, without the cid conversion the Qm.. CIDv0 will be force-lowercased and fail).

ps. Do you mind commenting in ipfs/kubo#4931 regarding the need for docker image working out of the box on rpi?

[jessicaschilling]

Would it be possible to add alt text for origin isolation to disambiguate what the padlock means? I don't want to imply too much security.

[lidel]

Good point. I don't think alt text is enough to clear this up.
Replaced 🔒 with 💚 just to be safe there is no security-releated confusion:

[autonome]

load events + timers + promises is some sticky business...

[lidel]

indeed: entire app.js needs to be re-written from scratch in modern JS (painfully needs rate-limiting via execution queues, similar to ones we have in js-libp2p-delegated-*)

@jessicaschilling just like with cid.ipfs.io, we should rewrite this app (both backend and frontend) at some point (2021 Q1/Q2-ish?)

[jessicaschilling]

@lidel There's a long tail of improvements to the gateway checker that never got completed here: #93

I'd suggest seeing what of that could be salvaged, or at least using that as the base issue for a rewrite. I'll add the notes from this PR to that issue.