Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let the gateway serve proofs of correct behavior. #6129

Open
Bren2010 opened this issue Mar 26, 2019 · 7 comments
Open

Let the gateway serve proofs of correct behavior. #6129

Bren2010 opened this issue Mar 26, 2019 · 7 comments
Labels
need/analysis Needs further analysis before proceeding topic/gateway Topic gateway

Comments

@Bren2010
Copy link
Contributor

Type:

feature

Description:

Motivation. Third-party IPFS gateways offer a lot of value to the ecosystem:

  • They let clients that are constrained by bandwidth or power usage access data from the network in a more efficient way.
  • A gateway can be used to answer requests if a client-side node hasn't had sufficient time to start-up yet or isn't working for unknown reasons (firewall?).
  • A gateway can act as an anonymizing layer, like a VPN.

Given the above, I'd like to propose that gateways have the ability to serve proofs that they're behaving correctly. This way, gateways can't maliciously modify content before serving it to their clients. A gateway can only refuse to answer a request, in which case another gateway can be tried.

How the feature should work. Requests to a gateway with the header X-Ipfs-Secure-Gateway: 1 should switch to a binary format containing the proof.

The first part of the response, called a preamble, relates the request path domain.com/a/b/c to the hash of the response body. It may contain a DNSSEC proof mapping domain.com to dnslink=/ipfs/Qm... or an IPNS record, as well as the contents of any intermediate directories /a/b/c.

The second part of the response, called the body, is a depth-first traversal of the file's DAG. The client can hash the first block and see that it equals the output of the preamble, and then check that the second block corresponds to the first link of the first block, and so on.

Background.

@Stebalien
Copy link
Member

Stebalien commented Mar 28, 2019

Thanks for writing this up, I'm currently traveling but I'll take a look when I get back (next Wednesday?) if not sooner.

(that is, this'll take some focus work to dig into so I'll need to allocate a chunk of distraction-free time)

@Stebalien
Copy link
Member

(we're trying to cut a release but I'll still try to look at this ASAP)

@Stebalien
Copy link
Member

Again, I apologize for the extensive delay. The release should be out shortly (#6223) and we'll re-enter the feature cycle.

@lanzafame lanzafame added the topic/gateway Topic gateway label Jul 16, 2019
@bertrandfalguiere
Copy link

What is the status on this?
Content-addressing is about "getting authentic data from any source", and this PR goes in the right direction. :)

@Stebalien
Copy link
Member

The status:

  1. I apologize for not communicating on this. I was over ambitious with my own time and should have managed expectations better.
  2. The network has some critical performance issues at the moment which are taking up the majority of my (and my team's) time.
  3. This is a pretty significant change and a large feature to maintain. It needs to be thoroughly reviewed, tested, documented, and discussed.

I'd like to revisit it when we get everything under control but I'm having trouble allocating time to it at the moment.


More generally, this feature doesn't really fit in with the overall goal of IPFS: to decentralize the internet. Basically, to make use of this feature, the user needs to install something. That leaves us with two choices:

  1. The user installs the validator extension and uses a centralized gateway.
  2. The user installs the IPFS companion extension and uses either a local gateway or the built-in, in-browser js-ipfs gateway.

The former helps provide better, secure access to the IPFS network but the latter actually works towards our goal of decentralizing the internet. If the user needs to install something anyways, I'd rather move closer to our goal.

On the other hand, I do understand that running IPFS locally is not always possible for performance reasons. That's why I'd like to revisit this once we get the larger-scale network performance issues under control.

@lidel
Copy link
Member

lidel commented Jun 26, 2020

Additional design work needs to happen here, and probably some dev on the browser vendor side as well.

Open problems:

  • PoC validator extension made by Cloudflare does not verify IPFS-based integrity correctly: it produces false-negatives if non-default parameter space was used during ipfs add (eg. different hash, trickle dag, chunk size etc)
  • Chromium-based browsers have more than 80% of the market atm and they do not have WebExtension API that makes it possible to verify proofs, making it infeasible in real world – see initial analysis on Verifiable HTTP Gateways in Verifiable HTTP Gateway Responses in-web-browsers#128

@lidel
Copy link
Member

lidel commented Aug 11, 2022

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/analysis Needs further analysis before proceeding topic/gateway Topic gateway
Projects
None yet
Development

No branches or pull requests

6 participants