Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

. #6983

Closed
wants to merge 42 commits into from
Closed

. #6983

wants to merge 42 commits into from

Conversation

lidel
Copy link
Member

@lidel lidel commented Mar 11, 2020

IGNORE THIS PR, @lidel b0rked it

See instead:
#6984

cryin_cat435

Stebalien and others added 30 commits March 4, 2020 14:33
(we're implementing an _actual_ proxy)

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Instead of adding a new fake header (that could be spoofed by the client...),
just read the original request URI from the request object.

This also removes support for suborigins. They have never been implemented in
browsers and it looks like efforts have stalled. We can add support back if we
need it but, well, maintaining support was going to be more trouble than it was
worth.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Transparently re-route all CONNECT requests back to the gateway. This way,
browsers can route IPFS-enabled websites to the daemon by setting the gateway as
a "proxy". We'll then handle these websites the same way we handle normal
DNSLink websites.

This _also_ means the IPFS Companion can route ipfs.io traffic to the local
daemon _without_ touching the URL (well, we need to replace the https part with
http).

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
Replace IPNSHostname based routing with general-purpose hostname-based routing
that also supports handling requests to "known gateways".

TODO: I'll break KnownGateways out into a config option in a later commit.

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
- "known gateways" can be now defined with or without port.
  if port is missing, a version defined without it will be used
  as a fallback
- subdomain gateway at *.ipfs.localhost is supported out of the box
- subdomain gateway redirects path-gateway requests to subdomain version
  with automatic conversion to CIDv1 in Base32

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
NoDNSLink disables DNS TXT lookups based on the Host header found in
request.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This enables us to mount /ipfs/ and /ipns/ on top of DNSLink
website in a way that prioritizes content-addressed paths.

Example:

Gateway: {
    PublicGateways: {
    ipfs.io: {
	PathPrefixes: [/ipfs, /ipns],
	UseSubdomains: false

Potential use for this is DNSLink website for IPFS Gateway that keeps
original URL (more elegant alternative to RootRedirect)
or just a regular website which uses content-addressing internally.

NoFetch could be used when loading of random CIDs is not desired.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This removes note about Clear-Site-Data.
Tracked in ipfs/in-web-browsers#157

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This adds support for resolving /ipns/<cidv1-libp2p-key>

cc #5287

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This covers *.ipfs.localhost and strict  *.ipns.localhost
What needs to be done next is improve UX of .ipns.localhost
and add separate file with tests specific to DNSLink

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
- gateway redirects to a valid libp2p-key CID
- CLI resolves as-is
- style: use 'hostname' in places where host header may have explicit port

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Reuses style from #6855

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Serve always returns a non-nil error:
https://golang.org/src/net/http/server.go?s=76163:76212#L2456

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Adds wrapper that runs HTTP request directly and checks if response
(headers or body) includes expected string.
Then the same is repeated in HTTP proxy mode.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
When request to a PublicGateway fails due to the lack
of valid muxer or of current path prefix is not on the Paths whitelist
for the hostname we return a vanilla HTTP 404 Not Found error.

This ensures that is the case, and also adds tests to safeguard
against regressions in the future.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
localhost returns redirect to subdomain,
switching to IP ensures path gateway is used for tests

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
/p2p path is used by experimental p2p http proxy:
https://github.com/ipfs/go-ipfs/blob/v0.4.23/docs/experimental-features.md#p2p-http-proxy

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
lidel added 12 commits March 4, 2020 14:40
License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This switches to peer.Decode as suggested in
#6096 (comment)
and adds tests for missing PeerID types.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This should make sharness test independent from DNS resolver on CI
machine + match what browser is sending more closely.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This is to fix error related to Graphsync config key missing.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
ipfs resolve of /ipns/{cidv1}
where CID has multicodec other than libp2p-key returns a meaningful error
that includes fixed multicodec

See also: #6096 (comment)

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This changes the way we test gateway requests that rely on hostname.

When direct connection is used, we remove hostname from original URL and
replace it  with raw IP to ensure value from Host header is used (no
false-positives).

When proxy connection is used, we follow semantics of HTTP proxy spec,
but try to decouple tests from OS-level DNS resolver where possible,
just to make sharness tests more portable.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Making it easier to copy&paste, as suggested in:
#6096 (comment)

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Prewarms namesys cache with static records for deteministic tests and
debugging. Useful for testing things like DNSLink without real DNS
lookup.

See docs/environment-variables.md

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This adds tests for DNSLink resolution on HTTP gateway.
Tests are deterministic, robust and portable thanks to IPFS_NS_MAP which
decouples test environment from production DNS infrastructure.

Key test is DNSLink behavior on {dnslink}.ipns.localhost,
which aims to be used by local IPFS node to provide Origin isolation
while maintaining user friendly address bar in the browser.

Subdomain gateway may use DNSLink for hosting its landing page.
User may choose to enable or block wildcard DNSLink gateway.

Tests are included for both.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Updated examples that can be copied and run in CLI

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
This moves static mappings defined via IPFS_NS_MAP
to a dedicated map that never expires.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
Context: #6975

When request is sent to http://localhost:8080/ipfs/$cid we check
User-Agent and disable subdomain redirect if it is a known cli tool that
does not follow redirects by default.

We also set Clear-Site-Data header on 'localhost' responses to ensure Origin
sandbox can't be abused.

License: MIT
Signed-off-by: Marcin Rataj <lidel@lidel.org>
@lidel lidel requested a review from Stebalien March 11, 2020 18:24
@lidel
Copy link
Member Author

lidel commented Mar 11, 2020

Oh boy, wrong target, ignore this PR, see #6984 instead.

@lidel lidel closed this Mar 11, 2020
@lidel lidel deleted the fix/curl-on-localhost-option-b branch March 11, 2020 18:25
@lidel lidel changed the title fix(subdomain-gw): curl on localhost (Option B: User-Agent) IGNOREME Mar 11, 2020
@lidel lidel removed the request for review from Stebalien March 11, 2020 18:26
@lidel lidel restored the fix/curl-on-localhost-option-b branch March 11, 2020 18:26
@ipfs ipfs locked as resolved and limited conversation to collaborators Mar 12, 2020
@lidel lidel changed the title IGNOREME . Mar 12, 2020
@hacdias hacdias deleted the fix/curl-on-localhost-option-b branch May 9, 2023 10:57
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants