Gateway requests responds with 403 - Forbidden

[fsdiogo]

When in the Download page, the user shouldn't have to have an IPFS instance or daemon running, so we'll use the ipfs.io gateway to fetch the files and download them.

The problem here is that when making the request the response is always 403 - Forbidden.

ipfs-share-files/src/pages/download/Download.js

Line 20 in 7ed1070

doFetchGatewayFileTree(params.hash)

ipfs-share-files/src/bundles/files.js

Lines 222 to 248 in 7ed1070

	doFetchGatewayFileTree: (hash) => async ({ dispatch, store }) => {
	const url = `https://ipfs.io/api/v0/ls?arg=${hash}`
	dispatch({ type: 'FILES_FETCH_GATEWAY_STARTED' })
	window.fetch(url)
	.then(ipfsFiles => {
	const files = {}
	for (const file of ipfsFiles) {
	const fileId = shortid.generate()
	const fileName = file.name
	const fileSize = file.size
	const fileHash = file.hash
	files[fileId] = {
	name: fileName,
	size: fileSize,
	hash: fileHash,
	progress: 100,
	pending: false
	}
	}
	dispatch({ type: 'FILES_FETCH_GATEWAY_FINISHED', payload: { files: files } })
	})
	.catch(err => dispatch({ type: 'FILES_FETCH_GATEWAY_FAILED', payload: { error: err.message } }))
	},

Both in Postman or directly in the browser the response comes OK, but not in the app.

[lidel]

@fsdiogo what are steps to reproduce this?
Checkout feat/add-download-page branch and run npm install && npm run start ?

[fsdiogo]

@lidel yes!

Then use a valid hash and go to:

http://localhost:3000/#/<hash>

That will open the download page and send a request to the gateway.

[lidel]

I eyeballed failing (403) and standalone request (200):

403 is triggered when either Origin or Referrer is present:

$ curl 'https://ipfs.io/api/v0/ls?arg=QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR' 
{"Objects":[{"Hash":"QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR","Links":[]}]}

$ curl 'https://ipfs.io/api/v0/ls?arg=QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR' -H 'Origin: http://localhost:3000'
403 - Forbidden

$ curl 'https://ipfs.io/api/v0/ls?arg=QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR' -H 'Referer: http://localhost:3000/'
403 - Forbidden

AFAIK its the fetch API itself adding those in default mode.
Need to run afk for today, may look into it tomorrow.

[fsdiogo]

When sending the request to a local JS and Go daemon it succeeds.

As @lidel mentioned, it's probably some header check in the gateway that fails and returns 403.

I'll expose this to the #ipfs-dev IRC channel to get more feedback.

[lidel]

Actually its even more bizarre!

• Origin: http://localhost:3000/ fails
• Origin: http://localhost:3000 fails
• Origin: http://localhost:8080/ fails
• Origin: http://localhost:8080 works fine

Looking at go-ipfs/core/corehttp/commands.go#L44-L49 it makes sense again(?): the patchCORSVars function makes sure the origin validation is performed using the port Gateway is actually listening. This may be why it worked for local node and did not for remote one, where go-ipfs is behind a reverse-proxy. But I am quite sure fetch worked in past...

---

Update from @lgierth confirms the error is caused by recent changes in ipfs.io infra:

012044 @lgierth │ yeah together with a recent change that moved declaration of the cors-related headers from go-ipfs into nginx
012057          │ without realizing that go-ipfs does more with these headers than just setting them on responses
012118          │ it actually validates the Origin on requests against these headers (of course)
012143          │ i'll try and fix it tonight

[fsdiogo]

Wow, thanks @lidel! It really is strange!

@lgierth can you please ping here when it is fixed?

[fsdiogo]

Huge thanks to @lidel and @lgierth, this is now fixed!