fix(gateway)!: no duplicate payload during subdomain redirects

[hacdias]

• fix(gateway)!: no duplicate payload during subdomain redirects boxo#326
• fix: remove body check for subdomain redirection gateway-conformance#65
• switch to boxo commit from boxo/main (or release)

Rationale

Based on ipfs/boxo#326 (comment):

We have made significant progress on writing specifications of the Path Gateway to Subdomain Gateway. It is time to remove hacks and unidiomatic HTTP behavior from Kubo's Gateway.

Previously, when performing redirects from Path gw to Subdomain gw, the response body would include the data. However, we have decided to discontinue this behavior for multiple reasons:

• Firstly, it was identified that this approach was a hack and not a recommended practice, antipattern when looking at HTTP Semantics. To ensure consistency and discourage others from implementing this non-standard behavior, we have removed it from the gateway-conformance requirements too (fix: remove body check for subdomain redirection gateway-conformance#65).

• Secondly, ipfs.io did not become a Subdomain Gateway. We've decoded to use a separate domain name for that, dweb.link. Thus, the redirect body hack was never deployed to ipfs.io.

• Additionally, including the CID data in the response body was found to be inefficient and could lead to increased bandwidth usage and latency for clients unable to drop the connection after receiving the Location header. In such cases, client would receive data twice.

If any automation or process relies on the old behavior, the response body will now include a text/html message, whichwill inform the user about the redirect and provide the URL of the new location. This change makes it easier for legacy users to identify and troubleshoot any issues that may arise due to the modified behavior.

Overall, these changes aim to improve the performance, consistency, and maintainability of the Path Gateway to Subdomain Gateway redirects, while providing clear information to users in case of any unexpected behavior.

[lidel]

lgtm: I've added changelog and restored sharness test to confirm we have human-readable body.
Feel free to merge once boxo issue below is resolved.

---

@hacdias FYI I was not able to bubble up commit form boxo main branch to this PR today because ipfs/boxo#273 got merged first and introduced breaking changes:

$ go mod tidy
[..]
github.com/ipfs/kubo/core/commands imports
	github.com/ipfs/boxo/provider/batched: package github.com/ipfs/boxo/provider/batched provided by github.com/ipfs/boxo at latest version v0.8.1 but not at required version v0.8.2-0.20230601215904-1464d19f3dba
[..]

My understanding is we need to merge #9886 first, then rebase this PR and bumping boxo should work again.