Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(fetch): properly redirect non-ascii location header url #2971

Merged
merged 8 commits into from
Mar 21, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions lib/web/fetch/util.js
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ function responseLocationURL (response, requestFragment) {
// Some websites respond location header in binary form without encoding them
Xvezda marked this conversation as resolved.
Show resolved Hide resolved
// and major browsers redirect them to correctly UTF-8 encoded addresses.
// Here, we handle that behavior in the same way.
location = new URL(decodeBinaryStringToUtf8(location), responseURL(response))
location = new URL(normalizeBinaryStringToUtf8(location), responseURL(response))
Xvezda marked this conversation as resolved.
Show resolved Hide resolved
}

// 4. If location is a URL whose fragment is null, then set location’s
Expand All @@ -61,10 +61,12 @@ function responseLocationURL (response, requestFragment) {
}

/**
* If string contains non-ASCII characters, assumes it's UTF-8 encoded and decodes it.
* Since UTF-8 is a superset of ASCII, this will work for ASCII strings as well.
* @param {string} value
* @returns {string}
*/
function decodeBinaryStringToUtf8 (value) {
function normalizeBinaryStringToUtf8 (value) {
return Buffer.from(value, 'binary').toString('utf8')
}
Copy link
Member

@lemire lemire Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  let b = Buffer.from(value, 'binary');
  if(Buffer.isAscii(b)) { return value; } // skip new string alloc.
  return b.toString('utf8'); // must alloc a new string

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to call Buffer.from(). Buffer.isAscii(b) is a static function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anonrig We get at the core of the issue here. The value is a string not a buffer. At some point it was wrongly interpreted as latin1 (maybe in http) and so we must now convert it to a buffer and then back to a string.

I think that this function should receive a Buffer and then the Buffer could be interpreted as UTF-8 and we'd be done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this is far beyond the scope of this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The irony is that we seem to try to purposefully (in http) prevent people from sending UTF-8 content as UTF-8... and then we go back and say "no, it is really UTF-8".

Much technical debt.


Expand Down
Loading