Skip to content

Commit

Permalink
[utils] Remove stray undocumented Host header in redirect (fix 46fde7c)
Browse files Browse the repository at this point in the history
  • Loading branch information
@dirkf
dirkf committed Jul 20, 2023
1 parent b2ba24b commit 1fa8b86
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion youtube_dl/utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2996,7 +2996,8 @@ def redirect_request(self, req, fp, code, msg, headers, newurl):
# Technically the Cookie header should be in unredirected_hdrs;
# however in practice some may set it in normal headers anyway.
# We will remove it here to prevent any leaks.
remove_headers = ['Cookie']
# Also remove unwanted and undocumented Host header for old URL
remove_headers = ['Cookie', 'Host']

This comment has been minimized.

Sign in to view

Copy link
@coletdjnz

coletdjnz Jul 21, 2023 •
edited
Loading

Contributor

In what cases does this apply? The Host header is not added to the Request unless done manually (which it shouldn't be)

https://datatracker.ietf.org/doc/html/rfc9110#section-15.4-6.3.1

# A 303 must either use GET or HEAD for subsequent request
# https://datatracker.ietf.org/doc/html/rfc7231#section-6.4.4
Expand Down

3 comments on commit 1fa8b86

@dirkf
Copy link
Contributor Author

@dirkf dirkf commented on 1fa8b86 Jul 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly with excessive purism, I used the Request.header_items() method, the common documented interface to the header list between Py2/3, when back-porting the yt-dlp redirect code. Apparently that method unexpectedly adds a Host header derived from the Request's URL, as if it was designed to gather the headers for transmission: ofc this is bad if the URL is being changed. Actually we do use Request.headers elsewhere, eg utils.update_Request(), and it appears to work back to 2.6.9 despite not being mentioned in the Py2 doc: possibly I could/should have used that routine instead of the literal back-port.

@coletdjnz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah Request.header_items() reads from unredirected_hdrs. Headers in that should not be added to redirected requests, so I'm not sure if header_items() is appropriate there.

Anyways, was just checking this isn't something we need to port to yt-dlp.

@dirkf
Copy link
Contributor Author

@dirkf dirkf commented on 1fa8b86 Jul 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somewhat reverted as part of 2b7dd3b.

Please sign in to comment.