resources.GetRemote: User-Agent header is ignored

[TheStaticTurtle]

What version of Hugo are you using: hugo v0.109.0-47b12b83e636224e5e601813ff3e6790c191e371+extended windows/amd64 BuildDate=2022-12-23T10:38:11Z VendorInfo=gohugoio
Does this issue reproduce with the latest release : Yes

Hello,
I’m currently doing some work that requires me to change the user agent of the client used for resources.GetRemote

Unfortunately, it seems that I'm unable to modify the User-Agent header, consider the following example:

{{ $opts := dict
    "method" "get"
    "headers" (dict
        "Authorization" "test-header"
        "User-Agent" "Mozilla/5.0" 
    )
}}
{{ with resources.GetRemote "https://127.0.0.1:8000" $opts }}
    {{ . }}
{{ end }}

The server sees:

GET / HTTP/1.1
Host: 127.0.0.1:8000
User-Agent: Hugo Static Site Generator
Authorization: test-header
Accept-Encoding: gzip

From what I can read in remote.go#L98 it seems that addDefaultHeaders is call first which sets the User-Agent header to Hugo Static Site Generator and then, addUserProvidedHeaders is called.

This functions uses req.Header.Add to add the header. The issue is that .Add does not override an existing header. Wouldn't it be preferable to use .Set which according to the documentation replaces any existing values associated with key?

Samuel,

[bep]

If I'm reading the code right, the correct fix would be to switch the order between addDefaultHeaders/addUserProvidedHeaders -- as addDefaultHeaders checks for preexisting values for the default keys before adding.

I don't think we can use Set, as there are cases where multiple header values for the same key is relevant.

[TheStaticTurtle]

True, inverting the function might be the best way to deal with this.

There are also some other weird behavior, still with the example in my first post, I can for example override the Accept-Encoding header but can't for Host (which could also be beneficial) and User-Agent, I can set multiple values for a header with (slice "value-a" "value-b") without any issue.
But addUserProvidedHeaders doesn't have any checks, so shouldn't it add a second value if the header was already present? Why isn't there two User-Agent, does the http package do some cleanup before executing?

[bep]

But addUserProvidedHeaders doesn't have any checks, so shouldn't it add a second value if the header was already present? Why isn't there two User-Agent, does the http package do some cleanup before executing?

Whatever it is, it's the http package that's doing this. They store this internally as a map of slices (If I remember correctly), but probably has some rules about headers that should have only one value (e.g. User-Agent).

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.