Skip to content

Commit

Permalink
should also look into headers for input_size (JuliaLang#167)
Browse files Browse the repository at this point in the history
If no content length is set while uploading some contents, Curl defaults to use
chunked transfer encoding. In some cases we want to prevent that because the
server may not support chunked transfers.

With this change, the request method will also look at the headers while
determining the input size and if found call `set_upload_size` as usual. So to
switch off chunked transfers, one must also know and set the content length
header while invoking `download` or `request` methods.

(cherry picked from commit ab628ab)
  • Loading branch information
tanmaykm authored and ericphanson committed Jan 27, 2022
1 parent 8469f35 commit f2a7ca9
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 0 deletions.
13 changes: 13 additions & 0 deletions src/Downloads.jl
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,10 @@ function request(
input = something(input, devnull)
output = something(output, devnull)
input_size = arg_read_size(input)
if input_size === nothing
# take input_size from content-length header if one is supplied
input_size = content_length(headers)
end
progress = p_func(progress, input, output)
arg_read(input) do input
arg_write(output) do output
Expand Down Expand Up @@ -395,4 +399,13 @@ arg_read_size(io::Base.GenericIOBuffer) = bytesavailable(io)
arg_read_size(::Base.DevNull) = 0
arg_read_size(::Any) = nothing

function content_length(headers::Union{AbstractVector, AbstractDict})
for (key, value) in headers
if lowercase(key) == "content-length" && isa(value, AbstractString)
return tryparse(Int, value)
end
end
return nothing
end

end # module
2 changes: 2 additions & 0 deletions test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -513,6 +513,8 @@ include("setup.jl")
@test Downloads.arg_read_size(IOBuffer("αa")) == 3
@test Downloads.arg_read_size(IOBuffer(codeunits("αa"))) == 3 # Issue #142
@test Downloads.arg_read_size(devnull) == 0
@test Downloads.content_length(["Accept"=>"*/*",]) === nothing
@test Downloads.content_length(["Accept"=>"*/*", "Content-Length"=>"100"]) == 100
end
end

Expand Down

0 comments on commit f2a7ca9

Please sign in to comment.