Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Should _http_request only close streams it has created? #468

Closed
nickrobinson251 opened this issue Sep 28, 2021 · 4 comments
Closed

Should _http_request only close streams it has created? #468

nickrobinson251 opened this issue Sep 28, 2021 · 4 comments

Comments

@nickrobinson251
Copy link
Contributor

nickrobinson251 commented Sep 28, 2021 •
edited
Loading

Follow up to #467 (comment) (see also #433)

Roughly, HTTP.jl v0.9.15 changed to not close streams within HTTP.request (see JuliaWeb/HTTP.jl#752, JuliaWeb/HTTP.jl#543, JuliaWeb/HTTP.jl#772).

Here in AWS.jl, we were relying on the previous HTTP behaviour of calling close on the response_stream, so #467 reintroduces the close in our internal AWS._http_request helper.

But this was done as simplest way to get back to the previous HTTP behaviour. We might instead want to re-evaluate indiscriminately calling close on the stream, which a user could have provided
@nickrobinson251 nickrobinson251 mentioned this issue Sep 28, 2021
Merged
@omus
Copy link
Member

omus commented Sep 28, 2021

Copied my response from JuliaWeb/HTTP.jl#772 (comment) as it is relevant here:

It should be pointed out that using response_stream for HTTP.jl 0.9.14 and below only really worked (I believe) with the unexported Base.BufferStream. The implementation of this stream seems to require the it be be closed in order for it to be read from. Most streams don't work this way.

I think the general rule should be that you shouldn't close IO instances that you didn't create.

@nickrobinson251 nickrobinson251 mentioned this issue Sep 28, 2021
Closed
@nickrobinson251
Copy link
Contributor Author

Care to make a PR? I think it's roughly Fredrik's suggested diff (#467 (comment)) plus a test that user-given IO-streams do not get closed?

@omus
Copy link
Member

omus commented Sep 28, 2021

Care to make a PR? I think it's roughly Fredrik's suggested diff (#467 (comment)) plus a test that user-given IO-streams do not get closed?

Sure. I'll be rolling it into #384 as I need to update it to not break the current behaviour and then this change can be opt-in

@omus omus mentioned this issue Sep 29, 2021
Merged
bors bot added a commit that referenced this issue Sep 29, 2021
@bors @omus
Merge #384
ca3ed0c 
384: Use `AWS.Response` to handle streaming/raw/parsing r=omus a=omus

The idea is to use a struct in AWS.jl that can be used to handle the automatic parsing that is currently used to turn XML/JSON into a dict while also giving the option of accessing the raw output as a string or stream without all the keywords currently needed to be specified.

Depends on:
- #457

Related:
- #346
- #348
- #438

Closes:
- #224
- #433
- #468 (when using `use_response_type` the passed in I/O is not closed)
- #471
- #433

Update: The tests in this PR run using the deprecated behaviour. Mainly I did that here to prove this change is non-breaking. For the updated tests see #458 

Co-authored-by: Curtis Vogt <curtis.vogt@gmail.com>
@omus
Copy link
Member

omus commented Sep 29, 2021

As of AWS.jl 1.63.0 (#384) streams are no longer closed when using the feature use_response_type (e.g. @service S3 use_response_type=true). If you enable this feature and pass in a Base.BufferStream you're responsible for closing it yourself.

When not enabling the use_response_type feature the current behaviour as of AWS 1.62.1 remains unchanged.

@omus omus closed this as completed Sep 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@omus @nickrobinson251