Skip to content
This repository has been archived by the owner on Jul 1, 2022. It is now read-only.

Always close HTTP responses in HttpSender #571

Merged

Conversation

mpetazzoni
Copy link
Contributor

@mpetazzoni mpetazzoni commented Nov 30, 2018

The Response object wasn't closed when the response is successful,
leading to connection leaks. This was revealed by seeing messages like
this in logs:

WARNING: A connection to https://<redacted>/ was leaked. Did you forget to close a response body?

Signed-off-by: Maxime Petazzoni max@signalfx.com

Which problem is this PR solving?

  • OkHttp3 reports connection leaks

Short description of the changes

  • This change makes sure that the Response object is always closed, even when the response is successful.

The Response object wasn't closed when the response is successful,
leading to connection leaks. This was revealed by seeing messages like
this in logs:

  WARNING: A connection to https://<redacted>/ was leaked. Did you forget to close a response body?

Signed-off-by: Maxime Petazzoni <max@signalfx.com>
@mpetazzoni
Copy link
Contributor Author

I'm not sure why this test is failing: https://travis-ci.org/jaegertracing/jaeger-client-java/jobs/462005629#L1693

✗ [endtoend] test_driver→ (services=java-udp testdriver=test_driver) ⇒ Fail: could not retrieve traces from query service

Any help here is appreciated.

@jpkrohling
Copy link
Collaborator

Looks like something got stuck somewhere, as I don't think it's related to this PR. I just started the job again, let's see how it goes.

@codecov
Copy link

codecov bot commented Dec 3, 2018

Codecov Report

Merging #571 into master will increase coverage by 0.3%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             master    #571     +/-   ##
==========================================
+ Coverage     89.49%   89.8%   +0.3%     
- Complexity      539     542      +3     
==========================================
  Files            68      68             
  Lines          1942    1942             
  Branches        249     249             
==========================================
+ Hits           1738    1744      +6     
+ Misses          130     126      -4     
+ Partials         74      72      -2
Impacted Files Coverage Δ Complexity Δ
...ing/internal/samplers/RemoteControlledSampler.java 90.69% <0%> (+1.16%) 18% <0%> (+1%) ⬆️
...gertracing/internal/reporters/LoggingReporter.java 90.9% <0%> (+9.09%) 5% <0%> (+1%) ⬆️
...rtracing/internal/reporters/CompositeReporter.java 100% <0%> (+28.57%) 7% <0%> (+1%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af610ad...d8781cf. Read the comment docs.

Copy link
Collaborator

@jpkrohling jpkrohling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks better than what we have, just need to clarify a couple of things.


String exceptionMessage = String.format("Could not send %d spans, response %d: %s",
spans.size(), response.code(), responseBody);
throw new SenderException(exceptionMessage, null, spans.size());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you need to close, in case the response isn't successful? Is there an alternative where try-with-resources is used, so that our code wouldn't need to care about the closing of the resources?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling response.body().string() will close the response, so this happens fine in the non-successful code path.

Ideally we'd use a try-with-resources, but we can't use it because this project requires Java 6 compatibility.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't wait to get rid of Java 6 support... (#511)

@jpkrohling jpkrohling merged commit f07f1ba into jaegertracing:master Dec 3, 2018
@mpetazzoni
Copy link
Contributor Author

Thanks! Would you consider doing a 0.32.1 with this fix? If not, when do you think the next release train will pick it up?

@mpetazzoni mpetazzoni deleted the http-sender-response-close branch December 3, 2018 17:10
@jpkrohling
Copy link
Collaborator

@pavolloffay can give you more info about releases, I'm a bit outdated on this area :)

@pavolloffay
Copy link
Member

The release works how it is described in RELEASE.md nothing has changed.

@mpetazzoni
Copy link
Contributor Author

@jpkrohling Would you be willing to release a 0.32.1 following the RELEASE.md instructions? I'd like to be able to point people at an available release that doesn't leak connections.

@jpkrohling
Copy link
Collaborator

jpkrohling commented Dec 14, 2018 via email

@objectiser
Copy link
Contributor

@mpetazzoni @jpkrohling I've started the process in #575 - just waiting for @pavolloffay (at kubecon) or another @jaegertracing/jaeger-maintainers to approve.

dmichel1 added a commit to dmichel1/opencensus-java that referenced this pull request Feb 21, 2019
Using the jaeger trace exporter we started seeing `WARNING: A connection to http://jaeger:8080/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE` in our application logs.

I traced this back to jaeger client release `0.33.0` that fixes this issue with closing the connection.

jaegertracing/jaeger-client-java#571
https://github.com/jaegertracing/jaeger-client-java/releases
dmichel1 added a commit to dmichel1/opencensus-java that referenced this pull request Feb 22, 2019
Using the jaeger trace exporter we started seeing `WARNING: A connection to http://jaeger:8080/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE` in our application logs.

I traced this back to jaeger client release `0.33.0` that fixes this issue with closing the connection.

The newer version of the jaeger client uses a new package name `io.jaegertracing`

jaegertracing/jaeger-client-java#571
https://github.com/jaegertracing/jaeger-client-java/releases
songy23 pushed a commit to census-instrumentation/opencensus-java that referenced this pull request Feb 22, 2019
* Bump jaeger trace exporter version

Using the jaeger trace exporter we started seeing `WARNING: A connection to http://jaeger:8080/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE` in our application logs.

I traced this back to jaeger client release `0.33.0` that fixes this issue with closing the connection.

The newer version of the jaeger client uses a new package name `io.jaegertracing`

jaegertracing/jaeger-client-java#571
https://github.com/jaegertracing/jaeger-client-java/releases

* Fix import control
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants