Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sporadic Elasticsearch test failures #444

Merged
merged 3 commits into from
May 27, 2020
Merged

Fix sporadic Elasticsearch test failures #444

merged 3 commits into from
May 27, 2020

Conversation

trask
Copy link
Member

@trask trask commented May 26, 2020

Hopefully resolves #426 and #438

@iNikem
Copy link
Contributor

iNikem commented May 27, 2020

Have you considered instead using https://www.testcontainers.org/modules/elasticsearch/ ?

@trask trask marked this pull request as ready for review May 27, 2020 04:44
@trask
Copy link
Member Author

trask commented May 27, 2020

No, but I totally agree with going in that direction.

I thought this was going to be an easy "fix" using Spock's @Retry annotation until I discovered that @Retry only applies to the tests themselves, and not to the setup, which is where these bind exceptions occur.

@iNikem
Copy link
Contributor

iNikem commented May 27, 2020

Your approach is certainly more applicable: the majority of tests which start some server can be solved that way. But it needs at least some documentation. Currently it leaves bad taste in my mouth: "Something is broken here, don't know what exactly, let us just retry".

Another solution, which requires more effort, is to run Elastic in Testcontainer. And probably to run only 1 container for all Elastic related tests, if that is possible. Will speed things up as well.

I am OK with this PR as a short term solution to flakiness, but I think we can do better in a long run.

@trask
Copy link
Member Author

trask commented May 27, 2020

👍 good point, added doc to withRetryOnBindException()

@trask trask merged commit 5c7e321 into open-telemetry:master May 27, 2020
iNikem added a commit that referenced this pull request May 31, 2020
* Define packages in exporter class loader (#409)

* Update docs about needing java 11 to build (#412)

* Update version to 0.3.0 (#413)

* Update the README (#414)

* Move build and configure to top as getting started section
* Add manual instrumentation section
* Document `@WithSpan` annotation
* Move developer specific information to CONTRIBUTING.md
* Cleanup formatting and use consistent spacing

* Update version to 0.4.0-SNAPSHOT (#415)

* Update CONTRIBUTING.md back to Java 11 (#419)

* Add Zipkin exporter support  (#411)

* #375 Add Zipkin exporter support

Signed-off-by: Sergei Malafeev <sergeymalafeev@gmail.com>

* #375 use OkHttpSender for Zipkin exporter

Signed-off-by: Sergei Malafeev <sergeymalafeev@gmail.com>

* #375 add Zipkin exporter to README

Signed-off-by: Sergei Malafeev <sergeymalafeev@gmail.com>

Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>

* Remove inactive maintainer (#420)

* Fix brolen anchor link (#422)

* Fix khttp instrumentation in case of absent or read-only headers map (#416)

* Add new approver (#429)

* Fix sqlNormalizerEnabled initialization (#432)

* Remove unused config (#424)

* Change names of servlet based server spans (#428)

* Add documentation describing non-obvious points of Servlet instrumentations

* Change names of servlet based server spans

* Fix java google format (#439)

* Remove deprecated usage from internal instrumentation (DataDog/dd-trace-java#1441)

* Update gradle to 6.4 (DataDog/dd-trace-java#1443)

* Migrate lettuce instrumentation away from deprecated finishSpanOnClose (DataDog/dd-trace-java#1445)

* Make Retrys consistent (DataDog/dd-trace-java#1442)

* Rename java packages for lettuce 4 and 5 to not have collisions (DataDog/dd-trace-java#1450)

* Adding an option to manually disable Kafka headers (DataDog/dd-trace-java#1448)

* Add version specific names to allow disabling only a specific version (DataDog/dd-trace-java#1456)

* Attempt to improve muzzle time by randomly ignoring versions until 10 remain (DataDog/dd-trace-java#1451)

* Save circle cache with name matching restore patterns (DataDog/dd-trace-java#1457)

* Wrap log statements using varargs to avoid object allocation (DataDog/dd-trace-java#1466)

* Grizzly-http and grizzly-client instrumentation (DataDog/dd-trace-java#1365)

* More refactoring for ScopeManager (DataDog/dd-trace-java#1467)

* Remove Java 9 and 10 from the build (DataDog/dd-trace-java#1473)

* Disable CI cache for muzzle (DataDog/dd-trace-java#1479)

* Separate out core instrumentation for AWS SDK to allow manual setup o… (#421)

* Separate out core instrumentation for AWS SDK to allow manual setup of instrumentation.

* Instrumentation core test

* Fix sporadic Elasticsearch test failures (#444)

* Fix sporadic grizzly test failure (#448)

* Fixes integration with latest version of Finatra (#450)

* #425 allow only one propagator per concern (#451)

* Updates to reflect new repo name (#454)

* Remove printlns that were accidentally committed (#459)

* Remove unnecessary version constant (#455)

Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>
Co-authored-by: Steve Flanders <steve@omnition.io>
Co-authored-by: Sergei Malafeev <sergeymalafeev@gmail.com>
Co-authored-by: Tyler Benson <tyler.benson@datadoghq.com>
Co-authored-by: Nikolay Martynov <mar.kolya@gmail.com>
Co-authored-by: Brian Devins-Suresh <brian.devinssuresh@datadoghq.com>
Co-authored-by: Richard Startin <richard.startin@datadoghq.com>
Co-authored-by: Anuraag Agrawal <anuraaga@gmail.com>
@trask trask deleted the fix-sporadic-elasticsearch-test-failures branch July 2, 2020 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sporadic test failure (1x): Elasticsearch2SpringTemplateTest
2 participants