Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pact consumer - when running a JUnit 4 spec with multiple tests every second pact test fails #342

Closed
andrewspinks opened this issue Oct 28, 2016 · 17 comments
Labels
bug Indicates an unexpected problem or unintended behavior

Comments

@andrewspinks
Copy link

We are using pact-jvm-provider-gradle_2.11::3.2.1 to test the downstream dependency in a springboot app. We have found that in one of our specs that has multiple pact tests every second test fails saying that the expected requests were not received. If we run them individually each test works fine, and also when we put a 4 second delay into a @before, then they all pass. We have also tried running them in a different order and it is always every second test that fails, regardless.

We have tried using versions 3.3.2, 3.3.1, 3.2.13 and 3.1.3 and all have the same problem.

@uglyog
Copy link
Member

uglyog commented Oct 29, 2016

I'm assuming that your test is not specifying a random port for the mock server?

@uglyog uglyog added the bug Indicates an unexpected problem or unintended behavior label Oct 29, 2016
@uglyog
Copy link
Member

uglyog commented Oct 29, 2016

Or are you talking about verifying the pacts. You mentioned pact-jvm-provider-gradle and 'one of our specs that has multiple pact tests'?

@uglyog
Copy link
Member

uglyog commented Oct 29, 2016

You have JUnit 4 spec in the title, so I'm assuming the issue is with the Junit library, not the groovy one?

@uglyog
Copy link
Member

uglyog commented Oct 29, 2016

I was not able to reproduce it (of course ;-) ) so, could you provide an example test and logs?

Here is the test I created: Defect342MultiTest.groovy

@andrewspinks
Copy link
Author

@uglyog Apologies, I copied the wrong dependency out of our gradle file. It should have been: au.com.dius:pact-jvm-consumer-groovy_2.11:3.2.1

I was trying to reproduce the issue in a simple environment when I noticed the problem has disappeared! Looking through our changes it seems that changing the configuration of our RestTemplate's requestFactory has magically made the problem go away. I'll dig a bit more to see if I can track down what was the underlying cause was...

@andrewspinks
Copy link
Author

@uglyog Here is a failing test. https://github.com/andrewspinks/pact-error/blob/master/src/test/groovy/com/example/PactSpec.groovy I will try and simplify it further to take out the springboot stuff and see if it still fails.

@uglyog
Copy link
Member

uglyog commented Nov 5, 2016

Awesome, thanks for the test project. Hopefully it is something as simple as requiring a wait for the mock server to shutdown at the end of the test.

@uglyog
Copy link
Member

uglyog commented Nov 5, 2016

Using a random port for each test does not have this issue, so it prob needs a wait for the mock server to be ready before running the test.

@uglyog
Copy link
Member

uglyog commented Nov 5, 2016

Ok, getting closer to the problem. It looks like an issue with the HTTP client, not the pact library.

From what I can gather, I think it is to do with persistent connections (as controlled by the Keep-Alive header). The HTTP client is sending a Keep-Alive header, and creating a persistent connection in a pool. Then at the end of the test, the mock server is shut down and the persistent connection is no longer valid.

The next test will try reuse the previous connection, which is stale as a new mock server has now started on that port, and results in a org.apache.http.NoHttpResponseException.

By putting the timeout in the startup, when the second test runs, the HTTP client will not reuse the old connection because it has probably exceeded the max stale check time in the pool, and will get a new one.

@uglyog
Copy link
Member

uglyog commented Nov 5, 2016

This fixes your test:

  @After
  void teardown() {
    Executor.closeIdleConnections()
  }

@andrewspinks
Copy link
Author

Wow, awesome Ron, thanks! Is there some troubleshooting doco that could be updated? Otherwise I'll close this issue.

@uglyog
Copy link
Member

uglyog commented Nov 6, 2016

We could add a note to the two project readmes.

@ricwal-richa
Copy link

ricwal-richa commented May 5, 2017

I am using au.com.dius:pact-jvm-provider-gradle_2.11:3.2.4 and getting 'Connection refused' for every single test (attaching stacktrace and debug mode files). Also is there a way to increase timeout or put a delay directly through a property defined in gradle build file ?
I am getting same error even for single individual test when using provider-gradle library for localhost.
Multitest_failure_debug.txt
Multitest_failure_stacktrace.txt
I am trying to run the sample test given under Defect342MultiTest.groovy

@uglyog
Copy link
Member

uglyog commented Dec 20, 2017

@ricwal-richa are you verifying an actual provider? It looks like there is nothing running on port 8080, hence the connection refused errors.

@peternelissen
Copy link

Adding the exception for SEO reasons. Searched a long time for this bug, maybe this helps people find it:

For me it only happened for two PUT requests after each other.

Caused by: java.net.SocketException: Software caused connection abort: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)

@asocaciu
Copy link

I want to point out another workaround for connections remaining open, if using the standard JDK http library
@BeforeAll public static void setup() { System.setProperty("http.keepAlive", "false"); }
Reference: https://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepalive.html

@uglyog
Copy link
Member

uglyog commented Jun 27, 2021

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Development

No branches or pull requests

5 participants