-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Duplicate DB containers with Tomcat DB connection pool and Testcontainers JDBC driver #159
Comments
Hi John Clearly there's something not working as it should in this case - apologies. I'll try and have a look into this in the next few days and get back to Richard |
The Driver is being instantiated multiple times, per pool connection. The tomcat connection pool has PooledConnection which holds a reference to a Driver instance which it creates using ...newInstance(); One option might be to make the jdbcUrlContainerCache static but I don't know how you would clean it up between tests etc. |
Setting the pool size to 1 didn't help me as liquibase and some other setup scripts doesn't play nice with just one connection in the pool, so I gave this a go as described above. https://github.com/robshep/testcontainers-java/commit/5e96a656d66e6241a7d430cdfebdca9a8bc89b50 Worked OK for me, for my initial smoketest. |
@robshep thank you for looking into this and for the patch. If newInstance() is being used to instantiate then it seems a static field is the only option, and it's roughly analagous to there being a singleton instance of ContainerDatabaseDriver with an instance variable, really. Re cleanup between tests perhaps we could go with an approach similar to using an H2 in memory database - remove the container from the cache when the last connection to it is closed. We could even borrow the Additionally a simple and sensible thing to do would be expose a method to tests that allows the DB to be programmatically cleared in a @ Before block, for example. What do you think? |
I think given that it is an integration testing component, not meant for production, and short lived, it doesn't need to be the cleanest or purest solution, as long as it gets the job done. I can live with a memory leak in the testing framework, but having an exposed method, as you suggest will allow conscientious maintenance. I just looked at the relationship between the individual test methods and my hibernate context, as prepared by spring boot. Spring boot provides a full application context before any methods are run, and you can inject spring components into the test classes. Here are my observations thus, using stacktraces to show where/how it is used, This one is built at the start of the unit test class, as the spring context is booting My "static" alteration defined above seem to allow this to work fine, with no memory leaking. But then upon each test method, the Here is where the Rule approach kicks in needlessly in this scenario
I've found that running without the Here is my config in spring boot's YAML format:
I've fiddled with (Note when using @rule, because it is not setup using the Driver.getConnection(), these containers don't make it into the cache) As part of the connection wrapper, it seems it cleans up the cache here, but i've yet to see it happening, the main container shutdown seems to occur due to the shutdown listener. A note on HikariCP with Spring Boot HikariCP has a setup which tests if a connection can be created in a constructor and throws an exception if it fails. This bumps into the testcontainer watching for the last connection to be closed in order to shut down the container.
A setting - only available in spring boot 1.4.0 disables this check in Hikari.
I'm sorry that's a bit rambling. I'm just trying to give a good picture of how testcontainers might play nice with Spring boot. I'll try some more experiemntation later on. Thanks for a great library by the way, i'm looking forward to using it extensively! R |
I've had some easier setup for a SpringBoot application by just creating the container manually, and binding it to a custom DataSource. https://gist.github.com/robshep/34b13e69e3bcac4caad582d9bb7407f7 I'm confident that I can wangle it to use SpringBoot's DataSourceBuilder to make it a bit more seamless and then allow much more trivial setup for springboot testing. |
Hi Rob Thanks for this analysis, and sorry for the slightly delayed response while I'll have to look into this further in a couple of days, but it sounds like Re use of @ Rule, as you've found you'd not want to use it in conjunction Still, I should probably point out that for situations where it makes Cheers Rich
|
Any movement on this issue, it has become a blocker for our development and CI solutions (causing timeouts) as the workaround for setting the connection pool size doesn't play well with liquibase as mentioned in the above comments. |
Extend test suite to include more rigorous testing of connection pools with containers
@hopperd sorry for the long delay on this - recent Docker compatibility updates have been taking up a lot of time :( I've just pushed into a branch with PR #195. This includes a simple fix, and improvements to the test suite to show the original problem. I'd hope to release this change in the next few days (following PR review), but in the meantime you could use jitpack to obtain a build based on the |
Extend test suite to include more rigorous testing of connection pools with containers
Extend test suite to include more rigorous testing of connection pools with containers
Extend test suite to include more rigorous testing of connection pools with containers
Extend test suite to include more rigorous testing of connection pools with containers
I just tried with version 1.1.4 and it appears to be fixed for me. Thanks! |
Glad to hear that! Sorry to all that it took so long to complete this fix.
|
When running a very simple test on a new Spring Boot project, the project is creating 10 postgresql containers. This appears to be because of the connection pooling. Below is the configuration used to reproduce the 10 container issue in Spring Boot.
To fix this:
Add
spring.datasource.max-active=1
to your testsapplication.properties
file.Is this an issue? Seems like it should just create 10 connections to the same container...
My properties file
My Test
mvn test output:
The text was updated successfully, but these errors were encountered: