JUnit 5 extension #887

britter · 2018-09-28T15:11:51Z

This is related to #636 but takes a different approach: instead of upgrading testcontainers core to JUnit 5 APIs, this introduces a new module containing a JUnit 5 extension that uses testcontainers core.

Static fields are shared among test methods (started only once and stopped at the end), while instance fields are started and stopped for every method. This approach has the advantage that it is relatively easy to implement. However, the downside of this is, that shared containers can't be used inside of @Nested test cases since these must not be declared as static inner classes and hence can not have static fields.

Feedback welcome @kiview, @sormuras, @nicolaiparlog, @marcphilipp

…apted from spock extension

kiview · 2018-09-28T21:00:08Z

Hey @britter, great that you've contributed the JUnit5 extension, I told you it's not too much work 😉
And cool approach similar to the Spock implementation, let's see what the others will think about this.

I'll do the review later this weekend 🙂

bsideup · 2018-09-29T10:18:29Z

Hi @britter! Thanks for contributing your vision of JUnit 5 support 👍
I'm totally okay to replace my PR with yours :) Could you please migrate the test cases tho? I've tried to reflect most of the use cases there, and (when I was thinking about the implementation) some use cases were not possible to cover with an extension, at least with the annotation-based one, but maybe I'm wrong :)

britter · 2018-09-29T11:38:18Z

Hi @bsideup,

sure thing. Which test cases do you mean? Those you added in #636?
The Codacy analysis fails but the failures seem to be false positives. Can you confirm?

nipafx

Since I was @'ed, I figured I'd give a quick review. I know little about Testcontainers (except that once they have proper JUnit 5 integration and no longer rely on JUnit 4, I will start using them immediately) and so focused on the Jupiter extension. Straightforward and well implemented. 👍

nipafx · 2018-09-30T08:30:50Z

modules/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/Testcontainers.java

+ * {@code @Testcontainers} is a JUnit Jupiter extension to activate automatic
+ * startup and stop of test containers used in a test case.
+ *
+ * <p>The test containers extension finds all fields of typ


Typo: "typ"

nipafx · 2018-09-30T08:33:22Z

...es/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/TestcontainersExtension.java

+class TestcontainersExtension implements BeforeAllCallback, BeforeEachCallback, AfterAllCallback, AfterEachCallback {
+
+    @Override
+    public void beforeAll(final ExtensionContext context) throws IllegalAccessException {


You could reduce code duplication between beforeAll and afterAll by having a method forEachSharedContainer(Consumer<Field>) that does the same thing as this method, but instead of startContainer calls Consumer::consume with the field.

nipafx · 2018-09-30T08:33:35Z

...es/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/TestcontainersExtension.java

+    }
+
+    @Override
+    public void beforeEach(final ExtensionContext context) throws IllegalAccessException {


Same as for beforeAll.

nipafx · 2018-09-30T08:51:20Z

modules/junit-jupiter/src/test/java/org/testcontainers/junit/jupiter/ComposeContainerIT.java

+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+@Testcontainers
+class ComposeContainerIT {


I don't know how Testcontainers in general or these new tests in particular are structured, so my point may be moot, but here it goes: I moved away from using test class names to distinguish unit and integration test. Instead I apply Jupiter tags (e.g. with @Tag("integration")) and configure the build tool to filter by tags instead of file names. If you're interested in that, let me know - I have more than these two cents to give. 😉

We build Testcontainers using Gradle and currently don't have dedicated source sets for Unit- and Integration-Tests. I would assume if we would retstructure our testing phases, we would use two distinct source sets for Unit- and Integration-Tests, so we can use it for all modules.

Alternatively, you could use tags and define two separate test tasks.

I would avoid IT suffix (a rudiment from Maven)

I think actually just copied over from the Spock tests 🙂

rnorth · 2018-09-30T09:45:16Z

@britter thanks for raising this! I haven't had a chance to review/play with this yet, but re the Codacy warnings: yes, please consider these false positives. There's nothing there that would worry me for test-scoped code.

kiview · 2018-10-01T08:26:30Z

modules/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/Testcontainers.java

+
+/**
+ * {@code @Testcontainers} is a JUnit Jupiter extension to activate automatic
+ * startup and stop of test containers used in a test case.


I would omit the 'test' from 'test container' here.

kiview · 2018-10-01T08:41:30Z

modules/junit-jupiter/src/test/java/org/testcontainers/junit/jupiter/MySqlContainerIT.java

+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * This test verifies, that setup and cleanup of containers works correctly.


I wonder right know, if this comment is still true for the existing Spock tests structure. Either way, this is kind of hackish and I think we should refactor this test in the Spock module and don't want to duplicate it for this new module :D

kiview

I think @bsideup is referring to the inheritance and abstract class tests:
https://github.com/testcontainers/testcontainers-java/pull/636/files#diff-d6b833ecb1659337d0093eab00102635

I'm very happy with the PR, since the integration surface is super slim and the API is the same than our Spock implementation.

One thing to keep in mind is, that VncRecording will not work right now. But we can add the support in a later PR.

britter · 2018-10-01T13:38:19Z

I'm doing another iteration. @nicolaiparlog has pointed me to the TestInstancePostProcessor extension point in JUnit Jupiter. This way I can implement shared containers as instance fields instead of static fields. This way the @Nested use case can also be implemented.

I'll also adapt the inheritance test case making sure that this use case is also supported!

britter · 2018-10-01T14:55:07Z

After another iteration I came up with a solution that is based on TestInstancePostProcessor. I've not pushed my solution, it has other draw backs which I'd like to outline below. I think we need to decide which approach we want to use and live with the drawbacks:

Solution 1 uses static fields to enable shared containers. This way shared containers are bound the the test class. The problem with this approach is, that shared containers can't be used inside nested test cases, because nested test cases must not be declared as static classes and can therefore not have static fields.

Solution 2 uses the TestInstancePostProcessor extension point to create shared containers. Shared containers can be declared as instance fields, so they are bound to the test instance rather then to the test class. The problem with this is, that containers can only be shared if the test is annotated with @TestInstance(Lifecycle.PER_CLASS) because otherwise it would be recreated every time the test class is instantiated. The the drawback here is, that users have to be aware of this and that they are forced to change the test instance lifecycle for shared containers to work. On the other hand with this approach shared containers also work in nested test cases.

I currently don't know which solution we should implement... So I'm happy for every feedback.

kiview · 2018-10-02T09:52:31Z

I think nested tests is a super convenient and powerful feature, however I'm not sure how many users will actually use them or know about this feature? Is it very prominent in JUnit5?

I have no real problem with **Solution 2 ** if we document it accordingly and would prefer it in order to support nested tests.

@rnorth @bsideup WDYT?

marcphilipp

Regarding solution 2: There's no way around the instance variable being instantiated twice. You could store the first one in the extension context of the nested test class and replace it using a TestInstancePostProcessor. However, I would consider that a hack. 😉

Thus, I'm in favor of using static fields and adding documentation that those won't work in nested classes.

marcphilipp · 2018-10-03T10:31:42Z

modules/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/Testcontainers.java

+ *
+ * <p>The test containers extension finds all fields of typ
+ * {@link org.testcontainers.lifecycle.Startable} and calls their container
+ * lifecylce methods. Containers declared as static fields will be shared


Typo: "lifecylce"

marcphilipp · 2018-10-03T10:34:06Z

...es/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/TestcontainersExtension.java

+    @Override
+    public void beforeAll(final ExtensionContext context) throws IllegalAccessException {
+        Class<?> testClass = context.getRequiredTestClass();
+        for (final Field field : testClass.getDeclaredFields()) {


This should take into account fields declared in super classes, shouldn't it? Same for the other methods in this class.

marcphilipp · 2018-10-03T10:36:50Z

...es/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/TestcontainersExtension.java

+    @Override
+    public void afterEach(final ExtensionContext context) throws IllegalAccessException {
+        Object testInstance = context.getRequiredTestInstance();
+        for (Field field : testInstance.getClass().getDeclaredFields()) {


Alternatively, you could store the started containers in the ExtensionContext and retrieve them from there. You could even store CloseableResource implementations and let Jupiter take care of stopping the containers.

marcphilipp · 2018-10-03T10:37:54Z

modules/junit-jupiter/src/test/java/org/testcontainers/junit/jupiter/ComposeContainerIT.java

+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+@Testcontainers
+class ComposeContainerIT {


Alternatively, you could use tags and define two separate test tasks.

kiview · 2018-10-26T06:16:47Z

Great, can you have a final look, @rnorth @bsideup?

modules/junit-jupiter/README.md

rnorth · 2018-10-27T17:02:40Z

This looks amazing - thank you all for contributing and refining this! I only had one minor comment on the docs; this is already a well polished PR by the time I've got to it 😄.

I'd like to have a little bit more of a play with this tonight, but I'll be looking forward to us merging and releasing this ASAP.

Thank you 🙇

rnorth · 2018-10-27T19:59:54Z

.../src/test/java/org/testcontainers/junit/jupiter/TestcontainersNestedRestaredContainerIT.java

+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+@Testcontainers
+class TestcontainersNestedRestaredContainerIT {


Tiny typo!

Suggested change

class TestcontainersNestedRestaredContainerIT {

class TestcontainersNestedRestartedContainerIT {

Co-Authored-By: britter <beneritter@gmail.com>

bsideup · 2018-10-29T20:22:55Z

...es/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/TestcontainersExtension.java

+        store.put(TEST_INSTANCE, testInstance);
+
+        findSharedContainers(testInstance)
+            .forEach(adapter -> store.getOrComputeIfAbsent(adapter.key, k -> adapter.start()));


Any reason to use field instead of getter?

The StoreAdapter class is a private class inside of TestcontainersExtension. So all fields are visible from TestcontainersExtension and there is no gain from an information hiding perspective if we add getters.

The problem I see is that it is inconsistent with the rest of our code base because we always use getters :)

bsideup · 2018-10-29T20:24:17Z

...es/junit-jupiter/src/main/java/org/testcontainers/junit/jupiter/TestcontainersExtension.java

+    }
+
+    private static Predicate<Field> isContainer() {
+        return field -> Startable.class.isAssignableFrom(field.getType()) && AnnotationSupport.isAnnotated(field, Container.class);


I would return any field annotated with @Container and throw and error if it is not Startable, otherwise it will simply ignore it

sormuras · 2018-10-29T21:54:27Z

I know, this PR is in a late and almost finished state -- but I promised @britter to spike a different way to handle shared containers, which is also thread safe. See https://github.com/junit-team/junit5-samples/compare/singleton-extension for a work-in-progress implementation using a StringBuilder as a shared resource.

Transferred to this container context, the test class could read like:

@ExtendWith(SingletonExtension.class)
class SingletonExtensionTests {

	@Test
	void test1(@Singleton(MySQL123.class) MySQLContainer shared) {}

	@Test
	void test2(@Singleton(MySQL123.class) MySQLContainer shared) {}

	@Test
	void test3(@New(MySQL123.class) MySQLContainer fresh) {}

}

The MySQL123 class is used as resource factory:

public class MySQL123 implements SingletonExtension.Resource<MySQLContainer> {

	private final MySQLContainer container = new MySQLContainer();

	@Override
	public MySQLContainer get() {
		return container;
	}
}

Edit: refactored the implementation and filed junit-team/junit5-samples#87 as a sample for discussion

Follow up to #887

* Small improvements to Jupiter extension code style Follow up to #887 * Only raise exception if annotated field is not Startable

jrehwaldt · 2018-12-17T11:36:13Z

I am trying to use this extension together with @SpringBootTest. How to ensure testcontainer starts first before executing SpringBootTest extension?

From what I see in the JUnit 5 documentation the annotation order matters. Since Java sorts annotations in bytecode alphabetically Spring will always run first unless it is possible to explicitly import @ExtendWith(TestcontainersExtension.class). Currently, this cannot be done due to TestcontainersExtension being package-private.

Question is: How to ensure testcontainers ordering relative to other extensions with the current extension mechanism?

bsideup · 2018-12-17T11:39:58Z

Hi @jrehwaldt!
Are you sure that you need JUnit 5 extension here? If you need containers for your Spring Boot app, you can easily add it inside a static block, test framework free:
https://testcontainers.gitbook.io/workshop/step-6-adding-redis

jrehwaldt · 2018-12-17T11:50:45Z

Thanks. That's exactly what I am currently doing. I got pointed to this new feature and thought I'd give it a shot. Imho, this extension provides test lifecycle support out of the box without copy and pasting start-stop logic in @[Before|After][All|Each] annotations from test to test. Unfortunately, it seems to me the interplay of multiple extensions wasn't part of the spec'ing, which is why I asked here for clarification.

Am I right in assuming the scenario I describe is simply not supported as of now? I believe making TestcontainersExtension public is all that's needed for users to be able to explicitly define the order of multiple extensions.

marcphilipp · 2018-12-18T06:11:03Z

@jrehwaldt Since this PR is merged, please open a new issue.

jrehwaldt · 2018-12-18T08:06:45Z

I did in #1017. Thank you.

britter added 9 commits September 28, 2018 11:41

Add JUnit Jupiter extension without implementation and simple test ad…

a057302

…apted from spock extension

Add first working implementation

7936f90

Adapt TestcontainersRestatBetweenTestsIT from spock extension

fff8dd0

Adapt PostgresContainerIT from spock extension

fbfac67

Adapt ComposeContainerIT from spock extension

92e2026

Adapt SharedComposeContainerIT from spock extension

5304cda

Adapt TestcontainersSharedContainerIT, thereby rewriting the extenstion

09719a9

Add JavaDoc

6b7bad6

Refactor common code into helper methods

728a50b

britter requested review from bsideup, kiview and rnorth as code owners September 28, 2018 15:11

nipafx reviewed Sep 30, 2018

View reviewed changes

kiview reviewed Oct 1, 2018

View reviewed changes

kiview requested changes Oct 1, 2018

View reviewed changes

Better documentation

3a939bf

britter mentioned this pull request Oct 1, 2018

Introduce JUnit Platform Test Kit junit-team/junit5#1392

Merged

7 tasks

britter added 2 commits October 1, 2018 17:04

Implement shared containers based on test instance post processor

18a16e2

Write shared container back to instance field

8d01abe

Write test for nested shared containers

475d349

marcphilipp reviewed Oct 3, 2018

View reviewed changes

Add implementation that works for nested restarted containers

88dc17a

britter changed the title ~~[WIP] JUnit 5 extension~~ JUnit 5 extension Oct 25, 2018

testcontainers deleted a comment Oct 25, 2018

kiview approved these changes Oct 26, 2018

View reviewed changes

rnorth reviewed Oct 27, 2018

View reviewed changes

modules/junit-jupiter/README.md Outdated Show resolved Hide resolved

rnorth reviewed Oct 27, 2018

View reviewed changes

rnorth mentioned this pull request Oct 28, 2018

Allow better support for test frameworks other than JUnit #87

Open

rnorth and others added 2 commits October 28, 2018 08:45

Shared containers have to be declared static

a6f9c40

Co-Authored-By: britter <beneritter@gmail.com>

Fix typo

835c68e

testcontainers deleted a comment Oct 28, 2018

kiview added the type/feature label Oct 29, 2018

kiview added this to the next milestone Oct 29, 2018

bsideup mentioned this pull request Oct 29, 2018

[WIP] JUnit 5 #636

Closed

bsideup reviewed Oct 29, 2018

View reviewed changes

britter added 2 commits October 30, 2018 10:37

Container can be declared final

1af5190

Rename tests

421c583

kiview merged commit d7c4708 into testcontainers:master Nov 2, 2018

kiview pushed a commit that referenced this pull request Nov 2, 2018

Small improvements to Jupiter extension code style

88d0a3e

Follow up to #887

kiview mentioned this pull request Nov 2, 2018

Small improvements to Jupiter extension code style #947

Merged

rnorth pushed a commit that referenced this pull request Nov 4, 2018

Small improvements to Jupiter extension code style (#947)

78a0f69

* Small improvements to Jupiter extension code style Follow up to #887 * Only raise exception if annotated field is not Startable

jrehwaldt mentioned this pull request Dec 18, 2018

Controlling execution order of JUnit 5 extension relative to other extensions not possible #1017

Closed

michael-simons mentioned this pull request Dec 18, 2018

JUnit 5 extension starts container to late. #1019

Closed

	class TestcontainersNestedRestaredContainerIT {
	class TestcontainersNestedRestartedContainerIT {

JUnit 5 extension #887

JUnit 5 extension #887

Conversation

britter commented Sep 28, 2018

kiview commented Sep 28, 2018

bsideup commented Sep 29, 2018

britter commented Sep 29, 2018

nipafx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rnorth commented Sep 30, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kiview left a comment

Choose a reason for hiding this comment

britter commented Oct 1, 2018

britter commented Oct 1, 2018

kiview commented Oct 2, 2018

marcphilipp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kiview commented Oct 26, 2018

rnorth commented Oct 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sormuras commented Oct 29, 2018 • edited Loading

jrehwaldt commented Dec 17, 2018

bsideup commented Dec 17, 2018

jrehwaldt commented Dec 17, 2018

marcphilipp commented Dec 18, 2018

jrehwaldt commented Dec 18, 2018

sormuras commented Oct 29, 2018 •

edited

Loading