Docker Compose and dependencies

[mdelapenya]

In the light of the amount of issues we received for the latest 0.16.0 release (#629 #634 #669 #687), where we introduced the native support for Docker Compose, including a bunch of transitive dependencies plus a replace directive using and old version of Docker, we open this discussion to debate where to go next with compose.

My first idea is to decouple code compose to a separate Go module (see #646), but we are trapped with a dependency version on the old Docker version. Please see our install instructions, the changelog, and docker/compose#9946 (comment) for further details.

I'm not really sure how much is the compose code used, but we are penalising regular users with compose dependencies. TBH, I did not consider how dependencies would impact downstream consumers when reviewing #476, the PR adding native compose support, but it should have been done there. Nevertheless, we have a few options, that I'd like to discuss here:

1. remove native compose code in 0.17.0, bringing back the library to a state where the dependencies are controlled, but without compose code. We would keep a native-compose branch with the very same code as main, which we would maintain and rebase/merge with current main for users that want to experiment with native compose.
2. review and eventually merge [Enhancement]: Move compose code to a separate Go module #646, which is separating compose to a module, but still keeping the dependency on the old Docker version. This is way reducing the amount of the direct and indirect dependencies of the project, which is desired. On the other hand, consumers would probably need the replacement of the fixed Docker version (which we already added in 0.16.0), therefore it's a simplification of the current state, although again not desired
3. do nothing and release 0.17.0 with the current state for compose, as the dependencies are documented in both the changelog for 0.16.0 and the docs site.

@baez90 @hhsnopek @testcontainers/oss-maintainers, I'd like to specially get your feedback on this 🙏 Feel free to comment here.

[kiview]

Regarding 2), this would only be required for users that consume the compose modules, correct? And it won't be necessary anymore, once the new Docker version is released, correct?

In this case, I'd propose doing 2), with the addition of marking the compose module as an incubating module. This is a term we use for modules in Testcontainers for Java, but it basically just means beta and potentially breaking changes in the future.

[mdelapenya]

Not exactly: because the compose module finally imports testcontainers-go, and both consume the Docker dependency, the lowest version of Docker will be fetched, therefore the main/core module will need the replace directives for Docker, hence the consumers of tc-go will need it too.

This replacement is way smaller than the current one, which bundled replaced imports for docker cli, k8s, etc.

[prskr]

First of all, I'm deeply sorry that the PR that originally intended to make working with docker-compose easier backfired so hard...

As already commented in #646 I think moving it to a new module is the right step also because this keeps extra maintenance compared to the extra branch reasonable. Also with something like that already in place other modules might follow in the future which could be nice as well?

Generally I hope that the Docker guys at some point move to a proper go.mod allowing all downstream projects to actually version without pain...

[mdelapenya]

Thanks @baez90 for your comments, no worries as there is there is no blame at all, I believe we all are learning from this!

I think we can merge #646 and mitigate the dependencies explosion that came with compose.

[mdelapenya]

Both #706 and #650 were merged and released in 0.17.0, please let us know if the dependency management is clear enough 🙏

PTAL at the release notes there, as we only need to replace Docker, removing a big amount of replacements (still needed if you use the compose module).

[mdelapenya]

0.18.0 was released yesterday, simplifying the dependencies with Docker. Thankfully we were able to remove the replace directive in the main module, although the compose one still needs some -fewer- of them.

[mdelapenya]

I think we can close this discussion, thanks all for participating!