Depends_on in version3

[nischay30]

Hi i would like to know what is the alternate of depends_on in docker-compose v3 as in the release notes you said we will jot port the depends_on feature in v3

[shin-]

depends_on still exists in v3, but healthcheck dependencies (and as a result, the extended syntax) will not be ported.

HTH

[nischay30]

But i have written a docker-compose v3 and i try to deploy on swarm but depends_on is not working as the container are not starting in the fashion in which they have to be started.

[shin-]

Are you using docker-compose or docker stack deploy?

[nischay30]

I am using docker stack deploy
and i try to deploy it on swarm of 7 machines

[stffabi]

What would be the equivalent on docker-compose v3 to achieve something like healthcheck dependencies? If you are going to drop that functionality in v3 one should basically never use it, or at least there should be a migration path for this.

What are the intentions to introduce it in a new docker-compose v2.1 and then dropping it for v3? I'm currently setting up different compose files for our applications, but I don't want to use features which are getting dropped in the next version and therefore preventing use from updating to a newer docker-compose file version.

[shin-]

At the moment, you should assume that the new depends_on syntax will not be ported to v3, as we have no current plans to do so.

I know that's not the answer a lot of people want, but I hope that helps give some clarity at least.

[mustafaakin]

Can I ask why it is not in plans? I assume it would be very useful to do so.

[hsmade]

It gives clarity, but doesn't explain. Can you elaborate on the why? And on the alternatives (if they exist)?
The depends_on gives us a really easy way to depend on a service, as opposed to handling it inside the container (which might mean wrapping a 3rd party image with some wait script and having to maintain that).

[dead10ck]

@shin- why did you implement it at all in 2.1, then? If people use it and come to depend on it, they will never be able to upgrade. With all due respect, that seems like very poor planning on your guys' part.

[brettmc]

So what is the supported depends_on syntax for v3? https://docs.docker.com/compose/compose-file/#version-3 does not mention depends_on, and when I use docker-compose v1.10 to deploy an application neither the v2 or v2.1 depends_on flavours work for me in a v3 compose file...

[shin-]

@mustafaakin

Can I ask why it is not in plans? I assume it would be very useful to do so.

@hsmade

Can you elaborate on the why? And on the alternatives (if they exist)?

From moby/moby#30404 (comment)

depends_on is a no-op when used with docker stack deploy. Swarm mode services are restarted when they fail, so there's no reason to delay their startup. Even if they fail a few times, they will eventually recover.

---

@brettmc

So what is the supported depends_on syntax for v3?

When using docker-compose, the supported syntax for v3 is the list syntax (similar to the one used for 2.0). If you're using docker stack deploy, dependencies will not be honored (see above for justification)

---

The version 3 format is the first step in moving away from the external docker-compose tool towards the integrated docker stack solution. The current implementation has its quirks which are being worked on. Support for the version 3 format in docker-compose is meant to help that transition. A lot of things have changed and improved in Docker since fig / Compose was first introduced, and that means a lot of the things that used to make sense have now become obsolete. docker stack is a fresh start using new concepts and sheds away some of the more unwieldy syntax and concepts, from volumes_from to depends_on.
If you have particular concerns about some of those changes, please report them over on the docker/docker repo where it is being actively iterated on.
If you are not yet ready to transition to Docker services and docker stack, feel free to keep using the v2 format. While it's reasonable to assume the project will be sunset at some point in the future, it will be announced well in advance. And after that the code will remain available and open source.

[wjv]

Thanks. Now it makes sense.

[mustafaakin]

Swarm mode services are restarted when they fail, so there's no reason to delay their startup. Even if they fail a few times, they will eventually recover.

IMHO this is not a good approach. Not all services can properly detect the other services they depend are not ready, they try many times then fail, so container might die later. We still need to introduce entrypoint wrapper scripts, which is not very nice. Healthy-check dependence was very nice, but it just does not support swarm mode.

[Marvel77]

Swarm mode services are restarted when they fail, so there's no reason to delay their startup. Even if they fail a few times, they will eventually recover.

Hey everyone!
Does it mean that if for example I have a service which runs and finishes its work very quickly (and should run only once by design) it will be started again and again and again... repeatedly?

[shin-]

@Marvel77 By default, yes, but you can override that behavior using the deploy.restart_policy section: https://docs.docker.com/compose/compose-file/#restartpolicy

[Marvel77]

@shin- Thank you very much!

[hsmade]

@mustafaakin Actually, the best practice is (IMHO) to have fault tolerant connection to the services you depend on. For example, if you use a database, you might delay startup until it's responding. But how do you handle a restart of the database? Your app should be able to recover from this, and then you don't need to depend on 'depend_on' either.
In a sense, the deprecation is a good thing. These conveniences make us lazy.

[mustafaakin]

@hsmade But almost all init (upstart,systemd) have depends on relationship. So it is not that being lazy, it is what makes sense. SSHD daemon does not start until you have your network device ready. I have no control over all the systems I have, yes I can make my systems be fault tolerant. But assume A depends on B. A takes a while to initialize, it is not very deterministic. So, how can you write a restart policy on B? restart forever? What if you do not want that?

This is a bigger problem than Compose, Compose just starts them. Swarm is what makes them run, so I think Swarm should handle this dependence on health relationship.

[hsmade]

@mustafaakin I don't think you can compare running micro services in a containerized environment with the classic init systems. That doesn't really make sense. I think the idea of micro services is that they are self-contained, minimal entities. They have a very clear definition of their API, etc. They should not assume any status of their external world. Yes, it would need a database, but no, you can't assume it's there. As I tried to say, knowing that your dependency is there on startup is the least of your problems, handling it well when it goes away is more important, and should solve the former as well.

But then again, these are my thoughts on the matter, and I could be wrong ;)

[mustafaakin]

Yeah, it does not make sense for microservices, but not everything is a microservice, I run Hadoop in a container. Docker is not for only microservices. As I said, its great for the environments I have control, but in the things that I do not have control, it requires wrapping the entrypoint of services. It was just solved with depends_on with healtcheck, I just think it would be great to have it in Swarm as well. Restarting contionously is just a lazy man's choice.

[ozlatkov]

Guys,

I think there is a kind of mis-conception of "fault-tolerant" and a kind of "first-time initialization".
Agree that the approach sounds good enough for the first one, but the latter is a real pain.
I would imagine that not only myself, but generally one usually would need to start services in a particular order as they depend on each other more or less.

Having continuous restarts at initialization phase and waiting for the services at some point start in the proper order by themselves is like a nightmare - one cannot plan any "downtime" for the startup process should the worse happen. Not only the worse, but sometimes there are like say "maintenance" time when something needs to change and nobody would be able to predict how much time it would take for the system to actually start, because different services are restarted by swarm over and over again waiting for the proper order.
I've tried and waited to see how it will go with 7 containers and for 20 minutes swarm still hasn't figured it out and the whole service was still down.
So one cannot even state how much time it would take to bring back and restore the whole infrastructure as it's really unknown how long the initial startup would take.
Sounds absurd to me.

I don't think I can quite enter in "production" without such a predictability even though it's supposed to happen rarely (full down) - best never happen.
But exactly if it still happens I would be on the spot and will be challenged to restore ASAP and right at that time and pressure I would have absolute no idea how much time my containers would start just because they continue to restart themselves during startup.

[shin-]

@ozlatkov This should really be posted on the docker/docker issue tracker, not here.

[ozlatkov]

@shin- Thank you! I've moved it to docker/docker tracker:

moby/moby#31333

[jacek99]

I think it is really bad that the Docker team assumes Docker Swarm is used. Compose is supposed to a totally functional standalone tool.

We do however use Docker Compose a lot in our testing pipelines and such a fundamental REMOVAL of features (without giving a workable alternative) is really having a dramatic negative effects on your users.

I am currently reviewing a very ugly PR from my team members where they are trying to come up with all sorts of workarounds to this (since we rely heavily on this functionality), including staying on Compose 2.X for all eternity.

Docker is supposed to help us get to our goals, not make it more difficult by removing critical features many teams already rely on.

Please reconsider porting all of this into Docker Compose 3.

Much appreciated.

[shin-]

For the 100th time now: There's no reason to use the v3 format if you don't intend to use swarm services.

[lukehau]

Does that mean the team is committing to supporting the 2.X formats for those that are using compose as a standalone development tool?

[jacek99]

Exactly my question: is the Compose team committed to supporting the v2 format forever?

We cannot standardize on Docker Compose if the v2 format is scheduled for deprecation any time.

[shaunwarman]

I feel like this forces all containers into an init container pattern, removes the docker restart policy and creates a hacky approach to startup order. Should it be assumed that >= v3 of compose will be moving in this direction to focus on stacks and creating application bundles? And if that's right, could you point me in the direction of how to maintain startup order in a docker stack? Is the wait-for-it.sh or dockerize the approach there?

[ntwrkguru]

What is the declarative equivalent of docker-compose.yml for a stack?

[VinceOPS]

Hi guys, what's the best practice if I intend to use docker stack and get rid of docker-compose?

This seems to be the solution, but having sort of hacky scripts to init containers doesn't sound like a good practice. Does it?

Thanks.

@mustafaakin Thanks for your downvote! Very helpful ❤️

[hutchic]

@VinceOPS I'm not sure about "best practice" but I've been using healthchecks and restart: always so it just cycles until it works ¯\(ツ)/¯

[ailjushkin]

@hutchic But as mentioned in the above conversation, it could not have an end date, some kind of a circular on-failure restart.

[tianhao-au]

Why the docker team remove this feature in version3 and reject to add those in?

[thaJeztah]

@tianhao-au see the discussion on moby/moby#31333 (and moby/moby#31333 (comment))

[thaJeztah]

For compose, I also left a suggestion in moby/moby#31333 (comment) (have a x-depends_on)

[wordstospend]

This change of feature is literally why I don't use docker-compose any longer. If I am using docker in production and docker-compose locally for dev environments I am now required to make my docker containers behave radically different in dev than in production. In production I rely on an orchestration system to insure the health and order of my deployments. In dev land that orchestration was being done by docker-compose. Now I'm writing a bunch of mangled scripts to check health of things to orchestrate my system. If I'm gonna do that why not just write some golang to manage my dockers and be done with it. A little bummed it was dropped. Just my 2p

[gtirloni]

Trying to use the latest docker-compose version to make things more future-proof and stumbled upon this issue. This is sad.