docker compose ps does not show services from docker-compose.override.yml

[roslov]

Tell us about your request
Since Docker Desktop 4.12.0, the docker compose ps command does not show the services from docker-compose.override.yml.

Which service(s) is this request for?
Docker Desktop 4.12.0 (85629)

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
I use both docker-compose.yml and docker-compose.override.yml.

When I start my project with docker compose up -d, all containers are running.

But when I run docker compose ps, I see only services located in docker-compose.yml, but no services from docker-compose.override.yml.

docker ps show all services.

This happens on Docker Desktop 4.12.0 (85629).

The previous problem did not have such a problem.

Are you currently working around the issue?
I use docker ps to see all containers. But it’s not so comfortable comparing to docker compose ps.

Additional context
—

[hyu]

Hi @roslov, thanks for submitting this issue! We're looking into this to see if a regression may have occurred in 4.12. I'll update when I have more insight.

[roslov]

Any updates on this?

[glours]

Hello @roslov
If you use the following command docker compose -p your-project-name ps you should be able to see all the containers associated to your stack.
We'll check if this is a regression from Compose v1 behaviour

[roslov]

Thank you. The -p key helped. But previously I’ve never used this key, and everything worked. Looks like a regression.

➜  docker compose -p deployment ps
NAME                                          COMMAND                  SERVICE                          STATUS              PORTS
deployment-api-1                              "docker-php-entrypoi…"   api                              running             9000/tcp
deployment-api-db-1                           "docker-entrypoint.s…"   api-db                           running             33060/tcp, 0.0.0.0:3310->3306/tcp
deployment-consumer-api-common-1              "docker-php-entrypoi…"   consumer-api-common              running             9000/tcp
deployment-consumer-core-common-1             "docker-php-entrypoi…"   consumer-core-common             running
deployment-consumer-core-common-topic-1       "docker-php-entrypoi…"   consumer-core-common-topic       running
deployment-consumer-core-google-ads-1         "docker-php-entrypoi…"   consumer-core-google-ads         running
deployment-consumer-core-seeding-alliance-1   "docker-php-entrypoi…"   consumer-core-seeding-alliance   running
deployment-consumer-stats-campaigns-1         "docker-php-entrypoi…"   consumer-stats-campaigns         running             9000/tcp
deployment-consumer-stats-stats-1             "docker-php-entrypoi…"   consumer-stats-stats             running             9000/tcp
deployment-core-1                             "docker-php-entrypoi…"   core                             running
deployment-cron-1                             "docker-php-entrypoi…"   cron                             running
deployment-docs-1                             "docker-php-entrypoi…"   docs                             running             9000/tcp
deployment-frontend-1                         "/docker-entrypoint.…"   frontend                         running             0.0.0.0:82->80/tcp
deployment-frontend-node-1                    "docker-entrypoint.s…"   frontend-node                    running
deployment-mailcatcher-1                      "mailcatcher --no-qu…"   mailcatcher                      running             1025/tcp, 0.0.0.0:1080->1080/tcp
deployment-mailer-1                           "docker-php-entrypoi…"   mailer                           running
deployment-main-db-1                          "docker-entrypoint.s…"   main-db                          running             0.0.0.0:3306->3306/tcp
deployment-memcached-1                        "docker-entrypoint.s…"   memcached                        running             0.0.0.0:11211->11211/tcp
deployment-nginx-1                            "/docker-entrypoint.…"   nginx                            running             0.0.0.0:80->80/tcp
deployment-rabbitmq-1                         "docker-entrypoint.s…"   rabbitmq                         running             4369/tcp, 5671-5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp
deployment-rpc-api-main-1                     "docker-php-entrypoi…"   rpc-api-main                     running             9000/tcp
deployment-rpc-core-token-1                   "docker-php-entrypoi…"   rpc-core-token                   running
deployment-stats-1                            "docker-php-entrypoi…"   stats                            running             9000/tcp
deployment-stats-db-1                         "docker-entrypoint.s…"   stats-db                         running             33060/tcp, 0.0.0.0:3312->3306/tcp
deployment-test-api-db-1                      "docker-entrypoint.s…"   test-api-db                      running             33060/tcp, 0.0.0.0:3311->3306/tcp
deployment-test-main-db-1                     "docker-entrypoint.s…"   test-main-db                     running             0.0.0.0:3308->3306/tcp
deployment-test-stats-db-1                    "docker-entrypoint.s…"   test-stats-db                    running             33060/tcp, 0.0.0.0:3313->3306/tcp

➜  docker compose ps
NAME                         COMMAND                  SERVICE             STATUS              PORTS
deployment-api-1             "docker-php-entrypoi…"   api                 running             9000/tcp
deployment-api-db-1          "docker-entrypoint.s…"   api-db              running             33060/tcp, 0.0.0.0:3310->3306/tcp
deployment-core-1            "docker-php-entrypoi…"   core                running
deployment-frontend-1        "/docker-entrypoint.…"   frontend            running             0.0.0.0:82->80/tcp
deployment-frontend-node-1   "docker-entrypoint.s…"   frontend-node       running
deployment-mailcatcher-1     "mailcatcher --no-qu…"   mailcatcher         running             1025/tcp, 0.0.0.0:1080->1080/tcp
deployment-main-db-1         "docker-entrypoint.s…"   main-db             running             0.0.0.0:3306->3306/tcp
deployment-memcached-1       "docker-entrypoint.s…"   memcached           running             0.0.0.0:11211->11211/tcp
deployment-nginx-1           "/docker-entrypoint.…"   nginx               running             0.0.0.0:80->80/tcp
deployment-rabbitmq-1        "docker-entrypoint.s…"   rabbitmq            running             4369/tcp, 5671-5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp
deployment-stats-db-1        "docker-entrypoint.s…"   stats-db            running             33060/tcp, 0.0.0.0:3312->3306/tcp
deployment-test-api-db-1     "docker-entrypoint.s…"   test-api-db         running             33060/tcp, 0.0.0.0:3311->3306/tcp
deployment-test-main-db-1    "docker-entrypoint.s…"   test-main-db        running             0.0.0.0:3308->3306/tcp
deployment-test-stats-db-1   "docker-entrypoint.s…"   test-stats-db       running             33060/tcp, 0.0.0.0:3313->3306/tcp

[jonDowdle]

I also have this issue. I've tried setting COMPOSE_PROJECT_NAME and doesn't fix this. I still have to add the -p flag which is pretty annoying. I experienced it on 4.14.1 and I just tested on 4.15 and it is still occuring. Here's my version info:

Server: Docker Desktop 4.15.0 (93002)
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 17:59:41 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.10
  GitCommit:        770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

I added this as a zsh alias until the bug is fixed. Hope it helps:

dc() {
  if [ -n "${COMPOSE_PROJECT_NAME}" ]; then
    docker compose -p $COMPOSE_PROJECT_NAME $@
  else 
    docker compose $@
  fi
}

[ndeloof]

I've not been able to reproduce with a simple override file, but found a scenario where I was able to: need to have profiles in my compose file (override is not involved) and run the app with such profiles enabled. Then docker compose ps is missing additional services. Do you have such a configuration ?

proposed a fix: docker/compose#10145

[roslov]

@ndeloof, you’re right. It is related to the profiles.

Here is the setup to reproduce:

#docker-compose.yml
version: "3.4"
services:
    nginx:
        image: nginx:latest
        restart: always
    nginx3:
        profiles:
            - full
        image: nginx:latest
        restart: always

#docker-compose.override.yml
version: "3.4"
services:
    nginx2:
        profiles:
            - full
        image: nginx:latest
        restart: always

$ docker compose --profile=full up -d
Network temp_default  Creating
Network temp_default  Created
Container temp-nginx3-1  Creating
Container temp-nginx-1  Creating
Container temp-nginx2-1  Creating
Container temp-nginx3-1  Created
Container temp-nginx2-1  Created
Container temp-nginx-1  Created
Container temp-nginx2-1  Starting
Container temp-nginx3-1  Starting
Container temp-nginx-1  Starting
Container temp-nginx2-1  Started
Container temp-nginx3-1  Started
Container temp-nginx-1  Started

$ docker compose ps
NAME                COMMAND                  SERVICE             STATUS              PORTS
temp-nginx-1        "/docker-entrypoint.…"   nginx               running             80/tcp

$ docker compose -p temp ps
NAME                COMMAND                  SERVICE             STATUS              PORTS
temp-nginx-1        "/docker-entrypoint.…"   nginx               running             80/tcp
temp-nginx2-1       "/docker-entrypoint.…"   nginx2              running             80/tcp
temp-nginx3-1       "/docker-entrypoint.…"   nginx3              running             80/tcp

$ docker compose --profile=full ps
NAME                COMMAND                  SERVICE             STATUS              PORTS
temp-nginx-1        "/docker-entrypoint.…"   nginx               running             80/tcp
temp-nginx2-1       "/docker-entrypoint.…"   nginx2              running             80/tcp
temp-nginx3-1       "/docker-entrypoint.…"   nginx3              running             80/tcp

Shouldn’t docker compose ps show all services if it is run from the same folder where docker compose --profile=full up -d was run?

[ndeloof]

@roslov docker compose ps should indeed show all services. Project loaded from compose.yaml (with profiles) is only relevant when it comes to filter by services, running docker compose ps SERVICE, where we validate SERVICE actually exists in project model

[ndeloof]

docker/compose#10145 being merged, this can be closed

[roslov]

I confirm that the bug is fixed. Docker Desktop v4.16.1 works correctly.

[roslov]

Still found another problem. The setup above is not downed by docker compose down completely without --profile=full.

$ docker compose down
Container temp-nginx-1  Stopping
Container temp-nginx-1  Stopping
Container temp-nginx-1  Stopped
Container temp-nginx-1  Removing
Container temp-nginx-1  Removed
Network temp_default  Removing
Network temp_default  Error
failed to remove network temp_default: Error response from daemon: error while removing network: network temp_default id fa36d97bfc845e349236a47e003d9c8ade3ecac8d0e4c4bf39f952ed180b969a has active endpoints

$ docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED              STATUS              PORTS     NAMES
a37c882190a8   nginx:latest   "/docker-entrypoint.…"   About a minute ago   Up About a minute   80/tcp    temp-nginx3-1
72f0cd34fac6   nginx:latest   "/docker-entrypoint.…"   About a minute ago   Up About a minute   80/tcp    temp-nginx2-1

[ndeloof]

@roslov this is the expected behavior: when ran with a compose file, down only consider the services declared in the file. If you want to "down the whole stack", use docker compose -p <project name> down

[hyu]

Closing this for now since the bug has been resolved as of docker/compose#10145 being merged into Docker Desktop v4.16.1.