[BUG] `docker compose ps --format json` no longer returns an array in 2.21.0 #10958

unikitty37 · 2023-09-01T09:04:46Z

Description

The release notes for 2.21.0 state "Changed docker compose ps and docker compose ps --format=json output to align with Docker CLI."

However, in prior versions, the result was a valid JSON array. 2.21.0 returns individual objects, one after the other, with no [ … ] wrapper. This doesn't seem to be valid JSON.

Although jq can handle this, Ruby's JSON.parse rejects it as invalid; I've not tried other parsers.

Was this intended?

Steps To Reproduce

With any Compose project, run docker compose up -d.
Run docker-compose ps --format json.

Versions before 2.21.0 will return an array of objects; version 2.21.0 will return sequential objects, separated only by a new line.

Compose Version

% docker compose version
Docker Compose version v2.21.0
% docker-compose version
Docker Compose version v2.21.0

Docker Environment

Client:
 Version:    24.0.5
 Context:    orbstack
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.1
    Path:     /Users/username/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /Users/username/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/username/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/username/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     /Users/username/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/username/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/username/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.20.0
    Path:     /Users/username/.docker/cli-plugins/docker-scout

Server:
 Containers: 22
  Running: 11
  Paused: 0
  Stopped: 11
 Images: 23
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: btrfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: fe457eb99ac0e27b3ce638175ef8e68a7d2bc373
 runc version: 82f18fe0e44a59034f3e1f45e475fa5636e539aa
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.4.13-orbstack-00090-ga4cfdc88d4df
 Operating System: OrbStack
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 11.67GiB
 Name: orbstack
 ID: 437f17f3-56ac-4f55-ad12-6154b025c473
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:
  http://192.168.69.99:13588/
 Live Restore Enabled: false
 Product License: Community Engine
 Default Address Pools:
   Base: 192.168.215.0/24, Size: 24
   Base: 192.168.228.0/24, Size: 24
   Base: 192.168.247.0/24, Size: 24
   Base: 192.168.207.0/24, Size: 24
   Base: 192.168.167.0/24, Size: 24
   Base: 192.168.107.0/24, Size: 24
   Base: 192.168.237.0/24, Size: 24
   Base: 192.168.148.0/24, Size: 24
   Base: 192.168.214.0/24, Size: 24
   Base: 192.168.165.0/24, Size: 24
   Base: 192.168.227.0/24, Size: 24
   Base: 192.168.181.0/24, Size: 24
   Base: 192.168.158.0/24, Size: 24
   Base: 192.168.117.0/24, Size: 24
   Base: 192.168.155.0/24, Size: 24
   Base: 192.168.147.0/24, Size: 24
   Base: 192.168.229.0/24, Size: 24
   Base: 192.168.183.0/24, Size: 24
   Base: 192.168.156.0/24, Size: 24
   Base: 192.168.97.0/24, Size: 24
   Base: 192.168.171.0/24, Size: 24
   Base: 192.168.186.0/24, Size: 24
   Base: 192.168.216.0/24, Size: 24
   Base: 192.168.242.0/24, Size: 24
   Base: 192.168.166.0/24, Size: 24
   Base: 192.168.239.0/24, Size: 24
   Base: 192.168.223.0/24, Size: 24
   Base: 192.168.164.0/24, Size: 24
   Base: 192.168.163.0/24, Size: 24
   Base: 192.168.172.0/24, Size: 24
   Base: 172.17.0.0/16, Size: 16
   Base: 172.18.0.0/16, Size: 16
   Base: 172.19.0.0/16, Size: 16
   Base: 172.20.0.0/14, Size: 16
   Base: 172.24.0.0/14, Size: 16
   Base: 172.28.0.0/14, Size: 16

Anything else?

json_files.zip

This is occurring with OrbStack; Docker Desktop is still providing Docker Compose version v2.20.2-desktop.1, and is thus unaffected for now.

The text was updated successfully, but these errors were encountered:

glours · 2023-09-01T09:08:34Z

Hey @unikitty37
This is expected and mentioned in the v2.21.0 release notes. We aligned the docker compose ps format to the docker ps one, for more details see #10918

unikitty37 · 2023-09-01T09:10:34Z

Got it — thanks! I was reading the release notes at https://docs.docker.com/compose/release-notes/, which don't have links to the underlying tickets. Would it be worth explicitly mentioning this there?

glours · 2023-09-01T09:13:06Z

Sure I'll open a PR to mention it there 👍

unikitty37 · 2023-09-01T09:13:48Z

Thanks!

philBrown · 2023-09-06T00:46:28Z

Doesn't this represent a breaking change, requiring a major version bump?

tingstad · 2023-09-07T19:17:46Z

Doesn't this represent a breaking change, requiring a major version bump?

It certainly broke several of our builds since GitHub recently updated its Ubuntu runner images to use 2.21.0.

veaviticus · 2023-09-08T21:06:44Z

The docker compose docs (https://docs.docker.com/engine/reference/commandline/compose_ps/#format) also still claim the output is array-formatted JSON, instead of a jsonlines-esque format.

docker/compose#10958 Signed-off-by: Andrew Haines <haines@cerbos.dev>

* chore(deps): Bump grpc from 1.57.0 to 1.58.0 Bumps [grpc](https://github.com/google/grpc) from 1.57.0 to 1.58.0. - [Release notes](https://github.com/google/grpc/releases) - [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md) - [Commits](grpc/grpc@v1.57.0...v1.58.0) --- updated-dependencies: - dependency-name: grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Bump bundler from 2.4.13 to 2.4.19 Signed-off-by: Andrew Haines <haines@cerbos.dev> * Handle breaking change to `docker compose ps` output format docker/compose#10958 Signed-off-by: Andrew Haines <haines@cerbos.dev> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andrew Haines <haines@cerbos.dev> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Andrew Haines <haines@cerbos.dev>

docker/compose#10958 Signed-off-by: Andrew Haines <haines@cerbos.dev>

* Bump node from 20.5.1 to 20.6.1 * Bump npm from 9.8.1 to 10.1.0 * Drop support for Node.js 16 * Handle breaking change to `docker compose ps` output format docker/compose#10958 * Handle updated invalid argument error details in Cerbos 0.30 Signed-off-by: Andrew Haines <haines@cerbos.dev>

Rikaelus · 2023-09-30T01:11:13Z

Not to be a prick but Docker doing something misleading and wrong doesn't seem to me to be just cause to break what worked and turn valid JSON into invalid JSON. Nor should this be considered JSON format when it isn't, in fact, valid JSON.

This definitely pressed my "principle of the thing" button. This is the kind of command people have buried in codebases that are now broken -- codebases that might now have to compensate to support both to remain compatible with multiple Docker Compose versions on host machines.

alexrosenfeld10 · 2023-10-17T16:12:38Z

I'm impacted by this as well. @unikitty37 at the very least you should reopen this until the docs are updated to show the no json array behavior.

See for explanation: * https://stackoverflow.com/questions/77142618/incorrect-json-format-from-docker-compose-ps * docker/compose#10958

Docker prior to version 2.21.0 returned valid json when calling `docker-compose ps --format json`. Newer versions now return a series of json objects. This bugfix utilizes jq in the cli to fix the issue. More info: docker/compose#10958

unikitty37 added kind/bug status/0-triage labels Sep 1, 2023

glours removed the status/0-triage label Sep 1, 2023

unikitty37 closed this as completed Sep 1, 2023

glours mentioned this issue Sep 1, 2023

add note about changes of compose ps --json format in Compose release notes 2.21.0 docker/docs#18088

Merged

haines added a commit to cerbos/cerbos-sdk-ruby that referenced this issue Sep 12, 2023

Handle breaking change to docker compose ps output format

4d91dc8

docker/compose#10958 Signed-off-by: Andrew Haines <haines@cerbos.dev>

haines added a commit to haines/cerbos-sdk-javascript that referenced this issue Sep 12, 2023

Handle breaking change to docker compose ps output format

5f5e4af

docker/compose#10958 Signed-off-by: Andrew Haines <haines@cerbos.dev>

haines added a commit to haines/cerbos-sdk-javascript that referenced this issue Sep 12, 2023

Handle breaking change to docker compose ps output format

50a6489

docker/compose#10958 Signed-off-by: Andrew Haines <haines@cerbos.dev>

haines added a commit to haines/cerbos-sdk-javascript that referenced this issue Sep 12, 2023

Handle breaking change to docker compose ps output format

3f3b2c2

docker/compose#10958 Signed-off-by: Andrew Haines <haines@cerbos.dev>

Georgezhang714 mentioned this issue Oct 11, 2023

### Description BeyondDimension/SteamTools#2948

Closed

basvandervlies added a commit to basvandervlies/scl_modules that referenced this issue Jan 3, 2024

docker compose ps json format has changed

4ad5a87

See for explanation: * https://stackoverflow.com/questions/77142618/incorrect-json-format-from-docker-compose-ps * docker/compose#10958

basvandervlies mentioned this issue Jan 3, 2024

docker compose ps json format has changed basvandervlies/scl_modules#9

Merged

basvandervlies added a commit to basvandervlies/scl_modules that referenced this issue Jan 3, 2024

docker compose ps json format has changed (#9)

d3b3aed

See for explanation: * https://stackoverflow.com/questions/77142618/incorrect-json-format-from-docker-compose-ps * docker/compose#10958

errpunk added a commit to errpunk/letsencrypt-docker-compose that referenced this issue Jan 27, 2024

bug: fix docker/compose#10958

f75a7b6

Sreejit-K mentioned this issue Jan 30, 2024

Frontend deployment by docker for Issuance Portal Sunbird-RC/community#811

Open

rdimaio mentioned this issue Apr 2, 2024

Testing: Flaky tests, No such file or directory: 'docker-compose' rucio/rucio#6623

Closed

jonaskistler mentioned this issue Aug 26, 2024

Docker compose returned JSON is empty since docker compose 2.21.0 avast/gradle-docker-compose-plugin#449

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] `docker compose ps --format json` no longer returns an array in 2.21.0 #10958

[BUG] `docker compose ps --format json` no longer returns an array in 2.21.0 #10958

unikitty37 commented Sep 1, 2023

glours commented Sep 1, 2023

unikitty37 commented Sep 1, 2023

glours commented Sep 1, 2023

unikitty37 commented Sep 1, 2023

philBrown commented Sep 6, 2023

tingstad commented Sep 7, 2023

veaviticus commented Sep 8, 2023 •

edited

Loading

Rikaelus commented Sep 30, 2023

alexrosenfeld10 commented Oct 17, 2023

[BUG] docker compose ps --format json no longer returns an array in 2.21.0 #10958

[BUG] docker compose ps --format json no longer returns an array in 2.21.0 #10958

Comments

unikitty37 commented Sep 1, 2023

Description

Steps To Reproduce

Compose Version

Docker Environment

Anything else?

glours commented Sep 1, 2023

unikitty37 commented Sep 1, 2023

glours commented Sep 1, 2023

unikitty37 commented Sep 1, 2023

philBrown commented Sep 6, 2023

tingstad commented Sep 7, 2023

veaviticus commented Sep 8, 2023 • edited Loading

Rikaelus commented Sep 30, 2023

alexrosenfeld10 commented Oct 17, 2023

[BUG] `docker compose ps --format json` no longer returns an array in 2.21.0 #10958

[BUG] `docker compose ps --format json` no longer returns an array in 2.21.0 #10958

veaviticus commented Sep 8, 2023 •

edited

Loading