delegate build to buildx bake

[ndeloof]

What I did
Use buildx raw json output (moby/buildkit#4113) so we can implement build as an external command.
Allows user to upgrade buildx and benefits new features/bug fixes without having to wait for compose to bump dependencies
Still relies on buildkit's progress UI, but we technically could redesign the build progress display by processing solver status on our own. If we do, we could fully remove dependency on buildx/buildkit

note: due to docker/buildx#2252, json parsing error must be ignored :'(

(not mandatory) A picture of a cute animal, if possible in relation to what you did

[crazy-max]

I wonder if we should call buildx in standalone mode to reduce overhead implied by docker command (and in particular docker proxy in DD).

[laurazard]

This also has implications wrt signal handling, CLI plugin management, etc. In this case we'd have CLI binary -> Compose plugin binary -> CLI binary -> Buildx plugin binary AND between the CLI and Compose and then between the 2nd CLI process and Buildx the CLI would be setting up a socket for plugin communication, etc. Need to think about what the tradeoffs are there/whether we should and what we can do on the CLI side to make this as painless as possible.

[ndeloof]

yes this I already identified, marked this as a Draft to avoid any confusion: there's a bunch of things missing here :)

[crazy-max]

👀

Suggested change

	cmd := exec.CommandContext(ctx, buildx.Path, "buildx", "bake", "--file", "-", "--progress", "rawjson", "--metadata-file", metadata.Name())
	cmd := exec.CommandContext(ctx, buildx.Path, "bake", "--file", "-", "--progress", "rawjson", "--metadata-file", metadata.Name())

[ndeloof]

as we propagate os.Env, DOCKER_CLI_PLUGIN_ORIGINAL_CLI_COMMAND is set and buildx get confused it's not actually running standalone. Need to exclude this variable to get buildx run standalone

[luciangabor]

would compose send to bake x-bakes too? https://docs.docker.com/build/bake/compose-file/
(if yes, I think this should have an "--opt-out" or an "--opt-in")

[ndeloof]

It could (this PR is 100% experimental) but AFAICT x-bake are not required, as all build features are now part of the compose-spec

[luciangabor]

It could be quite welcomed, but, when it comes to x-bakes, it's nice to be able to simply build and run with compose and bake with an output that extracts binaries, or set tags for pushes, or ... anything that's not about local dev (albeit it's possible to do that using distinct files)

[ndeloof]

docker compose build only would support a subset of bake features, typically is REQUIRE an image to be built, as the primary goal is to later run the app (also consider docker compose up --build), so "output that extracts binaries" would make no sense, and should only be used with docker buildx bake.

or set tags for pushes

Already supported if you run docker compose build --push

[thompson-shaun]

Did the stale timer expire?

[ndeloof]

revisited as #12300