Limit used memory/CPU or parallel jobs

[WolfspiritM]

Hi,

I tried to use buildx to speed up our pipeline.
We have a huge docker-compose with many services that need to be build and our buildserver is a bit limited with memory and cpu.

Once I started using buildx to build our project with docker buildx bake -f docker-compose.yml after a while the buildserver stopped responding and it was out of memory and cpu. It had many many build processes running in parallel.

Is there any way to limit the amount of ram and cpu used or at least specify the maximum allowed parallel jobs? Right now it seems to run everything at once which seems to overwhelm our buildserver.

[Aposhian]

This should be as simple as passing on runtime options to the buildkit container, right?

[Aposhian]

Also, docker buildx build already supports --memory and other throttling flags. Mind if I make a PR to add those same options to buildx bake? The only issue is still specifying the number of parallel jobs, since simply passing on --memory would give that much to each build job, right?

[MrHash]

In my testing setting CARGO_NET_GIT_FETCH_WITH_CLI=true resolves the OOM 137 error code during the cargo crate fetching stage using buildx

[tnaroska]

interested in this feature as well. There is max-parallelism option in buildkit.toml. However that has no effect on operations from individual RUN steps in the dockerfile.

It would be more useful to have options to limit the cpu/memory when creating a docker-container instance using buildx create

[aidapsibr]

interested in this feature as well. There is max-parallelism option in buildkit.toml. However that has no effect on operations from individual RUN steps in the dockerfile.

It would be more useful to have options to limit the cpu/memory when creating a docker-container instance using buildx create

I'm running into similar issues with the Kubernetes driver. If I limit the CPU I just end up CPU throttled. Seems like my build jobs aren't fanning out when I set replicas either, so if I bake too many images at once I just eventually get timeouts. I add this because the same issue could come up with the docker-container solution with that solution alone.

[crag-h4k]

In my testing setting CARGO_NET_GIT_FETCH_WITH_CLI=true resolves the OOM 137 error code during the cargo crate fetching stage using buildx

I'm still running into this same issue :(

[nicks]

Compose supports COMPOSE_PARALLEL_LIMIT. Maybe it would be good if buildx bake respected this as well?

https://docs.docker.com/compose/environment-variables/envvars/#compose_parallel_limit

Alternatively, maybe add a BUILDX_PARALLEL_LIMIT to match the compose CLI?

[Borda]

Also, docker buildx build already supports --memory and other throttling flags. Mind if I make a PR to add those same options to buildx bake?

That would be great!

[mrexodia]

In my opinion this feature is absolutely critical to using docker buildx. We are building LLVM for 2 architectures and because there are two architectures running in parallel the builds are failing left and right with OOM errors. CMake/Ninja can somewhat manage resources when they are the only ones using the resources, but with another container running there isn't much you can do. The best solution here would be to simply allow specifying -j 1, so the amount of parallel jobs is limited to 1.

My workaround for this is to run docker buildx for every platform separately first and only then combine everything into a single tag. This way the results are cached already, so the command will be very quick:

docker buildx build --platform linux/amd64 -t ghcr.io/org/repo:tag .
docker buildx build --platform linux/arm64 -t ghcr.io/org/repo:tag .
docker buildx build --platform linux/arm64,linux/amd64 -t ghcr.io/org/repo:tag .