-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add more buildkit docs #1493
add more buildkit docs #1493
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -121,6 +121,28 @@ registries. | |
When you're done with your build, you're ready to look into [*Pushing a | ||
repository to its registry*](https://docs.docker.com/engine/tutorials/dockerrepos/#/contributing-to-docker-hub). | ||
|
||
|
||
## BuildKit | ||
|
||
Starting with version 18.09, Docker supports a new backend for executing your | ||
builds that is provided by the [moby/buildkit](https://github.com/moby/buildkit) | ||
project. The BuildKit backend provides many benefits compared to the old | ||
implementation. For example, BuildKit can: | ||
|
||
* Detect and skip executing unused build stages | ||
* Parallelize building independent build stages | ||
* Incrementally transfer only the changed files in your build context between builds | ||
* Detect and skip transferring unused files in your build context | ||
* Use external Dockerfile implementations with many new features | ||
* Avoid side-effects with rest of the API (intermediate images and containers) | ||
* Prioritize your build cache for automatic pruning | ||
|
||
To use the BuildKit backend, you need to set an environment variable | ||
`DOCKER_BUILDKIT=1` on the CLI before invoking `docker build`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is now also configurable in the CLI configuration file? (if so, we should have a link to that configuration somewhere). Could be in a new section "configure the Docker CLI to use the BuildKit builder" (e.g.) Wondering which one we consider the "preferred" mechanism, and/or if we should call out (dis)advantages of both ways to configure. (I'm ok with addressing that in a follow-up) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Some of it was covered by #1314 |
||
|
||
To learn about the experimental Dockerfile syntax available to BuildKit-based | ||
builds [refer to the documentation in the BuildKit repository](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also for a follow-up; would love to have this integrated in the documentation somewhere. If we link to GitHub, it would be good to link to a specific commit / tag to prevent the link from breaking in future |
||
|
||
## Format | ||
|
||
Here is the format of the `Dockerfile`: | ||
|
@@ -224,10 +246,64 @@ following lines are all treated identically: | |
# dIrEcTiVe=value | ||
``` | ||
|
||
The following parser directive is supported: | ||
The following parser directives are supported: | ||
|
||
* `syntax` | ||
* `escape` | ||
|
||
## syntax | ||
|
||
# syntax=[remote image reference] | ||
|
||
For example: | ||
|
||
# syntax=docker/dockerfile | ||
# syntax=docker/dockerfile:1.0 | ||
# syntax=docker.io/docker/dockerfile:1 | ||
# syntax=docker/dockerfile:1.0.0-experimental | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should we include an example with a fully-qualified reference for Docker Hub as well? ( |
||
# syntax=example.com/user/repo:tag@sha256:abcdef... | ||
|
||
This feature is only enabled if the [BuildKit](#buildkit) backend is used. | ||
|
||
The syntax directive defines the location of the Dockerfile builder that is used for | ||
building the current Dockerfile. The BuildKit backend allows to seamlessly use | ||
external implementations of builders that are distributed as Docker images and | ||
execute inside a container sandbox environment. | ||
|
||
Custom Dockerfile implementation allows you to: | ||
- Automatically get bugfixes without updating the daemon | ||
- Make sure all users are using the same implementation to build your Dockerfile | ||
- Use the latest features without updating the daemon | ||
- Try out new experimental or third-party features | ||
|
||
### Official releases | ||
|
||
Docker distributes official versions of the images that can be used for building | ||
Dockerfiles under `docker/dockerfile` repository on Docker Hub. There are two | ||
channels where new images are released: stable and experimental. | ||
|
||
Stable channel follows semantic versioning. For example: | ||
|
||
- docker/dockerfile:1.0.0 - only allow immutable version 1.0.0 | ||
- docker/dockerfile:1.0 - allow versions 1.0.* | ||
- docker/dockerfile:1 - allow versions 1.*.* | ||
- docker/dockerfile:latest - latest release on stable channel | ||
|
||
The experimental channel uses incremental versioning with the major and minor | ||
component from the stable channel on the time of the release. For example: | ||
|
||
- docker/dockerfile:1.0.1-experimental - only allow immutable version 1.0.1-experimental | ||
- docker/dockerfile:1.0-experimental - latest experimental releases after 1.0 | ||
- docker/dockerfile:experimental - latest release on experimental channel | ||
|
||
You should choose a channel that best fits your needs. If you only want | ||
bugfixes, you should use `docker/dockerfile:1.0`. If you want to benefit from | ||
experimental features, you should use the experimental channel. If you are using | ||
the experimental channel, newer releases may not be backwards compatible, so it | ||
is recommended to use an immutable full version variant. | ||
|
||
For master builds and nightly feature releases refer to the description in [the source repository](https://github.com/moby/buildkit/blob/master/README.md). | ||
|
||
## escape | ||
|
||
# escape=\ (backslash) | ||
|
@@ -1627,6 +1703,38 @@ RUN echo "Hello World" | |
When building this Dockerfile, the `HTTP_PROXY` is preserved in the | ||
`docker history`, and changing its value invalidates the build cache. | ||
|
||
### Automatic platform ARGs in the global scope | ||
|
||
This feature is only available when using the [BuildKit](#buildkit) backend. | ||
|
||
Docker predefines a set of `ARG` variables with information on the platform of | ||
the node performing the build (build platform) and on the platform of the | ||
resulting image (target platform). The target platform can be specified with | ||
the `--platform` flag on `docker build`. | ||
|
||
The following `ARG` variables are set automatically: | ||
|
||
* `TARGETPLATFORM` - platform of the build result. Eg `linux/amd64`, `linux/arm/v7`, `windows/amd64`. | ||
* `TARGETOS` - OS component of TARGETPLATFORM | ||
* `TARGETARCH` - architecture component of TARGETPLATFORM | ||
* `TARGETVARIANT` - variant component of TARGETPLATFORM | ||
* `BUILDPLATFORM` - platform of the node performing the build. | ||
* `BUILDOS` - OS component of BUILDPLATFORM | ||
* `BUILDARCH` - OS component of BUILDPLATFORM | ||
* `BUILDVARIANT` - OS component of BUILDPLATFORM | ||
|
||
These arguments are defined in the global scope so are not automatically | ||
available inside build stages or for your `RUN` commands. To expose one of | ||
these arguments inside the build stage redefine it without value. | ||
|
||
For example: | ||
|
||
```Dockerfile | ||
FROM alpine | ||
ARG TARGETPLATFORM | ||
RUN echo "I'm building for $TARGETPLATFORM" | ||
``` | ||
|
||
### Impact on build caching | ||
|
||
`ARG` variables are not persisted into the built image as `ENV` variables are. | ||
|
@@ -1935,6 +2043,14 @@ required such as `zsh`, `csh`, `tcsh` and others. | |
|
||
The `SHELL` feature was added in Docker 1.12. | ||
|
||
## External implementation features | ||
|
||
This feature is only available when using the [BuildKit](#buildkit) backend. | ||
|
||
Docker build supports experimental features like cache mounts, build secrets and | ||
ssh forwarding that are enabled by using an external implementation of the | ||
builder with a syntax directive. To learn about these features, [refer to the documentation in BuildKit repository](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md). | ||
|
||
## Dockerfile examples | ||
|
||
Below you can see some examples of Dockerfile syntax. If you're interested in | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure I understand the "Prioritize". Is this the configurable garbage collection?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is gc and
prune --keep-storage
that removes unused based on priority.