-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Remove Stacks #167
RFC: Remove Stacks #167
Changes from all commits
f83859d
7f248d4
9b1d341
edeec28
f53558e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,159 @@ | ||
# Meta | ||
[meta]: #meta | ||
- Name: Remove Stacks | ||
- Start Date: 2021-06-16 | ||
- Author(s): sclevine | ||
- RFC Pull Request: (leave blank) | ||
- CNB Pull Request: (leave blank) | ||
- CNB Issue: (leave blank) | ||
- Supersedes: [RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md), many others | ||
|
||
# Summary | ||
[summary]: #summary | ||
|
||
This RFC proposes that we remove the "stack" and "mixin" concepts from the project and replace them with existing constructs in the container image ecosystem such as base images, Dockerfiles, and OS packages. This RFC also introduces additional functionality for customizing base images, as an alternative to stackpacks. | ||
|
||
# Motivation | ||
[motivation]: #motivation | ||
|
||
The "stack" and "mixin" concepts add unnecessary complexity to the project and make it difficult for new users and contributors to understand how buildpacks work. Compatibility guarantees that are strongly enforced by the stack contract could be replaced with metadata validations and warnings. | ||
|
||
Removing these concepts and relying on Dockerfiles for base image generation and manipulation applies buildpacks to the problem that they solve best: managing application runtimes and dependencies. | ||
|
||
# What it is | ||
[what-it-is]: #what-it-is | ||
|
||
Summary of changes: | ||
- Replace mixins with a CycloneDX-formatted list of packages. | ||
- Replace stackpacks with multi-purpose build-time and run-time Dockerfiles. | ||
- Replace stack metadata (including stack IDs) with canonical OS metadata. | ||
- Allow buildpacks to select a minimal runtime base image during detection. | ||
|
||
# How it Works | ||
[how-it-works]: #how-it-works | ||
|
||
## Base Image Metadata | ||
|
||
Instead of a stack ID, runtime and build-time base images are labeled with the following canonicalized metadata: | ||
- OS (e.g., "linux", `$GOOS`) | ||
- Architecture (e.g., "x86_64", `$GOARCH`) | ||
- Distribution (optional) (e.g., "ubuntu", `$ID`) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are there any cases where There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I only find the combination of |
||
- Version (optional) (e.g., "18.04", `$VERSION_ID`) | ||
|
||
For Linux-based images, each field should be canonicalized against values specified in `/etc/os-release` (`$ID` and `$VERSION_ID`). | ||
|
||
The `stacks` list in `buildpack.toml` is replaced by a `platforms` list, where each entry corresponds to a different buildpack image that is exported into a [manifest index](https://github.com/opencontainers/image-spec/blob/master/image-index.md). Each entry may contain multiple valid values for Distribution and/or Version, but only a single OS and Architecture. Each entry may also contain a list of package names (as PURL URLs without versions or qualifiers) that specify detect-time and build-time (but not runtime) OS package dependencies. Buildpacks may express runtime OS package dependencies during detection (see "Runtime Base Image Selection" below). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
How would this work? How would the exporter know which Distribution/Version combinations are valid?
What's the advantage in removing the ability to declare this statically? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Each entry represents a separate artifact. E.g., x86_64 Linux version of the buildpack is compatible with Ubuntu 18.04 and Ubuntu 16.04.
I didn't see much of an advantage to it, given that potential run images might be selected right before the build. Also, the static list in each platform in buildpack.toml could be used to specify build-time packages, which do need static validation (when creating a builder). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we need an example of what this There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let me make sure I understand the point correctly -- say the buildpack is something related to node.js, so it would need the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
How such installation will then take place, using which lib/tool ? Will the container running the lifecycle creator be able to execute a command to install a rpm which requires to be root ... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Not exactly -- this list would only be used before the build to ensure that the build-time base image contains OS packages that are required by the buildpack at build-time. E.g., it could be used to assert that curl is available at build time. If a platform wants to create a base image dynamically based on buildpack.toml contents, that sounds interesting, but tooling to do that is out-of-scope for this RFC. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Example platform table in: #172 |
||
|
||
App image builds fail if the build image and selected run image have mismatched metadata. We may consider introducing a flag to skip this validation. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am assuming "mismatched metadata" only applied to things like Architecture and Distribution, given we typically expect mismatched packages? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am also curious about what happens to stacks like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Happy to strike this requirement for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated the wording in #172 so that tiny can leave off the distro/version to be compatible with all distros/versions. Also mentioned that flags/labels could be used to skip validation in the future. |
||
|
||
When an app image is rebased, `pack rebase` will fail if the new run image and previous run image have mismatched metadata. This check may be skipped for Distribution and Version by passing a new `--force` flag to `pack rebase`. | ||
|
||
## Mixins | ||
|
||
The mixins label on each base image is replaced by a layer in each base image containing a single file consisting of a CycloneDX-formatted list of packages. Each package entry has a [PURL](https://github.com/package-url/purl-spec)-formatted ID that uniquely identifies the package. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this for run images too? How would the lifecycle get this information for selecting the run image? Think it's here:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, the label is supposed to be a reference to the layer that contains the SBoM. I should make that more clear. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am not 100% sold on combining the stack SBoM and packages list. I know the SBoM by definition does include the packages but it may also include a lot of other informaion (provenance, licenses) and it could be useful to pull out the piece that is required for validation into a more easily consumable format (and one that is less likely to change if for example we switch from cycloneDX to SPDX for the BOM) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Why should we implement logic to transform the data into a different format and put both formats on the image? We could just use one (ideally standardized, non-CNB-specific) format, and transform it when we need to validate.
If we commit to a format and change it, we're going to have to update the lifecycle to parse it regardless. |
||
|
||
### Validations | ||
|
||
Buildpack base image metadata and packages specified in `buildpack.toml`'s `platforms` list are validated against the runtime and build-time base images. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would it be possible to use YAML or JSON to declare such METADATA instead of the TOML format ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we'd need to introduce a separate file, since all buildpack configuration is currently in buildpack.toml. Alternatively, we could permit an alternate format version of the buildpack.toml file (e.g., buildpack.yml). Why do you prefer YAML or JSON? |
||
|
||
Runtime and build-time base image packages are no longer validated against each other. | ||
|
||
When an app image is rebased, `pack rebase` will fail if packages are removed from the new runtime base image. This check may be skipped by passing a new `--force` flag to `pack rebase`. | ||
|
||
## Runtime Base Image Selection | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @ekcasey Re: your comment on runtime base image selection and app-specified Dockerfiles not playing well together (i.e., app-specified Dockerfiles can't fulfill package requirements from buildpacks): what if we allow users to label the Dockerfiles with packages names (in version-less PURL format) that could be matched against (and thus remove) buildpacks-required packages? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (Note that the "label" would be something like a comment at the top of the Dockerfile, not an image label.) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Hmm, this fills the required purpose but it seems like it moving away from the simplicity of "just a Dockerfile" towards something that more closely resembled a list of "provided mixins"? I need to chew on this a little more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree, feels bad to me also |
||
|
||
Builders may specify an ordered list of runtime base images, where each entry may contain a list of runtime base image mirrors. | ||
|
||
Buildpacks may specify a list of package names (as PURL URLs without versions or qualifiers) in a `packages` table in the build plan. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @ekcasey some discussion on PURL version ranges: package-url/purl-spec#93 Some of the comments in those threads suggests that PURL could be used for querying, with qualifiers used for matching against ranges. Is it worth re-considering whether we move the build plan to PURL as well? Then we could represent the |
||
|
||
The first runtime base image that contains all required packages is selected. When mirrors are present, the runtime base image mirror matching the app image is always used, including for package queries. | ||
|
||
## Dockerfiles | ||
|
||
Note: kaniko, BuildKit, and/or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does "at the platform's discretion" mean that a platform can provide whatever mechanism it wants for buildpack users to select/provide There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The statement was only intended to suggest that the underlying technology for applying Dockerfiles is up to the platform. E.g., BuildKit if you're using the BuildKit frontent, kaniko if you're using kpack or tekton, etc. |
||
|
||
### App-specified Dockerfiles | ||
|
||
A buildpack app may have a build.Dockerfile and/or run.Dockerfile in its app directory. A run.Dockerfile is applied to the selected runtime base image after the detection phase. A build.Dockerfile is applied to the build-time base image before the detection phase. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this something we're specing, or is this a platform detail of Pack? I'd like to see these build/run Dockerfiles defined in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like @jabrown85's idea of putting all Dockerfile-related functionality into an extension spec. Do you mean the locations may be overriden in project.toml, or are you thinking inline? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was thinking of overriding the location, but now I'm interested in inline too |
||
|
||
Both Dockerfiles must accept `base_image` and `build_id` args. The `base_image` arg allows the lifecycle to specify the original base image. The `build_id` arg allows the app developer to bust the cache after a certain layer and must be defaulted to `0`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What exactly does lifecycle pass into There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The value of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looking at
and also
I wonder if maybe it's worth clarifying further how this would work. I'm assuming for the build image, the lifecycle could use kaniko during the existing build phase. But extending the run image would imply a new phase... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Could you describe a bit further how this would work? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See the examples below -- if you use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This is what I'm thinking as well. For pack, this phase could happen in parallel with the builder phase. Happy to add more detail. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Kaniko will imply to run the build process in a docker container or kubernetes pod. This is not needed using Google JIB - https://github.com/GoogleContainerTools/jib or buildah - https://github.com/containers/buildah There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These tools all use different approaches:
For in-cluster builds, kaniko's approach is least-privileged. For local builds, Docker/BuildKit (or buildah on Linux) all seem like good options. Happy to remove or extend the list of suggested technologies. |
||
|
||
A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of app-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before `run.Dockerfile` is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. | ||
|
||
### Platform-specified Dockerfiles | ||
|
||
The same Dockerfiles may be used to create new stacks or modify existing stacks outside of the app build process. For both app-specified and stack-modifying Dockerfiles, any specified labels override existing values. | ||
|
||
Dockerfiles that are used to create a stack must create a `/cnb/stack/genpkgs` executable that outputs a CycloneDX-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after any run.Dockerfile or build.Dockerfile is applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How would the validity of this binary be assured? Is this binary published by the project and included (by the platform?) in Dockerfile-extended stacks? Is it the responsibility of the Dockerfile writer to create (or validate) their own binary? I worry about a malicious binary that produces false information about the packages contained in a build/run image. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The binary would be root-owned and added by the Dockerfiles that created the build-time and runtime base images. The distribution-specific logic in the binary could be implemented for common distros by the CNB project. Given that all Dockerfiles can run as root, they must all be fully-trusted. If an untrusted Dockerfile is allowed to run, it could cause the binary to produce false information without touching the binary itself (e.g., via There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The approach proposed here which I suppose will rely on There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. From what I understand, this might not be necessary as the buildpacks lifecycle could use kaniko to execute the Dockerfile in the context of the build or run image (see #167 (comment) ). @sclevine is this correct? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct, kaniko can be used for in-cluster builds. The lifecycle already has build-time phases that require in-container root. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will the executable |
||
|
||
### Examples | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it possible to create, part of a github repo a more concrete example staring from an existing Buildpack and stacks and how it could be converted into Buildpacks DockerStacks tree of files ? |
||
|
||
run.Dockerfile used to create a runtime base image: | ||
|
||
``` | ||
ARG base_image | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this supposed to be a plain There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, but that would work also. The idea is to use the same format for all stack-related Dockerfiles (creating, pre-build extension, at-build extension). |
||
FROM ${base_image} | ||
ARG build_id=0 | ||
|
||
LABEL io.buildpacks.image.distro=ubuntu | ||
LABEL io.buildpacks.image.version=18.04 | ||
Comment on lines
+98
to
+99
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These could be derived automatically from There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Who would be responsible for adding this label? Would the lifecycle add it to the exported image? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
LABEL io.buildpacks.rebasable=true | ||
|
||
ENV CNB_USER_ID=1234 | ||
ENV CNB_GROUP_ID=1235 | ||
|
||
RUN groupadd cnb --gid ${CNB_GROUP_ID} && \ | ||
useradd --uid ${CNB_USER_ID} --gid ${CNB_GROUP_ID} -m -s /bin/bash cnb | ||
|
||
USER ${CNB_USER_ID}:${CNB_GROUP_ID} | ||
|
||
COPY genpkgs /cnb/stack/genpkgs | ||
``` | ||
|
||
run.Dockerfile present in an app directory that always installs the latest version of curl: | ||
``` | ||
ARG base_image | ||
FROM ${base_image} | ||
ARG build_id=0 | ||
|
||
LABEL io.buildpacks.rebasable=true | ||
|
||
RUN echo ${build_id} | ||
|
||
RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* | ||
``` | ||
|
||
Unsafe run.Dockerfile present in an app directory: | ||
``` | ||
ARG base_image | ||
FROM ${base_image} | ||
ARG build_id=0 | ||
|
||
LABEL io.buildpacks.rebasable=false | ||
|
||
RUN curl -L https://example.com/mypkg-install | sh # installs to /opt | ||
``` | ||
|
||
# Drawbacks | ||
[drawbacks]: #drawbacks | ||
|
||
- Involves breaking changes. | ||
- Buildpacks cannot install OS packages directly, only select runtime base images. | ||
|
||
# Alternatives | ||
[alternatives]: #alternatives | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think there's also a variant of stackpacks where we aren't so strict as the current RFC. All the complexity came in when we tried to put guardrails around them and ensure rebase always worked. The There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will add this as an alternative. I think our mistake is larger than trying to preserve rebase though. As mentioned in #167 (comment), I think stackpacks leave us on the hook to solve a hard problem. Even if we don't break rebase, how are we going to ensure that packages stay up-to-date? I'd rather implement the existing solution (and associated set of user expectations) first. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure, I agree the problem is larger (rebase is just an example here). My original vision for Stackpacks was something dead simple: a type of buildpack that runs as root. I don't think that's much different than a That said, I think the original very simple Stackpacks concept could co-exist with the Dockerfile mechanism. |
||
|
||
- Stackpacks | ||
- Keep stacks & mixins, but implement "Dockerfiles" | ||
- Ditch stacks & mixins, but skip "Dockerfiles" | ||
|
||
# Unresolved Questions | ||
[unresolved-questions]: #unresolved-questions | ||
|
||
- Should we use the build plan to allows buildpacks to specify package requirements? This allows, e.g., a requirement for "python" to be satisfied by either a larger runtime base image or by a buildpack. Opinion: no, too complex and difficult to match package names and plan entry names, e.g., python2.7 vs. python2 vs. python. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. agree. this is where stackpacks got messy. I don't think stackpacks themselves where the problem, but rather all the stuff like this that we tacked on. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IIUC the current proposal doesn't shut the door to doing something like that in the future, right? If so maybe we could revisit this question when we find that we need it. |
||
- Should packages be determined during the detect or build phase? Opinion: detect phase, so that a runtime base image's app-specified Dockerfiles may by applied in parallel to the buildpack build process. | ||
|
||
# Spec. Changes (OPTIONAL) | ||
[spec-changes]: #spec-changes | ||
|
||
This RFC requires extensive changes to all specifications. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to add the link to the project you refer here. I suppose that it corresponds to : https://cyclonedx.org/ ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's correct, will add a link in the smaller RFCs that will replace this one.