Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack removal: buildpack changes #336

Merged
merged 21 commits into from
Jul 13, 2023
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 81 additions & 41 deletions buildpack.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ This document specifies the interface between a lifecycle program and one or mor
- [Phase #1: Detection](#phase-1-detection)
- [Purpose](#purpose)
- [Process](#process)
- [Mixin Satisfaction](#mixin-satisfaction)
- [Order Resolution](#order-resolution)
- [Phase #2: Analysis](#phase-2-analysis)
- [Purpose](#purpose-1)
Expand All @@ -51,6 +50,7 @@ This document specifies the interface between a lifecycle program and one or mor
- [Process](#process-5)
- [Environment](#environment)
- [Provided by the Lifecycle](#provided-by-the-lifecycle)
- [Targets](#targets)
- [Layer Paths](#layer-paths)
- [Provided by the Platform](#provided-by-the-platform)
- [Provided by the Buildpacks](#provided-by-the-buildpacks)
Expand All @@ -71,10 +71,11 @@ This document specifies the interface between a lifecycle program and one or mor
- [Buildpack Plan (TOML)](#buildpack-plan-toml)
- [Layer Content Metadata (TOML)](#layer-content-metadata-toml)
- [buildpack.toml (TOML)](#buildpacktoml-toml)
- [Stacks](#stacks)
- [Targets](#targets-1)
- [Order](#order)
- [Exec.d Output (TOML)](#execd-output-toml)
- [Deprecations](#deprecations)
- [buildpack.toml (TOML) `stacks` Array](#buildpacktoml-toml-stacks-array)
- [Positional Arguments to `detect` and `build` Executables](#positional-arguments-to-detect-and-build-executables)
- [launch.toml (TOML) `bom` Array](#launchtoml-toml-bom-array)
- [build.toml (TOML) `bom` Array](#buildtoml-toml-bom-array)
Expand Down Expand Up @@ -132,10 +133,6 @@ A **layer directory** is a directory created by a component buildpack that conta
* a **build layer** contains child directories with paths that are added to the environment (e.g., `PATH`, `LD_LIBRARY_PATH`, etc.) for subsequent buildpacks in the same build
Any combination of the three layer types are valid for a particular layer directory.

A **stack** is a contract, implemented by a **build image** and **run image**, that guarantees properties of the **build environment** and **app image**. The provided stack is communicated to component buildpacks through the `CNB_STACK_ID` environment variable, enabling each component buildpack to modify its behavior when executed on different stacks.

A **mixin** is a named set of additions to a stack that can be used to make additive changes to the contract. Buildpacks can express their required mixins in `buildpack.toml`.

## Buildpack Interface

The following specifies the interface implemented by component buildpacks.
Expand Down Expand Up @@ -403,21 +400,8 @@ For each trial,

The lifecycle MAY execute each `/bin/detect` within a group in parallel.

The lifecycle MUST run `/bin/detect` for all extensions and buildpacks in a group in a container using common stack with a common set of mixins.
The lifecycle MUST fail detection if any of the buildpacks does not list that stack in `buildpack.toml`.
The lifecycle MUST fail detection if any of the buildpacks specifies a mixin associated with that stack in `buildpack.toml` that is not satisfied, see [Mixin Satisfaction](#mixin-satisfaction) below.

#### Mixin Satisfaction

A buildpack's mixin requirements must be satisfied by the stack in one of the following scenarios.

1) the stack provides the mixin `run:<mixin>` and the buildpack requires `run:<mixin>`
2) the stack provides the mixin `build:<mixin>` and the buildpack requires `build:<mixin>`
3) the stack provides the mixin `<mixin>` and the buildpack requires `<mixin>`
4) the stack provides the mixin `<mixin>` and the buildpack requires `build:<mixin>`
5) the stack provides the mixin `<mixin>` and the buildpack requires `run:<mixin>`
6) the stack provides the mixin `<mixin>` and the buildpack requires both `run:<mixin>` and `build:<mixin>`
7) the stack provides the mixins `build:<mixin>` and `run:<mixin>` the buildpack requires `<mixin>`
The lifecycle MUST run `/bin/detect` for all extensions and buildpacks in a group in a container using a common build environment.
If any buildpack in a group fails to declare a target in `buildpack.toml` matching the build-time and runtime base images, the lifecycle MUST fail detection for the group. For matching criteria, see [buildpack.toml](#buildpacktoml-toml).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we say that the the buildpack MUST declare the target but later in line 1106 it is a SHOULD.

Is this a required field or a recommended field?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added clarification in 7fd95e8

natalieparellano marked this conversation as resolved.
Show resolved Hide resolved

#### Order Resolution

Expand Down Expand Up @@ -690,7 +674,7 @@ The purpose of export is to create a new OCI image using a combination of remote
- The `<layers>` directories provided to each buildpack during the build phase,
- The `<app>` directory processed by the buildpacks during the build phase,
- The buildpack IDs associated with the buildpacks used during the build phase, in order of execution,
- A reference to the most recent version of the run image associated with the stack and mixins,
- A reference to the most recent version of the run image,
- A reference to the old OCI image processed during the analysis phase, if available, and
- A tag for a new OCI image,

Expand Down Expand Up @@ -763,9 +747,22 @@ The purpose of launch is to modify the running app environment using app-provide

### Provided by the Lifecycle

#### Targets

The following environment variables MUST be set by the lifecycle during the `detect` and `build` phases to describe the target runtime image, if inputs are provided.

| Env Variable | Description |
|-----------------------------|--------------------------------------------|
| `CNB_TARGET_ID` | Identifier for the target image (optional) |
| `CNB_TARGET_OS` | Target OS |
| `CNB_TARGET_ARCH` | Target architecture |
| `CNB_TARGET_ARCH_VARIANT` | Target architecture variant (optional) |
| `CNB_TARGET_DISTRO_NAME` | Target OS distribution name (optional) |
| `CNB_TARGET_DISTRO_VERISON` | Target OS distribution version (optional) |

#### Layer Paths

The following layer path environment variables MUST be set by the lifecycle during the build and launch phases in order to make buildpack dependencies accessible.
The following layer path environment variables MUST be set by the lifecycle during the `build` and `launch` phases in order to make buildpack dependencies accessible.

During the build phase, each variable designated for build MUST contain absolute paths of all previous buildpacks’ `<layers>/<layer>/` directories that are designated for build.

Expand All @@ -791,7 +788,6 @@ The following additional environment variables MUST NOT be overridden by the lif

| Env Variable | Description | Detect | Build | Launch |
|------------------------|---------------------------------------------------|--------|-------|--------|
| `CNB_STACK_ID` | Chosen stack ID | [x] | [x] | |
| `BP_*` | User-provided variable for buildpack | [x] | [x] | |
| `BPL_*` | User-provided variable for exec.d | | | [x] |
| `HOME` | Current user's home directory | [x] | [x] | [x] |
Expand All @@ -805,8 +801,6 @@ When `clear-env` in `buildpack.toml` is not set to `true` for a given buildpack,
1. For layer path environment variables, user-provided values are prepended before any existing values and are delimited by the OS path list separator.
2. For all other environment variables, user-provided values override any existing values.

Buildpacks MAY use the value of `CNB_STACK_ID` to modify their behavior when executed on different stacks.

The environment variable prefix `CNB_` is reserved.
It MUST NOT be used for environment variables that are not defined in this specification or approved extensions.

Expand Down Expand Up @@ -1062,9 +1056,13 @@ id = "<buildpack ID>"
version = "<buildpack version>"
optional = false

[[stacks]]
id = "<stack ID>"
mixins = ["<mixin name>"]
[[targets]]
os = "<OS name>"
arch = "<architecture>"
variant = "<architecture variant>"
[[targets.distributions]]
name = "<OS distribution name>"
version = "<OS distribution version>"

[metadata]
# buildpack-specific data
Expand All @@ -1084,8 +1082,6 @@ Buildpack authors MUST choose a globally unique ID, for example: "io.buildpacks.
- Each element MUST increase numerically.
- Buildpack authors will define what changes will increment `X`, `Y`, and `Z`.

If an `order` is specified, then `stacks` MUST NOT be specified.

**The buildpack API:**

*Key: `api = "<buildpack API version>"`*
Expand All @@ -1105,17 +1101,27 @@ The `[[buildpack.licenses]]` table is optional and MAY contain a list of buildpa
*Key: `sbom-formats = [ "<string>" ]`*
- MUST be supported SBOM media types as described in [Software-Bill-of-Materials](#software-bill-of-materials).

#### Stacks
#### Targets
hone marked this conversation as resolved.
Show resolved Hide resolved

A buildpack descriptor may specify `stacks`.
A buildpack descriptor SHOULD specify `targets`.

Each stack in `stacks` either:
- MUST identify a compatible stack:
- `id` MUST be set to a [valid stack ID](https://github.com/buildpacks/spec/blob/main/platform.md#stack-id).
- `mixins` MAY contain one or more mixin names.
- Or MUST indicate compatibility with any stack:
- `id` MUST be set to the special value `"*"`.
- `mixins` MUST be empty.
Each target in `targets`:
- MUST identify a compatible runtime environment:
- `os` and `arch` are required and MUST be valid identifiers as defined in the [OCI Image Specification](https://github.com/opencontainers/image-spec/blob/main/config.md)
- `variant` is optional and MUST be a valid identifier as defined in the [OCI Image Specification](https://github.com/opencontainers/image-spec/blob/main/config.md)
- `distributions` are optional and MUST describe the OS distributions supported by the buildpack
- For Linux-based images, `distributions.name` and `distributions.versions` should contain the values specified in `/etc/os-release` (`$ID` and `$VERSION_ID`), as the `os.version` field in an image config may contain combined distribution and version information
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
- For Windows-based images, `distributions.name` should be empty; `distributions.versions` should contain the value of `os.version` in the image config (e.g., `10.0.14393.1066`)
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved

If the `targets` list is empty, tools reading `buildpack.toml` will assume:
- `os = "linux"` and `arch = <matches any>` if `./bin/build` is present
- `os = "windows"` and `arch = <matches any>` if `./bin/build.bat` or `./bin/build.exe` are present
- `os = <matches any>` and `arch = <matches any>` for extensions
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved

Metadata specified in `[[targets]]` is validated against the runtime and build-time base images.
* A buildpack target satisfies a base image target when:
* `os` and `arch` match, and
* Optional fields either match exactly, or are left blank by at least one target (blank values match any other value)

#### Order

Expand All @@ -1138,6 +1144,41 @@ Each `key`:
## Deprecations
This section describes all the features that are deprecated.

### buildpack.toml (TOML) `stacks` Array

_Deprecated in Buildpack API 0.10._

The `stacks` array is deprecated.

```toml
[[stacks]]
id = "<stack ID>"
mixins = ["<mixin name>"]
```

Each stack in `stacks` either:
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
- MUST identify a compatible stack:
- `id` MUST be set to a [valid stack ID](https://github.com/buildpacks/spec/blob/main/platform.md#stack-id).
- `mixins` MAY contain one or more mixin names.
- Or MUST indicate compatibility with any stack:
- `id` MUST be set to the special value `"*"`.
- `mixins` MUST be empty.

If an `order` is specified, then `stacks` MUST NOT be specified.

Tools reading `buildpack.toml` will translate any section that sets `stacks.id = "io.buildpacks.stacks.bionic"` to:
natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we end up updating the lifecycle to traslate stack ids to targets based on the information present in the OCI labels/annotations and the files present in the images?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


```toml
[[targets]]
os = "linux"
arch = "amd64"
[[targets.distributions]]
name = "ubuntu"
version = "18.04"
```

natalieparellano marked this conversation as resolved.
Show resolved Hide resolved
Furthermore, any buildpack that contains `[[stacks]]` with `id = "*"` will match any target.

### Positional Arguments to `detect` and `build` Executables

_Deprecated in Buildpack API 0.8._
Expand Down Expand Up @@ -1206,7 +1247,6 @@ When the build is complete, a legacy build BOM describing the build container MA

If generated, this legacy build BOM MUST contain all `bom` entries in each `build.toml` at the end of each `/bin/build` execution, in adherence with the process and data format outlined in the [Platform Interface Specification](platform.md) for legacy BOM formats.


### Build Plan (TOML) `requires.version` Key

_Deprecated in Buildpack API 0.3._
Expand Down
10 changes: 10 additions & 0 deletions image_extension.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,12 +166,22 @@ keywords = [ "<string>" ]
[[extension.licenses]]
type = "<string>"
uri = "<uri>"

[[targets]]
os = "<OS name>"
arch = "<architecture>"
variant = "<architecture variant>"
[[targets.distributions]]
name = "<OS distribution name>"
version = "<OS distribution version>"
```

Image extension authors MUST choose a globally unique ID, for example: "io.buildpacks.apt".

The image extension `id`, `version`, `api`, and `licenses` entries MUST follow the requirements defined in the [Buildpack Interface Specification](buildpack.md).

An extension descriptor MAY specify `targets` following the requirements defined in the [Buildpack Interface Specification](buildpack.md).

### extend-config.toml (TOML)

```toml
Expand Down