From ea1d26f2ff647e8ab9c1381b2df2c37d9537f5c6 Mon Sep 17 00:00:00 2001 From: Iain Sproat Date: Thu, 12 Dec 2019 14:04:42 +0000 Subject: [PATCH 01/38] Include buildpack RFC0013 in specification - Add app source metadata format - [RFC0013](https://github.com/buildpack/rfcs/blob/master/text/0013-app-layer-metadata-source.md) [#170205764] Signed-off-by: Shane Huston Signed-off-by: Velda Conaty Signed-off-by: Emily Casey --- buildpack.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/buildpack.md b/buildpack.md index 11e43ab1..abeb662b 100644 --- a/buildpack.md +++ b/buildpack.md @@ -481,6 +481,8 @@ Subsequently, - The run image filesystem layers, - The executable component of the lifecycle that implements the launch phase, and - An `ENTRYPOINT` set to that component. +3. The lifecycle MAY add Config labels to the new OCI image, composed of + - A label with the key `"io.buildpacks.app.source"` with a [value describing the source location of the app](#io.buildpacks.app.source-oci-image-label). Finally, any `/` directories specified as `cache = true` in `/.toml` MAY be preserved for the next local build. For any `/.toml` files specifying both `cache = true` and `launch = true`, the lifecycle SHOULD store a checksum of the corresponding `/` directory so that it is associated with the locally cached directory. @@ -859,3 +861,19 @@ The stack ID: A buildpack descriptor that specifies `order` MUST be [resolvable](#order-resolution) into an ordering of buildpacks that implement the [Buildpack Interface](#buildpack-interface). A buildpack reference inside of a `group` MUST contain an `id` and `version`. + +### io.buildpacks.app.source OCI Image label +The value of this label: +- MUST be a string of escaped json complying with [RFC 8259](https://tools.ietf.org/html/rfc8259). +- when unescaped, MUST comply with the following schema: +```json +{ + "$schema": "http://json-schema.org/schema#", + "type": "object", + "properties": { + "type": {"type": "string"}, + "version": {}, + "metadata": {} + } +} +``` \ No newline at end of file From 330cc3fc2c3eb0190ae6b9c095079cb9ec285d5b Mon Sep 17 00:00:00 2001 From: Shane Huston Date: Thu, 12 Dec 2019 16:34:20 +0000 Subject: [PATCH 02/38] Revert "Include buildpack RFC0013 in specification" * Changes will be moved to the platform specification document This reverts commit 3dc1b445aa5d62081050b2ac48352287c3ef0cd9. Signed-off-by: Velda Conaty Signed-off-by: Emily Casey --- buildpack.md | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/buildpack.md b/buildpack.md index abeb662b..11e43ab1 100644 --- a/buildpack.md +++ b/buildpack.md @@ -481,8 +481,6 @@ Subsequently, - The run image filesystem layers, - The executable component of the lifecycle that implements the launch phase, and - An `ENTRYPOINT` set to that component. -3. The lifecycle MAY add Config labels to the new OCI image, composed of - - A label with the key `"io.buildpacks.app.source"` with a [value describing the source location of the app](#io.buildpacks.app.source-oci-image-label). Finally, any `/` directories specified as `cache = true` in `/.toml` MAY be preserved for the next local build. For any `/.toml` files specifying both `cache = true` and `launch = true`, the lifecycle SHOULD store a checksum of the corresponding `/` directory so that it is associated with the locally cached directory. @@ -861,19 +859,3 @@ The stack ID: A buildpack descriptor that specifies `order` MUST be [resolvable](#order-resolution) into an ordering of buildpacks that implement the [Buildpack Interface](#buildpack-interface). A buildpack reference inside of a `group` MUST contain an `id` and `version`. - -### io.buildpacks.app.source OCI Image label -The value of this label: -- MUST be a string of escaped json complying with [RFC 8259](https://tools.ietf.org/html/rfc8259). -- when unescaped, MUST comply with the following schema: -```json -{ - "$schema": "http://json-schema.org/schema#", - "type": "object", - "properties": { - "type": {"type": "string"}, - "version": {}, - "metadata": {} - } -} -``` \ No newline at end of file From abc48a4007198ebeb09c2fd77f323a2d5e350798 Mon Sep 17 00:00:00 2001 From: Iain Sproat Date: Thu, 12 Dec 2019 16:36:44 +0000 Subject: [PATCH 03/38] RFC0013 moved to platform specification * Moves the specification from Buildpack document to Platform document * Adds a new optional `-platform` flag to the lifecycle builder * Adds a new project.toml input file * Includes source, source.version, and source.metadata properties to project.toml * Provides a new section to specify the Config labels to be added to the OCI image * Includes a data type for the "io.buildpacks.project" Config label. [#170205764] Signed-off-by: Velda Conaty Signed-off-by: Emily Casey --- platform.md | 48 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/platform.md b/platform.md index 785aec01..e8bf9a61 100644 --- a/platform.md +++ b/platform.md @@ -81,7 +81,8 @@ The platform MUST ensure that: - The image config's `Env` field has the environment variable `CNB_GROUP_ID` set to the primary GID of the user specified in the `User` field. - The image config's `Label` field has the label `io.buildpacks.stack.id` set to the stack ID. - The image config's `Label` field has the label `io.buildpacks.stack.mixins` set to a JSON array containing mixin names for each mixin applied to the image. - + + #### Detection phase To initiate the detection phase, the platform MUST invoke the `/cnb/lifecycle/detector` executable with the user and environment defined in the build image config. Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. @@ -96,11 +97,12 @@ Where: - `-group` MUST specify output to a `group.toml` file path as defined in the [Data Format](#data-format) section. - `-plan` MUST specify output to a Build Plan as defined in the [Buildpack Interface Specification](buildpack.md). +#### Build phase To initiate the build phase, the platform MUST invoke the `/cnb/lifecycle/builder` executable with the user and environment defined in the build image config. Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. ```bash -/cnb/lifecycle/builder -buildpacks /cnb/by-id -group ./group.toml -plan ./plan.toml +/cnb/lifecycle/builder -buildpacks /cnb/by-id -group ./group.toml -plan ./plan.toml -project ./project.toml ``` Where: @@ -108,6 +110,7 @@ Where: - `-buildpacks` MUST specify input from a buildpacks directory as defined in the [Buildpacks Directory Layout](#buildpacks-directory-layout) section. - `-group` MUST specify input from a `group.toml` file path as defined in the [Data Format](#data-format) section. - `-plan` MUST specify input from a Build Plan as defined in the [Buildpack Interface Specification](buildpack.md). +- `-project` MAY specify input from a `project.toml` file path as defined in the [Data Format](#data-format) section. ### Run Image @@ -142,6 +145,13 @@ The buildpacks directory MUST contain unarchived buildpacks such that: - Each top-level directory is a buildpack ID. - Each second-level directory is a buildpack version. +## OCI Image +The OCI image containing the compiled application is generated by the Build Image during the [Build Phase](#build-phase). + +If the `/cnb/lifecycle/builder` is invoked with the optional `-project` argument, the lifecycle MUST add a `"io.buildpacks.project"` Config label to the new OCI image with a [value as described in the data-format](#io.buildpacks.app.source-oci-image-label) section. + + + ## Security Considerations The platform SHOULD run each phase of the lifecycle in an isolated container to prevent untrusted app and buildpack code from accessing storage credentials needed during the export and analysis phases. @@ -199,3 +209,37 @@ group = [ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. + +### project.toml (TOML) + +```toml +[source] +type = "" + +[source.version] + +[source.metadata] + +``` + +### io.buildpacks.project OCI Image label +The value of this label: +- MUST be a string of escaped json complying with [RFC 8259](https://tools.ietf.org/html/rfc8259). +- when unescaped, MUST comply with the following schema: + +```json +{ + "$schema": "http://json-schema.org/schema#", + "type": "object", + "properties": { + "source: { + "type": "object", + "properties":{ + "type": {"type": "string"}, + "version": {}, + "metadata": {} + } + } + } +} +``` \ No newline at end of file From 46933c24396edbe7b0975504da30e9f279874c19 Mon Sep 17 00:00:00 2001 From: Iain Sproat Date: Thu, 12 Dec 2019 16:41:28 +0000 Subject: [PATCH 04/38] Corrects error in json schema for io.buildpacks.project data type [#170205764] Signed-off-by: Velda Conaty Signed-off-by: Emily Casey --- platform.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform.md b/platform.md index e8bf9a61..5e9776bb 100644 --- a/platform.md +++ b/platform.md @@ -232,7 +232,7 @@ The value of this label: "$schema": "http://json-schema.org/schema#", "type": "object", "properties": { - "source: { + "source": { "type": "object", "properties":{ "type": {"type": "string"}, From abadb50e5cc0eae2a0e1f68590966c398abce5ab Mon Sep 17 00:00:00 2001 From: Iain Sproat Date: Fri, 13 Dec 2019 10:37:15 +0000 Subject: [PATCH 05/38] Moved flag from Build phase to Export phase * renamed flag to source-metadata * created an Export Phase section * removed redundant quotation marks from label key * renamed TOML file to source-metadata.toml [#170205764] Signed-off-by: Velda Conaty Signed-off-by: Emily Casey --- platform.md | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/platform.md b/platform.md index 5e9776bb..eac6762a 100644 --- a/platform.md +++ b/platform.md @@ -102,7 +102,7 @@ To initiate the build phase, the platform MUST invoke the `/cnb/lifecycle/builde Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. ```bash -/cnb/lifecycle/builder -buildpacks /cnb/by-id -group ./group.toml -plan ./plan.toml -project ./project.toml +/cnb/lifecycle/builder -buildpacks /cnb/by-id -group ./group.toml -plan ./plan.toml ``` Where: @@ -110,7 +110,18 @@ Where: - `-buildpacks` MUST specify input from a buildpacks directory as defined in the [Buildpacks Directory Layout](#buildpacks-directory-layout) section. - `-group` MUST specify input from a `group.toml` file path as defined in the [Data Format](#data-format) section. - `-plan` MUST specify input from a Build Plan as defined in the [Buildpack Interface Specification](buildpack.md). -- `-project` MAY specify input from a `project.toml` file path as defined in the [Data Format](#data-format) section. + +#### Export phase +To initiate the export phase, the platform MUST invoke the `/cnb/lifecycle/exporter` executable with the user and environment defined in the build image config. +Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. + +```bash +/cnb/lifecycle/exporter -source-metadata ./source-metadata.toml +``` + +Where: + +- `-source-metadata` MAY specify input from a `source-metadata.toml` file path as defined in the [Data Format](#data-format) section. ### Run Image @@ -146,11 +157,9 @@ The buildpacks directory MUST contain unarchived buildpacks such that: - Each second-level directory is a buildpack version. ## OCI Image -The OCI image containing the compiled application is generated by the Build Image during the [Build Phase](#build-phase). - -If the `/cnb/lifecycle/builder` is invoked with the optional `-project` argument, the lifecycle MUST add a `"io.buildpacks.project"` Config label to the new OCI image with a [value as described in the data-format](#io.buildpacks.app.source-oci-image-label) section. - +The OCI image containing the compiled application is generated by the Build Image during the [Export Phase](#export-phase). +If the `/cnb/lifecycle/exporter` is invoked with the optional `-source-metadata` argument, the lifecycle MUST add a `io.buildpacks.project` image config label to the new OCI image with a [value as described in the data-format](#io.buildpacks.app.source-oci-image-label-json) section. ## Security Considerations @@ -210,7 +219,7 @@ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. -### project.toml (TOML) +### source-metadata.toml (TOML) ```toml [source] @@ -222,7 +231,7 @@ type = "" ``` -### io.buildpacks.project OCI Image label +### io.buildpacks.project OCI Image label (JSON) The value of this label: - MUST be a string of escaped json complying with [RFC 8259](https://tools.ietf.org/html/rfc8259). - when unescaped, MUST comply with the following schema: From bc4f59b98004b5e7a7912cae3c65dac3e6dc3b64 Mon Sep 17 00:00:00 2001 From: Iain Sproat Date: Tue, 17 Dec 2019 15:55:42 +0000 Subject: [PATCH 06/38] RFC0013: renames `source-metadata` to `project-metadata` * follows comments received on PR ** remove whitespace ** add section to Table of Contents ** Renames `source-metadata` to `project-metadata` [#170205764] Signed-off-by: Carlo Colombo Signed-off-by: Velda Conaty Signed-off-by: Emily Casey --- platform.md | 33 ++++++--------------------------- 1 file changed, 6 insertions(+), 27 deletions(-) diff --git a/platform.md b/platform.md index eac6762a..7f8cd5d8 100644 --- a/platform.md +++ b/platform.md @@ -28,6 +28,7 @@ Examples of a platform might include: 6. [Data Format](#data-format) 1. [order.toml (TOML)](#order.toml-(toml)) 2. [group.toml (TOML)](#group.toml-(toml)) + 3. [project-metadata.toml (TOML)](#project-metadata.toml-(toml)) ## Platform API Version @@ -82,7 +83,7 @@ The platform MUST ensure that: - The image config's `Label` field has the label `io.buildpacks.stack.id` set to the stack ID. - The image config's `Label` field has the label `io.buildpacks.stack.mixins` set to a JSON array containing mixin names for each mixin applied to the image. - #### Detection phase +#### Detection phase To initiate the detection phase, the platform MUST invoke the `/cnb/lifecycle/detector` executable with the user and environment defined in the build image config. Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. @@ -116,12 +117,12 @@ To initiate the export phase, the platform MUST invoke the `/cnb/lifecycle/expor Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. ```bash -/cnb/lifecycle/exporter -source-metadata ./source-metadata.toml +/cnb/lifecycle/exporter -project-metadata ./project-metadata.toml ``` Where: -- `-source-metadata` MAY specify input from a `source-metadata.toml` file path as defined in the [Data Format](#data-format) section. +- `-project-metadata` MAY specify input from a `project-metadata.toml` file path as defined in the [Data Format](#data-format) section. ### Run Image @@ -159,7 +160,7 @@ The buildpacks directory MUST contain unarchived buildpacks such that: ## OCI Image The OCI image containing the compiled application is generated by the Build Image during the [Export Phase](#export-phase). -If the `/cnb/lifecycle/exporter` is invoked with the optional `-source-metadata` argument, the lifecycle MUST add a `io.buildpacks.project` image config label to the new OCI image with a [value as described in the data-format](#io.buildpacks.app.source-oci-image-label-json) section. +If the `/cnb/lifecycle/exporter` is invoked with the optional `-project-metadata` argument, the lifecycle MUST add a `io.buildpacks.project` image config label to the new OCI image. This label value is encoded JSON format of the `project-metadata.toml` contents, as described in the [data-format](#data-format) section. ## Security Considerations @@ -219,7 +220,7 @@ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. -### source-metadata.toml (TOML) +### project-metadata.toml (TOML) ```toml [source] @@ -229,26 +230,4 @@ type = "" [source.metadata] -``` - -### io.buildpacks.project OCI Image label (JSON) -The value of this label: -- MUST be a string of escaped json complying with [RFC 8259](https://tools.ietf.org/html/rfc8259). -- when unescaped, MUST comply with the following schema: - -```json -{ - "$schema": "http://json-schema.org/schema#", - "type": "object", - "properties": { - "source": { - "type": "object", - "properties":{ - "type": {"type": "string"}, - "version": {}, - "metadata": {} - } - } - } -} ``` \ No newline at end of file From d402d9d602311a6fc48d61978573936d7475a777 Mon Sep 17 00:00:00 2001 From: Iain Sproat Date: Thu, 23 Jan 2020 11:20:23 +0000 Subject: [PATCH 07/38] Removes project-metadata flag - Renames OCI image to Exported image [#170205764] Signed-off-by: Velda Conaty Signed-off-by: Emily Casey --- platform.md | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/platform.md b/platform.md index 7f8cd5d8..66e84063 100644 --- a/platform.md +++ b/platform.md @@ -114,15 +114,9 @@ Where: #### Export phase To initiate the export phase, the platform MUST invoke the `/cnb/lifecycle/exporter` executable with the user and environment defined in the build image config. -Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. - -```bash -/cnb/lifecycle/exporter -project-metadata ./project-metadata.toml -``` - -Where: -- `-project-metadata` MAY specify input from a `project-metadata.toml` file path as defined in the [Data Format](#data-format) section. +The platform MAY provide a [`project-metadata.toml`](#project-metadata.toml-(toml)) file in the platform directory. +The platform MAY provide a `-platform` flag or `CNB_PLATFORM_DIR` environment variable to specify the path of the platform directory. If not provided, the default is `/platform`. ### Run Image @@ -157,10 +151,10 @@ The buildpacks directory MUST contain unarchived buildpacks such that: - Each top-level directory is a buildpack ID. - Each second-level directory is a buildpack version. -## OCI Image -The OCI image containing the compiled application is generated by the Build Image during the [Export Phase](#export-phase). +## Exported Image +The Exported image containing the compiled application is generated by the Build Image during the [Export Phase](#export-phase). -If the `/cnb/lifecycle/exporter` is invoked with the optional `-project-metadata` argument, the lifecycle MUST add a `io.buildpacks.project` image config label to the new OCI image. This label value is encoded JSON format of the `project-metadata.toml` contents, as described in the [data-format](#data-format) section. +If the `/cnb/lifecycle/exporter` is provided with a `project-metadata.toml` file in the Platform directory, the lifecycle MUST add a `io.buildpacks.project` image config label to the Exported image. This label value is encoded JSON format of the `project-metadata.toml` contents, as described in the [data-format](#data-format) section. ## Security Considerations From d8469207f67e0be034615e21ff1a2fad7d73b6d8 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 13 May 2020 13:47:08 -0400 Subject: [PATCH 08/38] Specifies -project-metadata exporter flag * Derived from PR #68 * Paths and label names have been changes to match the concensus arrived at during implementation - buildpacks/lifecycle#230 - buildpacks/lifecycle#257 Signed-off-by: Emily Casey --- platform.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/platform.md b/platform.md index 66e84063..6a3d4b04 100644 --- a/platform.md +++ b/platform.md @@ -116,7 +116,7 @@ Where: To initiate the export phase, the platform MUST invoke the `/cnb/lifecycle/exporter` executable with the user and environment defined in the build image config. The platform MAY provide a [`project-metadata.toml`](#project-metadata.toml-(toml)) file in the platform directory. -The platform MAY provide a `-platform` flag or `CNB_PLATFORM_DIR` environment variable to specify the path of the platform directory. If not provided, the default is `/platform`. +The platform MAY provide a `-platform-project` flag or `CNB_PROJECT_METADATA_PATH` environment variable to specify the path to this file. If not provided, the default path is `./project-metadata.toml`. ### Run Image @@ -154,7 +154,7 @@ The buildpacks directory MUST contain unarchived buildpacks such that: ## Exported Image The Exported image containing the compiled application is generated by the Build Image during the [Export Phase](#export-phase). -If the `/cnb/lifecycle/exporter` is provided with a `project-metadata.toml` file in the Platform directory, the lifecycle MUST add a `io.buildpacks.project` image config label to the Exported image. This label value is encoded JSON format of the `project-metadata.toml` contents, as described in the [data-format](#data-format) section. +If the `/cnb/lifecycle/exporter` is provided with a `project-metadata.toml` file, the lifecycle MUST add a `io.buildpacks.project.metdata` image config label to the Exported image. This label value MUST be the encoded JSON format of the `project-metadata.toml` contents, as described in the [data-format](#data-format) section. ## Security Considerations From 06e9224d9517819029edf8e0e685189faf1286b5 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Tue, 2 Jun 2020 11:11:10 -0400 Subject: [PATCH 09/38] Describes keys in project-metadata.toml schema Signed-off-by: Emily Casey --- platform.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/platform.md b/platform.md index 6a3d4b04..e3e99d2e 100644 --- a/platform.md +++ b/platform.md @@ -224,4 +224,9 @@ type = "" [source.metadata] -``` \ No newline at end of file +``` +Where: +- All values are optional +- `type`, if present, SHOULD contain the type of location where the provided app source is stored (e.g `git`, `s3`) +- `version`, if present, SHOULD contain data uniquely identifying the particular version of the provided source +- `metadata` MAY contain additional arbitrary data about the provided source \ No newline at end of file From c0e423f5c83935937e19e65e492fe7cc61aa9d5a Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 10 Jun 2020 13:44:11 -0400 Subject: [PATCH 10/38] Adds comments for arbitrary data Signed-off-by: Emily Casey --- platform.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/platform.md b/platform.md index e3e99d2e..1c902652 100644 --- a/platform.md +++ b/platform.md @@ -221,8 +221,10 @@ Where: type = "" [source.version] +# arbitrary data describing version [source.metadata] +# arbitrary data ``` Where: From 9801c99cd7f63b70f437d71575e3989263b58838 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Mon, 15 Jun 2020 09:09:47 -0700 Subject: [PATCH 11/38] Bump Platform API to 0.3 Signed-off-by: Terence Lee --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b04d99d7..f30247a5 100644 --- a/README.md +++ b/README.md @@ -30,4 +30,4 @@ An implementation is compliant if it satisfies all the MUST, MUST NOT, REQUIRED, These documents currently specify: #### Buildpack API: 0.2 -#### Platform API: 0.2 +#### Platform API: 0.3 From 50ce7e9dab0f89e484095c45e87abac79b337012 Mon Sep 17 00:00:00 2001 From: Ben Hale Date: Wed, 17 Jun 2020 14:10:42 -0700 Subject: [PATCH 12/38] TOC Update Signed-off-by: Ben Hale --- platform.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/platform.md b/platform.md index 48ef7af2..faf166cc 100644 --- a/platform.md +++ b/platform.md @@ -19,10 +19,14 @@ Examples of a platform might include: - [Stacks](#stacks) - [Compatibility Guarantees](#compatibility-guarantees) - [Build Image](#build-image) + - [Detection phase](#detection-phase) + - [Build phase](#build-phase) + - [Export phase](#export-phase) - [Run Image](#run-image) - [Mixins](#mixins) - [Buildpacks](#buildpacks) - [Buildpacks Directory Layout](#buildpacks-directory-layout) + - [Exported Image](#exported-image) - [Security Considerations](#security-considerations) - [Additional Guidance](#additional-guidance) - [Environment](#environment) @@ -31,6 +35,7 @@ Examples of a platform might include: - [Data Format](#data-format) - [order.toml (TOML)](#ordertoml-toml) - [group.toml (TOML)](#grouptoml-toml) + - [project-metadata.toml (TOML)](#project-metadatatoml-toml) ## Platform API Version From 4926cbeadee228ffdc07f103286233bc19f40464 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Fri, 31 Jan 2020 14:23:18 -0500 Subject: [PATCH 13/38] use markdown number magic Signed-off-by: Emily Casey --- platform.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/platform.md b/platform.md index 785aec01..084f4296 100644 --- a/platform.md +++ b/platform.md @@ -7,27 +7,27 @@ A platform orchestrates a lifecycle to make buildpack functionality available to Examples of a platform might include: 1. A local CLI tool that uses buildpacks to create OCI images -2. A plugin for a continuous integration service that uses buildpacks to create OCI images -3. A cloud application platform that uses buildpacks to build source code before deployment +1. A plugin for a continuous integration service that uses buildpacks to create OCI images +1. A cloud application platform that uses buildpacks to build source code before deployment ## Table of Contents 1. [Platform API Version](#platform-api-version) 1. [Compatibility Verification](#compatibility-verification) -2. [Stacks](#stacks) +1. [Stacks](#stacks) 1. [Compatibility Guarantees](#compatibility-guarantees) - 2. [Build Image](#build-image) - 3. [Run Image](#run-image) -3. [Buildpacks](#buildpacks) + 1. [Build Image](#build-image) + 1. [Run Image](#run-image) +1. [Buildpacks](#buildpacks) 1. [Buildpacks Directory Layout](#buildpacks-directory-layout) -4. [Security Considerations](#security-considerations) -5. [Additional Guidance](#additional-guidance) +1. [Security Considerations](#security-considerations) +1. [Additional Guidance](#additional-guidance) 1. [Environment](#environment) - 2. [Run Image Rebasing](#run-image-rebasing) - 3. [Caching](#caching) -6. [Data Format](#data-format) + 1. [Run Image Rebasing](#run-image-rebasing) + 1. [Caching](#caching) +1. [Data Format](#data-format) 1. [order.toml (TOML)](#order.toml-(toml)) - 2. [group.toml (TOML)](#group.toml-(toml)) + 1. [group.toml (TOML)](#group.toml-(toml)) ## Platform API Version From b031510601805bb8a296c22b787413b335108514 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Fri, 31 Jan 2020 14:26:13 -0500 Subject: [PATCH 14/38] Adds analyzed.toml and stack.toml to data formats Signed-off-by: Emily Casey --- platform.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/platform.md b/platform.md index 084f4296..438c188a 100644 --- a/platform.md +++ b/platform.md @@ -28,6 +28,8 @@ Examples of a platform might include: 1. [Data Format](#data-format) 1. [order.toml (TOML)](#order.toml-(toml)) 1. [group.toml (TOML)](#group.toml-(toml)) + 1. [analyzed.toml (TOML)](#group.toml-(toml)) + 1. [stack.toml (TOML)](#group.toml-(toml)) ## Platform API Version @@ -199,3 +201,36 @@ group = [ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. + +### analyzed.toml (TOML) + +```toml +[image] + reference = "" + +[metadata] +# layer metadata +``` + +Where: +- `image.reference` MUST be EITHER a digest reference to an image in a docker registry or the ID of an image in a docker daemon +- `metadata` MUST be the TOML representation fo the layer [metadata label](#layer-metadata-label-json) + +### stack.toml (TOML) + +```toml +[run-image] + image = "" + +[run-image.mirrors] = ["", "mirror"] +``` + +Where: +- `run-image.image` MAY be a tag reference to an run image in a docker registry +- `run-image.mirrors` MUST NOT be present IF `run-image.image` is not present +- `run-image.mirrors` MAY contain one or more tag references to run images in docker registries +- all present `run-image-mirrors`: + * MUST resolve to a digest reference identical to that which `run-image.image` resolves + * MUST NOT refer to the same registry as does `run-image.image` or any other entries `run-image.mirrors` + + From db7bb0fdbe249818421cd50a9e733661147dfec8 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Thu, 13 Feb 2020 18:33:30 -0500 Subject: [PATCH 15/38] Documents lifecycle interface for build phases Signed-off-by: Emily Casey --- buildpack.md | 2 +- platform.md | 482 +++++++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 434 insertions(+), 50 deletions(-) diff --git a/buildpack.md b/buildpack.md index b9ccf512..6e60bb15 100644 --- a/buildpack.md +++ b/buildpack.md @@ -4,7 +4,7 @@ This document specifies the interface between a single lifecycle and one or more A lifecycle is a program that uses buildpacks to transform application source code into an OCI image containing the compiled application. -This is accomplished in four phases: +This is accomplished in five phases: 1. **Detection,** where an optimal selection of compatible buildpacks is chosen. 2. **Analysis,** where metadata about OCI layers generated during a previous build are made available to buildpacks. diff --git a/platform.md b/platform.md index 438c188a..3de69394 100644 --- a/platform.md +++ b/platform.md @@ -34,32 +34,38 @@ Examples of a platform might include: ## Platform API Version +**Current Version**: 0.3 + The Platform API version: - MUST be in form `.` or ``, where `` is equivalent to `.0` - - MUST describe the implemented Platform API. - - SHALL indicate compatibility with a given lifecycle according to the following rules: - - When `` is `0`, the platform is only compatible with lifecycles implementing that exact Platform API. - - When `` is greater than `0`, the platforms is only compatible with lifecycles implementing platform API - `.`, where `` of the lifecycle equals `` of the platform and `` of the lifecycle - is greater than or equal to `` of the platform. + - When `` is greater than `0` increments to `` SHALL exclusively indicate additive changes -### Compatibility Verification +### Lifecycle Compatibility -The lifecycle SHALL verify compatibility if the environment variable `CNB_PLATFORM_API` is set. The value of this -environment variable MUST be the version of the Platform API the platform implements. Compatibility verification SHALL -NOT occur if this environment variable is not set. Compatibility verification SHALL occur before any other validation. +IF `CNB_PLATFORM_API=[./` directory as specified in the [Buildpack Interface Specification](buildpack.md). +An **app image** refers to an OCI image generated by the lifecycle by extending the run image with any or all of the following: app layers, launch layers, launcher layers, image configuration + +A **launch layer** refers to a layer in the app image created from a `/` directory as specified in the [Buildpack Interface Specification](buildpack.md). -An **app layer** refers to a layer created from the `` directory as specified in the [Buildpack Interface Specification](buildpack.md). +An **app layer** refers to a layer in the app image created from the `` directory as specified in the [Buildpack Interface Specification](buildpack.md). +A **launcher layer** refers to a layer in the app OCI image containing the launcher itself and/or launcher configuration + +The **launcher** refers to a lifecycle binary packaged in the app image for the purpose of executing processes at runtime + +## Stacks ### Compatibility Guarantees Stack image authors SHOULD ensure that build image versions maintain [ABI-compatibility](https://en.wikipedia.org/wiki/Application_binary_interface) with previous versions, although violating this requirement will not change the behavior of previously built images containing app and launch layers. @@ -73,8 +79,6 @@ During build, platforms MUST use the same set of mixins for the run image as wer ### Build Image -The platform MUST execute the detection and build phases of the lifecycle on the build image. - The platform MUST ensure that: - The image config's `User` field is set to a non-root user with a writable home directory. @@ -84,37 +88,8 @@ The platform MUST ensure that: - The image config's `Label` field has the label `io.buildpacks.stack.id` set to the stack ID. - The image config's `Label` field has the label `io.buildpacks.stack.mixins` set to a JSON array containing mixin names for each mixin applied to the image. -To initiate the detection phase, the platform MUST invoke the `/cnb/lifecycle/detector` executable with the user and environment defined in the build image config. -Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. - -```bash -/cnb/lifecycle/detector -buildpacks /cnb/by-id -order /cnb/order.toml -group ./group.toml -plan ./plan.toml -``` - -Where: - -- `-buildpacks` MUST specify input from a buildpacks directory as defined in the [Buildpacks Directory Layout](#buildpacks-directory-layout) section. -- `-order` MUST specify input from an overriding `order.toml` file path as defined in the [Data Format](#data-format) section. -- `-group` MUST specify output to a `group.toml` file path as defined in the [Data Format](#data-format) section. -- `-plan` MUST specify output to a Build Plan as defined in the [Buildpack Interface Specification](buildpack.md). - -To initiate the build phase, the platform MUST invoke the `/cnb/lifecycle/builder` executable with the user and environment defined in the build image config. -Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. - -```bash -/cnb/lifecycle/builder -buildpacks /cnb/by-id -group ./group.toml -plan ./plan.toml -``` - -Where: - -- `-buildpacks` MUST specify input from a buildpacks directory as defined in the [Buildpacks Directory Layout](#buildpacks-directory-layout) section. -- `-group` MUST specify input from a `group.toml` file path as defined in the [Data Format](#data-format) section. -- `-plan` MUST specify input from a Build Plan as defined in the [Buildpack Interface Specification](buildpack.md). - ### Run Image -The platform MUST provide the lifecycle with a reference to the run image during the export phase. - The platform MUST ensure that: - The image config's `User` field is set to a user with the same UID and primary GID as in the build image. @@ -135,6 +110,308 @@ A platform MAY support any number of mixins for a given stack in order to suppor Changes introduced by mixins SHOULD be restricted to the addition of operating system software packages that are regularly patched with strictly backwards-compatible security fixes. However, mixins MAY consist of any changes that follow the [Compatibility Guarantees](#compatibility-guarantees). +## Lifecycle Interface + +### Build +To create an app image A platform MUST execute the following lifecycle phases + +1. Detection +1. Analysis +1. Cache Restoration +1. Build +1. Export + +either by executing the following phase specific lifecycle binaries +1. `/cnb/lifecycle/detector` +1. `/cnb/lifecycle/analyzer` +1. `/cnb/lifecycle/restorer` +1. `/cnb/lifecycle/builder` +1. `/cnb/lifecycle/exporter` + +or by executing +1. `/cnb/lifecycle/creator` + +### Run +`/cnb/lifecycle/launcher` is responsible for launching user and buildpack provided processes in the correct execution environment. +`/cnb/lifecycle/launcher` SHALL be the `ENTRYPOINT` for all app images. + + +### Phases + +The following is true of all lifecycle phases: + +1. command line inputs ALWAYS take precedence over other inputs +1. IF `CNB_PLATFORM_API=[.] \ + [-buildpacks ] \ + [-group ] \ + [-log-level ] \ + [-order ] \ + [-plan ] \ + [-platform ] +``` + +| Input | Environment Variable| Default Value | Description +|---------------|---------------------|-----------------|---------------------- +| `` | CNB_APP_DIR | /workspace | Path to application directory +| `` | CNB_BUILDPACKS_DIR | /cnb/buildpacks | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) +| `` | CNB_GROUP_PATH | ./group.toml | Path to output group file +| `` | CNB_LOG_LEVEL | info | Log Level +| `` | CNB_ORDER_PATH | ./order.toml | Path to order definition ( see [order.toml (TOML)](#order.toml-(toml))) +| `` | CNB_PLAN_PATH | ./plan.toml | Path to output build plan file +| `` | CNB_PLATFORM_DIR | /platform | Path to platform directory + +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Pass (0), or error (1-99, 101+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `` | Detected buildpack group ( see [group.toml (TOML)](#group.toml-(toml))) +| `` | Resolved Build Plan ( see data format in [Buildpack Interface Specification](buildpack.md)) + +### `analyzer` +Usage: +``` +/cnb/lifecycle/analyzer \ + [-analyzed ] \ + [-cache-dir ] \ + [-cache-image ] \ + [-gid ] \ + [-group ] \ + [-layers ] \ + [-skip-layers ] \ + [-uid ] \ + [-daemon ] +``` + +| Input | Environment Variable | Default Value | Description +|----------------|-----------------------|-------------------|---------------------- +| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to output analysis metadata ( see [analyzed.toml (TOML)](#analyzed.toml-(toml)) +| `` | `CNB_CACHE_DIR` | - | Location of cache, provided as a directory +| ``| `CNB_CACHE_IMAGE` | - | Location of cache, provided as an image +| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition ( see [group.toml (TOML)](#group.toml-(toml))) +| `` | - | - | Image reference to be analyzed (usually the result of the previous build) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| ``| `CNB_SKIP_LAYERS` | `false` | Do not write layer metadata +| `` | `CNB_USER_ID` | - | User that build phase will run as +| `` | `CNB_USE_DAEMON` | `false` | Analyze image from docker daemon + +IF `` is `false`, `` MUST be a valid registry reference +IF `` is `true`, `` MUST be a valid registry reference OR a docker image ID +The lifecycle MUST accept valid references to non-existent images without error. + +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Pass (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `` | Analysis metadata ( see [analyzed.toml (TOML)](#analyzed.toml-(toml)) +| `//.toml` files | Contain analyzed layer metadata ( see data format in [Buildpack Interface Specification](buildpack.md)) +| `//.sha` files | Contain sha256 of the uncompressed layer + +The lifecycle MUST write analysis metadata if `` is accessible. +If `` is `false` the lifecycle MUST analyze layers created by any buildpack present in the provided ``. +If `` is `true` the lifecycle MUST NOT analyze layers. + +#### Layer analysis +When analyzing a given layer the lifecycle SHALL: +* IF `build=true`, `cache=false`: +1. do nothing +* ELSE IF `launch=true`: +1. write layer metadata read from the analyzed image to `//.toml` +2. write the sha256 of the uncompressed layer from the analyzed image to `//.sha` +* ELSE IF `cache=true`: +1. write layer metadata read from the cache to `//.toml` +2. write the sha256 of the uncompressed layer from the cache to `//.sha` + +### `restorer` +Executable: +``` +/cnb/lifecycle/restorer \ + [-cache-dir ] + [-cache-image ] + [-gid ] \ + [-group ] \ + [-layers ] \ + [-uid ] +``` + +| Input | Environment Variable | Default Value | Description +|----------------|-----------------------|-----------------|---------------------- +| `` | `CNB_CACHE_DIR` | - | Path to a cache directory +| ``| `CNB_CACHE_IMAGE` | - | Reference to a cache image in an OCI image registry +| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition ( see [group.toml (TOML)](#group.toml-(toml))) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_USER_ID` | - | User that build phase will run as +| `//.sha` || | Files Containing the sha256 of the uncompressed layers +| `//.toml` || | Files Containing analyzed layer metadata ( see data format in [Buildpack Interface Specification](buildpack.md)) + +| Output | Description +|------------------------------------|---------------------------------------------- +| [exit status] | Pass (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `///*` | Restored layer contents + +#### Layer restoration +For each layer metadata file found in the `` the lifecycle: +1. MUST restore cached layer contents IF the cache contains a layer with a matching ID +1. MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a matching layer + +### `builder` +The platform MUST execute `builder` in the build environment + +Usage: +``` +/cnb/lifecycle/builder \ + [-app ] \ + [-buildpacks ] \ + [-group ] \ + [-layers ] \ + [-plan ] \ + [-platform ] +``` + +| Input | Env | Default Value | Description +|--------------|---------------------|-----------------|---------------------- +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file ( see [group.toml (TOML)](#group.toml-(toml))) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file ( see data format in [Buildpack Interface Specification](buildpack.md)) +| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory + +#### Outputs +| Output | Description +|--------------------------------------------|---------------------------------------------- +| [exit status] | Pass (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `//` | Layer contents buildpacks (see [Buildpack Interface Specfication](buildpack.md) +| `//.toml` | Layer metadata (see [Buildpack Interface Specfication](buildpack.md) +| `/config/metadata.toml` | Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) + +The lifecycle SHALL execute all buildpacks in the order defined in `` according to the rules of the [Buildpack Interface Specification](buildpack.md). +The lifecycle SHALL add all invoked buildpacks to `[[buildpacks]]` in `/config/metadata.toml`. +The lifecycle SHALL aggregate all `processes`, `slices` and BOM entries returned by buildpacks in `/config/metadata.toml`. + +### `exporter` +Usage: +``` +/cnb/lifecycle/exporter [...] + [-analyzed ] \ + [-app ] \ + [-cache-dir ] \ + [-cache-image ] \ + [-daemon ] \ + [-gid ] \ + [-group ] \ + [-image ] # Deprecated + [-launch-cache ] \ + [-launcher ] \ + [-layers ] \ + [-log-level ] \ + [-process-type ] \ + [-project-metadata ] \ + [-run-image ] \ + [-stack ] \ + [-uid ] +``` + +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|---------------------|--------------------------------------- +| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to analysis metadata ( see [analyzed.toml (TOML)](#analyzed.toml-(toml)) +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_CACHE_DIR` | - | Path to a cache directory +| `` | `CNB_CACHE_IMAGE` | - | Reference to a cache image in an OCI image registry +| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon +| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file ( see [group.toml (TOML)](#group.toml-(toml))) +| `` | - | - | An +| `` | `CNB_LAUNCH_CACHE_DIR`| - | Path to a cache directory containing launch layers +| `` | - | `/cnb/lifecycle/launcher` | Path to the `launcher` executable +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_LOG_LEVEL` ` | info | Log Level +| `` | `CNB_PROCESS_TYPE` | - | Default process type to set in the exported image +| ``| `CNB_PROCESS_TYPE` | `./project-metadata.toml` | Path to a project metadata file (see [project-metadata.toml (TOML)](#project-metadata.toml) +| `` | `CNB_RUN_IMAGE` | derived from | Default process type to set in the exported image +| `` | `CNB_STACK_PATH` | `/cnb/stack.toml` | Path to stack file (see [stack.toml (TOML)](#stack.toml-(toml)) +| `` | `CNB_USER_ID` | - | User that build phase will run as +| `/config/metadata.toml` |-|-| Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) + +At least one `` must be provided +Each `` MUST be a valid OCI image registry tag reference +If `` is `false` and more than one `` is provided they MUST refer to the same registry +If `` is not provided by the platform the value will be derived from the contents of `stack` +* If any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` +* If none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` + +#### Outputs +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Pass (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| | Exported app image (see [Buildpack Interface Specfication](buildpack.md) + +The lifecycle SHALL write the same app image to each `` tag +The app image: +* MUST be an extension of the `` + * all run-image layers and config values SHALL be preserved unless they conflict with the following +* MUST contain all buildpack provided launch layers as determined by the [Buildpack Interface Specfication](buildpack.md) +* MUST contain one or more app layers as determined by the [Buildpack Interface Specfication](buildpack.md) +* MUST contain a layer that includes the `` +* MUST contain a layer that includes `/config/metadata.toml` +* MUST have `ENTRYPOINT=` +* MUST contain the following `Env` entries + * `"CNB_LAYERS_DIR="` + * `"CNB_APP_DIR="` +* MUST contain the following `Env` entry, IF `` is set + * `"CNB_PROCESS_TYPE="` +* MUST contain the following labels +- `io.buildpacks.lifecycle.metadata`: see [lifecycle metadata label (JSON)](#lifecycle-metadata-label-(json)) +- `io.buildpacks.project.metadata`: the value of which SHALL be the json representation `` +- `io.buildpacks.build.metadata`: see [build metadata (JSON)](#build-metadata-label-(json)) + +IF a cache is provided the lifecycle SHALL +* write `io.buildpacks.cache.metadata` to the cache +* ensure all `cache=true` layers are written to the cache + +### `creator` +Usage: +``` +/cnb/lifecycle/create + [-t ...] + [-analyzed ] \ + [-app ] \ + [-cache-dir ] \ + [-cache-image ] \ + [-daemon ] \ + [-gid ] \ + [-group ] \ + [-image ] # Deprecated + [-launch-cache ] \ + [-launcher ] \ + [-layers ] \ + [-log-level ] \ + [-process-type ] \ + [-project-metadata ] \ + [-run-image ] \ + [-stack ] \ + [-uid ] +``` + ## Buildpacks ### Buildpacks Directory Layout @@ -216,21 +493,128 @@ Where: - `image.reference` MUST be EITHER a digest reference to an image in a docker registry or the ID of an image in a docker daemon - `metadata` MUST be the TOML representation fo the layer [metadata label](#layer-metadata-label-json) +### metadata.toml (TOML) +```toml +[[buildpacks]] +id = "" +version = "" +optional = false + +[[processes]] +type = "" +command = "" +args = [""] +direct = false + +[[slices]] +paths = [""] + +[bom] +``` + +Where: +- Both `id` and `version` MUST be present for each buildpack. + ### stack.toml (TOML) ```toml [run-image] image = "" -[run-image.mirrors] = ["", "mirror"] +[run-image.mirrors] = ["", ""] ``` Where: -- `run-image.image` MAY be a tag reference to an run image in a docker registry +- `run-image.image` MAY be a reference to a run image in a docker registry - `run-image.mirrors` MUST NOT be present IF `run-image.image` is not present - `run-image.mirrors` MAY contain one or more tag references to run images in docker registries -- all present `run-image-mirrors`: - * MUST resolve to a digest reference identical to that which `run-image.image` resolves - * MUST NOT refer to the same registry as does `run-image.image` or any other entries `run-image.mirrors` +- all `run-image.mirrors`: + * SHOULD reference an image with ID identical to that of `run-image.image` +- `run-image.image` and `run-image.mirrors.[]` SHOULD each refer to a unique registry + +### lifecycle metadata label (JSON) + +```json +{ + "app": [ + {"sha": ""} + ], + "config": { + "sha": "" + }, + "launcher": { + "sha": "" + }, + "buildpacks": [ + { + "key": "", + "version": "", + "layers": { + "": { + "sha": "", + "data": {}, + "build": false, + "launch": false, + "cache": false + } + } + } + ], + "runImage": { + "topLayer": "", + "reference": "" + }, + "stack": { + "runImage": { + "image": "cnbs/sample-stack-run:bionic" + } + } +} +``` + +### build metadata label (JSON) + +``` +{ + "processes": [ + { + "type": "", + "command": "", + "args": [ + "" + ], + "direct": false + } + ], + "buildpacks": [ + { + "id": "", + "version": "" + } + ], + "bom": [ + { + "name": "", + "version": "", + "metadata": { + # arbitrary buildpack provided metadata + }, + "buildpack": { + "id": "", + "version": "" + } + }, + ], + "launcher": { + "version": "", + "source": { + "git": { + "repository": "", + "commit": "" + } + } + } +} +``` From 7f6dfd72bed9c3e50d7d99877505eef843cd4fa5 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 10 Jun 2020 10:27:19 -0400 Subject: [PATCH 16/38] adds creator Signed-off-by: Emily Casey --- platform.md | 46 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/platform.md b/platform.md index 3de69394..6ef903a1 100644 --- a/platform.md +++ b/platform.md @@ -130,6 +130,9 @@ either by executing the following phase specific lifecycle binaries or by executing 1. `/cnb/lifecycle/creator` + +### Rebase +To rebase an app image A platform MUST execute the `/cnb/lifecycle/rebaser` or perform an equivalent operation. ### Run `/cnb/lifecycle/launcher` is responsible for launching user and buildpack provided processes in the correct execution environment. @@ -391,27 +394,41 @@ IF a cache is provided the lifecycle SHALL ### `creator` Usage: ``` -/cnb/lifecycle/create - [-t ...] - [-analyzed ] \ +/cnb/lifecycle/creator [-app ] \ [-cache-dir ] \ [-cache-image ] \ [-daemon ] \ [-gid ] \ - [-group ] \ - [-image ] # Deprecated [-launch-cache ] \ [-launcher ] \ [-layers ] \ [-log-level ] \ + [-order ] \ + [-platform ] \ + [-previous-image ] \ [-process-type ] \ [-project-metadata ] \ [-run-image ] \ + [-skip-restore ] \ [-stack ] \ + [-tag ...] \ [-uid ] ``` +Running `creator` SHALL be equivalent to running `detector`, `analzyer`, `restorer`, `builder` and `exporter` in order with identical inputs where they are accepted, with the following exceptions. + +| Input | Environment Variable| Default Value| Description +|-------------------|---------------------|--------------|---------------------- +| ``| `CNB_PREVIOUS_IMAGE`| `` | Image reference to be analyzed (usually the result of the previous build) +| `` | `CNB_SKIP_RESTORE` | `false` | Do not write layer metadata or restore cached layers +| `...` | - | - | Additional tag to apply to exported image + +If `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. +If the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` + +Outputs produced by `creator` and identical to those produced by `exporter`. + ## Buildpacks ### Buildpacks Directory Layout @@ -515,6 +532,25 @@ paths = [""] Where: - Both `id` and `version` MUST be present for each buildpack. +### project-metadata.toml (TOML) + +```toml +[source] +type = "" + +[source.version] +# arbitrary data + +[source.metadata] +# arbitrary data +``` + +Where: +- All values are optional +- `type`, if present, SHOULD contain the type of location where the provided app source is stored (e.g `git`, `s3`) +- `version`, if present, SHOULD contain data uniquely identifying the particular version of the provided source +- `metadata` MAY contain additional arbitrary data about the provided source + ### stack.toml (TOML) ```toml From 0473be3d8ad52d93068c2ba467dfb250410394d3 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Mon, 15 Jun 2020 10:42:54 -0400 Subject: [PATCH 17/38] Usage for rebaser Signed-off-by: Emily Casey --- platform.md | 46 ++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 2 deletions(-) diff --git a/platform.md b/platform.md index 6ef903a1..a7928875 100644 --- a/platform.md +++ b/platform.md @@ -295,7 +295,6 @@ Usage: | `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file ( see data format in [Buildpack Interface Specification](buildpack.md)) | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory -#### Outputs | Output | Description |--------------------------------------------|---------------------------------------------- | [exit status] | Pass (0), or error (1+) @@ -348,7 +347,7 @@ Usage: | `` | `CNB_LOG_LEVEL` ` | info | Log Level | `` | `CNB_PROCESS_TYPE` | - | Default process type to set in the exported image | ``| `CNB_PROCESS_TYPE` | `./project-metadata.toml` | Path to a project metadata file (see [project-metadata.toml (TOML)](#project-metadata.toml) -| `` | `CNB_RUN_IMAGE` | derived from | Default process type to set in the exported image +| `` | `CNB_RUN_IMAGE` | derived from | Run image reference | `` | `CNB_STACK_PATH` | `/cnb/stack.toml` | Path to stack file (see [stack.toml (TOML)](#stack.toml-(toml)) | `` | `CNB_USER_ID` | - | User that build phase will run as | `/config/metadata.toml` |-|-| Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) @@ -429,6 +428,49 @@ If the platform provides one or more `` inputs they SHALL be treated as add Outputs produced by `creator` and identical to those produced by `exporter`. +### `rebaser` +Usage: +``` +/cnb/lifecycle/rebaser [...] + [-daemon] + [-gid ] + [-image ] # Deprecated + [-log-level ] + [-run-image ] + [-uid ] +``` + +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|-----------------------|--------------------------------------- +| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon +| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as +| `` | - | - | App image to rebase +| `` | `CNB_LOG_LEVEL` ` | info | Log Level +| `` | `CNB_RUN_IMAGE` | derived from ``| Run image reference +| `` | `CNB_USER_ID` | - | User that build phase will run as + +At least one `` must be provided +Each `` MUST be a valid OCI image registry tag reference +If `` is `false` and more than one `` is provided they MUST refer to the same registry +If `` is not provided by the platform the value will be derived from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on `` +* If any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` +* If none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` + +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Pass (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `` | Rebased app image (see [Buildpack Interface Specfication](buildpack.md) + +The lifecycle SHALL write the same app image to each `` tag +The rebased app image SHALL be identical to ``, with the following modifications: +* Run image layers SHALL be defined as Layers in `` up to and including the layer with diff ID matching the value of `run-image.top-layer` from the `io.buildpacks.lifecycle.metadata` label +* Run image layers SHALL be replaced with the layers from the new `` +* The value of `io.buildpacks.lifecycle.metadata` SHALL be modified as follows + * `run-image.reference` SHALL uniquely identify `` + * `run-image.top-layer` SHALL be set to the uncompressed digest of the top layer in `` + ## Buildpacks ### Buildpacks Directory Layout From 33a257831d1b65f2514ce638eeff6dbb9f0fbe5a Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Mon, 15 Jun 2020 13:00:43 -0400 Subject: [PATCH 18/38] launcher usage Signed-off-by: Emily Casey --- platform.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/platform.md b/platform.md index a7928875..f53a300f 100644 --- a/platform.md +++ b/platform.md @@ -470,6 +470,41 @@ The rebased app image SHALL be identical to ``, with the following modifi * The value of `io.buildpacks.lifecycle.metadata` SHALL be modified as follows * `run-image.reference` SHALL uniquely identify `` * `run-image.top-layer` SHALL be set to the uncompressed digest of the top layer in `` + +### `launcher` +Usage: +``` +/cnb/lifecycle/launcher [--] [args...] +``` + +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|-----------------------|--------------------------------------- +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_PROCESS_TYPE` | `web` | `type` of process to launch +| `` | | `false` | Execution strategy for user provided process +| `...` | | - | User provided process +| `/config/metadata.toml` |- |-| Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) +| `///` |- |-| Launch Layers + +If `$1` is `--` `` is `true` and `` SHALL be `${@2:}` +If `$1` is anything other than `--`, `` is `false`, and `` SHALL be `$@` +If `` + +If `` are NOT provided `launcher` SHALL select the process with `type` equal to `` from `/config/metadata.toml` and launch it as described in the [Buildpack Interface Specification](buildpack.md). +If `` are provided `launcher` SHALL behaves as if it selected a process with the following process from `metadata.toml`. +``` +[[process]] +direct = +command = "args[0]" +args = ["args[1]"...] +``` + +The lifecycle SHOULD replace the lifecycle process in memory without forking it. +The process execution environment SHALL be identical to the lifecycle execution environment, with the following exceptions: +* `CNB_APP_DIR` SHALL NOT be set in the process environment +* `CNB_LAYERS_DIR` SHALL NOT be set in the process environment +* `CNB_PROCESS_TYPE` SHALL NOT be set in the process environment ## Buildpacks From ef660168377d9855398c121113bca7926686e7c8 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Mon, 15 Jun 2020 18:38:27 -0400 Subject: [PATCH 19/38] Fixes index * Adds details to data format * Alphabetizes Signed-off-by: Emily Casey --- platform.md | 347 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 201 insertions(+), 146 deletions(-) diff --git a/platform.md b/platform.md index f53a300f..05efd6e1 100644 --- a/platform.md +++ b/platform.md @@ -13,23 +13,43 @@ Examples of a platform might include: ## Table of Contents 1. [Platform API Version](#platform-api-version) - 1. [Compatibility Verification](#compatibility-verification) 1. [Stacks](#stacks) 1. [Compatibility Guarantees](#compatibility-guarantees) 1. [Build Image](#build-image) 1. [Run Image](#run-image) +1. [Lifecycle Interface](#lifecycle-interface) + 1. [Platform API Compatibility](#platform-api-compatibility) + 1. [Operations](#operations) + 1. [Build](#build) + 1. [Rebase](#rebase) + 1. [Launch](#launch) + 1. [Usage](#usage) + 1. [detector](#detector) + 1. [analyzer](#analyzer) + 1. [restorer](#restorer) + 1. [builder](#builder) + 1. [exporter](#exporter) + 1. [creator](#creator) + 1. [rebaser](#rebaser) + 1. [launcher](#launcher) 1. [Buildpacks](#buildpacks) 1. [Buildpacks Directory Layout](#buildpacks-directory-layout) 1. [Security Considerations](#security-considerations) 1. [Additional Guidance](#additional-guidance) 1. [Environment](#environment) - 1. [Run Image Rebasing](#run-image-rebasing) 1. [Caching](#caching) 1. [Data Format](#data-format) - 1. [order.toml (TOML)](#order.toml-(toml)) - 1. [group.toml (TOML)](#group.toml-(toml)) - 1. [analyzed.toml (TOML)](#group.toml-(toml)) - 1. [stack.toml (TOML)](#group.toml-(toml)) + 1. [Files](#files) + 1. [analyzed.toml (TOML)](#analyzedtoml-toml) + 1. [group.toml (TOML)](#grouptoml-toml) + 1. [metadata.toml (TOML)](#metadatatoml-toml) + 1. [order.toml (TOML)](#ordertoml-toml) + 1. [project-metadata.toml (TOML)](#project-metadatatoml-toml) + 1. [stack.toml (TOML)](#group.toml-(toml)) + 1. [Labels](#labels) + 1. [io.buildpacks.build.metadata (JSON)](#iobuildpacksbuildmetadata-json) + 1. [io.buildpacks.lifecycle.metadata (JSON)](#iobuildpackslifecyclemetadata-json) + 1. [io.buildpacks.project.metadata (JSON)](#iobuildpacksprojectmetadata-json) ## Platform API Version @@ -40,14 +60,6 @@ The Platform API version: - MUST be in form `.` or ``, where `` is equivalent to `.0` - When `` is greater than `0` increments to `` SHALL exclusively indicate additive changes -### Lifecycle Compatibility - -IF `CNB_PLATFORM_API=[.[.[.` | CNB_APP_DIR | /workspace | Path to application directory -| `` | CNB_BUILDPACKS_DIR | /cnb/buildpacks | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) -| `` | CNB_GROUP_PATH | ./group.toml | Path to output group file -| `` | CNB_LOG_LEVEL | info | Log Level -| `` | CNB_ORDER_PATH | ./order.toml | Path to order definition ( see [order.toml (TOML)](#order.toml-(toml))) -| `` | CNB_PLAN_PATH | ./plan.toml | Path to output build plan file -| `` | CNB_PLATFORM_DIR | /platform | Path to platform directory +| `` | `CNB_APP_DIR` | /workspace | Path to application directory +| `` | `CNB_BUILDPACKS_DIR` | /cnb/buildpacks | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) +| `` | `CNB_GROUP_PATH` | ./group.toml | Path to output group file +| `` | `CNB_LOG_LEVEL` | info | Log Level +| `` | `CNB_ORDER_PATH` | ./order.toml | Path to order definition ( see [order.toml (TOML)](#order.toml-(toml))) +| `` | `CNB_PLAN_PATH` | ./plan.toml | Path to output build plan file +| `` | `CNB_PLATFORM_DIR` | /platform | Path to platform directory | Output | Description |--------------------|---------------------------------------------- -| [exit status] | Pass (0), or error (1-99, 101+) +| [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) | `` | Detected buildpack group ( see [group.toml (TOML)](#group.toml-(toml))) | `` | Resolved Build Plan ( see data format in [Buildpack Interface Specification](buildpack.md)) -### `analyzer` +#### `analyzer` Usage: ``` /cnb/lifecycle/analyzer \ [-analyzed ] \ [-cache-dir ] \ [-cache-image ] \ + [-daemon] \ # sets [-gid ] \ [-group ] \ [-layers ] \ [-skip-layers ] \ [-uid ] \ - [-daemon ] ``` | Input | Environment Variable | Default Value | Description @@ -215,7 +237,7 @@ The lifecycle MUST accept valid references to non-existent images without error. | Output | Description |--------------------|---------------------------------------------- -| [exit status] | Pass (0), or error (1+) +| [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) | `` | Analysis metadata ( see [analyzed.toml (TOML)](#analyzed.toml-(toml)) @@ -226,7 +248,7 @@ The lifecycle MUST write analysis metadata if `` is accessible. If `` is `false` the lifecycle MUST analyze layers created by any buildpack present in the provided ``. If `` is `true` the lifecycle MUST NOT analyze layers. -#### Layer analysis +##### Layer analysis When analyzing a given layer the lifecycle SHALL: * IF `build=true`, `cache=false`: 1. do nothing @@ -237,8 +259,8 @@ When analyzing a given layer the lifecycle SHALL: 1. write layer metadata read from the cache to `//.toml` 2. write the sha256 of the uncompressed layer from the cache to `//.sha` -### `restorer` -Executable: +#### `restorer` +Usage: ``` /cnb/lifecycle/restorer \ [-cache-dir ] @@ -262,17 +284,17 @@ Executable: | Output | Description |------------------------------------|---------------------------------------------- -| [exit status] | Pass (0), or error (1+) +| [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) | `///*` | Restored layer contents -#### Layer restoration +##### Layer restoration For each layer metadata file found in the `` the lifecycle: 1. MUST restore cached layer contents IF the cache contains a layer with a matching ID 1. MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a matching layer -### `builder` +#### `builder` The platform MUST execute `builder` in the build environment Usage: @@ -297,7 +319,7 @@ Usage: | Output | Description |--------------------------------------------|---------------------------------------------- -| [exit status] | Pass (0), or error (1+) +| [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) | `//` | Layer contents buildpacks (see [Buildpack Interface Specfication](buildpack.md) @@ -308,7 +330,7 @@ The lifecycle SHALL execute all buildpacks in the order defined in `` acc The lifecycle SHALL add all invoked buildpacks to `[[buildpacks]]` in `/config/metadata.toml`. The lifecycle SHALL aggregate all `processes`, `slices` and BOM entries returned by buildpacks in `/config/metadata.toml`. -### `exporter` +#### `exporter` Usage: ``` /cnb/lifecycle/exporter [...] @@ -316,17 +338,16 @@ Usage: [-app ] \ [-cache-dir ] \ [-cache-image ] \ - [-daemon ] \ + [-daemon] \ # sets [-gid ] \ [-group ] \ - [-image ] # Deprecated [-launch-cache ] \ [-launcher ] \ [-layers ] \ [-log-level ] \ [-process-type ] \ [-project-metadata ] \ - [-run-image ] \ + [-run-image | -image ] \ # -image is Deprecated [-stack ] \ [-uid ] ``` @@ -337,7 +358,7 @@ Usage: | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | `` | `CNB_CACHE_DIR` | - | Path to a cache directory | `` | `CNB_CACHE_IMAGE` | - | Reference to a cache image in an OCI image registry -| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon +| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon | `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as | `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file ( see [group.toml (TOML)](#group.toml-(toml))) | `` | - | - | An @@ -359,13 +380,12 @@ If `` is not provided by the platform the value will be derived from * If any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` * If none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` -#### Outputs | Output | Description |--------------------|---------------------------------------------- -| [exit status] | Pass (0), or error (1+) +| `[exit status]` | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| | Exported app image (see [Buildpack Interface Specfication](buildpack.md) +| `` | Exported app image (see [Buildpack Interface Specfication](buildpack.md) The lifecycle SHALL write the same app image to each `` tag The app image: @@ -390,14 +410,14 @@ IF a cache is provided the lifecycle SHALL * write `io.buildpacks.cache.metadata` to the cache * ensure all `cache=true` layers are written to the cache -### `creator` +#### `creator` Usage: ``` /cnb/lifecycle/creator [-app ] \ [-cache-dir ] \ [-cache-image ] \ - [-daemon ] \ + [-daemon] \ # sets [-gid ] \ [-launch-cache ] \ [-launcher ] \ @@ -428,21 +448,20 @@ If the platform provides one or more `` inputs they SHALL be treated as add Outputs produced by `creator` and identical to those produced by `exporter`. -### `rebaser` +#### `rebaser` Usage: ``` -/cnb/lifecycle/rebaser [...] - [-daemon] - [-gid ] - [-image ] # Deprecated - [-log-level ] - [-run-image ] +/cnb/lifecycle/rebaser [...] \ + [-daemon] \ # sets + [-gid ] \ + [-log-level ] \ + [-run-image | -image ] \ # -image is Deprecated [-uid ] ``` | Input | Environment Variable | Default Value | Description |---------------------|-----------------------|-----------------------|--------------------------------------- -| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon +| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon | `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as | `` | - | - | App image to rebase | `` | `CNB_LOG_LEVEL` ` | info | Log Level @@ -470,11 +489,11 @@ The rebased app image SHALL be identical to ``, with the following modifi * The value of `io.buildpacks.lifecycle.metadata` SHALL be modified as follows * `run-image.reference` SHALL uniquely identify `` * `run-image.top-layer` SHALL be set to the uncompressed digest of the top layer in `` - -### `launcher` + +#### `launcher` Usage: ``` -/cnb/lifecycle/launcher [--] [args...] +/cnb/lifecycle/launcher [--] [ ...] ``` | Input | Environment Variable | Default Value | Description @@ -483,7 +502,8 @@ Usage: | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory | `` | `CNB_PROCESS_TYPE` | `web` | `type` of process to launch | `` | | `false` | Execution strategy for user provided process -| `...` | | - | User provided process +| `` | | - | User-provided command +| `...` | | - | Arguments to user-provided command | `/config/metadata.toml` |- |-| Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) | `///` |- |-| Launch Layers @@ -496,8 +516,8 @@ If `` are provided `launcher` SHALL behaves as if it selected a process wi ``` [[process]] direct = -command = "args[0]" -args = ["args[1]"...] +command = "" +args = [""...] ``` The lifecycle SHOULD replace the lifecycle process in memory without forking it. @@ -524,44 +544,34 @@ A more thorough explanation is provided in the [Buildpack Interface Specificatio ### Environment -User-provided environment variables intended for build and launch SHOULD NOT come from the same list. -The end-user SHOULD be encouraged to define them separately. The platform MAY determine the initial environment of the build phase, detection phase, and launch. The lifecycle MUST NOT assume that all platforms provide an identical environment. -### Run Image Rebasing - -Run image rebasing allows for fast stack updates for already-exported OCI images with minimal data transfer when those images are stored on a Docker registry. -When a new stack version with the same stack ID is available, the app layers and launch layers SHOULD be rebased on the new run image by updating the image's configuration to point at the new run image. -Once the new run image is present on the registry, filesystem layers SHOULD NOT be uploaded or downloaded. - -The new run image MUST have an identical stack ID and MUST include the exact same set of mixins. - -![Launch](img/launch.svg) - ### Caching -Each platform SHOULD implement caching so as to appropriately optimize performance. +If caching is enabled the platform is responsible for providing the lifecycle with access to the correct cache. +Whenever possible, the platform SHOULD provide the same cache to each rebuild of a given app image. Cache locality and availability MAY vary between platforms. ## Data Format -### order.toml (TOML) +### Files + +#### analyzed.toml (TOML) ```toml -[[order]] -[[order.group]] -id = "" -version = "" -optional = false +[image] + reference = "" + +[metadata] +# layer metadata ``` Where: +- `image.reference` MUST be EITHER a digest reference to an image in a docker registry or the ID of an image in a docker daemon +- `metadata` MUST be the TOML representation of the layer [metadata label](#layer-metadata-label-json) -- Both `id` and `version` MUST be present for each buildpack object in a group. -- The value of `optional` MUST default to false if not specified. - -### group.toml (TOML) +#### group.toml (TOML) ```toml group = [ @@ -573,21 +583,7 @@ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. -### analyzed.toml (TOML) - -```toml -[image] - reference = "" - -[metadata] -# layer metadata -``` - -Where: -- `image.reference` MUST be EITHER a digest reference to an image in a docker registry or the ID of an image in a docker daemon -- `metadata` MUST be the TOML representation fo the layer [metadata label](#layer-metadata-label-json) - -### metadata.toml (TOML) +#### metadata.toml (TOML) ```toml [[buildpacks]] id = "" @@ -608,8 +604,26 @@ paths = [""] Where: - Both `id` and `version` MUST be present for each buildpack. +- Where `processes` contains the complete set of processes contributed by all buildpacks +- Where `processes` contains the complete set of slice defined by all buildpacks +- Where `bom` contains the Bill of Materials + +#### order.toml (TOML) + +```toml +[[order]] +[[order.group]] +id = "" +version = "" +optional = false +``` -### project-metadata.toml (TOML) +Where: + +- Both `id` and `version` MUST be present for each buildpack object in a group. +- The value of `optional` MUST default to false if not specified. + +#### project-metadata.toml (TOML) ```toml [source] @@ -628,7 +642,7 @@ Where: - `version`, if present, SHOULD contain data uniquely identifying the particular version of the provided source - `metadata` MAY contain additional arbitrary data about the provided source -### stack.toml (TOML) +#### stack.toml (TOML) ```toml [run-image] @@ -645,7 +659,61 @@ Where: * SHOULD reference an image with ID identical to that of `run-image.image` - `run-image.image` and `run-image.mirrors.[]` SHOULD each refer to a unique registry -### lifecycle metadata label (JSON) +### Labels + +#### `io.buildpacks.build.metadata` (JSON) + +``` +{ + "processes": [ + { + "type": "", + "command": "", + "args": [ + "" + ], + "direct": false + } + ], + "buildpacks": [ + { + "id": "", + "version": "" + } + ], + "bom": [ + { + "name": "", + "version": "", + "metadata": { + # arbitrary buildpack provided metadata + }, + "buildpack": { + "id": "", + "version": "" + } + }, + ], + "launcher": { + "version": "", + "source": { + "git": { + "repository": "", + "commit": "" + } + } + } +} +``` +Where: +- `processes` MUST contain all buildpack contributed proccesses +- `buildpacks` MUST contain the detected group +- `bom` MUST contain the Bill of Materials +- `launcher.version` SHOULD contain the version of the `launcher` binary included in the app +- `luancher.source.git.repository` SHOULD contain the git repository containing the `launcher` source code +- `luancher.source.git.commit` SHOULD contain the git commit from which the given `launcher` was built + +#### `io.buildpacks.lifecycle.metadata` (JSON) ```json { @@ -684,50 +752,37 @@ Where: } } ``` - -### build metadata label (JSON) +Where: +- `app` MUST contains one entry per app slice layer where + - `sha` MUST contain the digest of the uncompressed layer +- `config.sha` MUST the digest of the uncompressed layer containing launcher config +- `launcher.sha` MUST the digest of the uncompressed layer containing the launcher binary +- `buildpacks` MUST contain one entry per buildpack that participated in the build where + - `key` is required and MUST contain the buildpack ID + - `version` is required and MUST contain the buidpack Version + - `layers` is required and MUST contain one entry per launch layer contributed by the given buildpack where + - the key MUST be the name of the layer + - the value MUST contain JSON representation of the `layer.toml` with an additional `sha` key, containing the digest of the uncompressed layer +- `run-image.topLayer` must contain the uncompressed digest of the top layer of the run-image +- `run-image.reference` MUST uniquely identify the run image. It MAY contain one of the following + - an image ID (the digest of the uncompressed config blob) + - a digest reference to a manifest stored in an OCI image registry +- `stack` MUST contain the json representation of `stack.toml` + +#### `io.buildpacks.project.metadata` (JSON) ``` { - "processes": [ - { - "type": "", - "command": "", - "args": [ - "" - ], - "direct": false - } - ], - "buildpacks": [ - { - "id": "", - "version": "" - } - ], - "bom": [ - { - "name": "", - "version": "", - "metadata": { - # arbitrary buildpack provided metadata - }, - "buildpack": { - "id": "", - "version": "" - } + "source": { + "type": "", - "source": { - "git": { - "repository": "", - "commit": "" - } + "metadata": { + # arbitrary data } } } ``` - +This label MUST contain the JSON representation of [`project-metadata.toml`](#project-metadatatoml-toml) From 32e77200a62ef8b6841ce45b288a839f69d84f4b Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Tue, 16 Jun 2020 17:07:43 -0400 Subject: [PATCH 20/38] some indentation fixes Signed-off-by: Emily Casey --- platform.md | 50 ++++++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/platform.md b/platform.md index 05efd6e1..778f0d48 100644 --- a/platform.md +++ b/platform.md @@ -20,18 +20,18 @@ Examples of a platform might include: 1. [Lifecycle Interface](#lifecycle-interface) 1. [Platform API Compatibility](#platform-api-compatibility) 1. [Operations](#operations) - 1. [Build](#build) - 1. [Rebase](#rebase) - 1. [Launch](#launch) + 1. [Build](#build) + 1. [Rebase](#rebase) + 1. [Launch](#launch) 1. [Usage](#usage) - 1. [detector](#detector) - 1. [analyzer](#analyzer) - 1. [restorer](#restorer) - 1. [builder](#builder) - 1. [exporter](#exporter) - 1. [creator](#creator) - 1. [rebaser](#rebaser) - 1. [launcher](#launcher) + 1. [detector](#detector) + 1. [analyzer](#analyzer) + 1. [restorer](#restorer) + 1. [builder](#builder) + 1. [exporter](#exporter) + 1. [creator](#creator) + 1. [rebaser](#rebaser) + 1. [launcher](#launcher) 1. [Buildpacks](#buildpacks) 1. [Buildpacks Directory Layout](#buildpacks-directory-layout) 1. [Security Considerations](#security-considerations) @@ -40,16 +40,16 @@ Examples of a platform might include: 1. [Caching](#caching) 1. [Data Format](#data-format) 1. [Files](#files) - 1. [analyzed.toml (TOML)](#analyzedtoml-toml) - 1. [group.toml (TOML)](#grouptoml-toml) - 1. [metadata.toml (TOML)](#metadatatoml-toml) - 1. [order.toml (TOML)](#ordertoml-toml) - 1. [project-metadata.toml (TOML)](#project-metadatatoml-toml) - 1. [stack.toml (TOML)](#group.toml-(toml)) + 1. [analyzed.toml (TOML)](#analyzedtoml-toml) + 1. [group.toml (TOML)](#grouptoml-toml) + 1. [metadata.toml (TOML)](#metadatatoml-toml) + 1. [order.toml (TOML)](#ordertoml-toml) + 1. [project-metadata.toml (TOML)](#project-metadatatoml-toml) + 1. [stack.toml (TOML)](#group.toml-(toml)) 1. [Labels](#labels) - 1. [io.buildpacks.build.metadata (JSON)](#iobuildpacksbuildmetadata-json) - 1. [io.buildpacks.lifecycle.metadata (JSON)](#iobuildpackslifecyclemetadata-json) - 1. [io.buildpacks.project.metadata (JSON)](#iobuildpacksprojectmetadata-json) + 1. [io.buildpacks.build.metadata (JSON)](#iobuildpacksbuildmetadata-json) + 1. [io.buildpacks.lifecycle.metadata (JSON)](#iobuildpackslifecyclemetadata-json) + 1. [io.buildpacks.project.metadata (JSON)](#iobuildpacksprojectmetadata-json) ## Platform API Version @@ -706,7 +706,7 @@ Where: } ``` Where: -- `processes` MUST contain all buildpack contributed proccesses +- `processes` MUST contain all buildpack contributed processes - `buildpacks` MUST contain the detected group - `bom` MUST contain the Bill of Materials - `launcher.version` SHOULD contain the version of the `launcher` binary included in the app @@ -753,16 +753,18 @@ Where: } ``` Where: -- `app` MUST contains one entry per app slice layer where +- `app` MUST contain one entry per app slice layer where - `sha` MUST contain the digest of the uncompressed layer - `config.sha` MUST the digest of the uncompressed layer containing launcher config - `launcher.sha` MUST the digest of the uncompressed layer containing the launcher binary - `buildpacks` MUST contain one entry per buildpack that participated in the build where - `key` is required and MUST contain the buildpack ID - `version` is required and MUST contain the buidpack Version - - `layers` is required and MUST contain one entry per launch layer contributed by the given buildpack where - - the key MUST be the name of the layer + - `layers` is required and MUST contain one entry per launch layer contributed by the given buildpack. + - For each entry in `layers`: + - the key MUST be the name of the layer - the value MUST contain JSON representation of the `layer.toml` with an additional `sha` key, containing the digest of the uncompressed layer + - the value MUST contain an additional `sha` key, containing the digest of the uncompressed layer - `run-image.topLayer` must contain the uncompressed digest of the top layer of the run-image - `run-image.reference` MUST uniquely identify the run image. It MAY contain one of the following - an image ID (the digest of the uncompressed config blob) From 355250ad1a7bc4005b494d5d6126c2c98846e4cf Mon Sep 17 00:00:00 2001 From: E Casey Date: Wed, 24 Jun 2020 17:57:03 -0400 Subject: [PATCH 21/38] Apply suggestions from code review Co-authored-by: Ben Hale Co-authored-by: Joe Kutner Signed-off-by: Emily Casey --- platform.md | 135 ++++++++++++++++++++++++++-------------------------- 1 file changed, 67 insertions(+), 68 deletions(-) diff --git a/platform.md b/platform.md index 778f0d48..2b30cf4b 100644 --- a/platform.md +++ b/platform.md @@ -125,9 +125,9 @@ However, mixins MAY consist of any changes that follow the [Compatibility Guaran ## Lifecycle Interface ### Platform API Compatibility -The platform SHOULD set `CNB_PLATFORM_API=[.[.]` in the lifecycle's execution environment -IF `CNB_PLATFORM_API=[.[.]` is set in the lifecycle's execution environment, the lifecycle MUST do one the following before attempting to parse other inputs 1. conform usage and behavior to the given version of the platform API specification 1. fail if it does not support the requested API version @@ -152,10 +152,10 @@ A platform MUST execute these phases EITHER by invoking the following phase-spec OR by executing `/cnb/lifecycle/creator`. #### Rebase -Run image rebasing allows for fast stack updates for already-exported OCI images without require a rebuild. A rebase requires minimal data transfer when those images are stored on a Docker registry. +Run image rebasing allows for fast stack updates for already-exported OCI images without requiring a rebuild. A rebase requires minimal data transfer when those images are stored on a Docker registry. When a new stack version with the same stack ID is available, the app layers and launch layers SHOULD be rebased on the new run image by updating the image's configuration to point at the new run image. -To rebase an app image A platform MUST execute the `/cnb/lifecycle/rebaser` OR perform an equivalent operation. +To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` OR perform an equivalent operation. #### Launch `/cnb/lifecycle/launcher` is responsible for launching user and buildpack provided processes in the correct execution environment. @@ -187,21 +187,21 @@ Usage: | Input | Environment Variable| Default Value | Description |---------------|---------------------|-----------------|---------------------- -| `` | `CNB_APP_DIR` | /workspace | Path to application directory -| `` | `CNB_BUILDPACKS_DIR` | /cnb/buildpacks | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) -| `` | `CNB_GROUP_PATH` | ./group.toml | Path to output group file -| `` | `CNB_LOG_LEVEL` | info | Log Level -| `` | `CNB_ORDER_PATH` | ./order.toml | Path to order definition ( see [order.toml (TOML)](#order.toml-(toml))) -| `` | `CNB_PLAN_PATH` | ./plan.toml | Path to output build plan file -| `` | `CNB_PLATFORM_DIR` | /platform | Path to platform directory +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to output group file +| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| `` | `CNB_ORDER_PATH` | `./order.toml` | Path to order definition (see [`order.toml` (TOML)](#order.toml-(toml))) +| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file +| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | Output | Description |--------------------|---------------------------------------------- | [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| `` | Detected buildpack group ( see [group.toml (TOML)](#group.toml-(toml))) -| `` | Resolved Build Plan ( see data format in [Buildpack Interface Specification](buildpack.md)) +| `` | Detected buildpack group (see [`group.toml` (TOML)](#group.toml-(toml))) +| `` | Resolved Build Plan (see data format in [Buildpack Interface Specification](buildpack.md)) #### `analyzer` Usage: @@ -220,15 +220,15 @@ Usage: | Input | Environment Variable | Default Value | Description |----------------|-----------------------|-------------------|---------------------- -| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to output analysis metadata ( see [analyzed.toml (TOML)](#analyzed.toml-(toml)) -| `` | `CNB_CACHE_DIR` | - | Location of cache, provided as a directory -| ``| `CNB_CACHE_IMAGE` | - | Location of cache, provided as an image -| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition ( see [group.toml (TOML)](#group.toml-(toml))) -| `` | - | - | Image reference to be analyzed (usually the result of the previous build) +| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to output analysis metadata (see [`analyzed.toml` (TOML)](#analyzed.toml-(toml)) +| `` | `CNB_CACHE_DIR` | | Location of cache, provided as a directory +| ``| `CNB_CACHE_IMAGE` | | Location of cache, provided as an image +| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml` (TOML)](#group.toml-(toml))) +| `` | | | Image reference to be analyzed (usually the result of the previous build) | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory | ``| `CNB_SKIP_LAYERS` | `false` | Do not write layer metadata -| `` | `CNB_USER_ID` | - | User that build phase will run as +| `` | `CNB_USER_ID` | | User that build phase will run as | `` | `CNB_USE_DAEMON` | `false` | Analyze image from docker daemon IF `` is `false`, `` MUST be a valid registry reference @@ -240,8 +240,8 @@ The lifecycle MUST accept valid references to non-existent images without error. | [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| `` | Analysis metadata ( see [analyzed.toml (TOML)](#analyzed.toml-(toml)) -| `//.toml` files | Contain analyzed layer metadata ( see data format in [Buildpack Interface Specification](buildpack.md)) +| `` | Analysis metadata (see [`analyzed.toml` (TOML)](#analyzed.toml-(toml)) +| `//.toml` files | Contain analyzed layer metadata (see data format in [Buildpack Interface Specification](buildpack.md)) | `//.sha` files | Contain sha256 of the uncompressed layer The lifecycle MUST write analysis metadata if `` is accessible. @@ -273,14 +273,14 @@ Usage: | Input | Environment Variable | Default Value | Description |----------------|-----------------------|-----------------|---------------------- -| `` | `CNB_CACHE_DIR` | - | Path to a cache directory -| ``| `CNB_CACHE_IMAGE` | - | Reference to a cache image in an OCI image registry -| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition ( see [group.toml (TOML)](#group.toml-(toml))) +| `` | `CNB_CACHE_DIR` | | Path to a cache directory +| ``| `CNB_CACHE_IMAGE` | | Reference to a cache image in an OCI image registry +| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml` (TOML)](#group.toml-(toml))) | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory -| `` | `CNB_USER_ID` | - | User that build phase will run as +| `` | `CNB_USER_ID` | | User that build phase will run as | `//.sha` || | Files Containing the sha256 of the uncompressed layers -| `//.toml` || | Files Containing analyzed layer metadata ( see data format in [Buildpack Interface Specification](buildpack.md)) +| `//.toml` || | Files Containing analyzed layer metadata (see data format in [Buildpack Interface Specification](buildpack.md)) | Output | Description |------------------------------------|---------------------------------------------- @@ -290,7 +290,7 @@ Usage: | `///*` | Restored layer contents ##### Layer restoration -For each layer metadata file found in the `` the lifecycle: +For each layer metadata file found in the `` directory the lifecycle: 1. MUST restore cached layer contents IF the cache contains a layer with a matching ID 1. MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a matching layer @@ -312,9 +312,9 @@ Usage: |--------------|---------------------|-----------------|---------------------- | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file ( see [group.toml (TOML)](#group.toml-(toml))) +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml` (TOML)](#group.toml-(toml))) | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory -| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file ( see data format in [Buildpack Interface Specification](buildpack.md)) +| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file (see data format in [Buildpack Interface Specification](buildpack.md)) | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | Output | Description @@ -324,7 +324,7 @@ Usage: | `/dev/stderr` | Logs (warnings, errors) | `//` | Layer contents buildpacks (see [Buildpack Interface Specfication](buildpack.md) | `//.toml` | Layer metadata (see [Buildpack Interface Specfication](buildpack.md) -| `/config/metadata.toml` | Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) +| `/config/metadata.toml` | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) The lifecycle SHALL execute all buildpacks in the order defined in `` according to the rules of the [Buildpack Interface Specification](buildpack.md). The lifecycle SHALL add all invoked buildpacks to `[[buildpacks]]` in `/config/metadata.toml`. @@ -354,24 +354,24 @@ Usage: | Input | Environment Variable | Default Value | Description |---------------------|-----------------------|---------------------|--------------------------------------- -| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to analysis metadata ( see [analyzed.toml (TOML)](#analyzed.toml-(toml)) +| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to analysis metadata (see [`analyzed.toml` (TOML)](#analyzed.toml-(toml)) | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory -| `` | `CNB_CACHE_DIR` | - | Path to a cache directory -| `` | `CNB_CACHE_IMAGE` | - | Reference to a cache image in an OCI image registry +| `` | `CNB_CACHE_DIR` | | Path to a cache directory +| `` | `CNB_CACHE_IMAGE` | | Reference to a cache image in an OCI image registry | `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon -| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file ( see [group.toml (TOML)](#group.toml-(toml))) -| `` | - | - | An -| `` | `CNB_LAUNCH_CACHE_DIR`| - | Path to a cache directory containing launch layers -| `` | - | `/cnb/lifecycle/launcher` | Path to the `launcher` executable +| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml` (TOML)](#group.toml-(toml))) +| `` | | | An +| `` | `CNB_LAUNCH_CACHE_DIR`| | Path to a cache directory containing launch layers +| `` | | `/cnb/lifecycle/launcher` | Path to the `launcher` executable | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory -| `` | `CNB_LOG_LEVEL` ` | info | Log Level -| `` | `CNB_PROCESS_TYPE` | - | Default process type to set in the exported image -| ``| `CNB_PROCESS_TYPE` | `./project-metadata.toml` | Path to a project metadata file (see [project-metadata.toml (TOML)](#project-metadata.toml) -| `` | `CNB_RUN_IMAGE` | derived from | Run image reference -| `` | `CNB_STACK_PATH` | `/cnb/stack.toml` | Path to stack file (see [stack.toml (TOML)](#stack.toml-(toml)) -| `` | `CNB_USER_ID` | - | User that build phase will run as -| `/config/metadata.toml` |-|-| Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) +| `` | `CNB_LOG_LEVEL` ` | `info` | Log Level +| `` | `CNB_PROCESS_TYPE` | | Default process type to set in the exported image +| ``| `CNB_PROCESS_TYPE` | `./project-metadata.toml` | Path to a project metadata file (see [`project-metadata.toml` (TOML)](#project-metadata.toml) +| `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference +| `` | `CNB_STACK_PATH` | `/cnb/stack.toml` | Path to stack file (see [`stack.toml` (TOML)](#stack.toml-(toml)) +| `` | `CNB_USER_ID` | | User that build phase will run as +| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) At least one `` must be provided Each `` MUST be a valid OCI image registry tag reference @@ -382,10 +382,10 @@ If `` is not provided by the platform the value will be derived from | Output | Description |--------------------|---------------------------------------------- -| `[exit status]` | Success (0), or error (1+) +| `[exit status]` | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| `` | Exported app image (see [Buildpack Interface Specfication](buildpack.md) +| `` | Exported app image (see [Buildpack Interface Specfication](buildpack.md) The lifecycle SHALL write the same app image to each `` tag The app image: @@ -441,7 +441,7 @@ Running `creator` SHALL be equivalent to running `detector`, `analzyer`, `restor |-------------------|---------------------|--------------|---------------------- | ``| `CNB_PREVIOUS_IMAGE`| `` | Image reference to be analyzed (usually the result of the previous build) | `` | `CNB_SKIP_RESTORE` | `false` | Do not write layer metadata or restore cached layers -| `...` | - | - | Additional tag to apply to exported image +| `...` | | | Additional tag to apply to exported image If `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. If the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` @@ -462,11 +462,11 @@ Usage: | Input | Environment Variable | Default Value | Description |---------------------|-----------------------|-----------------------|--------------------------------------- | `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon -| `` | `CNB_GROUP_ID` | - | Group of user that build phase will run as -| `` | - | - | App image to rebase -| `` | `CNB_LOG_LEVEL` ` | info | Log Level -| `` | `CNB_RUN_IMAGE` | derived from ``| Run image reference -| `` | `CNB_USER_ID` | - | User that build phase will run as +| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as +| `` | | | App image to rebase +| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference +| `` | `CNB_USER_ID` | | User that build phase will run as At least one `` must be provided Each `` MUST be a valid OCI image registry tag reference @@ -502,10 +502,10 @@ Usage: | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory | `` | `CNB_PROCESS_TYPE` | `web` | `type` of process to launch | `` | | `false` | Execution strategy for user provided process -| `` | | - | User-provided command -| `...` | | - | Arguments to user-provided command -| `/config/metadata.toml` |- |-| Build metadata (see [metadata.toml (TOML)](#metdata.toml-(toml)) -| `///` |- |-| Launch Layers +| `` | | | User-provided command +| `...` | | | Arguments to user-provided command +| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) +| `///` | | | Launch Layers If `$1` is `--` `` is `true` and `` SHALL be `${@2:}` If `$1` is anything other than `--`, `` is `false`, and `` SHALL be `$@` @@ -557,7 +557,7 @@ Cache locality and availability MAY vary between platforms. ### Files -#### analyzed.toml (TOML) +#### `analyzed.toml` (TOML) ```toml [image] @@ -571,7 +571,7 @@ Where: - `image.reference` MUST be EITHER a digest reference to an image in a docker registry or the ID of an image in a docker daemon - `metadata` MUST be the TOML representation of the layer [metadata label](#layer-metadata-label-json) -#### group.toml (TOML) +#### `group.toml` (TOML) ```toml group = [ @@ -583,7 +583,7 @@ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. -#### metadata.toml (TOML) +#### `metadata.toml` (TOML) ```toml [[buildpacks]] id = "" @@ -608,7 +608,7 @@ Where: - Where `processes` contains the complete set of slice defined by all buildpacks - Where `bom` contains the Bill of Materials -#### order.toml (TOML) +#### `order.toml` (TOML) ```toml [[order]] @@ -623,7 +623,7 @@ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. - The value of `optional` MUST default to false if not specified. -#### project-metadata.toml (TOML) +#### `project-metadata.toml` (TOML) ```toml [source] @@ -642,7 +642,7 @@ Where: - `version`, if present, SHOULD contain data uniquely identifying the particular version of the provided source - `metadata` MAY contain additional arbitrary data about the provided source -#### stack.toml (TOML) +#### `stack.toml` (TOML) ```toml [run-image] @@ -663,7 +663,7 @@ Where: #### `io.buildpacks.build.metadata` (JSON) -``` +```json { "processes": [ { @@ -773,7 +773,7 @@ Where: #### `io.buildpacks.project.metadata` (JSON) -``` +```json { "source": { "type": " Date: Fri, 26 Jun 2020 11:32:19 -0400 Subject: [PATCH 22/38] Use vscode-markdown to generate index Signed-off-by: Emily Casey --- buildpack.md | 2 +- platform.md | 84 ++++++++++++++++++++++++++++------------------------ 2 files changed, 46 insertions(+), 40 deletions(-) diff --git a/buildpack.md b/buildpack.md index 6e60bb15..7373323a 100644 --- a/buildpack.md +++ b/buildpack.md @@ -21,7 +21,7 @@ This is accomplished in two phases: 2. **Development,** where the lifecycle uses those buildpacks to create a containerized development environment. ## Table of Contents - + 1. [Buildpack Interface](#buildpack-interface) 1. [Key](#key) 2. [Detection](#detection) diff --git a/platform.md b/platform.md index 2b30cf4b..b2bc0aca 100644 --- a/platform.md +++ b/platform.md @@ -11,45 +11,51 @@ Examples of a platform might include: 1. A cloud application platform that uses buildpacks to build source code before deployment ## Table of Contents - -1. [Platform API Version](#platform-api-version) -1. [Stacks](#stacks) - 1. [Compatibility Guarantees](#compatibility-guarantees) - 1. [Build Image](#build-image) - 1. [Run Image](#run-image) -1. [Lifecycle Interface](#lifecycle-interface) - 1. [Platform API Compatibility](#platform-api-compatibility) - 1. [Operations](#operations) - 1. [Build](#build) - 1. [Rebase](#rebase) - 1. [Launch](#launch) - 1. [Usage](#usage) - 1. [detector](#detector) - 1. [analyzer](#analyzer) - 1. [restorer](#restorer) - 1. [builder](#builder) - 1. [exporter](#exporter) - 1. [creator](#creator) - 1. [rebaser](#rebaser) - 1. [launcher](#launcher) -1. [Buildpacks](#buildpacks) - 1. [Buildpacks Directory Layout](#buildpacks-directory-layout) -1. [Security Considerations](#security-considerations) -1. [Additional Guidance](#additional-guidance) - 1. [Environment](#environment) - 1. [Caching](#caching) -1. [Data Format](#data-format) - 1. [Files](#files) - 1. [analyzed.toml (TOML)](#analyzedtoml-toml) - 1. [group.toml (TOML)](#grouptoml-toml) - 1. [metadata.toml (TOML)](#metadatatoml-toml) - 1. [order.toml (TOML)](#ordertoml-toml) - 1. [project-metadata.toml (TOML)](#project-metadatatoml-toml) - 1. [stack.toml (TOML)](#group.toml-(toml)) - 1. [Labels](#labels) - 1. [io.buildpacks.build.metadata (JSON)](#iobuildpacksbuildmetadata-json) - 1. [io.buildpacks.lifecycle.metadata (JSON)](#iobuildpackslifecyclemetadata-json) - 1. [io.buildpacks.project.metadata (JSON)](#iobuildpacksprojectmetadata-json) + +- [Platform Interface Specification](#platform-interface-specification) + - [Table of Contents](#table-of-contents) + - [Platform API Version](#platform-api-version) + - [Terminology](#terminology) + - [Stacks](#stacks) + - [Compatibility Guarantees](#compatibility-guarantees) + - [Build Image](#build-image) + - [Run Image](#run-image) + - [Mixins](#mixins) + - [Lifecycle Interface](#lifecycle-interface) + - [Platform API Compatibility](#platform-api-compatibility) + - [Operations](#operations) + - [Build](#build) + - [Rebase](#rebase) + - [Launch](#launch) + - [Usage](#usage) + - [`detector`](#detector) + - [`analyzer`](#analyzer) + - [Layer analysis](#layer-analysis) + - [`restorer`](#restorer) + - [Layer restoration](#layer-restoration) + - [`builder`](#builder) + - [`exporter`](#exporter) + - [`creator`](#creator) + - [`rebaser`](#rebaser) + - [`launcher`](#launcher) + - [Buildpacks](#buildpacks) + - [Buildpacks Directory Layout](#buildpacks-directory-layout) + - [Security Considerations](#security-considerations) + - [Additional Guidance](#additional-guidance) + - [Environment](#environment) + - [Caching](#caching) + - [Data Format](#data-format) + - [Files](#files) + - [`analyzed.toml` (TOML)](#analyzedtoml-toml) + - [`group.toml` (TOML)](#grouptoml-toml) + - [`metadata.toml` (TOML)](#metadatatoml-toml) + - [`order.toml` (TOML)](#ordertoml-toml) + - [`project-metadata.toml` (TOML)](#project-metadatatoml-toml) + - [`stack.toml` (TOML)](#stacktoml-toml) + - [Labels](#labels) + - [`io.buildpacks.build.metadata` (JSON)](#iobuildpacksbuildmetadata-json) + - [`io.buildpacks.lifecycle.metadata` (JSON)](#iobuildpackslifecyclemetadata-json) + - [`io.buildpacks.project.metadata` (JSON)](#iobuildpacksprojectmetadata-json) ## Platform API Version From eb5aa02e9d2f000b13b6860e868fd84327149640 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Fri, 26 Jun 2020 11:37:24 -0400 Subject: [PATCH 23/38] Fixes stack.toml data format Signed-off-by: Emily Casey --- platform.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/platform.md b/platform.md index b2bc0aca..c14f1793 100644 --- a/platform.md +++ b/platform.md @@ -653,8 +653,7 @@ Where: ```toml [run-image] image = "" - -[run-image.mirrors] = ["", ""] + mirrors = ["", ""] ``` Where: From 19643d72bb7a3dae879e08c50d00971edd020e23 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Fri, 26 Jun 2020 12:59:58 -0400 Subject: [PATCH 24/38] Markdown style fixes * Use bullet points to break up hard to read paragraphs * Consistent use of - vs * * Consistent capitalization of IF * Consistent captialization of bullet point lines Signed-off-by: Emily Casey --- platform.md | 173 ++++++++++++++++++++++++++-------------------------- 1 file changed, 86 insertions(+), 87 deletions(-) diff --git a/platform.md b/platform.md index c14f1793..6d97d682 100644 --- a/platform.md +++ b/platform.md @@ -134,8 +134,8 @@ However, mixins MAY consist of any changes that follow the [Compatibility Guaran The platform SHOULD set `CNB_PLATFORM_API=[.]` in the lifecycle's execution environment IF `CNB_PLATFORM_API=[.]` is set in the lifecycle's execution environment, the lifecycle MUST do one the following before attempting to parse other inputs - 1. conform usage and behavior to the given version of the platform API specification - 1. fail if it does not support the requested API version + - Conform usage and behavior to the given version of the platform API specification + - Fail if it does not support the requested API version ### Operations @@ -171,10 +171,10 @@ To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` OR p The following is true of all lifecycle phases: -1. command line inputs ALWAYS take precedence over other inputs -1. IF `CNB_PLATFORM_API=[.[.` | `CNB_USER_ID` | | User that build phase will run as | `` | `CNB_USE_DAEMON` | `false` | Analyze image from docker daemon -IF `` is `false`, `` MUST be a valid registry reference -IF `` is `true`, `` MUST be a valid registry reference OR a docker image ID -The lifecycle MUST accept valid references to non-existent images without error. +- IF `` is `false`, `` MUST be a valid registry reference +- IF `` is `true`, `` MUST be a valid registry reference OR a docker image ID +- The lifecycle MUST accept valid references to non-existent images without error. | Output | Description |--------------------|---------------------------------------------- @@ -250,20 +250,20 @@ The lifecycle MUST accept valid references to non-existent images without error. | `//.toml` files | Contain analyzed layer metadata (see data format in [Buildpack Interface Specification](buildpack.md)) | `//.sha` files | Contain sha256 of the uncompressed layer -The lifecycle MUST write analysis metadata if `` is accessible. -If `` is `false` the lifecycle MUST analyze layers created by any buildpack present in the provided ``. -If `` is `true` the lifecycle MUST NOT analyze layers. +- The lifecycle MUST write analysis metadata if `` is accessible. +- IF `` is `false` the lifecycle MUST analyze layers created by any buildpack present in the provided ``. +- IF `` is `true` the lifecycle MUST NOT analyze layers. ##### Layer analysis When analyzing a given layer the lifecycle SHALL: -* IF `build=true`, `cache=false`: -1. do nothing -* ELSE IF `launch=true`: -1. write layer metadata read from the analyzed image to `//.toml` -2. write the sha256 of the uncompressed layer from the analyzed image to `//.sha` -* ELSE IF `cache=true`: -1. write layer metadata read from the cache to `//.toml` -2. write the sha256 of the uncompressed layer from the cache to `//.sha` +- IF `build=true`, `cache=false`: + - Do nothing +- ELSE IF `launch=true`: + - Write layer metadata read from the analyzed image to `//.toml` + - Write the sha256 of the uncompressed layer from the analyzed image to `//.sha` +- ELSE IF `cache=true`: + - Write layer metadata read from the cache to `//.toml` + - Write the sha256 of the uncompressed layer from the cache to `//.sha` #### `restorer` Usage: @@ -297,8 +297,8 @@ Usage: ##### Layer restoration For each layer metadata file found in the `` directory the lifecycle: -1. MUST restore cached layer contents IF the cache contains a layer with a matching ID -1. MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a matching layer +- MUST restore cached layer contents IF the cache contains a layer with a matching ID +- MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a matching layer #### `builder` The platform MUST execute `builder` in the build environment @@ -332,9 +332,9 @@ Usage: | `//.toml` | Layer metadata (see [Buildpack Interface Specfication](buildpack.md) | `/config/metadata.toml` | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) -The lifecycle SHALL execute all buildpacks in the order defined in `` according to the rules of the [Buildpack Interface Specification](buildpack.md). -The lifecycle SHALL add all invoked buildpacks to `[[buildpacks]]` in `/config/metadata.toml`. -The lifecycle SHALL aggregate all `processes`, `slices` and BOM entries returned by buildpacks in `/config/metadata.toml`. +- The lifecycle SHALL execute all buildpacks in the order defined in `` according to the rules of the [Buildpack Interface Specification](buildpack.md). +- The lifecycle SHALL add all invoked buildpacks to `[[buildpacks]]` in `/config/metadata.toml`. +- The lifecycle SHALL aggregate all `processes`, `slices` and BOM entries returned by buildpacks in `/config/metadata.toml`. #### `exporter` Usage: @@ -379,12 +379,12 @@ Usage: | `` | `CNB_USER_ID` | | User that build phase will run as | `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) -At least one `` must be provided -Each `` MUST be a valid OCI image registry tag reference -If `` is `false` and more than one `` is provided they MUST refer to the same registry -If `` is not provided by the platform the value will be derived from the contents of `stack` -* If any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` -* If none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` +- At least one `` must be provided +- Each `` MUST be a valid OCI image registry tag reference +- IF `` is `false` and more than one `` is provided they MUST refer to the same registry +- IF `` is not provided by the platform the value will be derived from the contents of `stack` + - IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` + - IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` | Output | Description |--------------------|---------------------------------------------- @@ -393,28 +393,28 @@ If `` is not provided by the platform the value will be derived from | `/dev/stderr` | Logs (warnings, errors) | `` | Exported app image (see [Buildpack Interface Specfication](buildpack.md) -The lifecycle SHALL write the same app image to each `` tag -The app image: -* MUST be an extension of the `` - * all run-image layers and config values SHALL be preserved unless they conflict with the following -* MUST contain all buildpack provided launch layers as determined by the [Buildpack Interface Specfication](buildpack.md) -* MUST contain one or more app layers as determined by the [Buildpack Interface Specfication](buildpack.md) -* MUST contain a layer that includes the `` -* MUST contain a layer that includes `/config/metadata.toml` -* MUST have `ENTRYPOINT=` -* MUST contain the following `Env` entries - * `"CNB_LAYERS_DIR="` - * `"CNB_APP_DIR="` -* MUST contain the following `Env` entry, IF `` is set - * `"CNB_PROCESS_TYPE="` -* MUST contain the following labels -- `io.buildpacks.lifecycle.metadata`: see [lifecycle metadata label (JSON)](#lifecycle-metadata-label-(json)) -- `io.buildpacks.project.metadata`: the value of which SHALL be the json representation `` -- `io.buildpacks.build.metadata`: see [build metadata (JSON)](#build-metadata-label-(json)) - -IF a cache is provided the lifecycle SHALL -* write `io.buildpacks.cache.metadata` to the cache -* ensure all `cache=true` layers are written to the cache +- The lifecycle SHALL write the same app image to each `` tag +- The app image: + - MUST be an extension of the `` + - All run-image layers and config values SHALL be preserved unless they conflict with the following + - MUST contain all buildpack provided launch layers as determined by the [Buildpack Interface Specfication](buildpack.md) + - MUST contain one or more app layers as determined by the [Buildpack Interface Specfication](buildpack.md) + - MUST contain a layer that includes the `` + - MUST contain a layer that includes `/config/metadata.toml` + - MUST have `ENTRYPOINT=` + - MUST contain the following `Env` entries + - `"CNB_LAYERS_DIR="` + - `"CNB_APP_DIR="` + - MUST contain the following `Env` entry, IF `` is set + - `"CNB_PROCESS_TYPE="` + - MUST contain the following labels + - `io.buildpacks.lifecycle.metadata`: see [lifecycle metadata label (JSON)](#lifecycle-metadata-label-(json)) + - `io.buildpacks.project.metadata`: the value of which SHALL be the json representation `` + - `io.buildpacks.build.metadata`: see [build metadata (JSON)](#build-metadata-label-(json)) + +IF a cache is provided the lifecycle SHALL: +- Write `io.buildpacks.cache.metadata` to the cache +- Ensure all `cache=true` layers are written to the cache #### `creator` Usage: @@ -449,8 +449,8 @@ Running `creator` SHALL be equivalent to running `detector`, `analzyer`, `restor | `` | `CNB_SKIP_RESTORE` | `false` | Do not write layer metadata or restore cached layers | `...` | | | Additional tag to apply to exported image -If `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. -If the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` +- IF `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. +- IF the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` Outputs produced by `creator` and identical to those produced by `exporter`. @@ -474,12 +474,12 @@ Usage: | `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference | `` | `CNB_USER_ID` | | User that build phase will run as -At least one `` must be provided -Each `` MUST be a valid OCI image registry tag reference -If `` is `false` and more than one `` is provided they MUST refer to the same registry -If `` is not provided by the platform the value will be derived from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on `` -* If any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` -* If none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` +- At least one `` must be provided +- Each `` MUST be a valid OCI image registry tag reference +- IF `` is `false` and more than one `` is provided they MUST refer to the same registry +- IF `` is not provided by the platform the value will be derived from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on `` + - IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` + - IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` | Output | Description |--------------------|---------------------------------------------- @@ -488,13 +488,13 @@ If `` is not provided by the platform the value will be derived from | `/dev/stderr` | Logs (warnings, errors) | `` | Rebased app image (see [Buildpack Interface Specfication](buildpack.md) -The lifecycle SHALL write the same app image to each `` tag -The rebased app image SHALL be identical to ``, with the following modifications: -* Run image layers SHALL be defined as Layers in `` up to and including the layer with diff ID matching the value of `run-image.top-layer` from the `io.buildpacks.lifecycle.metadata` label -* Run image layers SHALL be replaced with the layers from the new `` -* The value of `io.buildpacks.lifecycle.metadata` SHALL be modified as follows - * `run-image.reference` SHALL uniquely identify `` - * `run-image.top-layer` SHALL be set to the uncompressed digest of the top layer in `` +- The lifecycle SHALL write the same app image to each `` tag +- The rebased app image SHALL be identical to ``, with the following modifications: + - Run image layers SHALL be defined as Layers in `` up to and including the layer with diff ID matching the value of `run-image.top-layer` from the `io.buildpacks.lifecycle.metadata` label + - Run image layers SHALL be replaced with the layers from the new `` + - The value of `io.buildpacks.lifecycle.metadata` SHALL be modified as follows + - `run-image.reference` SHALL uniquely identify `` + - `run-image.top-layer` SHALL be set to the uncompressed digest of the top layer in `` #### `launcher` Usage: @@ -513,12 +513,11 @@ Usage: | `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) | `///` | | | Launch Layers -If `$1` is `--` `` is `true` and `` SHALL be `${@2:}` -If `$1` is anything other than `--`, `` is `false`, and `` SHALL be `$@` -If `` - -If `` are NOT provided `launcher` SHALL select the process with `type` equal to `` from `/config/metadata.toml` and launch it as described in the [Buildpack Interface Specification](buildpack.md). -If `` are provided `launcher` SHALL behaves as if it selected a process with the following process from `metadata.toml`. +- IF `$1` is `--` `` is `true` and `` SHALL be `${@2:}` +- IF `$1` is anything other than `--`, `` is `false`, and `` SHALL be `$@` +- IF `` +- IF `` are NOT provided `launcher` SHALL select the process with `type` equal to `` from `/config/metadata.toml` and launch it as described in the [Buildpack Interface Specification](buildpack.md). +- IF `` are provided `launcher` SHALL behaves as if it selected a process with the following process from `metadata.toml`. ``` [[process]] direct = @@ -526,11 +525,11 @@ command = "" args = [""...] ``` -The lifecycle SHOULD replace the lifecycle process in memory without forking it. -The process execution environment SHALL be identical to the lifecycle execution environment, with the following exceptions: -* `CNB_APP_DIR` SHALL NOT be set in the process environment -* `CNB_LAYERS_DIR` SHALL NOT be set in the process environment -* `CNB_PROCESS_TYPE` SHALL NOT be set in the process environment +- The lifecycle SHOULD replace the lifecycle process in memory without forking it. +- The process execution environment SHALL be identical to the lifecycle execution environment, with the following exceptions: + - `CNB_APP_DIR` SHALL NOT be set in the process environment + - `CNB_LAYERS_DIR` SHALL NOT be set in the process environment + - `CNB_PROCESS_TYPE` SHALL NOT be set in the process environment ## Buildpacks @@ -555,7 +554,7 @@ The lifecycle MUST NOT assume that all platforms provide an identical environmen ### Caching -If caching is enabled the platform is responsible for providing the lifecycle with access to the correct cache. +IF caching is enabled the platform is responsible for providing the lifecycle with access to the correct cache. Whenever possible, the platform SHOULD provide the same cache to each rebuild of a given app image. Cache locality and availability MAY vary between platforms. @@ -660,8 +659,8 @@ Where: - `run-image.image` MAY be a reference to a run image in a docker registry - `run-image.mirrors` MUST NOT be present IF `run-image.image` is not present - `run-image.mirrors` MAY contain one or more tag references to run images in docker registries -- all `run-image.mirrors`: - * SHOULD reference an image with ID identical to that of `run-image.image` +- All `run-image.mirrors`: + - SHOULD reference an image with ID identical to that of `run-image.image` - `run-image.image` and `run-image.mirrors.[]` SHOULD each refer to a unique registry ### Labels @@ -767,13 +766,13 @@ Where: - `version` is required and MUST contain the buidpack Version - `layers` is required and MUST contain one entry per launch layer contributed by the given buildpack. - For each entry in `layers`: - - the key MUST be the name of the layer - - the value MUST contain JSON representation of the `layer.toml` with an additional `sha` key, containing the digest of the uncompressed layer - - the value MUST contain an additional `sha` key, containing the digest of the uncompressed layer + - The key MUST be the name of the layer + - The value MUST contain JSON representation of the `layer.toml` with an additional `sha` key, containing the digest of the uncompressed layer + - The value MUST contain an additional `sha` key, containing the digest of the uncompressed layer - `run-image.topLayer` must contain the uncompressed digest of the top layer of the run-image - `run-image.reference` MUST uniquely identify the run image. It MAY contain one of the following - - an image ID (the digest of the uncompressed config blob) - - a digest reference to a manifest stored in an OCI image registry + - An image ID (the digest of the uncompressed config blob) + - A digest reference to a manifest stored in an OCI image registry - `stack` MUST contain the json representation of `stack.toml` #### `io.buildpacks.project.metadata` (JSON) From f3c1c917598d0c0b9a02d2baca64f07af0f43d32 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Fri, 26 Jun 2020 14:26:17 -0400 Subject: [PATCH 25/38] Adds missing -log-level flags Signed-off-by: Emily Casey --- platform.md | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/platform.md b/platform.md index 6d97d682..14eaa09f 100644 --- a/platform.md +++ b/platform.md @@ -220,6 +220,7 @@ Usage: [-gid ] \ [-group ] \ [-layers ] \ + [-log-level ] \ [-skip-layers ] \ [-uid ] \ ``` @@ -229,13 +230,14 @@ Usage: | `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to output analysis metadata (see [`analyzed.toml` (TOML)](#analyzed.toml-(toml)) | `` | `CNB_CACHE_DIR` | | Location of cache, provided as a directory | ``| `CNB_CACHE_IMAGE` | | Location of cache, provided as an image +| `` | `CNB_USE_DAEMON` | `false` | Analyze image from docker daemon | `` | `CNB_GROUP_ID` | | Group of user that build phase will run as | `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml` (TOML)](#group.toml-(toml))) | `` | | | Image reference to be analyzed (usually the result of the previous build) | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_LOG_LEVEL` | `info` | Log Level | ``| `CNB_SKIP_LAYERS` | `false` | Do not write layer metadata | `` | `CNB_USER_ID` | | User that build phase will run as -| `` | `CNB_USE_DAEMON` | `false` | Analyze image from docker daemon - IF `` is `false`, `` MUST be a valid registry reference - IF `` is `true`, `` MUST be a valid registry reference OR a docker image ID @@ -274,6 +276,7 @@ Usage: [-gid ] \ [-group ] \ [-layers ] \ + [-log-level ] \ [-uid ] ``` @@ -284,6 +287,7 @@ Usage: | `` | `CNB_GROUP_ID` | | Group of user that build phase will run as | `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml` (TOML)](#group.toml-(toml))) | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_LOG_LEVEL` | `info` | Log Level | `` | `CNB_USER_ID` | | User that build phase will run as | `//.sha` || | Files Containing the sha256 of the uncompressed layers | `//.toml` || | Files Containing analyzed layer metadata (see data format in [Buildpack Interface Specification](buildpack.md)) @@ -310,16 +314,18 @@ Usage: [-buildpacks ] \ [-group ] \ [-layers ] \ + [-log-level ] \ [-plan ] \ [-platform ] ``` -| Input | Env | Default Value | Description -|--------------|---------------------|-----------------|---------------------- +| Input | Env | Default Value | Description +|----------------|-----------------------|-------------------|---------------------- | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) | `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml` (TOML)](#group.toml-(toml))) | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_LOG_LEVEL` | `info` | Log Level | `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file (see data format in [Buildpack Interface Specification](buildpack.md)) | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory @@ -371,10 +377,10 @@ Usage: | `` | `CNB_LAUNCH_CACHE_DIR`| | Path to a cache directory containing launch layers | `` | | `/cnb/lifecycle/launcher` | Path to the `launcher` executable | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory -| `` | `CNB_LOG_LEVEL` ` | `info` | Log Level +| `` | `CNB_LOG_LEVEL` ` | `info` | Log Level | `` | `CNB_PROCESS_TYPE` | | Default process type to set in the exported image | ``| `CNB_PROCESS_TYPE` | `./project-metadata.toml` | Path to a project metadata file (see [`project-metadata.toml` (TOML)](#project-metadata.toml) -| `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference +| `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference | `` | `CNB_STACK_PATH` | `/cnb/stack.toml` | Path to stack file (see [`stack.toml` (TOML)](#stack.toml-(toml)) | `` | `CNB_USER_ID` | | User that build phase will run as | `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) @@ -388,10 +394,10 @@ Usage: | Output | Description |--------------------|---------------------------------------------- -| `[exit status]` | Success (0), or error (1+) +| `[exit status]` | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| `` | Exported app image (see [Buildpack Interface Specfication](buildpack.md) +| `` | Exported app image (see [Buildpack Interface Specfication](buildpack.md) - The lifecycle SHALL write the same app image to each `` tag - The app image: @@ -465,14 +471,14 @@ Usage: [-uid ] ``` -| Input | Environment Variable | Default Value | Description -|---------------------|-----------------------|-----------------------|--------------------------------------- -| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon -| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as -| `` | | | App image to rebase -| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|------------------------|--------------------------------------- +| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon +| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as +| `` | | | App image to rebase +| `` | `CNB_LOG_LEVEL` | `info` | Log Level | `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference -| `` | `CNB_USER_ID` | | User that build phase will run as +| `` | `CNB_USER_ID` | | User that build phase will run as - At least one `` must be provided - Each `` MUST be a valid OCI image registry tag reference @@ -510,8 +516,8 @@ Usage: | `` | | `false` | Execution strategy for user provided process | `` | | | User-provided command | `...` | | | Arguments to user-provided command -| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) -| `///` | | | Launch Layers +| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) +| `///` | | | Launch Layers - IF `$1` is `--` `` is `true` and `` SHALL be `${@2:}` - IF `$1` is anything other than `--`, `` is `false`, and `` SHALL be `$@` From 1664efd499954d64346b0b1d43e0fff480145ffa Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Fri, 26 Jun 2020 14:45:38 -0400 Subject: [PATCH 26/38] API compat fixes * Removes duplicated compatibility rules * Clarifies language * Other minor cleanup Signed-off-by: Emily Casey --- platform.md | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/platform.md b/platform.md index 14eaa09f..5849672d 100644 --- a/platform.md +++ b/platform.md @@ -60,9 +60,9 @@ Examples of a platform might include: ## Platform API Version -**Current Version**: 0.3 +This document specifies Platform API version `0.3` -The Platform API version: +Platform API versions: - MUST be in form `.` or ``, where `` is equivalent to `.0` - When `` is greater than `0` increments to `` SHALL exclusively indicate additive changes @@ -131,11 +131,11 @@ However, mixins MAY consist of any changes that follow the [Compatibility Guaran ## Lifecycle Interface ### Platform API Compatibility -The platform SHOULD set `CNB_PLATFORM_API=[.]` in the lifecycle's execution environment +The platform SHOULD set `CNB_PLATFORM_API=` in the lifecycle's execution environment -IF `CNB_PLATFORM_API=[.]` is set in the lifecycle's execution environment, the lifecycle MUST do one the following before attempting to parse other inputs - - Conform usage and behavior to the given version of the platform API specification - - Fail if it does not support the requested API version +IF `CNB_PLATFORM_API` is set in the lifecycle's execution environment, the lifecycle: + - MUST EITHER conform to the matching version of this specification OR + - Fail if it does not support `` ### Operations @@ -169,12 +169,10 @@ To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` OR p ### Usage -The following is true of all lifecycle phases: +All lifecycle phases: -- Command line inputs ALWAYS take precedence over other inputs -- IF `CNB_PLATFORM_API=[.`| `CNB_SKIP_LAYERS` | `false` | Do not write layer metadata | `` | `CNB_USER_ID` | | User that build phase will run as -- IF `` is `false`, `` MUST be a valid registry reference -- IF `` is `true`, `` MUST be a valid registry reference OR a docker image ID +- IF `` is `false`, `` MUST be a valid registry reference +- IF `` is `true`, `` MUST be a valid registry reference OR a docker image ID - The lifecycle MUST accept valid references to non-existent images without error. | Output | Description @@ -387,7 +385,7 @@ Usage: - At least one `` must be provided - Each `` MUST be a valid OCI image registry tag reference -- IF `` is `false` and more than one `` is provided they MUST refer to the same registry +- IF `` is `false` and more than one `` is provided they MUST refer to the same registry - IF `` is not provided by the platform the value will be derived from the contents of `stack` - IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` - IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` @@ -482,7 +480,7 @@ Usage: - At least one `` must be provided - Each `` MUST be a valid OCI image registry tag reference -- IF `` is `false` and more than one `` is provided they MUST refer to the same registry +- IF `` is `false` and more than one `` is provided they MUST refer to the same registry - IF `` is not provided by the platform the value will be derived from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on `` - IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` - IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` From 1456c8334f4a3b6eb8aaf8e9bcd99fa4c2a5c38c Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Mon, 29 Jun 2020 16:07:42 -0400 Subject: [PATCH 27/38] Adds plan.toml data format and other improvements * Defines additional terminology - define heavily used OCI distribution terms (image, tag, and digest referecen) - define DiffID and ImageID * Moves run-image resolution to a separate section * Longer, more detailed description of the rebase operation * Fixes inaccuracies in process environment description * Better description of -uid and -gid Signed-off-by: Emily Casey --- platform.md | 196 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 128 insertions(+), 68 deletions(-) diff --git a/platform.md b/platform.md index 5849672d..17807787 100644 --- a/platform.md +++ b/platform.md @@ -16,6 +16,8 @@ Examples of a platform might include: - [Table of Contents](#table-of-contents) - [Platform API Version](#platform-api-version) - [Terminology](#terminology) + - [CNB Terminology](#cnb-terminology) + - [Additional Terminology](#additional-terminology) - [Stacks](#stacks) - [Compatibility Guarantees](#compatibility-guarantees) - [Build Image](#build-image) @@ -38,6 +40,7 @@ Examples of a platform might include: - [`creator`](#creator) - [`rebaser`](#rebaser) - [`launcher`](#launcher) + - [Run Image Resolution](#run-image-resolution) - [Buildpacks](#buildpacks) - [Buildpacks Directory Layout](#buildpacks-directory-layout) - [Security Considerations](#security-considerations) @@ -50,6 +53,7 @@ Examples of a platform might include: - [`group.toml` (TOML)](#grouptoml-toml) - [`metadata.toml` (TOML)](#metadatatoml-toml) - [`order.toml` (TOML)](#ordertoml-toml) + - [`plan.toml` (TOML)](#plantoml-toml) - [`project-metadata.toml` (TOML)](#project-metadatatoml-toml) - [`stack.toml` (TOML)](#stacktoml-toml) - [Labels](#labels) @@ -68,20 +72,42 @@ Platform API versions: ## Terminology -A **stack** is a contract defined by a base run OCI image and a base build OCI image that are only updated with security patches. -Stack images can be modified with mixins in order to make additive changes to the contract. +#### CNB Terminology +A **buildpack** refers to software compliant with the [Buildpack Interface Specification](https://github.com/buildpacks/spec/blob/main/buildpack.md) -A **mixin** is a named set of additions to a stack that avoid changing the behavior of buildpacks or apps that do not depend on the mixin. +A **stack** is a contract defined by a **build image** and a **run image** that are only updated with security patches. -An **app image** refers to an OCI image generated by the lifecycle by extending the run image with any or all of the following: app layers, launch layers, launcher layers, image configuration +A **build image** is an OCI image that provides the base of the **build environment** + +A **run image** is an OCI image that provides the base from which **app images** are built + +The **build environment** refers to the containerized environment in which the lifecycle executes buildpacks. + +A **mixin** is a named set of additions to a stack that can be used to make additive changes to the contract. + +An **app image** refers to an OCI image generated by the lifecycle by extending the **run image** with any or all of the following: **app layers**, **launch layers**, **launcher layers**, image configuration A **launch layer** refers to a layer in the app image created from a `/` directory as specified in the [Buildpack Interface Specification](buildpack.md). An **app layer** refers to a layer in the app image created from the `` directory as specified in the [Buildpack Interface Specification](buildpack.md). +A **run image layer** refers to a layer in the **app image** originating from the **run image** + A **launcher layer** refers to a layer in the app OCI image containing the launcher itself and/or launcher configuration -The **launcher** refers to a lifecycle binary packaged in the app image for the purpose of executing processes at runtime +The **launcher** refers to a lifecycle executable packaged in the app image for the purpose of executing processes at runtime + +#### Additional Terminology +An **image reference** refers to either a **tag reference** or **digest reference** + +A **tag reference** refers to an identifier of form `/:` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. + +A **digest reference** refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form `/@` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. + +The following is a non-exhaustive list of terms defined in the [OCI Image Format Specification](https://github.com/opencontainers/image-spec) used throughout this document: +* **image config** https://github.com/opencontainers/image-spec/blob/master/config.md#oci-image-configuration +* **imageID** - https://github.com/opencontainers/image-spec/blob/master/config.md#imageid +* **DiffID** - https://github.com/opencontainers/image-spec/blob/master/config.md#layer-diffid ## Stacks ### Compatibility Guarantees @@ -140,12 +166,12 @@ IF `CNB_PLATFORM_API` is set in the lifecycle's execution environment, the lifec ### Operations #### Build -A single app image build consists of the following phases: +A single app image build* consists of the following phases: 1. Detection 1. Analysis 1. Cache Restoration -1. Build +1. Build* 1. Export A platform MUST execute these phases EITHER by invoking the following phase-specific lifecycle binaries in order: @@ -157,9 +183,16 @@ A platform MUST execute these phases EITHER by invoking the following phase-spec OR by executing `/cnb/lifecycle/creator`. +> \* **build** is an overloaded term that refers to both a single phase and the operation comprised of the above phases. +> The meaning of any particular instance of the word **build** must be assessed in context + #### Rebase -Run image rebasing allows for fast stack updates for already-exported OCI images without requiring a rebuild. A rebase requires minimal data transfer when those images are stored on a Docker registry. -When a new stack version with the same stack ID is available, the app layers and launch layers SHOULD be rebased on the new run image by updating the image's configuration to point at the new run image. +When a new stack version with the same stack ID is available, an updated app image SHOULD be generated from the existing app image config by replacing the run image layers in the existing `RootFS` with the layers from the new run image. +This is referred to as rebasing the app, launch, and launcher layers onto the new run image layers. +When layers are rebased, any app image metadata referenceing to the original run image MUST be updated to reference to the new run image. +This entire operation is referred to as rebasing the app image. + +Rebasing allows for fast stack updates for app images without requiring a rebuild. A rebase requires minimal data transfer when the app and run images are colocated on a Docker registry. To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` OR perform an equivalent operation. @@ -193,10 +226,10 @@ Usage: |---------------|---------------------|-----------------|---------------------- | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to output group file +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to output group definition | `` | `CNB_LOG_LEVEL` | `info` | Log Level -| `` | `CNB_ORDER_PATH` | `./order.toml` | Path to order definition (see [`order.toml` (TOML)](#order.toml-(toml))) -| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file +| `` | `CNB_ORDER_PATH` | `./order.toml` | Path to order definition (see [`order.toml` (TOML)](#ordertoml-(toml))) +| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output resolved build plan | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | Output | Description @@ -204,8 +237,12 @@ Usage: | [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| `` | Detected buildpack group (see [`group.toml` (TOML)](#group.toml-(toml))) -| `` | Resolved Build Plan (see data format in [Buildpack Interface Specification](buildpack.md)) +| `` | Detected buildpack group (see [`group.toml`](#grouptoml-toml)) +| `` | Resolved Build Plan (see [`plan.toml`](#plantoml-toml)) + +The lifecycle: +- SHALL detect a single group from `` and write it to `` using the [detection process](buildpack.md#phase-1-detection) outlined in the Buildpack Interface Specification +- SHALL write the resolved build plan from the detected group to `` #### `analyzer` Usage: @@ -225,20 +262,20 @@ Usage: | Input | Environment Variable | Default Value | Description |----------------|-----------------------|-------------------|---------------------- -| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to output analysis metadata (see [`analyzed.toml` (TOML)](#analyzed.toml-(toml)) +| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to output analysis metadata (see [`analyzed.toml` (TOML)](#analyzedtoml-toml) | `` | `CNB_CACHE_DIR` | | Location of cache, provided as a directory | ``| `CNB_CACHE_IMAGE` | | Location of cache, provided as an image | `` | `CNB_USE_DAEMON` | `false` | Analyze image from docker daemon -| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` | `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml` (TOML)](#group.toml-(toml))) | `` | | | Image reference to be analyzed (usually the result of the previous build) -| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | `` | `CNB_LOG_LEVEL` | `info` | Log Level -| ``| `CNB_SKIP_LAYERS` | `false` | Do not write layer metadata -| `` | `CNB_USER_ID` | | User that build phase will run as +| ``| `CNB_SKIP_LAYERS` | `false` | Do not perform layer analysis +| `` | `CNB_USER_ID` | | UID of the stack `User` -- IF `` is `false`, `` MUST be a valid registry reference -- IF `` is `true`, `` MUST be a valid registry reference OR a docker image ID +- IF `` is `false`, `` MUST be a valid image reference +- IF `` is `true`, `` MUST be EITHER a valid image reference OR an ImageID - The lifecycle MUST accept valid references to non-existent images without error. | Output | Description @@ -246,13 +283,13 @@ Usage: | [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| `` | Analysis metadata (see [`analyzed.toml` (TOML)](#analyzed.toml-(toml)) -| `//.toml` files | Contain analyzed layer metadata (see data format in [Buildpack Interface Specification](buildpack.md)) -| `//.sha` files | Contain sha256 of the uncompressed layer +| `` | Analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) +| `//.sha` | Files containing the DiffID of each analyzed layer +| `//.toml` | Files containing the layer content metadata of each analyzed layer (see data format in [Buildpack Interface Specification]((buildpack.md))) -- The lifecycle MUST write analysis metadata if `` is accessible. -- IF `` is `false` the lifecycle MUST analyze layers created by any buildpack present in the provided ``. -- IF `` is `true` the lifecycle MUST NOT analyze layers. +- The lifecycle MUST write [analysis metadata](#analyzedtoml-toml) to `` IF `` is accessible. +- IF `` is `false` the lifecycle MUST analyze any app image layers or cached layers created by any buildpack present in the provided ``. +- IF `` is `true` the lifecycle MUST NOT perform layer analysis. ##### Layer analysis When analyzing a given layer the lifecycle SHALL: @@ -260,10 +297,10 @@ When analyzing a given layer the lifecycle SHALL: - Do nothing - ELSE IF `launch=true`: - Write layer metadata read from the analyzed image to `//.toml` - - Write the sha256 of the uncompressed layer from the analyzed image to `//.sha` + - Write the layer DiffID from the analyzed image to `//.sha` - ELSE IF `cache=true`: - Write layer metadata read from the cache to `//.toml` - - Write the sha256 of the uncompressed layer from the cache to `//.sha` + - Write the layer DiffID from the cache to `//.sha` #### `restorer` Usage: @@ -282,13 +319,13 @@ Usage: |----------------|-----------------------|-----------------|---------------------- | `` | `CNB_CACHE_DIR` | | Path to a cache directory | ``| `CNB_CACHE_IMAGE` | | Reference to a cache image in an OCI image registry -| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml` (TOML)](#group.toml-(toml))) -| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml`](#grouptoml-toml)) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | `` | `CNB_LOG_LEVEL` | `info` | Log Level -| `` | `CNB_USER_ID` | | User that build phase will run as -| `//.sha` || | Files Containing the sha256 of the uncompressed layers -| `//.toml` || | Files Containing analyzed layer metadata (see data format in [Buildpack Interface Specification](buildpack.md)) +| `` | `CNB_USER_ID` | | UID of the stack `User` +| `//.sha` || | Files containing the DiffID of each analyzed layer +| `//.toml` || | Files containing the layer content metadata of each analyzed layer (see data format in [Buildpack Interface Specification]((buildpack.md))) | Output | Description |------------------------------------|---------------------------------------------- @@ -298,9 +335,9 @@ Usage: | `///*` | Restored layer contents ##### Layer restoration -For each layer metadata file found in the `` directory the lifecycle: -- MUST restore cached layer contents IF the cache contains a layer with a matching ID -- MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a matching layer +For each layer metadata file found in the `` directory, the lifecycle: +- MUST restore cached layer contents IF the cache contains a layer with matching DiffID +- MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a layer with matching DiffID #### `builder` The platform MUST execute `builder` in the build environment @@ -321,10 +358,10 @@ Usage: |----------------|-----------------------|-------------------|---------------------- | `` | `CNB_APP_DIR` | `/workspace` | Path to application directory | `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml` (TOML)](#group.toml-(toml))) -| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml`](#grouptoml-toml)) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | `` | `CNB_LOG_LEVEL` | `info` | Log Level -| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output build plan file (see data format in [Buildpack Interface Specification](buildpack.md)) +| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to resolved build plan (see [`plan.toml`](#plantoml-toml)) | `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory | Output | Description @@ -332,12 +369,12 @@ Usage: | [exit status] | Success (0), or error (1+) | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) -| `//` | Layer contents buildpacks (see [Buildpack Interface Specfication](buildpack.md) +| `//` | Layer contents (see [Buildpack Interface Specfication](buildpack.md) | `//.toml` | Layer metadata (see [Buildpack Interface Specfication](buildpack.md) -| `/config/metadata.toml` | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) +| `/config/metadata.toml` | Build metadata (see [`metadata.toml`](#metadatatoml-toml)) -- The lifecycle SHALL execute all buildpacks in the order defined in `` according to the rules of the [Buildpack Interface Specification](buildpack.md). -- The lifecycle SHALL add all invoked buildpacks to `[[buildpacks]]` in `/config/metadata.toml`. +- The lifecycle SHALL execute all buildpacks in the order defined in `` according process outlined in the [Buildpack Interface Specification](buildpack.md). +- The lifecycle SHALL add all invoked buildpacks to`/config/metadata.toml`. - The lifecycle SHALL aggregate all `processes`, `slices` and BOM entries returned by buildpacks in `/config/metadata.toml`. #### `exporter` @@ -369,8 +406,8 @@ Usage: | `` | `CNB_CACHE_DIR` | | Path to a cache directory | `` | `CNB_CACHE_IMAGE` | | Reference to a cache image in an OCI image registry | `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon -| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as -| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml` (TOML)](#group.toml-(toml))) +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml`](#group.toml-(toml))) | `` | | | An | `` | `CNB_LAUNCH_CACHE_DIR`| | Path to a cache directory containing launch layers | `` | | `/cnb/lifecycle/launcher` | Path to the `launcher` executable @@ -378,17 +415,15 @@ Usage: | `` | `CNB_LOG_LEVEL` ` | `info` | Log Level | `` | `CNB_PROCESS_TYPE` | | Default process type to set in the exported image | ``| `CNB_PROCESS_TYPE` | `./project-metadata.toml` | Path to a project metadata file (see [`project-metadata.toml` (TOML)](#project-metadata.toml) -| `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference +| `` | `CNB_RUN_IMAGE` | resolved from `` | Run image reference | `` | `CNB_STACK_PATH` | `/cnb/stack.toml` | Path to stack file (see [`stack.toml` (TOML)](#stack.toml-(toml)) -| `` | `CNB_USER_ID` | | User that build phase will run as +| `` | `CNB_USER_ID` | | UID of the stack `User` | `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) - At least one `` must be provided -- Each `` MUST be a valid OCI image registry tag reference +- Each `` MUST be a valid tag reference - IF `` is `false` and more than one `` is provided they MUST refer to the same registry -- IF `` is not provided by the platform the value will be derived from the contents of `stack` - - IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` - - IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` +- IF `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of `stack` | Output | Description |--------------------|---------------------------------------------- @@ -400,10 +435,11 @@ Usage: - The lifecycle SHALL write the same app image to each `` tag - The app image: - MUST be an extension of the `` - - All run-image layers and config values SHALL be preserved unless they conflict with the following - - MUST contain all buildpack provided launch layers as determined by the [Buildpack Interface Specfication](buildpack.md) + - All run-image layers SHALL be preserved + - All run-image config values SHALL be preserved unless this conflict with another requirement + - MUST contain all buildpack-provided launch layers as determined by the [Buildpack Interface Specfication](buildpack.md) - MUST contain one or more app layers as determined by the [Buildpack Interface Specfication](buildpack.md) - - MUST contain a layer that includes the `` + - MUST contain a layer that includes `` - MUST contain a layer that includes `/config/metadata.toml` - MUST have `ENTRYPOINT=` - MUST contain the following `Env` entries @@ -416,9 +452,9 @@ Usage: - `io.buildpacks.project.metadata`: the value of which SHALL be the json representation `` - `io.buildpacks.build.metadata`: see [build metadata (JSON)](#build-metadata-label-(json)) -IF a cache is provided the lifecycle SHALL: -- Write `io.buildpacks.cache.metadata` to the cache -- Ensure all `cache=true` layers are written to the cache +IF a cache is provided the lifecycle: +- SHALL write the contents of all cached layers to the cache +- SHALL record the DiffID and layer content metadata of all cached layers in the cache #### `creator` Usage: @@ -456,7 +492,7 @@ Running `creator` SHALL be equivalent to running `detector`, `analzyer`, `restor - IF `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. - IF the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` -Outputs produced by `creator` and identical to those produced by `exporter`. +Outputs produced by `creator` are identical to those produced by `exporter`. #### `rebaser` Usage: @@ -472,18 +508,16 @@ Usage: | Input | Environment Variable | Default Value | Description |---------------------|-----------------------|------------------------|--------------------------------------- | `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon -| `` | `CNB_GROUP_ID` | | Group of user that build phase will run as +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` | `` | | | App image to rebase | `` | `CNB_LOG_LEVEL` | `info` | Log Level | `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference -| `` | `CNB_USER_ID` | | User that build phase will run as +| `` | `CNB_USER_ID` | | UID of the stack `User` - At least one `` must be provided -- Each `` MUST be a valid OCI image registry tag reference +- Each `` MUST be a valid tag reference - IF `` is `false` and more than one `` is provided they MUST refer to the same registry -- IF `` is not provided by the platform the value will be derived from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on `` - - IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` - - IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` +- IF `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on ``. | Output | Description |--------------------|---------------------------------------------- @@ -514,7 +548,7 @@ Usage: | `` | | `false` | Execution strategy for user provided process | `` | | | User-provided command | `...` | | | Arguments to user-provided command -| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) +| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml`](#metdata.toml-(toml)) | `///` | | | Launch Layers - IF `$1` is `--` `` is `true` and `` SHALL be `${@2:}` @@ -530,11 +564,17 @@ args = [""...] ``` - The lifecycle SHOULD replace the lifecycle process in memory without forking it. -- The process execution environment SHALL be identical to the lifecycle execution environment, with the following exceptions: +- The process execution environment SHALL inherit all variables from lifecycle execution environment, with the following exceptions: - `CNB_APP_DIR` SHALL NOT be set in the process environment - `CNB_LAYERS_DIR` SHALL NOT be set in the process environment - `CNB_PROCESS_TYPE` SHALL NOT be set in the process environment +### Run Image Resolution + +Given stack metadata containing `run-image.image` and a set of `run-image.mirrors`. The `` for a given `` shall be resolved as follows: +- IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` +- IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` + ## Buildpacks ### Buildpacks Directory Layout @@ -632,6 +672,26 @@ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. - The value of `optional` MUST default to false if not specified. +#### `plan.toml` (TOML) +```toml +[[entries]] + + [[entries.providers]] + id = "" + version = "buildpack Version" + + [[entries.requires]] + name = "" + [entries.requires.metadata] + # arbitrary data describing the required dependency +``` +Where: +- `entries` MAY be empty +- Each entry: + - MUST contain at least one buildpack in `providers` + - MUST contain at least one dependency requirement in `requires` + - MUST exclusively contain dependency requirements with the same `` + #### `project-metadata.toml` (TOML) ```toml From 61b2eb24fcc6f58e21fe0bd763f5ec8d3d5c8a2b Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Mon, 29 Jun 2020 17:47:05 -0400 Subject: [PATCH 28/38] Adds Registry Authentication Signed-off-by: Emily Casey --- platform.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/platform.md b/platform.md index 17807787..cb4be93a 100644 --- a/platform.md +++ b/platform.md @@ -41,6 +41,7 @@ Examples of a platform might include: - [`rebaser`](#rebaser) - [`launcher`](#launcher) - [Run Image Resolution](#run-image-resolution) + - [Registry Authentication](#registry-authentication) - [Buildpacks](#buildpacks) - [Buildpacks Directory Layout](#buildpacks-directory-layout) - [Security Considerations](#security-considerations) @@ -575,6 +576,18 @@ Given stack metadata containing `run-image.image` and a set of `run-image.mirror - IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` - IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` +### Registry Authentication + +The platform MAY set `CNB_REGISTRY_AUTH` in the lifecycle execution environment, where value of `CNB_REGISTRY_AUTH`, MUST be valid JSON object and MAY contain any number of `` to `` mappings. +IF `CNB_REGISTRY_AUTH` is set AND `` matches the registry of an image reference, the lifecycle SHOULD set the value of the `Authorization` HTTP header to `` when attempting to read or write the image located at the given reference. + +IF `CNB_REGISTRY_AUTH` is unset AND a docker [config.json](https://docs.docker.com/engine/reference/commandline/cli/#configjson-properties) file is present, the lifecycle SHOULD use the contents of this file to authenticate with any matching registry. +The lifecycle SHOULD adhere to established docker conventions when checking for the existence of or interpreting the contents of a `config.json` file. + +The lifecycle MAY provide other mechanisms by which a platform can supply registry credentials. + +The lifecycle MUST attempt to authenticate anonymously if no matching credentials are found. + ## Buildpacks ### Buildpacks Directory Layout From e68cee1e19773b316401d336f51b0a8a165f9cb8 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 1 Jul 2020 10:55:04 -0400 Subject: [PATCH 29/38] Adds missing image input description to exporter Signed-off-by: Emily Casey --- platform.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform.md b/platform.md index cb4be93a..5c99a4ee 100644 --- a/platform.md +++ b/platform.md @@ -409,7 +409,7 @@ Usage: | `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon | `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` | `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml`](#group.toml-(toml))) -| `` | | | An +| `` | | | Tag reference to which the app image will be written | `` | `CNB_LAUNCH_CACHE_DIR`| | Path to a cache directory containing launch layers | `` | | `/cnb/lifecycle/launcher` | Path to the `launcher` executable | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory From dfb91743e8f0984dacda1007561bdb163536c948 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Thu, 2 Jul 2020 10:15:43 -0400 Subject: [PATCH 30/38] bold build environment * adds missing requirement that creator must run in the build environment Signed-off-by: Emily Casey --- platform.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/platform.md b/platform.md index 5c99a4ee..0fd2c0e1 100644 --- a/platform.md +++ b/platform.md @@ -209,7 +209,7 @@ All lifecycle phases: - MUST give command line inputs precedence over other inputs #### `detector` -The platform MUST execute `detector` in the build environment +The platform MUST execute `detector` in the **build environment** Usage: ``` @@ -341,7 +341,7 @@ For each layer metadata file found in the `` directory, the lifecycle: - MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a layer with matching DiffID #### `builder` -The platform MUST execute `builder` in the build environment +The platform MUST execute `builder` in the **build environment** Usage: ``` @@ -458,6 +458,8 @@ IF a cache is provided the lifecycle: - SHALL record the DiffID and layer content metadata of all cached layers in the cache #### `creator` +The platform MUST execute `creator` in the **build environment** + Usage: ``` /cnb/lifecycle/creator From 2e160296aa618d19f2ee5212cc68e339ae726604 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Thu, 2 Jul 2020 12:39:06 -0400 Subject: [PATCH 31/38] Adds more details about environment * Describe how a platform can configure buildpack and process environments Signed-off-by: Emily Casey --- platform.md | 41 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 38 insertions(+), 3 deletions(-) diff --git a/platform.md b/platform.md index 0fd2c0e1..f46496a1 100644 --- a/platform.md +++ b/platform.md @@ -47,6 +47,10 @@ Examples of a platform might include: - [Security Considerations](#security-considerations) - [Additional Guidance](#additional-guidance) - [Environment](#environment) + - [Buildpack Environment](#buildpack-environment) + - [Stack-Provided Variables](#stack-provided-variables) + - [User-Provided Variables](#user-provided-variables) + - [Launch Environment](#launch-environment) - [Caching](#caching) - [Data Format](#data-format) - [Files](#files) @@ -607,9 +611,40 @@ A more thorough explanation is provided in the [Buildpack Interface Specificatio ## Additional Guidance ### Environment - -The platform MAY determine the initial environment of the build phase, detection phase, and launch. -The lifecycle MUST NOT assume that all platforms provide an identical environment. +#### Buildpack Environment +##### Stack-Provided Variables +The following variables SHOULD be set in the lifecycle execution environment and SHALL be directly inherited by the buildpack without modification: +| Env Variable | Description +|-----------------|-------------------------------------- +| `CNB_STACK_ID` | Chosen stack ID +| `HOME` | Current user's home directory + +The following variables SHOULD be set in the lifecycle execution environment and MAY be modified by prior buildpacks before they are provided to a given buildpack: + +| Env Variable | Layer Path | Contents +|-------------------|--------------|------------------ +| `PATH` | `/bin` | binaries +| `LD_LIBRARY_PATH` | `/lib` | shared libraries +| `LIBRARY_PATH` | `/lib` | static libraries +| `CPATH` | `/include` | header files +| `PKG_CONFIG_PATH` | `/pkgconfig` | pc files + +The platform SHOULD NOT assume any other stack-provided environment variables are inherited by the buildpack. + +##### User-Provided Variables +User-provided environment variables MUST be supplied by the platform as files in the `/env/` directory. +Each file SHALL define a single environment variable, where the file name defines the key and the file contents define the value. + +User-provided environment variables MAY be modified by prior buildpacks before they are provided to a given buildpack. + +The platform SHOULD NOT set user-provided environment variables directly in the lifecycle execution environment. + +#### Launch Environment +User-provided modifications to the process execution environment SHOULD be set directly in the lifecycle execution environment. + +The process SHALL inherit both stack-provided and user-provided variables from the lifecycle execution environment with the following exceptions: +* `CNB_APP_DIR`, `CNB_LAYERS_DIR` and `CNB_PROCESS_TYPE` SHALL NOT be set in the process execution environment. +* The lifecycle SHALL apply buildpack-provided modifications to the environment as outlined in the [Buildpack Interface Specification](buildpack.md). ### Caching From 1b7fd4088755125e0958d3bbbf2fbd4ca3460bdc Mon Sep 17 00:00:00 2001 From: E Casey Date: Tue, 7 Jul 2020 13:30:46 -0400 Subject: [PATCH 32/38] Apply suggestions from code review Co-authored-by: Stephen Levine Signed-off-by: Emily Casey --- platform.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/platform.md b/platform.md index f46496a1..0881e68a 100644 --- a/platform.md +++ b/platform.md @@ -69,7 +69,7 @@ Examples of a platform might include: ## Platform API Version -This document specifies Platform API version `0.3` +This document specifies Platform API version `0.3`. Platform API versions: - MUST be in form `.` or ``, where `` is equivalent to `.0` @@ -166,7 +166,7 @@ The platform SHOULD set `CNB_PLATFORM_API=` in the lifecyc IF `CNB_PLATFORM_API` is set in the lifecycle's execution environment, the lifecycle: - MUST EITHER conform to the matching version of this specification OR - - Fail if it does not support `` + - MUST fail if it does not support `` ### Operations @@ -192,12 +192,12 @@ OR by executing `/cnb/lifecycle/creator`. > The meaning of any particular instance of the word **build** must be assessed in context #### Rebase -When a new stack version with the same stack ID is available, an updated app image SHOULD be generated from the existing app image config by replacing the run image layers in the existing `RootFS` with the layers from the new run image. +When an updated run image with the same stack ID is available, an updated app image SHOULD be generated from the existing app image config by replacing the run image layers in the existing app image with the layers from the new run image. This is referred to as rebasing the app, launch, and launcher layers onto the new run image layers. When layers are rebased, any app image metadata referenceing to the original run image MUST be updated to reference to the new run image. This entire operation is referred to as rebasing the app image. -Rebasing allows for fast stack updates for app images without requiring a rebuild. A rebase requires minimal data transfer when the app and run images are colocated on a Docker registry. +Rebasing allows for fast runtime OS-level dependency updates for app images without requiring a rebuild. A rebase requires minimal data transfer when the app and run images are colocated on a Docker registry that supports [Cross Repository Blob Mounts](https://docs.docker.com/registry/spec/api/#cross-repository-blob-mount). To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` OR perform an equivalent operation. @@ -703,9 +703,9 @@ paths = [""] Where: - Both `id` and `version` MUST be present for each buildpack. -- Where `processes` contains the complete set of processes contributed by all buildpacks -- Where `processes` contains the complete set of slice defined by all buildpacks -- Where `bom` contains the Bill of Materials +- `processes` contains the complete set of processes contributed by all buildpacks +- `processes` contains the complete set of slice defined by all buildpacks +- `bom` contains the Bill of Materials #### `order.toml` (TOML) From 74f23609231e8321be0d0246fbf02ac7c9e5c799 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Tue, 7 Jul 2020 15:01:08 -0400 Subject: [PATCH 33/38] Launcher clarifications and style fixes * don't capitalize non RFC 2119 words * small gramatiical fixes * improved launcher process selection Signed-off-by: Emily Casey --- platform.md | 150 +++++++++++++++++++++++++++------------------------- 1 file changed, 77 insertions(+), 73 deletions(-) diff --git a/platform.md b/platform.md index 0881e68a..16690da3 100644 --- a/platform.md +++ b/platform.md @@ -40,6 +40,7 @@ Examples of a platform might include: - [`creator`](#creator) - [`rebaser`](#rebaser) - [`launcher`](#launcher) + - [Process Selection](#process-selection) - [Run Image Resolution](#run-image-resolution) - [Registry Authentication](#registry-authentication) - [Buildpacks](#buildpacks) @@ -78,32 +79,32 @@ Platform API versions: ## Terminology #### CNB Terminology -A **buildpack** refers to software compliant with the [Buildpack Interface Specification](https://github.com/buildpacks/spec/blob/main/buildpack.md) +A **buildpack** refers to software compliant with the [Buildpack Interface Specification](https://github.com/buildpacks/spec/blob/main/buildpack.md). A **stack** is a contract defined by a **build image** and a **run image** that are only updated with security patches. -A **build image** is an OCI image that provides the base of the **build environment** +A **build image** is an OCI image that provides the base of the **build environment**. -A **run image** is an OCI image that provides the base from which **app images** are built +A **run image** is an OCI image that provides the base from which **app images** are built. The **build environment** refers to the containerized environment in which the lifecycle executes buildpacks. A **mixin** is a named set of additions to a stack that can be used to make additive changes to the contract. -An **app image** refers to an OCI image generated by the lifecycle by extending the **run image** with any or all of the following: **app layers**, **launch layers**, **launcher layers**, image configuration +An **app image** refers to an OCI image generated by the lifecycle by extending the **run image** with any or all of the following: **app layers**, **launch layers**, **launcher layers**, image configuration. A **launch layer** refers to a layer in the app image created from a `/` directory as specified in the [Buildpack Interface Specification](buildpack.md). An **app layer** refers to a layer in the app image created from the `` directory as specified in the [Buildpack Interface Specification](buildpack.md). -A **run image layer** refers to a layer in the **app image** originating from the **run image** +A **run image layer** refers to a layer in the **app image** originating from the **run image**. -A **launcher layer** refers to a layer in the app OCI image containing the launcher itself and/or launcher configuration +A **launcher layer** refers to a layer in the app OCI image containing the **launcher** itself and/or launcher configuration. -The **launcher** refers to a lifecycle executable packaged in the app image for the purpose of executing processes at runtime +The **launcher** refers to a lifecycle executable packaged in the **app image** for the purpose of executing processes at runtime. #### Additional Terminology -An **image reference** refers to either a **tag reference** or **digest reference** +An **image reference** refers to either a **tag reference** or **digest reference**. A **tag reference** refers to an identifier of form `/:` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. @@ -112,7 +113,7 @@ A **digest reference** refers to a [content addressable](https://en.wikipedia.o The following is a non-exhaustive list of terms defined in the [OCI Image Format Specification](https://github.com/opencontainers/image-spec) used throughout this document: * **image config** https://github.com/opencontainers/image-spec/blob/master/config.md#oci-image-configuration * **imageID** - https://github.com/opencontainers/image-spec/blob/master/config.md#imageid -* **DiffID** - https://github.com/opencontainers/image-spec/blob/master/config.md#layer-diffid +* **diffID** - https://github.com/opencontainers/image-spec/blob/master/config.md#layer-diffid ## Stacks ### Compatibility Guarantees @@ -164,8 +165,8 @@ However, mixins MAY consist of any changes that follow the [Compatibility Guaran The platform SHOULD set `CNB_PLATFORM_API=` in the lifecycle's execution environment -IF `CNB_PLATFORM_API` is set in the lifecycle's execution environment, the lifecycle: - - MUST EITHER conform to the matching version of this specification OR +If `CNB_PLATFORM_API` is set in the lifecycle's execution environment, the lifecycle: + - MUST either conform to the matching version of this specification or - MUST fail if it does not support `` ### Operations @@ -179,14 +180,14 @@ A single app image build* consists of the following phases: 1. Build* 1. Export -A platform MUST execute these phases EITHER by invoking the following phase-specific lifecycle binaries in order: +A platform MUST execute these phases either by invoking the following phase-specific lifecycle binaries in order: 1. `/cnb/lifecycle/detector` 1. `/cnb/lifecycle/analyzer` 1. `/cnb/lifecycle/restorer` 1. `/cnb/lifecycle/builder` 1. `/cnb/lifecycle/exporter` -OR by executing `/cnb/lifecycle/creator`. +or by executing `/cnb/lifecycle/creator`. > \* **build** is an overloaded term that refers to both a single phase and the operation comprised of the above phases. > The meaning of any particular instance of the word **build** must be assessed in context @@ -199,7 +200,7 @@ This entire operation is referred to as rebasing the app image. Rebasing allows for fast runtime OS-level dependency updates for app images without requiring a rebuild. A rebase requires minimal data transfer when the app and run images are colocated on a Docker registry that supports [Cross Repository Blob Mounts](https://docs.docker.com/registry/spec/api/#cross-repository-blob-mount). -To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` OR perform an equivalent operation. +To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` or perform an equivalent operation. #### Launch `/cnb/lifecycle/launcher` is responsible for launching user and buildpack provided processes in the correct execution environment. @@ -279,8 +280,8 @@ Usage: | ``| `CNB_SKIP_LAYERS` | `false` | Do not perform layer analysis | `` | `CNB_USER_ID` | | UID of the stack `User` -- IF `` is `false`, `` MUST be a valid image reference -- IF `` is `true`, `` MUST be EITHER a valid image reference OR an ImageID +- **If** `` is `false`, `` MUST be a valid image reference +- **If** `` is `true`, `` MUST be either a valid image reference or an imageID - The lifecycle MUST accept valid references to non-existent images without error. | Output | Description @@ -289,23 +290,23 @@ Usage: | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) | `` | Analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) -| `//.sha` | Files containing the DiffID of each analyzed layer +| `//.sha` | Files containing the diffID of each analyzed layer | `//.toml` | Files containing the layer content metadata of each analyzed layer (see data format in [Buildpack Interface Specification]((buildpack.md))) -- The lifecycle MUST write [analysis metadata](#analyzedtoml-toml) to `` IF `` is accessible. -- IF `` is `false` the lifecycle MUST analyze any app image layers or cached layers created by any buildpack present in the provided ``. -- IF `` is `true` the lifecycle MUST NOT perform layer analysis. +- The lifecycle MUST write [analysis metadata](#analyzedtoml-toml) to `` if `` is accessible. +- **If** `` is `false` the lifecycle MUST analyze any app image layers or cached layers created by any buildpack present in the provided ``. +- **If** `` is `true` the lifecycle MUST NOT perform layer analysis. ##### Layer analysis When analyzing a given layer the lifecycle SHALL: -- IF `build=true`, `cache=false`: +- **If** `build=true`, `cache=false`: - Do nothing -- ELSE IF `launch=true`: +- **Else if** `launch=true`: - Write layer metadata read from the analyzed image to `//.toml` - - Write the layer DiffID from the analyzed image to `//.sha` -- ELSE IF `cache=true`: + - Write the layer diffID from the analyzed image to `//.sha` +- **Else if** `cache=true`: - Write layer metadata read from the cache to `//.toml` - - Write the layer DiffID from the cache to `//.sha` + - Write the layer diffID from the cache to `//.sha` #### `restorer` Usage: @@ -329,7 +330,7 @@ Usage: | `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory | `` | `CNB_LOG_LEVEL` | `info` | Log Level | `` | `CNB_USER_ID` | | UID of the stack `User` -| `//.sha` || | Files containing the DiffID of each analyzed layer +| `//.sha` || | Files containing the diffID of each analyzed layer | `//.toml` || | Files containing the layer content metadata of each analyzed layer (see data format in [Buildpack Interface Specification]((buildpack.md))) | Output | Description @@ -341,8 +342,8 @@ Usage: ##### Layer restoration For each layer metadata file found in the `` directory, the lifecycle: -- MUST restore cached layer contents IF the cache contains a layer with matching DiffID -- MUST remove layer metadata IF `cache=true` AND the cache DOES NOT contain a layer with matching DiffID +- MUST restore cached layer contents if the cache contains a layer with matching diffID +- MUST remove layer metadata if `cache=true` AND the cache DOES NOT contain a layer with matching diffID #### `builder` The platform MUST execute `builder` in the **build environment** @@ -427,8 +428,8 @@ Usage: - At least one `` must be provided - Each `` MUST be a valid tag reference -- IF `` is `false` and more than one `` is provided they MUST refer to the same registry -- IF `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of `stack` +- **If** `` is `false` and more than one `` is provided they MUST refer to the same registry +- **If** `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of `stack` | Output | Description |--------------------|---------------------------------------------- @@ -450,16 +451,16 @@ Usage: - MUST contain the following `Env` entries - `"CNB_LAYERS_DIR="` - `"CNB_APP_DIR="` - - MUST contain the following `Env` entry, IF `` is set + - MUST contain the following `Env` entry, if `` is set - `"CNB_PROCESS_TYPE="` - MUST contain the following labels - `io.buildpacks.lifecycle.metadata`: see [lifecycle metadata label (JSON)](#lifecycle-metadata-label-(json)) - `io.buildpacks.project.metadata`: the value of which SHALL be the json representation `` - `io.buildpacks.build.metadata`: see [build metadata (JSON)](#build-metadata-label-(json)) -IF a cache is provided the lifecycle: +If a cache is provided the lifecycle: - SHALL write the contents of all cached layers to the cache -- SHALL record the DiffID and layer content metadata of all cached layers in the cache +- SHALL record the diffID and layer content metadata of all cached layers in the cache #### `creator` The platform MUST execute `creator` in the **build environment** @@ -496,8 +497,8 @@ Running `creator` SHALL be equivalent to running `detector`, `analzyer`, `restor | `` | `CNB_SKIP_RESTORE` | `false` | Do not write layer metadata or restore cached layers | `...` | | | Additional tag to apply to exported image -- IF `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. -- IF the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` +- **If** `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. +- **If** the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` Outputs produced by `creator` are identical to those produced by `exporter`. @@ -523,8 +524,8 @@ Usage: - At least one `` must be provided - Each `` MUST be a valid tag reference -- IF `` is `false` and more than one `` is provided they MUST refer to the same registry -- IF `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on ``. +- **If** `` is `false` and more than one `` is provided they MUST refer to the same registry +- **If** `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on ``. | Output | Description |--------------------|---------------------------------------------- @@ -547,47 +548,50 @@ Usage: /cnb/lifecycle/launcher [--] [ ...] ``` -| Input | Environment Variable | Default Value | Description -|---------------------|-----------------------|-----------------------|--------------------------------------- -| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory -| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory -| `` | `CNB_PROCESS_TYPE` | `web` | `type` of process to launch -| `` | | `false` | Execution strategy for user provided process -| `` | | | User-provided command -| `...` | | | Arguments to user-provided command -| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml`](#metdata.toml-(toml)) -| `///` | | | Launch Layers - -- IF `$1` is `--` `` is `true` and `` SHALL be `${@2:}` -- IF `$1` is anything other than `--`, `` is `false`, and `` SHALL be `$@` -- IF `` -- IF `` are NOT provided `launcher` SHALL select the process with `type` equal to `` from `/config/metadata.toml` and launch it as described in the [Buildpack Interface Specification](buildpack.md). -- IF `` are provided `launcher` SHALL behaves as if it selected a process with the following process from `metadata.toml`. -``` -[[process]] -direct = -command = "" -args = [""...] -``` - -- The lifecycle SHOULD replace the lifecycle process in memory without forking it. -- The process execution environment SHALL inherit all variables from lifecycle execution environment, with the following exceptions: - - `CNB_APP_DIR` SHALL NOT be set in the process environment - - `CNB_LAYERS_DIR` SHALL NOT be set in the process environment - - `CNB_PROCESS_TYPE` SHALL NOT be set in the process environment +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|----------------|--------------------------------------- +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_PROCESS_TYPE` | `web` | `type` of process to launch +| `` | | | Process execution strategy +| `` | | | Command to execute +| `` | | | Arguments to command +| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml`](#metdata.toml-(toml)) +| `///` | | | Launch Layers + +A command (``), arguments to that command (``), and an execution strategy (``) comprise a process definition. Processes MAY be buildpack-defined or user-defined. + +The launcher: +- MUST [select](#process-selection) a single process +- MUST construct the process execution environment as described in [Launch Environment](#launch-environment) +- MUST execute the process as specified in the [Buildpack Interface Specfication](buildpack.md) +- SHOULD replace the lifecycle with the process in memory without forking it. + +##### Process Selection +- **If** zero positional arguments are provided to the **launcher**, the lifecycle + - MUST select the process with `type` equal to `` from `/config/metadata.toml` +- **Else** + - **If** `$1` is `--` + - `` SHALL be `true` + - `` SHALL be `$2` + - `` SHALL be `${@3:}` + - **Else** + - `` SHALL be `false` + - `` SHALL be `$1` + - `` SHALL be `${@2:}` ### Run Image Resolution Given stack metadata containing `run-image.image` and a set of `run-image.mirrors`. The `` for a given `` shall be resolved as follows: -- IF any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` -- IF none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` +- **If** any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` +- **If** none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` ### Registry Authentication -The platform MAY set `CNB_REGISTRY_AUTH` in the lifecycle execution environment, where value of `CNB_REGISTRY_AUTH`, MUST be valid JSON object and MAY contain any number of `` to `` mappings. -IF `CNB_REGISTRY_AUTH` is set AND `` matches the registry of an image reference, the lifecycle SHOULD set the value of the `Authorization` HTTP header to `` when attempting to read or write the image located at the given reference. +The platform MAY set `CNB_REGISTRY_AUTH` in the lifecycle execution environment, where value of `CNB_REGISTRY_AUTH` MUST be valid JSON object and MAY contain any number of `` to `` mappings. +If `CNB_REGISTRY_AUTH` is set and `` matches the registry of an image reference, the lifecycle SHOULD set the value of the `Authorization` HTTP header to `` when attempting to read or write the image located at the given reference. -IF `CNB_REGISTRY_AUTH` is unset AND a docker [config.json](https://docs.docker.com/engine/reference/commandline/cli/#configjson-properties) file is present, the lifecycle SHOULD use the contents of this file to authenticate with any matching registry. +If `CNB_REGISTRY_AUTH` is unset and a docker [config.json](https://docs.docker.com/engine/reference/commandline/cli/#configjson-properties) file is present, the lifecycle SHOULD use the contents of this file to authenticate with any matching registry. The lifecycle SHOULD adhere to established docker conventions when checking for the existence of or interpreting the contents of a `config.json` file. The lifecycle MAY provide other mechanisms by which a platform can supply registry credentials. @@ -648,7 +652,7 @@ The process SHALL inherit both stack-provided and user-provided variables from t ### Caching -IF caching is enabled the platform is responsible for providing the lifecycle with access to the correct cache. +If caching is enabled the platform is responsible for providing the lifecycle with access to the correct cache. Whenever possible, the platform SHOULD provide the same cache to each rebuild of a given app image. Cache locality and availability MAY vary between platforms. @@ -667,7 +671,7 @@ Cache locality and availability MAY vary between platforms. ``` Where: -- `image.reference` MUST be EITHER a digest reference to an image in a docker registry or the ID of an image in a docker daemon +- `image.reference` MUST be either a digest reference to an image in a docker registry or the ID of an image in a docker daemon - `metadata` MUST be the TOML representation of the layer [metadata label](#layer-metadata-label-json) #### `group.toml` (TOML) @@ -771,7 +775,7 @@ Where: Where: - `run-image.image` MAY be a reference to a run image in a docker registry -- `run-image.mirrors` MUST NOT be present IF `run-image.image` is not present +- `run-image.mirrors` MUST NOT be present if `run-image.image` is not present - `run-image.mirrors` MAY contain one or more tag references to run images in docker registries - All `run-image.mirrors`: - SHOULD reference an image with ID identical to that of `run-image.image` From ba086dfa947c98937714cb11554e769c4ced1703 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Tue, 7 Jul 2020 18:11:05 -0400 Subject: [PATCH 34/38] Don't modify the buildpack spec Signed-off-by: Emily Casey --- buildpack.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/buildpack.md b/buildpack.md index 196b5820..11e43ab1 100644 --- a/buildpack.md +++ b/buildpack.md @@ -4,7 +4,7 @@ This document specifies the interface between a single lifecycle and one or more A lifecycle is a program that uses buildpacks to transform application source code into an OCI image containing the compiled application. -This is accomplished in five phases: +This is accomplished in four phases: 1. **Detection,** where an optimal selection of compatible buildpacks is chosen. 2. **Analysis,** where metadata about OCI layers generated during a previous build are made available to buildpacks. @@ -14,7 +14,7 @@ This is accomplished in five phases: The `ENTRYPOINT` of the OCI image contains logic implemented by the lifecycle that executes during the **Launch** phase. ## Table of Contents - + 1. [Buildpack Interface](#buildpack-interface) 1. [Key](#key) 2. [Detection](#detection) From c181659654ff5d95f5009f8195ee55dd0bff6445 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Tue, 7 Jul 2020 18:14:30 -0400 Subject: [PATCH 35/38] Use javascript instead of json formatting for labels A hack so we can use illegal comments and still get syntax highlighting. Signed-off-by: Emily Casey --- platform.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/platform.md b/platform.md index 16690da3..01bb6b60 100644 --- a/platform.md +++ b/platform.md @@ -785,7 +785,7 @@ Where: #### `io.buildpacks.build.metadata` (JSON) -```json +```javascript { "processes": [ { @@ -808,7 +808,7 @@ Where: "name": "", "version": "", "metadata": { - # arbitrary buildpack provided metadata + // arbitrary buildpack provided metadata }, "buildpack": { "id": "", @@ -837,7 +837,7 @@ Where: #### `io.buildpacks.lifecycle.metadata` (JSON) -```json +```javascript { "app": [ {"sha": ""} @@ -895,15 +895,15 @@ Where: #### `io.buildpacks.project.metadata` (JSON) -```json +```javascript { "source": { "type": " Date: Wed, 8 Jul 2020 13:46:32 -0400 Subject: [PATCH 36/38] Positional args must go last * Fixes inaccurate commandline usage Signed-off-by: Emily Casey --- platform.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/platform.md b/platform.md index 01bb6b60..a81fabc4 100644 --- a/platform.md +++ b/platform.md @@ -253,7 +253,7 @@ The lifecycle: #### `analyzer` Usage: ``` -/cnb/lifecycle/analyzer \ +/cnb/lifecycle/analyzer \ [-analyzed ] \ [-cache-dir ] \ [-cache-image ] \ @@ -264,6 +264,7 @@ Usage: [-log-level ] \ [-skip-layers ] \ [-uid ] \ + ``` | Input | Environment Variable | Default Value | Description @@ -386,7 +387,7 @@ Usage: #### `exporter` Usage: ``` -/cnb/lifecycle/exporter [...] +/cnb/lifecycle/exporter \ [-analyzed ] \ [-app ] \ [-cache-dir ] \ @@ -402,7 +403,8 @@ Usage: [-project-metadata ] \ [-run-image | -image ] \ # -image is Deprecated [-stack ] \ - [-uid ] + [-uid ] \ + [...] ``` | Input | Environment Variable | Default Value | Description @@ -467,7 +469,7 @@ The platform MUST execute `creator` in the **build environment** Usage: ``` -/cnb/lifecycle/creator +/cnb/lifecycle/creator \ [-app ] \ [-cache-dir ] \ [-cache-image ] \ @@ -486,7 +488,8 @@ Usage: [-skip-restore ] \ [-stack ] \ [-tag ...] \ - [-uid ] + [-uid ] \ + ``` Running `creator` SHALL be equivalent to running `detector`, `analzyer`, `restorer`, `builder` and `exporter` in order with identical inputs where they are accepted, with the following exceptions. @@ -505,12 +508,13 @@ Outputs produced by `creator` are identical to those produced by `exporter`. #### `rebaser` Usage: ``` -/cnb/lifecycle/rebaser [...] \ +/cnb/lifecycle/rebaser \ [-daemon] \ # sets [-gid ] \ [-log-level ] \ [-run-image | -image ] \ # -image is Deprecated - [-uid ] + [-uid ] \ + [...] ``` | Input | Environment Variable | Default Value | Description From 79bed0960265e0bd01c58170fb53358249867d6a Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Tue, 14 Jul 2020 12:09:30 -0400 Subject: [PATCH 37/38] Specify build reproducibily Signed-off-by: Emily Casey --- platform.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/platform.md b/platform.md index a81fabc4..a8940d67 100644 --- a/platform.md +++ b/platform.md @@ -53,6 +53,7 @@ Examples of a platform might include: - [User-Provided Variables](#user-provided-variables) - [Launch Environment](#launch-environment) - [Caching](#caching) + - [Build Reproducibility](#build-reproducibility) - [Data Format](#data-format) - [Files](#files) - [`analyzed.toml` (TOML)](#analyzedtoml-toml) @@ -459,6 +460,9 @@ Usage: - `io.buildpacks.lifecycle.metadata`: see [lifecycle metadata label (JSON)](#lifecycle-metadata-label-(json)) - `io.buildpacks.project.metadata`: the value of which SHALL be the json representation `` - `io.buildpacks.build.metadata`: see [build metadata (JSON)](#build-metadata-label-(json)) +- To ensure [build reproducibility](#build-reproducibility), the lifecycle: + - SHOULD set the modification time of all files in newly created layers to a constant value + - SHOULD set the `created` time in image config to a constant value If a cache is provided the lifecycle: - SHALL write the contents of all cached layers to the cache @@ -545,6 +549,8 @@ Usage: - The value of `io.buildpacks.lifecycle.metadata` SHALL be modified as follows - `run-image.reference` SHALL uniquely identify `` - `run-image.top-layer` SHALL be set to the uncompressed digest of the top layer in `` +- To ensure [build reproducibility](#build-reproducibility), the lifecycle: + - Set the `created` time in image config to a constant #### `launcher` Usage: @@ -660,6 +666,25 @@ If caching is enabled the platform is responsible for providing the lifecycle wi Whenever possible, the platform SHOULD provide the same cache to each rebuild of a given app image. Cache locality and availability MAY vary between platforms. +### Build Reproducibility +When given identical inputs all build and rebase operations: + - SHOULD produce app images with identical imageIDs + - **If** exporting directly to a registry + - SHOULD produce app images with identical manifest digests + - MAY output other non-reproducible artifacts + +To achieve reproducibility the lifecycle SHOULD set the following to a constant, rather than an accurate value: +- file modification times in generated layers +- image creation time + +Because compressions algorithms and manifest whitespace affect the image digest, an app image exported to the docker daemon and subsequently pushed to a registry MAY have a different digest than an app image exported directly to a registry by the lifecycle, even when all other inputs are held constant. + +If buildpacks do not generate layer contents or layer metadata reproducibly, builds MAY NOT be reproducibile even when identical source code and buildpacks are provided to the lifecycle. + +All app image labels SHOULD contain only reproducible values. + +For more information on build reproducibility see [https://reproducible-builds.org/](https://reproducible-builds.org/) + ## Data Format ### Files From 7cf284fa9e9b7d54d0c62a08ca87f5a864341a28 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 15 Jul 2020 14:59:06 -0400 Subject: [PATCH 38/38] Backout accidental changes to specs besides platform Signed-off-by: Emily Casey --- buildpack.md | 82 ++++++++++------ distribution.md | 12 +-- extensions/bindings.md | 44 ++++++--- extensions/project-descriptor.md | 156 +++++++++++++++++++++++++++++++ 4 files changed, 248 insertions(+), 46 deletions(-) create mode 100644 extensions/project-descriptor.md diff --git a/buildpack.md b/buildpack.md index 11e43ab1..60c5c433 100644 --- a/buildpack.md +++ b/buildpack.md @@ -15,31 +15,59 @@ The `ENTRYPOINT` of the OCI image contains logic implemented by the lifecycle th ## Table of Contents -1. [Buildpack Interface](#buildpack-interface) - 1. [Key](#key) - 2. [Detection](#detection) - 3. [Build](#build) - 4. [Layer Types](#layer-types) -2. [App Interface](#app-interface) -3. [Phase #1: Detection](#phase-1-detection) -4. [Phase #2: Analysis](#phase-2-analysis) -5. [Phase #3: Build](#phase-3-build) -6. [Phase #4: Export](#phase-4-export) -7. [Launch](#launch) -8. [Environment](#environment) - 1. [Provided by the Lifecycle](#provided-by-the-lifecycle) - 2. [Provided by the Platform](#provided-by-the-platform) - 3. [Provided by the Buildpacks](#provided-by-the-buildpacks) -9. [Security Considerations](#security-considerations) - 1. [Assumptions of Trust](#assumptions-of-trust) - 2. [Requirements](#requirements) -10. [Data Format](#data-format) - 1. [launch.toml (TOML)](#launchtoml-toml) - 2. [Build Plan (TOML)](#build-plan-toml) - 3. [Buildpack Plan (TOML)](#buildpack-plan-toml) - 4. [Bill-of-Materials (TOML)](#bill-of-materials-toml) - 5. [Layer Content Metadata (TOML)](#layer-content-metadata-toml) - 6. [buildpack.toml (TOML)](#buildpacktoml-toml) +- [Buildpack Interface Specification](#buildpack-interface-specification) + - [Table of Contents](#table-of-contents) + - [Buildpack Interface](#buildpack-interface) + - [Key](#key) + - [Detection](#detection) + - [Build](#build) + - [Layer Types](#layer-types) + - [Launch Layers](#launch-layers) + - [Build Layers](#build-layers) + - [Other Layers](#other-layers) + - [App Interface](#app-interface) + - [Phase #1: Detection](#phase-1-detection) + - [Purpose](#purpose) + - [Process](#process) + - [Order Resolution](#order-resolution) + - [Phase #2: Analysis](#phase-2-analysis) + - [Purpose](#purpose-1) + - [Process](#process-1) + - [Phase #3: Build](#phase-3-build) + - [Purpose](#purpose-2) + - [Process](#process-2) + - [Buildpack Plan Entry Refinements](#buildpack-plan-entry-refinements) + - [Layers](#layers) + - [Phase #4: Export](#phase-4-export) + - [Purpose](#purpose-3) + - [Process](#process-3) + - [Launch](#launch) + - [Purpose](#purpose-4) + - [Process](#process-4) + - [Environment](#environment) + - [Provided by the Lifecycle](#provided-by-the-lifecycle) + - [Layer Paths](#layer-paths) + - [Provided by the Platform](#provided-by-the-platform) + - [Provided by the Buildpacks](#provided-by-the-buildpacks) + - [Environment Variable Modification Rules](#environment-variable-modification-rules) + - [Delimiter](#delimiter) + - [Prepend](#prepend) + - [Append](#append) + - [Override](#override) + - [Default](#default) + - [Security Considerations](#security-considerations) + - [Assumptions of Trust](#assumptions-of-trust) + - [Requirements](#requirements) + - [Data Format](#data-format) + - [launch.toml (TOML)](#launchtoml-toml) + - [store.toml (TOML)](#storetoml-toml) + - [Build Plan (TOML)](#build-plan-toml) + - [Buildpack Plan (TOML)](#buildpack-plan-toml) + - [Bill-of-Materials (TOML)](#bill-of-materials-toml) + - [Layer Content Metadata (TOML)](#layer-content-metadata-toml) + - [buildpack.toml (TOML)](#buildpacktoml-toml) + - [Buildpack Implementations](#buildpack-implementations) + - [Order Buildpacks](#order-buildpacks) ## Buildpack Interface @@ -93,8 +121,8 @@ Executable: `/bin/build `, Working Dir: `< | `/dev/stdout` | Logs (info) | `/dev/stderr` | Logs (warnings, errors) | `` | Refinements to the [Buildpack Plan](#buildpack-plan-toml) (TOML) -| `/launch.toml` | App metadata (see [launch.toml](#launch.toml-toml)) -| `/store.toml` | Persistent metadata (see [store.toml](#store.toml-toml)) +| `/launch.toml` | App metadata (see [launch.toml](#launchtoml-toml)) +| `/store.toml` | Persistent metadata (see [store.toml](#storetoml-toml)) | `/.toml` | Layer metadata (see [Layer Content Metadata](#layer-content-metadata-toml)) | `//bin/` | Binaries for launch and/or subsequent buildpacks | `//lib/` | Shared libraries for launch and/or subsequent buildpacks diff --git a/distribution.md b/distribution.md index 124886b8..00cc18f7 100644 --- a/distribution.md +++ b/distribution.md @@ -5,11 +5,11 @@ This document specifies the artifact format, delivery mechanism, and order resol ## Table of Contents -1. [Order Resolution](#order-resolution) -2. [Artifact Format](#artifact-format) - 1. [Buildpack](#buildpack) - 2. [Buildpackage](#buildpackage) - +- [Distribution Specification](#distribution-specification) + - [Table of Contents](#table-of-contents) + - [Artifact Format](#artifact-format) + - [Buildpack](#buildpack) + - [Buildpackage](#buildpackage) ## Artifact Format @@ -54,4 +54,4 @@ For each listed stack, all associated buildpacks MUST be a candidate for detecti Each stack ID MUST only be present once. For a given stack, the `mixins` list MUST enumerate mixins such that no included buildpacks are missing a mixin for the stack. -Fewer stack entries as well as additional mixins for a stack entry MAY be specified. \ No newline at end of file +Fewer stack entries as well as additional mixins for a stack entry MAY be specified. diff --git a/extensions/bindings.md b/extensions/bindings.md index 6fae1aee..d4d39119 100644 --- a/extensions/bindings.md +++ b/extensions/bindings.md @@ -1,11 +1,27 @@ # Bindings -Bindings are exposed inside of a container during all three phases of the lifecycle; detect, build, and launch. The contents of bindings MUST NOT be part of the image created after the detect and build phases. + +Bindings are exposed inside of a container during the detect, build, and launch phases of the lifecycle. The contents of bindings MUST NOT be part of the image created after the detect and build phases. + +## Table of Contents + +- [Bindings](#bindings) + - [Table of Contents](#table-of-contents) + - [Detect and Build Phases](#detect-and-build-phases) + - [Metadata](#metadata) + - [Secret](#secret) + - [Example Directory Structure](#example-directory-structure) + - [Launch Phase](#launch-phase) + - [Metadata](#metadata-1) + - [Secret](#secret-1) + - [Example Directory Structure](#example-directory-structure-1) ## Detect and Build Phases -During the detect and build phases, the lifecycle MUST provide any bindings as files in `/bindings/` with directory names matching the name of the binding. Binding names MUST match `[a-z0-9\-\.]{1,253}`. + +Before initiating the detect or build phases on the build-image, the platform MUST provide any bindings as files in `/bindings/` with directory names matching the name of the binding. Binding names MUST match `[a-z0-9\-\.]{1,253}`. ### Metadata -Within each binding directory the lifecycle MUST provide a `metadata` directory containing `kind`, `provider`, and `tags` files. The value of the `kind` file MUST contain an abstract classification of the binding. The value of the `provider` file MUST identify the provider of this binding. The value of the `tags` file MUST contain a newline delimited collection of tags attached to the binding. + +Within each binding directory the platform MUST provide a `metadata` directory containing `kind` and `provider` files. The value of the `kind` file MUST contain an abstract classification of the binding. The value of the `provider` file MUST identify the provider of this binding. In addition to the required files, the `metadata` directory MAY contain additional metadata about the binding with file names and contents matching the metadata names and contents. @@ -14,7 +30,8 @@ The collection of files within the directory MAY change between detect and build The contents of the files MAY change between detect and build phase pairs. The contents of the files MUST NOT change during the detect and build phase pair. ### Secret -Within each binding directory the lifecycle MAY provide a `secret` directory containing the secret associated with the binding with filenames matching the secret key names. + +Within each binding directory the platform MAY provide a `secret` directory containing the secret associated with the binding with filenames matching the secret key names. During the detect and build phases, if the `secret` directory exists, the contents of the files MAY be one of the following: @@ -27,6 +44,7 @@ If the `secret` directory exists, the contents of the files MAY change between d ### Example Directory Structure + ```plain └── bindings @@ -34,14 +52,12 @@ If the `secret` directory exists, the contents of the files MAY change between d │ └── metadata │ ├── connection-count │ ├── kind - │ ├── provider - │ └── tags + │ └── provider └── secondary-db ├── metadata │ ├── connection-count │ ├── kind - │ ├── provider - │ └── tags + │ └── provider └── secret ├── endpoint ├── password @@ -49,10 +65,12 @@ If the `secret` directory exists, the contents of the files MAY change between d ``` ## Launch Phase -During the launch phase, the lifecycle MUST provide any bindings as files in `$CNB_BINDINGS/` with directory names matching the name of the binding. Binding names MUST match `[a-z0-9\-\.]{1,253}`. The `CNB_BINDINGS` environment variable MUST be declared and can point to any valid filesystem location. + +During the launch phase, the platform MUST provide any bindings as files in `$CNB_BINDINGS/` with directory names matching the name of the binding. Binding names MUST match `[a-z0-9\-\.]{1,253}`. The `CNB_BINDINGS` environment variable MUST be declared and can point to any valid filesystem location. ### Metadata -Within each binding directory the lifecycle MUST provide a `metadata` directory containing `kind`, `provider`, and `tags` files. The value of the `kind` file MUST contain an abstract classification of the binding. The value of the `provider` file MUST identify the provider of this binding. The value of the `tags` file MUST contain a newline delimited collection of tags attached to the binding. + +Within each binding directory the platform MUST provide a `metadata` directory containing `kind` and `provider` files. The value of the `kind` file MUST contain an abstract classification of the binding. The value of the `provider` file MUST identify the provider of this binding. In addition to the required files, the `metadata` directory MAY contain additional metadata about the binding with file names and contents matching the metadata names and contents. @@ -61,7 +79,8 @@ The collection of files within the directory MAY change between launches. The c The contents of the files MAY change between launches. The contents of the files MAY change during the launch phase. ### Secret -Within each binding directory the lifecycle MUST provide a `secret` directory containing the secret associated with the binding with filenames matching the secret key names. + +Within each binding directory the platform MUST provide a `secret` directory containing the secret associated with the binding with filenames matching the secret key names. During the launch phase, the contents of the files MAY be one of the following: @@ -73,6 +92,7 @@ The collection of files within the directory MAY change between launches. The c The contents of the files MAY change between launches. The contents of the files MAY change during the launch phase. ### Example Directory Structure + ```plain custom-bindings-location ├── primary-db @@ -80,7 +100,6 @@ custom-bindings-location │ │ ├── connection-count │ │ ├── kind │ │ ├── provider -│ │ └── tags │ └── secret │ ├── endpoint │ ├── password @@ -90,7 +109,6 @@ custom-bindings-location │ ├── connection-count │ ├── kind │ ├── provider - │ └── tags └── secret ├── endpoint ├── password diff --git a/extensions/project-descriptor.md b/extensions/project-descriptor.md new file mode 100644 index 00000000..1aed03bb --- /dev/null +++ b/extensions/project-descriptor.md @@ -0,0 +1,156 @@ +# Project Descriptor + +A project descriptor is a file that MAY contain configuration for apps, services, functions, and buildpacks. By default, the file SHOULD be named `project.toml` and located in the root directory of a project's repository. A platform SHOULD read the file to enrich the build process. + +## Schema + +The TOML schema of the project descriptor is the following: + +```toml +[project] +id = "" # machine readble +name = "" # human readable +version = "" +authors = [""] +documentation-url = "" +source-url = "" + +[[project.licenses]] +type = "" +uri = "" + +[build] +include = [""] +exclude = [""] +[[build.buildpacks]] +id = "" +version = "" +uri = "" +[[build.env]] +name = "" +value = "" +[metadata] +# additional arbitrary keys allowed +``` + +The following sections describe each part of the schema in detail. + +## `[project]` + +The top-level `[project]` table MAY contain configuration about the repository, including `id` and `version`. It MAY also include metadata about how it is authored, documented, and version controlled. + +The `project.id` + +```toml +[project] +id = "" +name = "" +version = "" +``` + +* `id` - (optional) the machine readable identifier of the project (ex. "com.example.myservice") +* `name` - (optional) the human readable name of the project (ex. "My Example Service") +* `version` - (optional) and arbitrary string representing the version of the project +* `authors` - (optional) the names and/or email addresses of the project's authors +* `documentation-url` - (optional) a URL to the documentation for the project +* `source-url` - (optional) a URL to the source code for the project + +## `[[project.licenses]]` + +An optional list of project licenses. + +* `type` - This MAY use the [SPDX 2.1 license expression](https://spdx.org/spdx-specification-21-web-version), but is not limited to identifiers in the [SPDX Licenses List](https://spdx.org/licenses/). +* `uri` - If this project is using a nonstandard license, then this key MAY be specified in lieu of or in addition to `type` to point to the license. + +## `[build.include]` and `[build.exclude]` + +A optional list of files to include in the build (while excluding everything else): + +```toml +[build] +include = [ + "cmd/", + "go.mod", + "go.sum", + "*.go" +] +``` + +A list of files to exclude from the build (while including everything else): + +```toml +[build] +exclude = [ + "spec/" +] +``` + +The `.gitignore` pattern is used in both cases. The `exclude` and `include` keys are mutually exclusive, and if both are present the Lifecycle will error out. + +Any files that are excluded (either via `include` or `exclude`) MUST BE excluded before the build (i.e. not only exluded from the final image). + +If both `exclude` and `include` are defined, the build process MUST result in an error. + +## `[[build.buildpacks]]` + +The build table MAY contain an array of buildpacks. The schema for this table is: + +```toml +[[build.buildpacks]] +id = "" +version = "" +uri = ")" +``` + +This defines the buildpacks that a platform should use on the repo. + +Either an `id` or a `uri` MUST be included, but MUST NOT include both. If `uri` is provided, `version` MUST NOT be allowed. + +## `[[build.env]]` + +Used to set environment variables at build time, for example: + +```toml +[[build.env]] +name = "JAVA_OPTS" +value = "-Xmx1g" +``` + +## `[metadata]` + +This table includes a some defined keys, but additional keys are not validated. It can be used to add platform specific metadata. For example: + +```toml +[metadata.heroku] +pipeline = "foobar" +``` + +## Example + +```toml +[project] +id = "io.buildpacks.my-app" +version = "0.1" + +[build] +include = [ + "cmd/", + "go.mod", + "go.sum", + "*.go" +] + +[[build.buildpacks]] +id = "io.buildpacks/java" +version = "1.0" + +[[build.buildpacks]] +id = "io.buildpacks/nodejs" +version = "1.0" + +[metadata] +foo = "bar" + +[metadata.fizz] +buzz = ["a", "b", "c"] +``` \ No newline at end of file