Merge pull request #664 from buildpacks/consolidate-common-concepts

Consolidate common concepts

config.toml

@@ -91,3 +91,29 @@ featureKatacoda = false
   [security.http]
     methods = ['(?i)GET|POST']
     urls = ['.*']
+
+# For more information, see https://gohugo.io/getting-started/directory-structure/#union-file-system
+[[module.mounts]]
+    source = 'content'
+    target = 'content'
+[[module.mounts]]
+    source = 'content/docs/.common/concepts/buildpack.md'
+    target = 'content/docs/for-app-developers/concepts/buildpack.md'
+[[module.mounts]]
+    source = 'content/docs/.common/concepts/buildpack.md'
+    target = 'content/docs/for-buildpack-authors/concepts/buildpack.md'
+[[module.mounts]]
+    source = 'content/docs/.common/concepts/buildpack.md'
+    target = 'content/docs/for-platform-operators/concepts/buildpack.md'
+[[module.mounts]]
+    source = 'content/docs/.common/concepts/extension.md'
+    target = 'content/docs/for-buildpack-authors/concepts/extension.md'
+[[module.mounts]]
+    source = 'content/docs/.common/concepts/extension.md'
+    target = 'content/docs/for-platform-operators/concepts/extension.md'
+[[module.mounts]]
+    source = 'content/docs/.common/concepts/composite-buildpack.md'
+    target = 'content/docs/for-buildpack-authors/concepts/composite-buildpack.md'
+[[module.mounts]]
+    source = 'content/docs/.common/concepts/composite-buildpack.md'
+    target = 'content/docs/for-platform-operators/concepts/composite-buildpack.md'

content/docs/.common/concepts/buildpack.md

@@ -0,0 +1,54 @@
++++
+title="What is a buildpack?"
+weight=1
++++
+
+A `buildpack` is software that transforms application source code into runnable artifacts
+by analyzing the code and determining the best way to build it.
+
+<!--more-->
+
+![buildpacks](/images/what.svg)
+
+## Why buildpacks?
+
+Buildpacks allow application developers to focus on what they do best - writing code, without having to worry about image security, optimizing container images, or container build strategy.
+
+How much time have you spent struggling to wrangle yet another Dockerfile? Copying and pasting random Dockerfile snippets into every project? Buildpacks can help! They are a better approach to building container images for applications.
+
+## What do they look like?
+
+Typical buildpacks consist of at least three files:
+
+* `buildpack.toml` -- provides metadata about the buildpack, containing information such as its name, ID, and version.
+* `bin/detect` -- performs [detect](#detect).
+* `bin/build` -- performs [build](#build).
+
+## How do they work?
+
+![how](/images/how.svg)
+
+**Each buildpack has two jobs to do**
+
+### Detect
+
+The buildpack determines if it is needed or not.
+
+For example:
+
+- A Python buildpack may look for a `requirements.txt` or a `setup.py` file.
+- A Node buildpack may look for a `package-lock.json` file.
+
+### Build
+
+The buildpack transforms application source code in some way, for example by
+
+- Setting build-time and run-time environment variables.
+- Downloading dependencies.
+- Running source code compilation (if needed).
+- Configuring the application entrypoint and any startup scripts.
+
+For example:
+
+- A Python buildpack may run `pip install -r requirements.txt` if it detected a `requirements.txt` file.
+- A Node buildpack may run `npm install` if it detected a `package-lock.json` file.

content/docs/.common/concepts/composite-buildpack.md

@@ -0,0 +1,10 @@
++++
+title="What is a composite buildpack?"
+weight=99
++++
+
+A composite buildpack, also sometimes called a "meta-buildpack",
+is a buildpack that does not contain any `./bin/detect` or `./bin/build` binaries,
+but instead references other buildpacks in its `buildpack.toml` via the `[[order]]` array.
+
+This is useful for composing more complex detection strategies.

content/docs/.common/concepts/extension.md

@@ -0,0 +1,84 @@
+
++++
+title="What is an image extension?"
+aliases=[
+  "/docs/features/dockerfiles"
+]
+weight=99
++++
+
+An `image extension` is software that generates Dockerfiles that can be used to extend base images for buildpacks builds.
+
+<!--more-->
+
+## Why image extensions?
+
+Buildpacks can do a lot, but there are some things buildpacks can't do. They can't install operating system packages,
+for example. Why not?
+
+Buildpacks do not run as the `root` user and cannot make arbitrary changes to the filesystem. This enhances security,
+enables buildpack interoperability, and preserves the ability to rebase - but it comes at a cost. Base image authors
+must anticipate the OS-level dependencies that will be needed at build and run-time ahead of time, and this isn't always
+possible.
+
+This has been a longstanding source of [discussion](https://github.com/buildpacks/rfcs/pull/173) within the CNB project:
+how can we preserve the benefits of buildpacks while enabling more powerful capabilities?
+
+### Buildpacks and Dockerfiles can work together
+
+Buildpacks are often presented as an alternative to Dockerfiles, but we think buildpacks and Dockerfiles can work
+together.
+
+Buildpacks are optimized for creating layers that are efficient and logically mapped to the dependencies that they
+provide. By contrast, Dockerfiles are the most-used and best-understood mechanism for constructing base images and
+installing OS-level dependencies for containers.
+
+The CNB Dockerfiles feature allows Dockerfiles to "provide" dependencies that buildpacks "require" through a
+shared [build plan], by introducing the concept of image extensions.
+
+## What do they look like?
+
+Image extensions are buildpack-like components that use a restricted `Dockerfile` syntax to expand base images. Their
+purpose is to generate Dockerfiles that can be used to extend the builder or run images prior to buildpacks builds.
+
+An image extension could be defined with the following directory:
+
+```
+.
+├── extension.toml <- similar to a buildpack buildpack.toml
+├── bin
+│   ├── detect     <- similar to a buildpack ./bin/detect
+│   ├── generate   <- similar to a buildpack ./bin/build
+```
+
+* The `extension.toml` provides metadata about the extension, containing information such as its name, ID, and version.
+* `./bin/detect` performs [detect](#detect). It analyzes application source code to determine if the extension
+  is needed and contributes build plan entries.
+* `./bin/generate` performs [generate](#generate) (a new lifecycle phase that happens after `detect`). It
+  outputs either or both of `build.Dockerfile` or `run.Dockerfile` for extending the builder or run image.
+
+## How do they work?
+
+**Each image extension has two jobs to do**
+
+### Detect
+
+The extension determines if it is needed or not.
+
+Like buildpacks, extensions participate in the `detect` phase - analyzing application source code to determine if they
+are needed. During `detect`, extensions can contribute to
+the [build plan] - recording dependencies that they are able to "
+provide" (though unlike buildpacks, they can't "require" anything).
+
+If the provided order contains extensions, the output of `detect` will be a group of image extensions and a group of
+buildpacks that together produce a valid build plan. Image extensions only generate Dockerfiles - they don't create
+layers or participate in the `build` phase.
+
+### Generate
+
+The extension outputs Dockerfiles that can be used to extend either or both of the build-time base image and the runtime base image.
+
+For more information and to see a build in action,
+see [authoring an image extension](/docs/for-buildpack-authors/tutorials/basic-extension).
+
+[build plan]: /docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan

content/docs/for-app-developers/how-to/_index.md

@@ -1,6 +1,5 @@
 +++
 title="How To"
 weight=3
-
 include_summaries=true
 +++

content/docs/for-buildpack-authors/concepts/_index.md

@@ -1,6 +1,5 @@
 +++
 title="Concepts"
 weight=2
-
 include_summaries=true
 +++

content/docs/for-buildpack-authors/concepts/buildpack-group.md

@@ -57,4 +57,4 @@ The [Operator's Guide][operator-guide] has more information on creating builders
 [order-resolution]: https://github.com/buildpacks/spec/blob/main/buildpack.md#order-resolution
 [operator-guide]: /docs/for-platform-operators/
 [builder]: /docs/for-platform-operators/concepts/builder/
-[meta-buildpack]: /docs/for-platform-operators/concepts/buildpack/#meta-buildpack
+[meta-buildpack]: /docs/for-platform-operators/concepts/composite-buildpack

content/docs/for-buildpack-authors/concepts/caching-strategies.md

@@ -14,7 +14,7 @@ There are three types of layers that can be contributed to an image
 
 * `build` layers -- the directory will be accessible by subsequent buildpacks,
 * `cache` layers -- the directory will be included in the cache,
-* `launch` layers -- the directory will be included in the run image as a single layer,
+* `launch` layers -- the directory will be included in the final app image as a single layer,
 
 In this section we look at caching each layer type.
 

content/docs/for-buildpack-authors/how-to/_index.md

@@ -1,6 +1,5 @@
 +++
 title="How To"
 weight=3
-
 include_summaries=true
 +++

content/docs/for-buildpack-authors/how-to/distribute-buildpacks/craft-order.md

@@ -0,0 +1,10 @@
++++
+title="Craft a buildpack order"
+weight=99
++++
+
+<!--more-->
+
+This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
+
+If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)

content/docs/for-buildpack-authors/how-to/distribute-buildpacks/package-buildpack.md

@@ -4,7 +4,7 @@ title="Package a buildpack or extension"
 aliases=[
   "/docs/buildpack-author-guide/package-a-buildpack"
 ]
-weight=5
+weight=4
 summary="Learn how to package your buildpack or extension for distribution."
 +++
 

content/docs/for-buildpack-authors/how-to/migrate/buildpack-api-0.8-0.9.md

@@ -33,6 +33,6 @@ For older platforms (Platform API version 0.9 and below), arguments in `command`
 
 ### Image extensions are supported (experimental)
 
-Platform 0.10 introduces image extensions as experimental components for customizing build and run-time base images (see [here](/docs/for-platform-operators/concepts/dockerfiles) for more information).
+Platform 0.10 introduces image extensions as experimental components for customizing build and run-time base images (see [here](/docs/for-platform-operators/concepts/extension) for more information).
 
 For more information, see our tutorial on [authoring an image extension](/docs/for-buildpack-authors/tutorials/basic-extension).

content/docs/for-buildpack-authors/how-to/write-buildpacks/add-sbom.md

@@ -1,6 +1,6 @@
 +++
 title="Provide a Software Bill-of-Materials"
-weight=2
+weight=5
 +++
 
 Buildpacks can provide a [Software `Bill-of-Materials`](https://en.wikipedia.org/wiki/Software_bill_of_materials) (SBOM)

content/docs/for-buildpack-authors/how-to/write-buildpacks/create-layer.md

@@ -1,10 +1,50 @@
 +++
 title="Create dependency layers"
-weight=99
+weight=3
 +++
 
+Each directory created by the buildpack under the `CNB_LAYERS_DIR` can be used as a layer in the final app image or build cache.
+
 <!--more-->
 
-This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
+That is, each directory can be used for any of the following purposes:
+
+| Layer Type |                                                                                                             |
+|------------|-------------------------------------------------------------------------------------------------------------|
+| `Launch`   | the directory will be included in the **final app image** as a single layer                                 |
+| `Cache`    | the directory will be included in the **build cache** and restored to the `CNB_LAYERS_DIR` on future builds |
+| `Build`    | the directory will be accessible to **buildpacks that follow** in the build (via the environment)           |
+
+A buildpack can control how a layer will be used by creating a `<layer>.toml` with a name matching the directory it describes in the `CNB_LAYERS_DIR`.
+
+### Example
+
+A buildpack might create a `$CNB_LAYERS_DIR/python` directory and a `$CNB_LAYERS_DIR/python.toml` with the following contents:
+
+```
+launch = true
+cache = true
+build = true
+```
+
+In this example:
+* the final app image will contain a layer with `python`, as this is needed to run the app
+* the `$CNB_LAYERS_DIR/python` directory will be pre-created for future builds, avoiding the need to re-download this large dependency
+* buildpacks that follow in the build will be able to use `python`
+
+### Example
+
+This is a simple `./bin/build` script for a buildpack that runs Python's `pip` package manager to resolve dependencies:
+
+```
+#!/bin/sh
+
+PIP_LAYER="$CNB_LAYERS_DIR/pip"
+mkdir -p "$PIP_LAYER/modules" "$PIP_LAYER/env"
+
+pip install -r requirements.txt -t "$PIP_LAYER/modules" \
+  --install-option="--install-scripts=$PIP_LAYER/bin" \
+  --exists-action=w --disable-pip-version-check --no-cache-dir
 
-If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
+echo "launch = true" > "$PIP_LAYER.toml"
+```