diff --git a/_implementors/contributing.md b/_implementors/contributing.md index 69d896cb..8060bb71 100644 --- a/_implementors/contributing.md +++ b/_implementors/contributing.md @@ -3,7 +3,7 @@ layout: implementors title: "How to contribute to the Development Container Specification" shortTitle: "Contributing" author: Microsoft -index: 5 +index: 7 --- We're excited for your contributions to the Dev Container Specification! This document outlines how you can get involved. @@ -50,4 +50,4 @@ We use the following [labels](https://github.com/microsoft/dev-container-spec/la - `proposal`: Issues under discussion, still collecting feedback. - `finalization`: Proposals we intend to make part of the spec. -[Milestones](https://github.com/microsoft/dev-container-spec/milestones) use a "month year" pattern (i.e. January 2022). If a finalized proposal is added to a milestone, it is intended to be merged during that milestone. \ No newline at end of file +[Milestones](https://github.com/microsoft/dev-container-spec/milestones) use a "month year" pattern (i.e. January 2022). If a finalized proposal is added to a milestone, it is intended to be merged during that milestone. diff --git a/_implementors/features-distribution.md b/_implementors/features-distribution.md new file mode 100644 index 00000000..02e45e56 --- /dev/null +++ b/_implementors/features-distribution.md @@ -0,0 +1,142 @@ +--- +layout: implementors +title: "Dev container Features contribution and discovery [proposal]" +shortTitle: "Features distribution" +author: Microsoft +index: 6 +--- + +> Note: This section provides information on a currently active proposal. See the [Features distribution proposal in the spec repo](https://github.com/devcontainers/spec/issues/61) for input and links to other proposed improvements. + +This specification defines a pattern where community members and organizations can author and self-publish [dev container 'features'](./features.md). + +Goals include: + +- For community authors, a "self-served" mechanism for dev container feature publishing, either publicly or privately. +- For users, the ability to validate the integrity of previously fetched assets. +- For users, the ability for a user to pin to a particular version (absolute, or semantic version) of a feature to allow for consistent, repeatable environments. +- The ability to standardize publishing such that [supporting tools](../supporting.md) may implement mechanisms for feature discoverability. + +## Source Code + +Features source code is stored in a git repository. + +For ease of authorship and maintenance, [1..n] features can share a single git repository. This set of features is referred to as a collection, and will share the same [`devcontainer-collection.json`](#devcontainer-collection.json) file and 'namespace' (eg. `/`). + +Source code for the set follows the example file structure below: + +``` +. +├── README.md +├── src +│ ├── dotnet +│ │ ├── devcontainer-feature.json +│ │ ├── install.sh +│ │ └── ... +| ├ +│ ├── go +│ │ ├── devcontainer-feature.json +│ │ └── install.sh +| ├── ... +│ │ ├── devcontainer-feature.json +│ │ └── install.sh +├── test +│ ├── dotnet +│ │ ├── test.sh +│ │ └── ... +│ └── go +│ | └── test.sh +| ├── ... +│ │ └── test.sh +├── ... +``` + +Where `src` is a directory containing a sub-folder with the name of the feature (e.g. `src/dotnet` or `src/go`) with at least a file named `devcontainer-feature.json` that contains the feature metadata, and an `install.sh` script that implementing tools will use as the entrypoint to install the feature. Each sub-directory should be named such that it matches the `id` field of the `devcontainer-feature.json`. Other files can also be included in the feature's sub-directory, and will be included during the [packaging step](#packaging) alongside the two required files. Any files that are not part of the feature's sub-directory (e.g. outside of `src/dotnet`) will not included in the [packaging step](#packaging). + +Optionally, a mirrored `test` directory can be included with an accompanying `test.sh` script. Implementing tools may use this to run tests against the given feature. + +## Versioning + +Each feature is individually [versioned according to the semver specification](https://semver.org/). The `version` property in the respective `devcontainer-feature.json` file is parsed to determine if the feature should be republished. + +Tooling that handles publishing features will not republish features if that exact version has already been published; however, tooling must republish major and minor versions in accordance with the semver specification. + +## Packaging + +Features are distributed as tarballs. The tarball contains the entire contents of the feature sub-directory, including the `devcontainer-feature.json`, `install.sh`, and any other files in the directory. + +The tarball is named `devcontainer-feature-.tgz`, where `` is the feature's `id` field. + +A reference implementation for packaging and distributing features is provided as a GitHub Action (https://github.com/devcontainers/action). + +### devcontainer-collection.json + +The `devcontainer-collection.json` is an auto-generated metadata file. + +| Property | Type | Description | +| :--- | :--- | :--- | +| sourceInformation | object | Metadata from the implementing packaging tool. | +| features | array | The list of features that are contained in this collection.| + +Each features's `devcontainer-feature.json` metadata file is appended into the `features` top-level array. + +## Distribution + +There are several supported ways to distribute features. Distribution is handled by the implementing packaging tool. + +A user references a distributed feature in a `devcontainer.json` as defined in ['referencing a feature'](./features.md#referencing-a-feature). + +### OCI Registry + +An OCI registry that implements the [OCI Artifact Distribution Specification](https://github.com/opencontainers/distribution-spec) serves as the primary distribution mechanism for features. + +Each packaged feature is pushed to the registry following the naming convention `//[:version]`, where version is the major, minor, and patch version of the feature, according to the semver specification. + +> The `namespace` is a unique indentifier for the collection of features. There are no strict rules for the `namespace`; however, one pattern is to set `namespace` equal to source repository's `/`. + +A custom media type `application/vnd.devcontainers` and `application/vnd.devcontainers.layer.v1+tar` are used as demonstrated below. + +For example, the `go` feature in the `devcontainers/features` namespace at version `1.2.3` would be pushed to the ghcr.io OCI registry. + +_NOTE: The example below uses [`oras`](https://oras.land/) for demonstration purposes. A supporting tool should directly implement the required functionality from the aforementioned OCI artifact distribution specification._ +```bash +# ghcr.io/devcontainers/features/go:1 +REGISTRY=ghcr.io +NAMESPACE=devcontainers/features +FEATURE=go + +ARTIFACT_PATH=devcontainer-feature-go.tgz + +for VERSION in 1 1.2 1.2.3 latest +do + oras push ${REGISTRY}/${NAMESPACE}/${FEATURE}:${VERSION} \ + --manifest-config /dev/null:application/vnd.devcontainers \ + ./${ARTIFACT_PATH}:application/vnd.devcontainers.layer.v1+tar +done +``` + +`Namespace` is the globally identifiable name for the collection of features. (eg: `owner/repo` for the source code's git repository). + +The auto-generated `devcontainer-collection.json` is pushed to the registry with the same `namespace` as above and no accompanying `feature` name. The collection file is always tagged as `latest`. + +```bash +# ghcr.io/devcontainers/features +REGISTRY=ghcr.io +NAMESPACE=devcontainers/features + +oras push ${REGISTRY}/${NAMESPACE}:latest \ + --manifest-config /dev/null:application/vnd.devcontainers \ + ./devcontainer-collection.json.:application/vnd.devcontainers.layer.v1+json +``` + +### Directly Reference Tarball + +A feature can be referenced directly in a user's [`devcontainer.json`](./spec.md/#a-hrefdevcontainerjson-namedevcontainerjson-classanchor-devcontainerjson-a) file by an HTTP or HTTPS URI that points to the tarball from the [package step](#packaging). + +### Addendum: Locally Referenced + +To aid in feature authorship, or in instances where a feature should not be published externally, individual features can be referenced locally from the project's file tree. + +A feature can be referenced directly in a user's [`devcontainer.json`](./spec.md/#a-hrefdevcontainerjson-namedevcontainerjson-classanchor-devcontainerjson-a) by relative path _inside_ the project directory. A local feature may not be referenced outside of the project directory (`../` is not allowed), nor is an absolute path allowed. + +The provided relative path is a path to the folder containing the feature's `devcontainer-feature.json` and `install.sh` file. \ No newline at end of file diff --git a/_implementors/features.md b/_implementors/features.md new file mode 100644 index 00000000..0954cd7a --- /dev/null +++ b/_implementors/features.md @@ -0,0 +1,267 @@ +--- +layout: implementors +title: "Dev container Features reference [proposal]" +shortTitle: "Features" +author: Microsoft +index: 5 +--- + +> Note: This section provides information on a currently active proposal. See the [Features proposal in the spec repo](https://github.com/devcontainers/spec/issues/61) for input and links to other proposed improvements. + +Development container "Features" are self-contained, shareable units of installation code and development container configuration. The name comes from the idea that referencing one of them allows you to quickly and easily add more tooling, runtime, or library "features" into your development container for you or your collaborators to use. + +> While 'features' may be installed on top of any base image, the implementation of a feature might restrict it to a subset of possible base images. +> +> For example, some features may be authored to work with a certain linux distro (e.g. debian-based images that use the `apt` package manager). + +Feature metadata is captured by a `devcontainer-feature.json` file in the root folder of the feature. + +## Folder Structure + +A feature is a self contained entity in a folder with at least a `devcontainer-feature.json` and `install.sh` entrypoint script. Additional files are permitted and are packaged along side the required files. + +``` ++-- feature +| +-- devcontainer-feature.json +| +-- install.sh +| +-- (other files) +``` + +## devcontainer-feature.json properties + +the `devcontainer-feature.json` file defines information about the feature to be used by any supporting tools and the way the feature will be executed. + +The properties of the file are as follows: + +| Property | Type | Description | +| :--- | :--- | :--- | +| id | string | Id of the feature/definition. The id should be unique in the context of the repository/published package where the feature exists and must match the name of the directory where the `devcontainer-feature.json` resides. | +| version | string | The semantic version of the feature. | +| name | string | Name of the feature/definition. | +| description | string | Description of the feature/definition. | +| documentationURL | string | Url that points to the documentation of the feature. | +| licenseURL | string | Url that points to the license of the feature. | +| keywords | array | List of keywords relevant to a user that would search for this definition/feature. | +| options | object | Possible options to be passed as environment variables to the execution of the scripts | +| containerEnv | object | A set of name value pairs that sets or overrides environment variables. | +| privileged | boolean | If privileged mode is required by the feature. | +| init | boolean | If it's necessary to run `init`. | +| capAdd | array | Additional capabilities needed by the feature. | +| securityOpt | array | Security options needed by the feature. | +| entrypoint | string | Set if the feature requires an entrypoint. | +| customizations | object | Product specific properties, each namespace under `customizations` is treated as a separate set of properties. For each of this sets the object is parsed, values are replaced while arrays are set as a union. | +| installsAfter | array | Array of Id's of features that should execute before this one. Allows control for feature authors on soft dependencies between different features. | + +Options + +| Property | Type | Description | +| :--- | :--- | :--- | +| id | string | Id of the option. | +| id.type | string | Type of the option. | +| id.enum | string array | Possible values. | +| id.default | string | Default value for the option. | +| id.description | string | Description for the option. | + +## devcontainer.json properties + +Features are referenced in a user's [`devcontainer.json`](/docs/specs/devcontainer-reference.md#devcontainerjson) under the top level `features` object. + +A user can specify an arbitrary number of features. At build time, these features will be installed in an order defined by a combination of the [installation order rules and implementation](#Installation-Order). + +A single feature is provided as a key/value pair, where the key is the feature identifier, and the value is an object containing "options" (or empty for "default"). Each key in the feature object must be unique. + +These options are sourced as environment variables at build-time, as specified in [Option Resolution](#Option-Resolution). + +Below is a valid `features` object provided as an example. +```jsonc +"features": { + "ghcr.io/user/repo/go": {}, + "ghcr.io/user/repo1/go:1": {}, + "ghcr.io/user/repo2/go:latest": {}, + "https://github.com/user/repo/releases/devcontainer-feature-go.tgz": { + "optionA": "value" + }, + "./myGoFeature": { + "optionA": true, + "optionB": "hello", + "version" : "1.0.0" + } +} +``` + +> Note: The `:latest` version annotation is added implicitly if omitted. To pin to a specific package version ([example](https://github.com/devcontainers/features/pkgs/container/features/go/versions)), append it to the end of the feature. + +An option's value can be provided as either a `string` or `boolean`, and should match what is expected by the feature in the `devcontainer-feature.json` file. + +As a shorthand, the value of a `feature` can be provided as a single string. This string is mapped to an option called `version`. In the example below, both examples are equivalent. + +```jsonc +"features": { + "ghcr.io/owner/repo/go": "1.18" +} +``` +```jsonc +"features": { + "ghcr.io/owner/repo/go": { + "version": "1.18" + } +} +``` + +### Referencing a feature + +The `id` format specified dicates how a supporting tool will locate and download a given feature. `id` is one of the following: + +| Type | Description | Example | +| :--- | :--- | :--- | +| `//[:]` | Reference to feature in OCI registry(*) | ghcr.io/user/repo/go
ghcr.io/user/repo/go:1
ghcr.io/user/repo/go:latest| +| `https://` | Direct HTTPS URI to a tarball. | https://github.com/user/repo/releases/devcontainer-feature-go.tgz | +| `./`| A relative directory to folder containing a devcontainer-feature.json. | ./myGoFeature | + +` +(*) OCI registry must implement the [OCI Artifact Distribution Specification](https://github.com/opencontainers/distribution-spec). Some implementors can be [found here](https://oras.land/implementors/). + +## Versioning + +Each feature is individually [versioned according to the semver specification](https://semver.org/). The `version` property in the respective `devcontainer-feature.json` file is updated to increment the feature's version. + +Tooling that handles releasing features will not republish features if that exact version has already been published; however, tooling must republish major and minor versions in accordance with the semver specification. + +## Authoring + +Features can be authored in a number of languages, the most straightforward being bash scripts. If a feature is authored in a different language information about it should be included in the metadata so that users can make an informed choice about it. + +Reference information about the application required to execute the feature should be included in `devcontainer-feature.json` in the metadata section. + +Applications should default to `/bin/sh` for features that do not include this information. + +If the feature is included in a folder as part of the repository that contains `devcontainer.json`, no other steps are necessary. + +## Release + +_For information on distribution features, see [the dev container features distribution proposal](./features-distribution.md)._ + +## Execution + +### Installation Order + +By default, features are installed on top of a base image in an order determined as optimal by the implementing tool. + +If any of the following properties are provided in the feature's `devcontainer-feature.json`, or the user's `devcontainer.json`, the order indicated by these propert(ies) are respected (with decreasing precedence). + +1. The `overrideFeatureInstallOrder` property in user's `devcontainer.json`. Allows users to control the order of execution of their features. +2. The `installsAfter` property defined as part of a feature's `devcontainer-feature.json`. + +#### (1) overrideFeatureInstallOrder + +This property is declared by the user in their `devcontainer.json` file. + +Any feature IDs listed in this array will be installed before all other features, in the provided order. Any omitted features will be installed in an order selected by the implementing tool, or ordered via the `installsAfter` property _after_ any features listed in the `overrideFeatureInstallOrder` array, if applicable. + +All feature `id` provided in `overrideFeatureInstallOrder` must also exist in the `features` property of a user's `devcontainer.json`. + +| Property | Type | Description | +| :--- | :--- | :--- | +| overrideFeatureInstallOrder | array | Array consisting of the feature `id` of features in the order the user wants them to be installed. | + +#### (2) installsAfter + +This property is defined in an individual feature's `devcontainer-feature.json` file by the feature author. `installsAfter` allows an author to provide the tooling hints on loose dependencies between features. + +After `overrideFeatureInstallOrder` is resolved, any remaining features that declare an `installsAfter` must be installed after the features declared in the property, provided that the features have also been declared in the `features` property. + +| Property | Type | Description | +| :--- | :--- | :--- | +| installsAfter | array | Array consisting of the feature `id` that should be installed before the given feature | + +### Option Resolution + +A feature's 'options' - specified as the value of a single feature key/value pair in the user's `devcontainer.json` - are passed to the feature as environment variables. + +A supporting tool will parse the `options` object provided by the user. If a value is provided for a feature, it will be emitted to a file named `devcontainer-features.env` following the format `=`. + +To ensure a option that is valid as an environment variable, the follow substitutions are performed. + +```javascript +(str: string) => str + .replace(/[^\w_]/g, '_') + .replace(/^[\d_]+/g, '_') + .toUpperCase(); +``` + +This file is sourced at build-time for the feature `install.sh` entrypoint script to handle. + +Any options defined by a feature's `devcontainer-feature.json` that are omitted in the user's `devcontainer.json` will be implicitly exported as its default value. + +### Option Resolution Example + +Suppose a `python` feature has the following `options` parameters declared in the `devcontainer-feature.json` file: + +```jsonc +// ... +"options": { + "version": { + "type": "string", + "enum": ["latest", "3.10", "3.9", "3.8", "3.7", "3.6"], + "default": "latest", + "description": "Select a Python version to install." + }, + "pip": { + "type": "boolean", + "default": true, + "description": "Installs pip" + }, + "optimize": { + "type": "boolean", + "default": true, + "description": "Optimize python installation" + } +} +``` + +The user's `devcontainer.json` declared the python feature like so + +```jsonc + +"features": { + "python": { + "version": "3.10", + "pip": false + } +} +``` +The emitted environment variables will be: + +```env +VERSION="3.10" +PIP="false" +OPTIMIZE="true" +``` + +These will be sourced and visible to the `install.sh` entrypoint script. The following `install.sh`... + +```bash +#!/usr/bin/env bash + +echo "Version is $VERSION" +echo "Pip? $PIP" +echo "Optimize? $OPTIMIZE" +``` + +...output the following: +``` +Version is 3.10 +Pip? false +Optimize? true +``` + +### Implementation Notes + +There are several things to keep in mind for an application that implements features: + +- The order of execution of features is determined by the application, based on the `installAfter` property used by feature authors. It can be overridden by users if necessary with the `overrideFeatureInstallOrder` in `devcontainer.json`. +- Features are used to create an image that can be used to create a container or not. +- Parameters like `privileged`, `init` are included if just 1 feature requires them. +- Parameters like `capAdd`, `securityOp` are concatenated. +- `containerEnv` is added before the feature is executed as `ENV` commands in the Dockerfile. +- Each feature script executes as its own layer to aid in caching and rebuilding. \ No newline at end of file diff --git a/_implementors/json_reference.md b/_implementors/json_reference.md index b85c9210..0dbd2287 100644 --- a/_implementors/json_reference.md +++ b/_implementors/json_reference.md @@ -54,7 +54,9 @@ The focus of `devcontainer.json` is to describe how to enrich a container for th | `updateRemoteUserUID` | boolean | On Linux, if `containerUser` or `remoteUser` is specified, the container user's UID/GID will be updated to match the local user's UID/GID to avoid permission problems with bind mounts. Defaults to `true`.
Requires the container be recreated / rebuilt for updates to take effect. | | `userEnvProbe` | enum | Indicates the type of shell to use to "probe" for user environment variables to include in `devcontainer.json` supporting services' / tools' processes: `none`, `interactiveShell`, `loginShell`, or `loginInteractiveShell` (default). The specific shell used is based on the default shell for the user (typically bash). For example, bash interactive shells will typically include variables set in `/etc/bash.bashrc` and `~/.bashrc` while login shells usually include variables from `/etc/profile` and `~/.profile`. Setting this property to `loginInteractiveShell` will get variables from all four files. | | `overrideCommand` | boolean | Tells `devcontainer.json` supporting services / tools whether they should run `/bin/sh -c "while sleep 1000; do :; done"` when starting the container instead of the container's default command (since the container can shut down if the default command fails). Set to `false` if the default command must run for the container to function properly. Defaults to `true` for when using an image Dockerfile and `false` when referencing a Docker Compose file. | -| `features` (currently in preview) | object | An object of dev container features and related options to be added into your primary container. The specific options that are available varies by feature, so see its documentation for additional details. For example:
`"features": {"github-cli": "latest"}`

⚠️ Currently in preview. | +| `features` (proposed) | object | An object of dev container features and related options to be added into your primary container. The specific options that are available varies by feature, so see [its documentation](./features.md#a-hrefdevcontainer-json-properties-namedevcontainer-json-properties-classanchor-devcontainerjson-properties-a) for additional details. For example:
`"features": {"github-cli": "latest"}`

⚠️ Support is currently in preview as the [spec](./features.md) is not yet final. | + +Maybe we specifically refer to this as proposed to tie the two together? | `shutdownAction` | enum | Indicates whether `devcontainer.json` supporting tools should stop the containers when the related tool window is closed / shut down.
Values are `none`, `stopContainer` (default for image or Dockerfile), and `stopCompose` (default for Docker Compose). | | `customizations` | object | Product specific properties, defined in [supporting tools](supporting-tools.md) | {: .table .table-bordered .table-responsive} diff --git a/_implementors/spec.md b/_implementors/spec.md index 68852d8d..df910f7d 100644 --- a/_implementors/spec.md +++ b/_implementors/spec.md @@ -67,6 +67,12 @@ In addition to the configuration options explained above, there are other settin A complete list of available metadata properties and their purposes can be found in the [`devcontainer.json` reference](https://aka.ms/devcontainer.json). However, we will describe the critical ones below in more detail. +## Features (preview) + +Development container "Features" are self-contained, shareable units of installation code and development container configuration. The name comes from the idea that referencing one of them allows you to quickly and easily add more tooling, runtime, or library "features" into your development container for you or your collaborators to use. + +They are applied to container images as a secondary build step and can affect a number of dev container configuration settings. See the [features documentation](./features.md) for more details. + ## Environment variables Environment variables can be set at different points in the dev container lifecycle. With this in mind, **development containers** support two classes of environment variables: