Skip to content

Commit

Permalink
Merge branch 'main' into isolated-change
Browse files Browse the repository at this point in the history
  • Loading branch information
archen committed Aug 18, 2023
2 parents 4cdad24 + 79f1d8f commit 880ba02
Show file tree
Hide file tree
Showing 40 changed files with 600 additions and 89 deletions.
1 change: 1 addition & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ jobs:
uses: bazel-contrib/.github/.github/workflows/release_ruleset.yaml@2ded56de883b35b1e18b37566fd406625153f1e2
with:
release_files: rules_oci-*.tar.gz
prerelease: false
29 changes: 28 additions & 1 deletion .github/workflows/release_prep.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,34 @@ git archive --format=tar --prefix=${PREFIX}/ ${TAG} | gzip > $ARCHIVE
SHA=$(shasum -a 256 $ARCHIVE | awk '{print $1}')

cat << EOF
WORKSPACE snippet:
## Using bzlmod with Bazel 6 or later:
1. Add \`common --enable_bzlmod\` to \`.bazelrc\`.
2. Add to your \`MODULE.bazel\` file:
\`\`\`starlark
bazel_dep(name = "rules_oci", version = "${TAG:1}")
# For testing, we also recommend https://registry.bazel.build/modules/container_structure_test
oci = use_extension("@rules_oci//oci:extensions.bzl", "oci")
# Declare external images you need to pull, for example:
oci.pull(
name = "distroless_base",
# 'latest' is not reproducible, but it's convenient.
# During the build we print a WARNING message that includes recommended 'digest' and 'platforms'
# values which you can use here in place of 'tag' to pin for reproducibility.
tag = "latest",
image = "gcr.io/distroless/base",
platforms = ["linux/amd64"],
)
# For each oci.pull call, repeat the "name" here to expose them as dependencies.
use_repo(oci, "distroless_base")
\`\`\`
## Using WORKSPACE:
\`\`\`starlark
Expand Down
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

This is a "barebones" alternative to [rules_docker](https://github.com/bazelbuild/rules_docker).

A lot of companies have already done a successful migration from `rules_docker`. Please let us know about yours on our adoption discussion!
<https://github.com/bazel-contrib/rules_oci/discussions/299>

We start from first principles and avoided some pitfalls we learned in maintaining that repo:

- Use a toolchain consisting of off-the-shelf, pre-built layer and container manipulation tools.
Expand All @@ -15,12 +18,11 @@ _Need help?_ This ruleset has support provided by https://aspect.dev.

## Installation

- Bazel >= 6.2.0 with `--enable_bzlmod`: start from <https://registry.bazel.build/modules/rules_oci>
- Others: Copy the WORKSPACE snippet into your `WORKSPACE` file from a release: <https://github.com/bazel-contrib/rules_oci/releases>
See the install instructions on the release notes: <https://github.com/bazel-contrib/rules_oci/releases>

To use a commit rather than a release, you can point at any SHA of the repo.

For example to use commit `abc123`:
With bzlmod, you can use `archive_override` or `git_override`. For `WORKSPACE`, you modify the `http_archive` call; for example to use commit `abc123` with a `WORKSPACE` file:

1. Replace `url = "https://github.com/bazel-contrib/rules_oci/releases/download/v0.1.0/rules_oci-v0.1.0.tar.gz"`
with a GitHub-provided source archive like `url = "https://github.com/bazel-contrib/rules_oci/archive/abc123.tar.gz"`
Expand All @@ -43,6 +45,7 @@ rules_oci does not contain language-specific rules, but we do have limited docum
- [Rust](docs/rust.md)
- [Scala](docs/scala.md)
- [WASM](https://github.com/bazel-contrib/rules_oci/tree/main/e2e/wasm) (see https://docs.docker.com/desktop/wasm/)
- [Static Content](docs/static_content.md) (such as a html/javascript frontend)

> Your language not listed above? Please contribute engineering resources or financially through our Sponsor link!
Expand Down
10 changes: 6 additions & 4 deletions docs/image.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ oci_image(
| <a id="oci_image_rule-base"></a>base | Label to an oci_image target to use as the base. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
| <a id="oci_image_rule-cmd"></a>cmd | Default arguments to the <code>entrypoint</code> of the container. These values act as defaults and may be replaced by any specified when creating a container. | List of strings | optional | [] |
| <a id="oci_image_rule-entrypoint"></a>entrypoint | A list of arguments to use as the <code>command</code> to execute when the container starts. These values act as defaults and may be replaced by an entrypoint specified when creating a container. | List of strings | optional | [] |
| <a id="oci_image_rule-env"></a>env | Default values to the environment variables of the container. These values act as defaults and are merged with any specified when creating a container. Entries replace the base environment variables if any of the entries has conflicting keys. To merge entries with keys specified in the base, <code>${KEY}</code> or <code>$KEY</code> syntax may be used. | <a href="https://bazel.build/docs/skylark/lib/dict.html">Dictionary: String -> String</a> | optional | {} |
| <a id="oci_image_rule-env"></a>env | A file containing the default values for the environment variables of the container. These values act as defaults and are merged with any specified when creating a container. Entries replace the base environment variables if any of the entries has conflicting keys. To merge entries with keys specified in the base, <code>${KEY}</code> or <code>$KEY</code> syntax may be used. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
| <a id="oci_image_rule-labels"></a>labels | A file containing a dictionary of labels. Each line should be in the form <code>name=value</code>. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
| <a id="oci_image_rule-os"></a>os | The name of the operating system which the image is built to run on. eg: <code>linux</code>, <code>windows</code>. See $GOOS documentation for possible values: https://go.dev/doc/install/source#environment | String | optional | "" |
| <a id="oci_image_rule-tars"></a>tars | List of tar files to add to the image as layers. Do not sort this list; the order is preserved in the resulting image. Less-frequently changed files belong in lower layers to reduce the network bandwidth required to pull and push.<br><br> The authors recommend [dive](https://github.com/wagoodman/dive) to explore the layering of the resulting image. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |
Expand All @@ -90,16 +90,17 @@ oci_image(
## oci_image

<pre>
oci_image(<a href="#oci_image-name">name</a>, <a href="#oci_image-labels">labels</a>, <a href="#oci_image-annotations">annotations</a>, <a href="#oci_image-kwargs">kwargs</a>)
oci_image(<a href="#oci_image-name">name</a>, <a href="#oci_image-labels">labels</a>, <a href="#oci_image-annotations">annotations</a>, <a href="#oci_image-env">env</a>, <a href="#oci_image-kwargs">kwargs</a>)
</pre>

Macro wrapper around [oci_image_rule](#oci_image_rule).

Allows labels and annotations to be provided as a dictionary, in addition to a text file.
See https://github.com/opencontainers/image-spec/blob/main/annotations.md

Label/annotation keys like `org.opencontainers.image.created` and `org.opencontainers.image.version`
may be supplied with non-deterministic information when bazel is run with `--stamp`; see the example in
Label/annotation/env can by configured using either dict(key->value) or a file that contains key=value pairs
(one per line). The file can be preprocessed using (e.g. using `jq`) to supply external (potentially not
deterministic) information when running with `--stamp` flag. See the example in
[/examples/labels/BUILD.bazel](https://github.com/bazel-contrib/rules_oci/blob/main/examples/labels/BUILD.bazel).


Expand All @@ -111,6 +112,7 @@ may be supplied with non-deterministic information when bazel is run with `--sta
| <a id="oci_image-name"></a>name | name of resulting oci_image_rule | none |
| <a id="oci_image-labels"></a>labels | Labels for the image config. See documentation above. | <code>None</code> |
| <a id="oci_image-annotations"></a>annotations | Annotations for the image config. See documentation above. | <code>None</code> |
| <a id="oci_image-env"></a>env | Environment variables provisioned by default to the running container. See documentation above. | <code>None</code> |
| <a id="oci_image-kwargs"></a>kwargs | other named arguments to [oci_image_rule](#oci_image_rule) | none |


2 changes: 1 addition & 1 deletion docs/image_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Build a multi-architecture OCI compatible container image.

It takes number of `oci_image`s to create a fat multi-architecture image.

Requires `wc` and `shasum` to be installed on the execution machine.
Requires `wc` and either `sha256sum` or `shasum` to be installed on the execution machine.

```starlark
oci_image(
Expand Down
3 changes: 2 additions & 1 deletion docs/pull.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ oci_image(
## oci_pull

<pre>
oci_pull(<a href="#oci_pull-name">name</a>, <a href="#oci_pull-image">image</a>, <a href="#oci_pull-repository">repository</a>, <a href="#oci_pull-registry">registry</a>, <a href="#oci_pull-platforms">platforms</a>, <a href="#oci_pull-digest">digest</a>, <a href="#oci_pull-tag">tag</a>, <a href="#oci_pull-reproducible">reproducible</a>)
oci_pull(<a href="#oci_pull-name">name</a>, <a href="#oci_pull-image">image</a>, <a href="#oci_pull-repository">repository</a>, <a href="#oci_pull-registry">registry</a>, <a href="#oci_pull-platforms">platforms</a>, <a href="#oci_pull-digest">digest</a>, <a href="#oci_pull-tag">tag</a>, <a href="#oci_pull-reproducible">reproducible</a>, <a href="#oci_pull-is_bzlmod">is_bzlmod</a>)
</pre>

Repository macro to fetch image manifest data from a remote docker registry.
Expand All @@ -69,5 +69,6 @@ in rules like `oci_image`.
| <a id="oci_pull-digest"></a>digest | the digest string, starting with "sha256:", "sha512:", etc. If omitted, instructions for pinning are provided. | <code>None</code> |
| <a id="oci_pull-tag"></a>tag | a tag to choose an image from the registry. Exactly one of <code>tag</code> and <code>digest</code> must be set. Since tags are mutable, this is not reproducible, so a warning is printed. | <code>None</code> |
| <a id="oci_pull-reproducible"></a>reproducible | Set to False to silence the warning about reproducibility when using <code>tag</code>. | <code>True</code> |
| <a id="oci_pull-is_bzlmod"></a>is_bzlmod | whether the oci_pull is being called from a module extension | <code>False</code> |


2 changes: 1 addition & 1 deletion docs/push.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ stamp_tags(
oci_push(
image = ":app_image",
repository = "ghcr.io/<OWNER>/image",
tags = ":stamped",
remote_tags = ":stamped",
)
```

Expand Down
115 changes: 115 additions & 0 deletions docs/static_content.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Serving Static Content

This is useful for creating an image to serve static content, such as the output of building your
frontend javascript.

In this example we'll use the [docker nginx image](https://hub.docker.com/_/nginx), but you could
use any other static content webserver the same way.

## Example

Pull our base image.

**./WORKSPACE**

```python
load("@rules_oci//oci:pull.bzl", "oci_pull")
oci_pull(
name = "nginx_debian_slim",
digest = "sha256:6b06964cdbbc517102ce5e0cef95152f3c6a7ef703e4057cb574539de91f72e6",
image = "docker.io/library/nginx",
)
```

Next lets create our static content files.

**./frontend/index.html**

```html

<!DOCTYPE html>
<html>
<body>

<h1>Our Homepage</h1>

<p>Hello from index.html</p>

</body>
</html>
```

**./frontend/textfile.txt**

```txt
This is text file.
```

And finally the build rules for our image.

**./frontend/BUILD**

```python
load("@rules_oci//oci:defs.bzl", "oci_image", "oci_tarball")
load("@rules_pkg//:pkg.bzl", "pkg_tar")
filegroup(
name = "static",
srcs = ["index.html", "textfile.txt"],
)

pkg_tar(
name = "static_tar",
srcs = [":static"],
package_dir = "/usr/share/nginx/html"
)

oci_image(
name = "frontend_image",
base = "@nginx_debian_slim",
tars = [
":static_tar",
],
# Intentionally omit cmd/entrypoint to default to the base nginx container's cmd/entrypoint.
# entrypoint = [],
# cmd = [],
)
oci_tarball(
name = "frontend_tarball",
image = ":frontend_image",
repo_tags = ["ourfrontend:latest"],
)


```

If you want to customize the nginx.conf you could create `./frontend/nginx.conf` and add this to
`./frontend/BUILD`.

```python

pkg_tar(
name = "nginx_conf_tar",
srcs = [":nginx.conf"],
package_dir = "/etc/nginx",
)

# ...
oci_image(
#...
tars = [
":static_tar",
":nginx_conf_tar
],
# ...
)

```

## Try running the container with docker

```bash
bazel run :frontend_tarball
docker run --rm -p 8080:80 "ourfrontend:latest"
```

Wait for nginx to start in your container, and then go to `localhost:8080` and `localhost:8080/example.txt` to see your static content.
5 changes: 2 additions & 3 deletions docs/tarball.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ oci_tarball(
and then run it in a container like so:

```
bazel build //path/to:tarball
docker load --input $(bazel cquery --output=files //path/to:tarball)
bazel run :tarball
docker run --rm my-repository:latest
```

Expand All @@ -41,6 +40,6 @@ Passing anything other than oci_image to the image attribute will lead to build
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="oci_tarball-name"></a>name | A unique name for this target. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
| <a id="oci_tarball-image"></a>image | Label of a directory containing an OCI layout, typically <code>oci_image</code> | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | required | |
| <a id="oci_tarball-repo_tags"></a>repo_tags | a file containing repo_tags, one per line. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
| <a id="oci_tarball-repo_tags"></a>repo_tags | a file containing repo_tags, one per line. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | required | |


6 changes: 1 addition & 5 deletions e2e/smoke/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,7 @@ load("@container_structure_test//:defs.bzl", "container_structure_test")

oci_image(
name = "image",
architecture = select({
"@platforms//cpu:arm64": "arm64",
"@platforms//cpu:x86_64": "amd64",
}),
base = "@distroless_base",
cmd = [
"--arg1",
"--arg2",
Expand All @@ -17,7 +14,6 @@ oci_image(
env = {
"ENV": "/test",
},
os = "linux",
)

tags = [
Expand Down
14 changes: 14 additions & 0 deletions e2e/smoke/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,17 @@ local_path_override(
module_name = "rules_oci",
path = "../..",
)

oci = use_extension("@rules_oci//oci:extensions.bzl", "oci")

oci.pull(
name = "distroless_base",
digest = "sha256:ccaef5ee2f1850270d453fdf700a5392534f8d1a8ca2acda391fbb6a06b81c86",
image = "gcr.io/distroless/base",
platforms = [
"linux/amd64",
"linux/arm64",
],
)

use_repo(oci, "distroless_base")
13 changes: 13 additions & 0 deletions e2e/smoke/WORKSPACE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,16 @@ oci_register_toolchains(
# Note that it does not support docker-format images.
# zot_version = LATEST_ZOT_VERSION,
)

# You can pull your base images using oci_pull like this:
load("@rules_oci//oci:pull.bzl", "oci_pull")

oci_pull(
name = "distroless_base",
digest = "sha256:ccaef5ee2f1850270d453fdf700a5392534f8d1a8ca2acda391fbb6a06b81c86",
image = "gcr.io/distroless/base",
platforms = [
"linux/amd64",
"linux/arm64",
],
)
3 changes: 1 addition & 2 deletions e2e/wasm/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,7 @@ build_test(

# In order to run the image you need to follow instructions at https://docs.docker.com/desktop/wasm/ first.
# then run the following;
# `bazel build :tarball`
# `docker load -i bazel-bin/tarball/tarball.tar`
# `bazel run :tarball``
# `docker run --runtime=io.containerd.wasmedge.v1 --platform=wasi/wasm32 --pull=never gcr.io/wasm:latest`
oci_tarball(
name = "tarball",
Expand Down
32 changes: 26 additions & 6 deletions examples/deb/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,14 +1,34 @@
load("//oci:defs.bzl", "oci_image")
load("//oci:defs.bzl", "oci_image", "oci_tarball")

_ARCH = [
"amd64",
"arm64",
]

# Workaround: crane doesn't do the right thing with .tar.xz compression
# so we simply decompress explicitly first.
[
genrule(
name = "decompress_" + architecture,
srcs = ["@bash_{}//:layer".format(architecture)],
outs = ["_{}.tar".format(architecture)],
cmd = "xz --decompress --stdout $< >$@",
)
for architecture in _ARCH
]

[
oci_image(
name = "image_" + architecture,
architecture = architecture,
os = "linux",
tars = ["@bash_{}//:layer".format(architecture)],
tars = ["_{}.tar".format(architecture)],
)
for architecture in [
"amd64",
"arm64",
]
for architecture in _ARCH
]

oci_tarball(
name = "tarball",
image = ":image_amd64",
repo_tags = ["test:test"],
)
Loading

0 comments on commit 880ba02

Please sign in to comment.