Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial Tilt support #297

Merged
merged 1 commit into from
Jul 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,5 @@ install.sh

# documentation website asset folder
docs/_site

ncdc marked this conversation as resolved.
Show resolved Hide resolved
.tiltbuild/
66 changes: 66 additions & 0 deletions Tiltfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# This loads a helper function that isn't part of core Tilt that simplifies restarting the process in the container
# when files changes.
load('ext://restart_process', 'docker_build_with_restart')

# Treat the main binary as a local resource, so we can automatically rebuild it when any of the deps change. This
# builds it locally, targeting linux, so it can run in a linux container.
local_resource(
'manager_binary',
cmd='''
mkdir -p .tiltbuild/bin
CGO_ENABLED=0 GOOS=linux go build -o .tiltbuild/bin/manager ./cmd/manager
''',
deps=['api', 'cmd/manager', 'internal', 'pkg', 'go.mod', 'go.sum']
)

# Configure our image build. If the file in live_update.sync (.tiltbuild/bin/manager) changes, Tilt
# copies it to the running container and restarts it.
docker_build_with_restart(
# This has to match an image in the k8s_yaml we call below, so Tilt knows to use this image for our Deployment,
# instead of the actual image specified in the yaml.
ref='quay.io/operator-framework/operator-controller:devel',
# This is the `docker build` context, and because we're only copying in the binary we've already had Tilt build
# locally, we set the context to the directory containing the binary.
context='.tiltbuild/bin',
# We use a slimmed-down Dockerfile that only has $binary in it.
dockerfile_contents='''
FROM gcr.io/distroless/static:debug
EXPOSE 8080
WORKDIR /
COPY manager manager
''',
# The set of files Tilt should include in the build. In this case, it's just the binary we built above.
only='manager',
# If .tiltbuild/bin/manager changes, Tilt will copy it into the running container and restart the process.
live_update=[
sync('.tiltbuild/bin/manager', '/manager'),
],
# The command to run in the container.
entrypoint="/manager",
)

# Tell Tilt what to deploy by running kustomize and then doing some manipulation to make things work for Tilt.
objects = decode_yaml_stream(kustomize('config/default'))
for o in objects:
# For Tilt's live_update functionality to work, we have to run the container as root. Remove any PSA labels to allow
# this.
if o['kind'] == 'Namespace' and 'labels' in o['metadata']:
labels_to_delete = [label for label in o['metadata']['labels'] if label.startswith('pod-security.kubernetes.io')]
for label in labels_to_delete:
o['metadata']['labels'].pop(label)

if o['kind'] != 'Deployment':
# We only need to modify Deployments, so we can skip this
continue

# For Tilt's live_update functionality to work, we have to run the container as root. Otherwise, Tilt won't
# be able to untar the updated binary in the container's file system (this is how live update
# works). If there are any securityContexts, remove them.
if "securityContext" in o['spec']['template']['spec']:
o['spec']['template']['spec'].pop('securityContext')
for c in o['spec']['template']['spec']['containers']:
if "securityContext" in c:
c.pop('securityContext')

# Now apply all the yaml
k8s_yaml(encode_yaml_stream(objects))
82 changes: 82 additions & 0 deletions tilt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Rapid iterative development with Tilt

[Tilt](https://tilt.dev) is a tool that enables rapid iterative development of containerized workloads.

Here is an example workflow without Tilt for modifying some source code and testing those changes in a cluster:

1. Modify the source code.
2. Build the container image.
3. Either push the image to a registry or load it into your kind cluster.
4. Deploy all the appropriate Kubernetes manifests for your application.
1. Or, if this is an update, you'd instead scale the Deployment to 0 replicas, scale back to 1, and wait for the
new pod to be running.

This process can take minutes, depending on how long each step takes.

Here is the same workflow with Tilt:

1. Run `tilt up`
2. Modify the source code
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to emphasize "with your preferred editor", so people don't think you need to use the Tilt interface.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's likely people will be confused.

3. Wait for Tilt to update the container with your changes

This ends up taking a fraction of the time, sometimes on the order of a few seconds!

## Installing Tilt

Follow Tilt's [instructions](https://docs.tilt.dev/install.html) for installation.

## Installing rukpak and catalogd

operator-controller requires [rukpak](https://github.com/operator-framework/rukpak) and
[catalogd](https://github.com/operator-framework/catalogd). Please make sure they're installed, either normally or via
their own Tiltfiles, before proceeding. If you want to use Tilt, make sure you specify a unique `--port` flag to each
`tilt up` invocation.

## Starting Tilt

This is typically as short as:

```shell
tilt up
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Does this need to be run inside the git repo?
  2. Does this "take over" the shell/terminal? (i.e. recommend opening another terminal?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and yes

```

**NOTE:** if you are using Podman, at least as of v4.5.1, you need to do this:

```shell
DOCKER_BUILDKIT=0 tilt up
```

Otherwise, you'll see an error when Tilt tries to build your image that looks similar to:

```text
Build Failed: ImageBuild: stat /var/tmp/libpod_builder2384046170/build/Dockerfile: no such file or directory
```

When Tilt starts, you'll see something like this in your terminal:

```text
Tilt started on http://localhost:10350/
v0.33.1, built 2023-06-28

(space) to open the browser
(s) to stream logs (--stream=true)
(t) to open legacy terminal mode (--legacy=true)
ncdc marked this conversation as resolved.
Show resolved Hide resolved
(ctrl-c) to exit
```

Typically, you'll want to press the space bar to have it open the UI in your web browser.

Shortly after starting, Tilt processes the `Tiltfile`, resulting in:

- Building the go binaries
- Building the images
- Loading the images into kind
- Running kustomize and applying everything except the Deployments that reference the images above
- Modifying the Deployments to use the just-built images
- Creating the Deployments

## Making code changes

Any time you change any of the files listed in the `deps` section in the `<binary name>_binary` `local_resource`,
Tilt automatically rebuilds the go binary. As soon as the binary is rebuilt, Tilt pushes it (and only it) into the
appropriate running container, and then restarts the process.
Loading