Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make /var a runtime volume #779

Merged
merged 4 commits into from
Aug 20, 2019
+14 −23
Merged

make /var a runtime volume #779

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4ace036
do not add volume when creating node image
BenTheElder Aug 19, 2019
4179434
make all of /var an anonymous volume, use long flags when creating nodes
BenTheElder Aug 19, 2019
6660b04
make shared from /
BenTheElder Aug 20, 2019
ead0872
bump base image
BenTheElder Aug 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions images/base/entrypoint
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,9 @@ fix_mount() {
# and this flag also happens to make /sys rw, amongst other things
mount -o remount,ro /sys

echo 'INFO: making mounts shared for "/", "/run", "/var/lib/containerd"'
echo 'INFO: making mounts shared'
# for mount propagation
# TODO(bentheelder): determine which exact mounts we need to do this on
mount --make-shared /
mount --make-shared /run
mount --make-shared /var/lib/containerd
mount --make-rshared /
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is recursive, and from upstream (difficult to find docs for this though...)
we don't know what mounts users will try to propagate, so everything is made shared

we need this change not just because it's more correct, but because /var/lib/containerd is no longer a mount point

}

fix_machine_id() {
Expand Down
11 changes: 1 addition & 10 deletions pkg/build/node/node.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ import (
const DefaultImage = "kindest/node:latest"

// DefaultBaseImage is the default base image used
const DefaultBaseImage = "kindest/base:v20190708-022110d@sha256:8acfd3b9b8a3a42385a761f8c6aa3bdad4241cac220c12d3309f1b7a6d70af24"
const DefaultBaseImage = "kindest/base:v20190819-26e1eb5@sha256:e609eaa7853289ef603db647ae9568b32093b2347f839a2117d98a08bfc7ab17"

// DefaultMode is the default kubernetes build mode for the built image
// see pkg/build/kube.Bits
Expand Down Expand Up @@ -320,15 +320,6 @@ func (c *BuildContext) buildImage(dir string) error {
// Save the image changes to a new image
cmd := exec.Command(
"docker", "commit",
/*
The snapshot storage must be a volume to avoid overlay on overlay

NOTE: we do this last because changing a volume with a docker image
must occur before defining it.

See: https://docs.docker.com/engine/reference/builder/#volume
*/
"--change", `VOLUME [ "/var/lib/containerd" ]`,
// we need to put this back after changing it when running the image
"--change", `ENTRYPOINT [ "/usr/local/bin/entrypoint", "/sbin/init" ]`,
containerID, c.image,
Expand Down
19 changes: 11 additions & 8 deletions pkg/cluster/nodes/create.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,23 +133,26 @@ func CreateWorkerNode(name, image, clusterLabel string, mounts []cri.Mount, port
// effectively be paused until we call actuallyStartNode(...)
func createNode(name, image, clusterLabel, role string, mounts []cri.Mount, portMappings []cri.PortMapping, extraArgs ...string) (handle *Node, err error) {
runArgs := []string{
"-d", // run the container detached
"-t", // allocate a tty for entrypoint logs
"--detach", // run the container detached
"--tty", // allocate a tty for entrypoint logs
// running containers in a container requires privileged
// NOTE: we could try to replicate this with --cap-add, and use less
// privileges, but this flag also changes some mounts that are necessary
// including some ones docker would otherwise do by default.
// for now this is what we want. in the future we may revisit this.
"--privileged",
"--security-opt", "seccomp=unconfined", // also ignore seccomp
// runtime temporary storage
"--tmpfs", "/tmp", // various things depend on working /tmp
"--tmpfs", "/run", // systemd wants a writable /run
// some k8s things want /lib/modules
"-v", "/lib/modules:/lib/modules:ro",
// ensure pods etc. are not on container filesystem
// TODO: we could do this in the image instead
// However this would leave old images with this issue
"-v", "/var/lib/kubelet",
// runtime persistent storage
// this ensures that E.G. pods, logs etc. are not on the container
// filesystem, which is not only better for performance, but allows
// running kind in kind for "party tricks"
// (please don't depend on doing this though!)
"--volume", "/var",
// some k8s things want to read /lib/modules
"--volume", "/lib/modules:/lib/modules:ro",
"--hostname", name, // make hostname match container name
"--name", name, // ... and set the container name
// label the node with the cluster ID
Expand Down