Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not make read-only mounts recursively read-only by default (also updates Docker client module to v25) #311

Merged
merged 4 commits into from
Mar 14, 2024
Merged

Do not make read-only mounts recursively read-only by default (also updates Docker client module to v25) #311

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
77d8e43
go mod tidy
AkihiroSuda Mar 9, 2024
bf1a9b9
Use new mount API (`github.com/docker/docker/api/types/mount`)
AkihiroSuda Mar 9, 2024
24a43e2
go.mod: github.com/docker/docker v25.0.4+incompatible
AkihiroSuda Mar 9, 2024
a16ab45
Do not make read-only mounts recursively read-only by default
AkihiroSuda Mar 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 27 additions & 3 deletions core/container_create.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,10 @@ import (
"path/filepath"

"github.com/Mirantis/cri-dockerd/libdocker"
"github.com/docker/docker/api/types"
dockerbackend "github.com/docker/docker/api/types/backend"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... this feels "wrong". Did we not expose the right options in the API types? I think the backend is not expected to be used externally (should we consider making that internal?).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's expected in cri-dockerd though if it's implementing API and Backend (but I honestly would have to look at how cri-dockerd is implemented). Thought I'd at least leave a comment that this stood out to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dockerbackend.ContainerCreateConfig is used here

cri-dockerd/libdocker/kube_docker_client.go

Lines 143 to 167 in 8ae7a0e

func (d *kubeDockerClient) CreateContainer(
opts dockertypes.ContainerCreateConfig,
) (*dockercontainer.CreateResponse, error) {
ctx, cancel := context.WithTimeout(context.Background(), d.timeout)
defer cancel()
// we provide an explicit default shm size as to not depend on docker daemon.
if opts.HostConfig != nil && opts.HostConfig.ShmSize <= 0 {
opts.HostConfig.ShmSize = defaultShmSize
}
createResp, err := d.client.ContainerCreate(
ctx,
opts.Config,
opts.HostConfig,
opts.NetworkingConfig,
nil,
opts.Name,
)
if ctxErr := contextError(ctx); ctxErr != nil {
return nil, ctxErr
}
if err != nil {
return nil, err
}
return &createResp, nil
}

I guess it is possible to remove this dependency, but it is beyond the topic of this PR.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it looks like the client (still) has this awkward signature to pass all those individual structs as separate arguments, but uses a local (non-exported) "ad-hoc" struct to create the equivalent for the request (grouping them in a single struct); https://github.com/moby/moby/blob/e61c425cc283ed85a6a87ab4750b52389aea4021/client/container_create.go#L15-L24

type configWrapper struct {
	*container.Config
	HostConfig       *container.HostConfig
	NetworkingConfig *network.NetworkingConfig
}

// ContainerCreate creates a new container based on the given configuration.
// It can be associated with a name, but it's not mandatory.
func (cli *Client) ContainerCreate(ctx context.Context, config *container.Config, hostConfig *container.HostConfig, networkingConfig *network.NetworkingConfig, platform *ocispec.Platform, containerName string) (container.CreateResponse, error) {
	var response container.CreateResponse

https://github.com/moby/moby/blob/e61c425cc283ed85a6a87ab4750b52389aea4021/client/container_create.go#L75-L81

	body := configWrapper{
		Config:           config,
		HostConfig:       hostConfig,
		NetworkingConfig: networkingConfig,
	}

	serverResp, err := cli.post(ctx, "/containers/create", query, body, **nil)

We should probably consider to either change that signature, and define a CreateOptions struct for the client, or (perhaps) for cri-dockerd to (for now) define a similar struct on its side for the time being

/cc @neersighted

"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/strslice"
"github.com/opencontainers/selinux/go-selinux/label"
v1 "k8s.io/cri-api/pkg/apis/runtime/v1"
)

Expand Down Expand Up @@ -64,8 +65,9 @@ func (ds *dockerService) CreateContainer(
image = iSpec.Image
}
containerName := makeContainerName(sandboxConfig, config)
mounts := config.GetMounts()
terminationMessagePath, _ := config.Annotations["io.kubernetes.container.terminationMessagePath"]
createConfig := types.ContainerCreateConfig{
createConfig := dockerbackend.ContainerCreateConfig{
Name: containerName,
Config: &container.Config{
Entrypoint: strslice.StrSlice(config.Command),
Expand All @@ -85,13 +87,35 @@ func (ds *dockerService) CreateContainer(
},
},
HostConfig: &container.HostConfig{
Binds: libdocker.GenerateMountBindings(config.GetMounts(), terminationMessagePath),
Mounts: libdocker.GenerateMountBindings(mounts, terminationMessagePath),
RestartPolicy: container.RestartPolicy{
Name: "no",
},
},
}

// Only request relabeling if the pod provides an SELinux context. If the pod
// does not provide an SELinux context relabeling will label the volume with
// the container's randomly allocated MCS label. This would restrict access
// to the volume to the container which mounts it first.
if selinuxOpts := config.GetLinux().GetSecurityContext().GetSelinuxOptions(); selinuxOpts != nil {
mountLabel, err := selinuxMountLabel(selinuxOpts)
if err != nil {
return nil, fmt.Errorf("unable to generate SELinux mount label: %v", err)
}
if mountLabel != "" {
// Equates to "Z" in the old bind API
const shared = false
for _, m := range mounts {
if m.SelinuxRelabel {
if err := label.Relabel(m.HostPath, mountLabel, shared); err != nil {
return nil, fmt.Errorf("unable to relabel %q with %q: %v", m.HostPath, mountLabel, err)
}
}
}
}
}

hc := createConfig.HostConfig
err = ds.updateCreateConfig(
&createConfig,
Expand Down
8 changes: 5 additions & 3 deletions core/helpers_linux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,10 @@ package core

import (
"fmt"

"github.com/blang/semver"
dockertypes "github.com/docker/docker/api/types"
dockerbackend "github.com/docker/docker/api/types/backend"
dockercontainer "github.com/docker/docker/api/types/container"
"github.com/sirupsen/logrus"
runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1"
Expand All @@ -34,7 +36,7 @@ func DefaultMemorySwap() int64 {
}

func (ds *dockerService) updateCreateConfig(
createConfig *dockertypes.ContainerCreateConfig,
createConfig *dockerbackend.ContainerCreateConfig,
config *runtimeapi.ContainerConfig,
sandboxConfig *runtimeapi.PodSandboxConfig,
podSandboxID string, securityOptSep rune, apiVersion *semver.Version) error {
Expand Down Expand Up @@ -97,12 +99,12 @@ func getNetworkNamespace(c *dockertypes.ContainerJSON) (string, error) {

type containerCleanupInfo struct{}

// applyPlatformSpecificDockerConfig applies platform-specific configurations to a dockertypes.ContainerCreateConfig struct.
// applyPlatformSpecificDockerConfig applies platform-specific configurations to a dockerbackend.ContainerCreateConfig struct.
// The containerCleanupInfo struct it returns will be passed as is to performPlatformSpecificContainerCleanup
// after either the container creation has failed or the container has been removed.
func (ds *dockerService) applyPlatformSpecificDockerConfig(
*runtimeapi.CreateContainerRequest,
*dockertypes.ContainerCreateConfig,
*dockerbackend.ContainerCreateConfig,
) (*containerCleanupInfo, error) {
return nil, nil
}
Expand Down
22 changes: 12 additions & 10 deletions core/helpers_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ import (
"github.com/Mirantis/cri-dockerd/libdocker"

dockertypes "github.com/docker/docker/api/types"
dockermount "github.com/docker/docker/api/types/mount"
dockerregistry "github.com/docker/docker/api/types/registry"
dockernat "github.com/docker/go-connections/nat"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
Expand Down Expand Up @@ -140,7 +142,7 @@ func TestParsingCreationConflictError(t *testing.T) {

func TestEnsureSandboxImageExists(t *testing.T) {
sandboxImage := "gcr.io/test/image"
authConfig := dockertypes.AuthConfig{Username: "user", Password: "pass"}
authConfig := dockerregistry.AuthConfig{Username: "user", Password: "pass"}
for desc, test := range map[string]struct {
injectImage bool
imgNeedsAuth bool
Expand Down Expand Up @@ -326,15 +328,15 @@ func TestGenerateMountBindings(t *testing.T) {
Propagation: runtimeapi.MountPropagation_PROPAGATION_BIDIRECTIONAL,
},
}
expectedResult := []string{
"/mnt/1:/var/lib/mysql/1",
"/mnt/2:/var/lib/mysql/2:ro",
"/mnt/3:/var/lib/mysql/3:Z",
"/mnt/4:/var/lib/mysql/4",
"/mnt/5:/var/lib/mysql/5:rslave",
"/mnt/6:/var/lib/mysql/6:rshared",
"/mnt/7:/var/lib/mysql/7",
"/mnt/8:/var/lib/mysql/8:ro,Z,rshared",
expectedResult := []dockermount.Mount{
{Type: dockermount.TypeBind, Source: "/mnt/1", Target: "/var/lib/mysql/1", BindOptions: &dockermount.BindOptions{CreateMountpoint: true}},
{Type: dockermount.TypeBind, Source: "/mnt/2", Target: "/var/lib/mysql/2", ReadOnly: true, BindOptions: &dockermount.BindOptions{CreateMountpoint: true, ReadOnlyNonRecursive: true}},
{Type: dockermount.TypeBind, Source: "/mnt/3", Target: "/var/lib/mysql/3", BindOptions: &dockermount.BindOptions{CreateMountpoint: true}}, // Relabeling is not handled here
{Type: dockermount.TypeBind, Source: "/mnt/4", Target: "/var/lib/mysql/4", BindOptions: &dockermount.BindOptions{CreateMountpoint: true}},
{Type: dockermount.TypeBind, Source: "/mnt/5", Target: "/var/lib/mysql/5", BindOptions: &dockermount.BindOptions{CreateMountpoint: true, Propagation: dockermount.PropagationRSlave}},
{Type: dockermount.TypeBind, Source: "/mnt/6", Target: "/var/lib/mysql/6", BindOptions: &dockermount.BindOptions{CreateMountpoint: true, Propagation: dockermount.PropagationRShared}},
{Type: dockermount.TypeBind, Source: "/mnt/7", Target: "/var/lib/mysql/7", BindOptions: &dockermount.BindOptions{CreateMountpoint: true}},
{Type: dockermount.TypeBind, Source: "/mnt/8", Target: "/var/lib/mysql/8", ReadOnly: true, BindOptions: &dockermount.BindOptions{CreateMountpoint: true, ReadOnlyNonRecursive: true, Propagation: dockermount.PropagationRShared}}, // Relabeling is not handled here
}
result := libdocker.GenerateMountBindings(mounts, "")

Expand Down
7 changes: 4 additions & 3 deletions core/helpers_unsupported.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (

"github.com/blang/semver"
dockertypes "github.com/docker/docker/api/types"
dockerbackend "github.com/docker/docker/api/types/backend"
"github.com/sirupsen/logrus"
runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1"
)
Expand All @@ -47,7 +48,7 @@ func (ds *dockerService) getSandBoxSecurityOpts(separator rune) []string {
}

func (ds *dockerService) updateCreateConfig(
createConfig *dockertypes.ContainerCreateConfig,
createConfig *dockerbackend.ContainerCreateConfig,
config *runtimeapi.ContainerConfig,
sandboxConfig *runtimeapi.PodSandboxConfig,
podSandboxID string, securityOptSep rune, apiVersion *semver.Version) error {
Expand All @@ -66,12 +67,12 @@ func getNetworkNamespace(c *dockertypes.ContainerJSON) (string, error) {

type containerCleanupInfo struct{}

// applyPlatformSpecificDockerConfig applies platform-specific configurations to a dockertypes.ContainerCreateConfig struct.
// applyPlatformSpecificDockerConfig applies platform-specific configurations to a dockerbackend.ContainerCreateConfig struct.
// The containerCleanupInfo struct it returns will be passed as is to performPlatformSpecificContainerCleanup
// after either the container creation has failed or the container has been removed.
func (ds *dockerService) applyPlatformSpecificDockerConfig(
*runtimeapi.CreateContainerRequest,
*dockertypes.ContainerCreateConfig,
*dockerbackend.ContainerCreateConfig,
) (*containerCleanupInfo, error) {
return nil, nil
}
Expand Down
12 changes: 7 additions & 5 deletions core/helpers_windows.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,15 @@ import (
"crypto/rand"
"encoding/hex"
"fmt"
"golang.org/x/sys/windows/registry"
"os"
"regexp"
"runtime"

"golang.org/x/sys/windows/registry"

"github.com/blang/semver"
dockertypes "github.com/docker/docker/api/types"
dockerbackend "github.com/docker/docker/api/types/backend"
dockercontainer "github.com/docker/docker/api/types/container"
dockerfilters "github.com/docker/docker/api/types/filters"
"github.com/sirupsen/logrus"
Expand All @@ -42,7 +44,7 @@ func DefaultMemorySwap() int64 {
}

func (ds *dockerService) updateCreateConfig(
createConfig *dockertypes.ContainerCreateConfig,
createConfig *dockerbackend.ContainerCreateConfig,
config *runtimeapi.ContainerConfig,
sandboxConfig *runtimeapi.PodSandboxConfig,
podSandboxID string, securityOptSep rune, apiVersion *semver.Version) error {
Expand Down Expand Up @@ -147,12 +149,12 @@ type containerCleanupInfo struct {
gMSARegistryValueName string
}

// applyPlatformSpecificDockerConfig applies platform-specific configurations to a dockertypes.ContainerCreateConfig struct.
// applyPlatformSpecificDockerConfig applies platform-specific configurations to a dockerbackend.ContainerCreateConfig struct.
// The containerCleanupInfo struct it returns will be passed as is to performPlatformSpecificContainerCleanup
// after either the container creation has failed or the container has been removed.
func (ds *dockerService) applyPlatformSpecificDockerConfig(
request *runtimeapi.CreateContainerRequest,
createConfig *dockertypes.ContainerCreateConfig,
createConfig *dockerbackend.ContainerCreateConfig,
) (*containerCleanupInfo, error) {
cleanupInfo := &containerCleanupInfo{}

Expand All @@ -173,7 +175,7 @@ func (ds *dockerService) applyPlatformSpecificDockerConfig(
// https://github.com/moby/moby/pull/38777
func applyGMSAConfig(
config *runtimeapi.ContainerConfig,
createConfig *dockertypes.ContainerCreateConfig,
createConfig *dockerbackend.ContainerCreateConfig,
cleanupInfo *containerCleanupInfo,
) error {
var credSpec string
Expand Down
3 changes: 2 additions & 1 deletion core/image.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import (

dockertypes "github.com/docker/docker/api/types"
dockerfilters "github.com/docker/docker/api/types/filters"
dockerregistry "github.com/docker/docker/api/types/registry"
"github.com/docker/docker/pkg/jsonmessage"

"github.com/sirupsen/logrus"
Expand Down Expand Up @@ -137,7 +138,7 @@ func (ds *dockerService) PullImage(
) (*runtimeapi.PullImageResponse, error) {
image := r.GetImage()
auth := r.GetAuth()
authConfig := dockertypes.AuthConfig{}
authConfig := dockerregistry.AuthConfig{}

if auth != nil {
authConfig.Username = auth.Username
Expand Down
23 changes: 13 additions & 10 deletions core/sandbox_helpers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,20 @@ package core

import (
"fmt"
"os"
"strings"
"time"

"github.com/Mirantis/cri-dockerd/libdocker"
"github.com/Mirantis/cri-dockerd/utils"
"github.com/Mirantis/cri-dockerd/utils/errors"
"k8s.io/kubernetes/pkg/credentialprovider"
"os"
"strings"
"time"

"github.com/Mirantis/cri-dockerd/config"
dockertypes "github.com/docker/docker/api/types"
dockerbackend "github.com/docker/docker/api/types/backend"
dockercontainer "github.com/docker/docker/api/types/container"
dockerregistry "github.com/docker/docker/api/types/registry"
"github.com/sirupsen/logrus"
runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1"
)
Expand Down Expand Up @@ -169,7 +172,7 @@ func (ds *dockerService) getPodSandboxDetails(
func (ds *dockerService) applySandboxLinuxOptions(
hc *dockercontainer.HostConfig,
lc *runtimeapi.LinuxPodSandboxConfig,
createConfig *dockertypes.ContainerCreateConfig,
createConfig *dockerbackend.ContainerCreateConfig,
image string,
separator rune,
) error {
Expand Down Expand Up @@ -207,11 +210,11 @@ func (ds *dockerService) applySandboxResources(
return nil
}

// makeSandboxDockerConfig returns dockertypes.ContainerCreateConfig based on runtimeapi.PodSandboxConfig.
// makeSandboxDockerConfig returns dockerbackend.ContainerCreateConfig based on runtimeapi.PodSandboxConfig.
func (ds *dockerService) makeSandboxDockerConfig(
c *runtimeapi.PodSandboxConfig,
image string,
) (*dockertypes.ContainerCreateConfig, error) {
) (*dockerbackend.ContainerCreateConfig, error) {
// Merge annotations and labels because docker supports only labels.
labels := makeLabels(c.GetLabels(), c.GetAnnotations())
// Apply a label to distinguish sandboxes from regular containers.
Expand All @@ -222,7 +225,7 @@ func (ds *dockerService) makeSandboxDockerConfig(
hc := &dockercontainer.HostConfig{
IpcMode: dockercontainer.IpcMode("shareable"),
}
createConfig := &dockertypes.ContainerCreateConfig{
createConfig := &dockerbackend.ContainerCreateConfig{
Name: makeSandboxName(c),
Config: &dockercontainer.Config{
Hostname: c.Hostname,
Expand Down Expand Up @@ -374,7 +377,7 @@ func rewriteFile(filePath, stringToWrite string) error {

func recoverFromCreationConflictIfNeeded(
client libdocker.DockerClientInterface,
createConfig dockertypes.ContainerCreateConfig,
createConfig dockerbackend.ContainerCreateConfig,
err error,
) (*dockercontainer.CreateResponse, error) {
matches := conflictRE.FindStringSubmatch(err.Error())
Expand Down Expand Up @@ -421,7 +424,7 @@ func ensureSandboxImageExists(client libdocker.DockerClientInterface, image stri
if !withCredentials {
logrus.Infof("Pulling the image without credentials. Image: %v", image)

err := client.PullImage(image, dockertypes.AuthConfig{}, dockertypes.ImagePullOptions{})
err := client.PullImage(image, dockerregistry.AuthConfig{}, dockertypes.ImagePullOptions{})
if err != nil {
return fmt.Errorf("failed pulling image %q: %v", image, err)
}
Expand All @@ -431,7 +434,7 @@ func ensureSandboxImageExists(client libdocker.DockerClientInterface, image stri

var pullErrs []error
for _, currentCreds := range creds {
authConfig := dockertypes.AuthConfig(currentCreds)
authConfig := dockerregistry.AuthConfig(currentCreds)
err := client.PullImage(image, authConfig, dockertypes.ImagePullOptions{})
// If there was no error, return success
if err == nil {
Expand Down
24 changes: 24 additions & 0 deletions core/selinux_util.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package core
import (
"fmt"

"github.com/opencontainers/selinux/go-selinux/label"
runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1"
)

Expand Down Expand Up @@ -88,3 +89,26 @@ func modifySecurityOption(config []string, name, value string) []string {

return config
}

func selinuxMountLabel(selinuxOpts *runtimeapi.SELinuxOption) (string, error) {
var opts []string
if selinuxOpts != nil {
if selinuxOpts.User != "" {
opts = append(opts, "user:"+selinuxOpts.User)
}
if selinuxOpts.Role != "" {
opts = append(opts, "role:"+selinuxOpts.Role)
}
if selinuxOpts.Type != "" {
opts = append(opts, "type:"+selinuxOpts.Type)
}
if selinuxOpts.Level != "" {
opts = append(opts, "level:"+selinuxOpts.Level)
}
}
if len(opts) == 0 {
opts = []string{"disable"}
}
_, s, err := label.InitLabels(opts)
return s, err
}
15 changes: 15 additions & 0 deletions core/selinux_util_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ package core
import (
"reflect"
"testing"

"github.com/opencontainers/selinux/go-selinux"
"github.com/stretchr/testify/assert"
)

func TestModifySecurityOptions(t *testing.T) {
Expand Down Expand Up @@ -57,3 +60,15 @@ func TestModifySecurityOptions(t *testing.T) {
}
}
}

func TestSELinuxMountLabel(t *testing.T) {
selinuxOpts := inputSELinuxOptions()
s, err := selinuxMountLabel(selinuxOpts)
assert.NoError(t, err)
t.Logf("mount label for %+v=%q", selinuxOpts, s)
if selinux.GetEnabled() {
assert.NotEmpty(t, s)
} else {
assert.Empty(t, s)
}
}
Loading
Loading