Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support containerd/cri #98

Merged
merged 1 commit into from
Aug 5, 2020
Merged

support containerd/cri #98

merged 1 commit into from
Aug 5, 2020

Conversation

dweomer
Copy link

@dweomer dweomer commented Jul 9, 2020

Adapts changes suggested by @crosbymichael:

Unsure if this works because #97. Any pointers to resolve the installation issue would be greatly appreciated.

Copy link

@crosbymichael crosbymichael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crosbymichael
Copy link

I remember getting those same errors when I tried to build the package. I think I resolved it by copying the file into the systems install, or it was different for each disto version. I can’t remember off the top of my head right now but maybe someone else has a quick suggestion on how to install.

@dweomer
Copy link
Author

dweomer commented Jul 9, 2020

Hmm, @ibuildthecloud pointed me at https://github.com/containers/container-selinux/tree/RHEL7.5, will give this a try.

@dweomer
Copy link
Author

dweomer commented Jul 9, 2020

BTW, thank you @crosbymichael for taking a look at this. I had another question with regards to your changes. Isn't something like the following also going to be necessary?

/var/run/containerd/[^/]*/sandboxes/[^/]*/shm(/.*)? gen_context(container_runtime_tmpfs_t,s0)

Copy link
Member

@rhatdan rhatdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also can you add snapshots to container.if function container_filetrans_named_content

container.fc Outdated Show resolved Hide resolved
@rhatdan
Copy link
Member

rhatdan commented Jul 10, 2020

Also add transition rule for containerd in container_filetrans_named_content in container.if

@dweomer
Copy link
Author

dweomer commented Jul 10, 2020

@rhatdan I appreciate the feedback. To make this possible for me to test do you have guidance on how to install this policy from source on centos7?


I see that you closed #97 saying I should use the RHEL7.5 branch but as per #97 (comment), it doesn't seem to work.

@rhatdan
Copy link
Member

rhatdan commented Jul 13, 2020

Check out the repo and then checkout the RHEL7.5 branch.

@rhatdan
Copy link
Member

rhatdan commented Jul 17, 2020

It would be best to test this policy on Fedora 32 or Centos 8 and then once we know it works backport fixes to the RHEL7.5 branch for centos7.

@dweomer
Copy link
Author

dweomer commented Jul 17, 2020

It would be best to test this policy on Fedora 32 or Centos 8 and then once we know it works backport fixes to the RHEL7.5 branch for centos7.

Agreed. Working on it (still have some failures running critest against containerd/cri the 1.4.0-beta but I did switch to centos8):

[vagrant@centos8 cri]$ sudo grep denied /var/log/audit/audit.log
type=AVC msg=audit(1594982029.763:1626): avc:  denied  { search } for  pid=40743 comm="top" name="13" dev="proc" ino=146110 scontext=system_u:system_r:container_t:s0:c575,c634 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=dir permissive=0
type=AVC msg=audit(1594982034.152:1637): avc:  denied  { open } for  pid=41942 comm="nginx" path="pipe:[149304]" dev="pipefs" ino=149304 scontext=system_u:system_r:container_t:s0:c526,c939 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982034.154:1638): avc:  denied  { open } for  pid=41942 comm="nginx" path="pipe:[149304]" dev="pipefs" ino=149304 scontext=system_u:system_r:container_t:s0:c526,c939 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982034.202:1639): avc:  denied  { open } for  pid=42019 comm="nginx" path="pipe:[150010]" dev="pipefs" ino=150010 scontext=system_u:system_r:container_t:s0:c387,c969 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982034.202:1640): avc:  denied  { open } for  pid=42019 comm="nginx" path="pipe:[150010]" dev="pipefs" ino=150010 scontext=system_u:system_r:container_t:s0:c387,c969 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982034.961:1649): avc:  denied  { open } for  pid=42540 comm="httpd" path="pipe:[151536]" dev="pipefs" ino=151536 scontext=system_u:system_r:container_t:s0:c474,c1017 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982036.639:1652): avc:  denied  { open } for  pid=43236 comm="nginx" path="pipe:[154567]" dev="pipefs" ino=154567 scontext=system_u:system_r:container_t:s0:c531,c546 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982036.642:1653): avc:  denied  { open } for  pid=43236 comm="nginx" path="pipe:[154567]" dev="pipefs" ino=154567 scontext=system_u:system_r:container_t:s0:c531,c546 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982036.743:1655): avc:  denied  { search } for  pid=43356 comm="top" name="1" dev="proc" ino=155021 scontext=system_u:system_r:container_t:s0:c4,c5 tcontext=system_u:system_r:container_t:s0:c872,c943 tclass=dir permissive=0
type=AVC msg=audit(1594982037.050:1658): avc:  denied  { open } for  pid=43587 comm="nginx" path="pipe:[155583]" dev="pipefs" ino=155583 scontext=system_u:system_r:container_t:s0:c378,c736 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982037.053:1659): avc:  denied  { open } for  pid=43587 comm="nginx" path="pipe:[155583]" dev="pipefs" ino=155583 scontext=system_u:system_r:container_t:s0:c378,c736 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982047.129:1689): avc:  denied  { search } for  pid=47108 comm="top" name="1" dev="proc" ino=169075 scontext=system_u:system_r:container_t:s0 tcontext=system_u:system_r:container_t:s0:c203,c354 tclass=dir permissive=0
type=AVC msg=audit(1594982047.173:1690): avc:  denied  { open } for  pid=47107 comm="httpd" path="pipe:[168717]" dev="pipefs" ino=168717 scontext=system_u:system_r:container_t:s0:c184,c344 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982050.799:1700): avc:  denied  { search } for  pid=48061 comm="top" name="13" dev="proc" ino=172508 scontext=system_u:system_r:container_t:s0:c115,c413 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=dir permissive=0
type=AVC msg=audit(1594982051.765:1703): avc:  denied  { open } for  pid=48325 comm="nginx" path="pipe:[173580]" dev="pipefs" ino=173580 scontext=system_u:system_r:container_t:s0:c419,c429 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982051.766:1704): avc:  denied  { open } for  pid=48325 comm="nginx" path="pipe:[173580]" dev="pipefs" ino=173580 scontext=system_u:system_r:container_t:s0:c419,c429 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982056.986:1720): avc:  denied  { open } for  pid=49942 comm="nginx" path="pipe:[179666]" dev="pipefs" ino=179666 scontext=system_u:system_r:container_t:s0:c984,c995 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982056.986:1721): avc:  denied  { open } for  pid=49942 comm="nginx" path="pipe:[179666]" dev="pipefs" ino=179666 scontext=system_u:system_r:container_t:s0:c984,c995 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982064.204:1744): avc:  denied  { open } for  pid=52433 comm="httpd" path="pipe:[188857]" dev="pipefs" ino=188857 scontext=system_u:system_r:container_t:s0:c674,c982 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982069.362:1756): avc:  denied  { search } for  pid=53832 comm="top" name="14" dev="proc" ino=195602 scontext=system_u:system_r:container_t:s0:c295,c468 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=dir permissive=0
type=AVC msg=audit(1594982074.452:1763): avc:  denied  { open } for  pid=54711 comm="nginx" path="pipe:[198342]" dev="pipefs" ino=198342 scontext=system_u:system_r:container_t:s0:c215,c717 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
type=AVC msg=audit(1594982074.453:1764): avc:  denied  { open } for  pid=54711 comm="nginx" path="pipe:[198342]" dev="pipefs" ino=198342 scontext=system_u:system_r:container_t:s0:c215,c717 tcontext=unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 tclass=fifo_file permissive=0
[vagrant@centos8 cri]$ 

The pipefs entries look a lot like https://bugzilla.redhat.com/show_bug.cgi?id=1456963 but that is supposed to be fixed?

@rhatdan
Copy link
Member

rhatdan commented Jul 20, 2020

The issue here is, if we want the container processes to open a shared fifo_file, it needs to be labeled correctly. In this case the fifo_file has a range of MCS labels associated with it, that is blocked by MCS Protections.
If containerd/docker wants to create this fifo file, it should be calling setfscreatecon(CONTAINER_LABEL) before creating the fifo file, then the container would be allowed to access it.

Adapts changes suggested by @crosbymichael:
- containerd/cri#1246 (comment)

With some bits grafted from https://github.com/rancher/k3s-selinux by:
- @erikwilson
- @ibuildthecloud

Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
@dweomer
Copy link
Author

dweomer commented Aug 5, 2020

@rhatdan I've added the transition(s) to container_filetrans_named_content in container.if as requested. These changes resolve 10 of the 11 outstanding critest failures that I was seeing with containerd on fedora32 with selinux enforcing. I am working on verifying the last but I believe that failure is on purpose and should not hold up this PR. Thank you for the assistance!

@rhatdan
Copy link
Member

rhatdan commented Aug 5, 2020

LGTM

@rhatdan rhatdan merged commit ec6130c into containers:master Aug 5, 2020
@dweomer dweomer deleted the containerd branch August 6, 2020 08:47
dweomer added a commit to dweomer/containerd that referenced this pull request Aug 10, 2020
`vagrant up` will build and install containerd and all dependencies,
setting up proper SELinux contexts on the runc and containerd binaries.
The VM is configured to be SELinux Enforcing by default but this gets
changed during various CI passes via a matrix param to Disabled and
Permissive before running tests. I have an open PR to fix the
container-selinux policy for containerd at
containers/container-selinux#98 which once
accepted we will want to update the CI matrix to use Enforcing mode
instead of Permissive.

All tests currently pass in SELinux permissive mode with containerd
configured with `enable_selinux=true`. To see which tests are failing
with SELinux enforcing and an already spun up VM:
`SELINUX=Enforcing vagrant up --provision-with=selinux,test-cri`
To test SELinux enforcing in a new VM:
`vagrant destroy -force; SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-cri`

The `selinux` shell provisioner, parameterized by the SELINUX envvar,
will configure the system as you would expect, with the side effect that
containerd is configured with `enable_selinux=true` via
`/etc/containerd/config.toml` for Permissive or Enforcing modes and
`enable_selinux=false` when SELINUX=Disabled.

Provided that virtualization is suported, this Vagrantfile and provisioners
make it easy to test containerd/cri for conformance under SELinux on
non-SELinux systems.

Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
tussennet pushed a commit to tussennet/containerd that referenced this pull request Sep 11, 2020
`vagrant up` will build and install containerd and all dependencies,
setting up proper SELinux contexts on the runc and containerd binaries.
The VM is configured to be SELinux Enforcing by default but this gets
changed during various CI passes via a matrix param to Disabled and
Permissive before running tests. I have an open PR to fix the
container-selinux policy for containerd at
containers/container-selinux#98 which once
accepted we will want to update the CI matrix to use Enforcing mode
instead of Permissive.

All tests currently pass in SELinux permissive mode with containerd
configured with `enable_selinux=true`. To see which tests are failing
with SELinux enforcing and an already spun up VM:
`SELINUX=Enforcing vagrant up --provision-with=selinux,test-cri`
To test SELinux enforcing in a new VM:
`vagrant destroy -force; SELINUX=Enforcing vagrant up --provision-with=shell,selinux,test-cri`

The `selinux` shell provisioner, parameterized by the SELINUX envvar,
will configure the system as you would expect, with the side effect that
containerd is configured with `enable_selinux=true` via
`/etc/containerd/config.toml` for Permissive or Enforcing modes and
`enable_selinux=false` when SELINUX=Disabled.

Provided that virtualization is suported, this Vagrantfile and provisioners
make it easy to test containerd/cri for conformance under SELinux on
non-SELinux systems.

Signed-off-by: Jacob Blain Christen <jacob@rancher.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants