Formally add support for SELinux (Enforced mode)

[davidnuzik]

We need to expand our testing and identify any issues that prevent us from formally supporting SELinux (in enforced mode) on CentOS. Please note that SELinux is expected to work -- this issue is for tracking testing effort so we can formally support it.

We should review existing GitHub issues, but we need to execute some testing and identify any other issues. As needed, we'll need to resolve these issues so we may fully support SELinux on CentOS.

Formal CentOS (general OS) support is also needed, which is tracked separately here: #1371

[davidnuzik]

@ShylajaDevadiga I have assigned this issue to you for now. This will require some testing and discovery. We need to identify any/all SELinux CentOS issues that prevent us from formally supporting CentOS with SELinux support in our next release. Work with me as needed.

[davidnuzik]

As a reminder we must support IPv6 as well.

[westurner]

• https://wiki.centos.org/HowTos/SELinux#Troubleshooting_SELinux
• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-fixing_problems-allowing_access_audit2allow
  • https://danwalsh.livejournal.com/24750.html
• https://stopdisablingselinux.com/

Is this more complicated than running tests in permissive mode, running audit2allow, and reviewing the generated policy for what needs to be relabled so that k3s will run with selinux in enforcing mode?

• https://www.google.com/search?q=openshift+selinux
  • https://fedorapeople.org/~dwalsh/SELinux/Presentations/openshift_selinux.ogv
  • https://blog.openshift.com/openshift-protects-against-nasty-container-exploit/
  • https://blog.openshift.com/securing-kubernetes/

    The main thing to understand about SELinux integration with OpenShift is that, by default, OpenShift runs each container as a random uid and is isolated with SELinux MCS labels. The easiest way of thinking about MCS labels is they are a dynamic way of getting SELinux separation without having to create policy files and run restorecon.

    If you are wondering why we need SELinux and namespaces at the same time, the way I view it is namespaces provide the nice abstraction but are not designed from a security first perspective. SELinux is the brick wall that’s going to stop you if you manage to break out of (accidentally or on purpose) from the namespace abstraction.

    CGroups is the remaining piece of the puzzle. Its primary purpose isn’t security, but I list it because it regulates that different containers stay within their allotted space for compute resources (cpu, memory, I/O). So without cgroups, you can’t be confident your application won’t be stomped on by another application on the same node.

  • https://docs.openshift.com/container-platform/4.3/installing/installing_bare_metal/installing-bare-metal.html#installation-requirements-user-infra_installing-bare-metal ... lots of vCPUs, RAM, and storage
• "Container Domains (Types)" https://danwalsh.livejournal.com/81756.html by @rhatdan
  • dnf install -y setools-console; seinfo -acontainer_domain -x
    • container_t, container_logreader_t, container_userns_t
    • spc_t "Super-Privileged Container"
  • container_file_t
  • container_share_t
  • udica builds selinux policy for containers
    • https://github.com/containers/udica
    • https://fedoramagazine.org/use-udica-to-build-selinux-policy-for-containers/
      • podman inspect 37a3635afb8f > container.json; udica -j container.json my_container
      • semodule -i my_container.cil /usr/share/udica/templates/{base_container.cil,net_container.cil,home_container.cil}
      • podman run --security-opt label=type:my_container.process -v /home:/home:ro -v /var/spool:/var/spool:rw -p 21:21 -it fedora bash
• https://github.com/containers/container-selinux
  • https://github.com/containers/container-selinux/blob/master/Makefile
  • https://github.com/containers/container-selinux/blob/master/container.fc
  • https://github.com/containers/container-selinux/blob/master/container.if
  • https://github.com/containers/container-selinux/blob/master/container.te

$ cat container.te | egrep '^\s*type'
typealias container_runtime_t alias docker_t;
type container_runtime_exec_t alias docker_exec_t;
type spc_t, container_domain;
type container_auth_t alias docker_auth_t;
type container_auth_exec_t alias docker_auth_exec_t;
type spc_var_run_t;
type container_var_lib_t alias docker_var_lib_t;
type container_home_t alias docker_home_t;
type container_config_t alias docker_config_t;
type container_lock_t alias docker_lock_t;
type container_log_t alias docker_log_t;
type container_runtime_tmp_t alias docker_tmp_t;
type container_runtime_tmpfs_t alias docker_tmpfs_t;
type container_var_run_t alias docker_var_run_t;
type container_plugin_var_run_t alias docker_plugin_var_run_t;
type container_unit_file_t alias docker_unit_file_t;
type container_devpts_t alias docker_devpts_t;
typealias container_ro_file_t alias { container_share_t docker_share_t };
type container_port_t alias docker_port_t;
	type cephfs_t;
		type systemd_logind_t;
	type container_t;
typeattribute container_t container_domain, container_net_domain;
	type cgroup_t;
	type usermodehelper_t;
		type sysctl_kernel_ns_last_pid_t;
	type iptables_t;
		type unconfined_service_t;
		type unconfined_service_exec_t;
typeattribute  container_userns_t sandbox_net_domain;
		type proc_t, proc_kcore_t;
		type sysctl_t, sysctl_irq_t;
typeattribute container_logreader_t container_net_domain;
		type sysadm_t, staff_t, user_t;
	type init_t;

...
rpm -ql setools-console
man sesearch
sesearch -A -s 'container_.*' -rs -t 'container_.*' -rt | grep 'container' | wc -l
sesearch -A -t 'spc_.*' -rt -s 'spc_.*' -rs | grep 'spc'
seinfo --all | egrep 'kubernetes|openshift|container|docker|spc_' | less
seinfo -a container_domain -x

[westurner]

• remove the "Set SELinux to disabled state" task from the ansible prereq role: https://github.com/rancher/k3s/blob/0374c4f63d056df01c3e9e8cf4d77a6461169070/contrib/ansible/roles/prereq/tasks/main.yml#L2-L5

[cjellick]

@westurner the main bit of work involves getting MCS label support into containerd (k3s's embedded contianer runtime): See this WIP PR from @ibuildthecloud:
containerd/cri#1246

[erikwilson]

Merged #1448

[davidnuzik]

Closing this issue in favor of #533