Add ConfigMap support for seccomp custom profiles

[pjbgf]

This proposal focuses on allowing users to load user-defined seccomp profiles from ConfigMap objects instead of loading them from physical files placed on nodes.

Feature: #135

For context:
kubernetes/kubernetes#52827
kubernetes/kubernetes#20870

/sig-node
/sig-auth

/priority important-longterm

/assign @tallclair
/cc @jessfraz

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pjbgf
To complete the pull request process, please assign dchen1107
You can assign the PR to them by writing /assign @dchen1107 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• keps/sig-node/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dchen1107]

We discussed this proposal at today's SIG Node meeting. It is a usability improve for seccomp feature and would like to bundle it together with seccomp GA in v1.18 even there is no strict dependencies.

[tallclair]

I haven't had a chance to look through the details yet, but I think it would be helpful to add a section (maybe under alternatives) justifying the use of a ConfigMap over a CRD. There's a good list of when each is appropriate here: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#should-i-use-a-configmap-or-a-custom-resource, and so noting which of those bullets apply in this situation would be helpful.

[pjbgf]

@tallclair I have added the reasoning for choosing ConfigMap vs CRDs. PTAL

[pjbgf]

@tallclair PTAL

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[palnabarun]

/remove-lifecycle stale

[palnabarun]

/milestone clear

[evrardjp]

I am on the fence. This proposal is indeed more convient, and simple for users. However, I have to admit that passing big blobs to container runtimes for each container doesn't sound right to me.

The file based approach is simple, and just require configuration management.
I am pretty sure a few of those already exist, so that we won't need to do these changes in the container runtimes :p

[pjbgf]

@saschagrunert @evrardjp thank you for taking the time reviewing the KEP.

I don't mind changing the direction of this KEP to go for a file-based approach (which is already implemented and used by "localhost/" profiles). But given that seccomp is a security feature, I believe we should mitigate (or have some protection against) "profile race tampering".

In a file-based approach, between the creation of the file (by the kubelet, based off the ConfigMap/CRD) and its consumption (by the container runtime), the profile can be tampered with effectively disabling its protections. The end-users would be oblivious to this, as the containers would still run. It may be even harder to diagnose if it happens on a single node within their cluster.

At present there is no such protection, but I believe this would be a valid "requirement" for GA.
Implementing a tamper-resistant file based approach, could require further changes to the CRI (and the container run-times), as they would have to receive and validate the payload hashes.

On the other hand, passing the entire profile to the CRI removes such need. As a level of "tamper resistance" could be obtained at transport level, which is something important between the kubelet and CRI regardless of this feature and therefore would be off-scope of this KEP.

The "least-effort" compromise could be to go for the file-based approach, using the existing infrastructure and use file permissions to protect against this. Having this well-documented and working together with kubeadmin/kops/kubespray to provide the community with an automated way to ensure file permissions are up to par. However, that feels to me like the weakest link in the seccomp chain implementation.

Please do share your thoughts and potential workarounds. I am quite keen to get this progressed and pushed over the line sooner rather than later.

[saschagrunert]

@saschagrunert @evrardjp thank you for taking the time reviewing the KEP.

I don't mind changing the direction of this KEP to go for a file-based approach (which is already implemented and used by "localhost/" profiles). But given that seccomp is a security feature, I believe we should mitigate (or have some protection against) "profile race tampering".

I acknowledge your arguments and the only issue I see is answering the question on: How much information we want to transfer via the CRI? We right now have a state in the CRI where some information are transferred directly and others via local paths. If we say we generally want to move away from referencing local paths then I'm totally fine with going your proposed way.

---

Another more general point: We'd like to support you with your efforts to make Kubernetes more easily secure-able. With taking the seccomp GA KEP as well as the new proposed release cycle into account, do you think we could bring this feature with 1.19 (together)?

We're planning to work in parallel on a more high level seccomp-operator in a separate repository. The idea is to provide a much higher level abstraction for seccomp (via the CRD) and sync the state into either files (like now) or config-maps (like your proposed KEP). What do you think about that? I'd love to see us working together on the overall story.

(Later on we could promote your KEP to GA by adding the CRD from the seccomp-operator project, but this will be a tough run. Shipping a CRD via a feature gate is right now an unresolved question in Kubernetes.)

[pjbgf]

Another more general point: We'd like to support you with your efforts to make Kubernetes more easily secure-able. With taking the seccomp GA KEP as well as the new proposed release cycle into account, do you think we could bring this feature with 1.19 (together)?

That's brilliant, I do think we can deliver this for 1.19, probably with a few small changes (i.e. use file-based profiles). The tamper-resistance can be a KEP on its own to consider Seccomp Profiles and AppArmor, which would consider the CRI impact.

The seccomp operator sounds like a great idea and I would be keen to part take. Around the subject, but not as comprehensive as the operator, I got another KEP open that would bring some Kubernetes native profiles types. And on a side note, I have a pet project that auto-generates seccomp profiles reasonably for some workloads. Putting this all together may result in a better end-user experience.

I think it may be worth having a catch-up to consider the "big-picture" and decide what pieces of the puzzle we shall focus first to achieve that. From mid-next week I will have plenty of extra time to focus on this. I will reach you out on slack.

[saschagrunert]

The seccomp operator sounds like a great idea and I would be keen to part take. Around the subject, but not as comprehensive as the operator, I got another KEP open that would bring some Kubernetes native profiles types. And on a side note, I have a pet project that auto-generates seccomp profiles reasonably for some workloads. Putting this all together may result in a better end-user experience.

That sounds great, seccomp profile recording via OCI hooks is mentioned in the RFC of the operator too. I think this would be a very valuable addition in the future. 👍

I think it may be worth having a catch-up to consider the "big-picture" and decide what pieces of the puzzle we shall focus first to achieve that. From mid-next week I will have plenty of extra time to focus on this. I will reach you out on slack.

Yes, I setup a scheduled Zoom meeting for the whole topic, everyone if free to join:

Time: Apr 30, 2020 02:00 PM Amsterdam, Berlin, Rome, Stockholm, Vienna
Meeting: https://zoom.us/j/91379118387

[pjbgf]

Yes, I setup a scheduled Zoom meeting for the whole topic, everyone if free to join:
Time: Apr 30, 2020 02:00 PM Amsterdam, Berlin, Rome, Stockholm, Vienna
Meeting: https://zoom.us/j/91379118387

cc @tallclair

[pjbgf]

@saschagrunert I created a google doc to set an initial agenda. In it I also added a mind-map for me to talk through my understanding of the current state/outstanding work. I added a point for you to talk a bit about the seccomp-operator, feel free to add more stuff to it.

[pjbgf]

This KEP is being withdrawn in favour of the seccomp-operator, which will eventually bring similar features out of tree.