Roll back problematic pod labeling feature

[JAORMX]

What type of PR is this?

/kind cleanup
/kind feature

What this PR does / why we need it:

Some time ago I introduced a feature for the SPO to label denials for
pods. While this seemed like a good idea at the time, it introduces too
much privileges for te SPOd (update and patch all pods).

While there hasn't been any security incidents reported for the SPOd,
the drawbacks of giving such privileges are more critical than the
features the labeling would have enabled. So, let's get rid of this
feature and revisit the problem another time.

This fully removes the flag as there is no documented usage of the feature.

Which issue(s) this PR fixes:

None

Does this PR have test?

The test was removed.

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Pod denials labeling feature was removed.

[codecov-commenter]

Codecov Report

Merging #1088 (3f9e128) into main (ad89007) will increase coverage by 0.51%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1088      +/-   ##
==========================================
+ Coverage   50.13%   50.65%   +0.51%     
==========================================
  Files          42       42              
  Lines        4789     4740      -49     
==========================================
  Hits         2401     2401              
+ Misses       2309     2262      -47     
+ Partials       79       77       -2     

[jhrozek]

/test pull-security-profiles-operator-test-e2e

[JAORMX]

/retest

[jhrozek]

So I'm fine removing this functionality (AFAIK nothing and nobody uses it now), but I'd like to discuss alternatives, because I think it's something users will ask for.

The use-case I care about the most is:

• a developer records an initial policy in CI
• an SRE deploys the app and the policy. Both acknowledge that the initial policy version is not 100% correct and want to find out AVCs/seccomp denials
• so far I was thinking about adding a "permissive" flag to be added to the profiles which would change the defaultAction to SCMP_ACT_LOG for seccomp profiles and inherit from typepermissive process for SELinux policies.
• this would produce logs, then the question is how to gather them and there we'd probably do what your approach did, but maybe label the profiles instead of the workloads? I think this would be more palatable from the RBAC perspective. Maybe we could issue events, too, not sure what would be best for SREs in the end.

[JAORMX]

@jhrozek I was actually thinking about doing events instead. But struggled to introduce them without adding a lot of extra complexity to the code-base. We might need to do some refactoring.

[jhrozek]

@jhrozek I was actually thinking about doing events instead. But struggled to introduce them without adding a lot of extra complexity to the code-base. We might need to do some refactoring.

Let's file an issue, then

[jhrozek]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JAORMX, jhrozek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JAORMX,jhrozek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[JAORMX]

/retest