feat: Allow configuration of pod's security context for DevWorkspaces on Kubernetes

[amisevsk]

What does this PR do?

Adds field podSecurityContext to the DevWorkspaceOperatorConfig CR to allow overriding securityContext in all workspace-related pods when running on Kubernetes.

Also sets a default fsGroup = 1234 to avoid issues on some k8s clusters (see: eclipse-che/che#20965). Note that this is what is done in Eclipse Che as well as far as I can tell so should not cause issues.

Configuration value is ignored on OpenShift.

What issues does this PR fix or reference?

Closes #718
Related: eclipse-che/che#20965
Related: eclipse-che/che#20963

Is it tested? How?

1. On Kubernetes, start a DevWorkspace with default config. Pods created for workspace (including cleanup job pods) should use fsGroup=1234
2. Configure .config.workspace.podSecurityContext in the devworkspace-operator-config DevWorkspaceOperatorConfig and make sure change is propagated to DevWorkspaces (may need to trigger a reconcile for running workspaces)

PR Checklist

• E2E tests pass (when PR is ready, comment /test v8-devworkspace-operator-e2e, v8-che-happy-path to trigger)
  • v8-devworkspace-operator-e2e: DevWorkspace e2e test
  • v8-che-happy-path: Happy path for verification integration with Che

[nils-mosbach]

Thanks!
Kubernetes 20 supports a property that allows changing owners only once if the root node does not match: fsGroupChangePolicy: "OnRootMismatch"

I haven't used this before, but that could speed up things if projects contain a lot of npm dependencies. What do you think?

See: https://kubernetes.io/blog/2020/12/14/kubernetes-release-1.20-fsgroupchangepolicy-fsgrouppolicy/

[amisevsk]

@nils-mosbach Thanks for that link, I wasn't aware of this change! This is something I believe might solve a longterm complaint about workspace startup time for us (and Eclipse Che), as our use cases (especially with npm projects ;)) generally involve many files in the PVC.

The DevWorkspace Operator depends on the Kubernetes 1.21.3 API, which means that this PR implicitly supports setting fsGroupChangePolicy:

apiVersion: controller.devfile.io/v1alpha1
kind: DevWorkspaceOperatorConfig
metadata:
  name: devworkspace-operator-config
config:
  workspace:
    podSecurityContext:
      fsGroupChangePolicy: "OnRootMismatch" 
      # Other fields need to be set too

The impact of this on DevWorkspace startup time could be tested using the next version of DWO as soon as this PR is merged. However, I don't have access to non-toy Kubernetes clusters (only OpenShift, where I believe this option would be ignored by the cluster). If you have a chance to test it out on your cluster, I'd love to hear the result!

[amisevsk]

/test v8-devworkspace-operator-e2e

[amisevsk]

/test v8-devworkspace-operator-e2e

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amisevsk, ibuziuk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [amisevsk]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[nils-mosbach]

Great, thanks a lot! I'll give it at go as soon as we manage to get devworkspaces running under Kubernetes. A version that I patched myself seemed to resolve the issue.

[nils-mosbach]

I've just updated devworkspace controller to quay.io/devfile/devworkspace-controller:next.
#20965 and #20963 are resolved in my case. I'll close the issues. Thanks, a lot!

[amisevsk]

I'm glad to hear it 😄