Use fsGroup instead of initContainer for setting appropriate file owners.

[AleksandarSavchev]

How to categorize this issue?

/area quality
/kind enhancement

What would you like to be added:
Currently etcd uses initContainers to change file owners:

etcd-druid/pkg/component/etcd/statefulset/statefulset.go

Lines 491 to 525 in 4cb6e5c

	sts.Spec.Template.Spec.InitContainers = []corev1.Container{
	{
	Name: "change-permissions",
	Image: c.values.InitContainerImage,
	ImagePullPolicy: corev1.PullIfNotPresent,
	Command: []string{"sh", "-c", "--"},
	Args: []string{"chown -R 65532:65532 /var/etcd/data"},
	VolumeMounts: getEtcdVolumeMounts(c.values),
	SecurityContext: &corev1.SecurityContext{
	RunAsGroup: pointer.Int64(0),
	RunAsNonRoot: pointer.Bool(false),
	RunAsUser: pointer.Int64(0),
	},
	},
	}
	if c.values.BackupStore != nil {
	// Special container to change permissions of backup bucket folder to 65532 (nonroot)
	// Only used with local provider
	prov, _ := utils.StorageProviderFromInfraProvider(c.values.BackupStore.Provider)
	if prov == utils.Local {
	sts.Spec.Template.Spec.InitContainers = append(sts.Spec.Template.Spec.InitContainers, corev1.Container{
	Name: "change-backup-bucket-permissions",
	Image: c.values.InitContainerImage,
	ImagePullPolicy: corev1.PullIfNotPresent,
	Command: []string{"sh", "-c", "--"},
	Args: []string{fmt.Sprintf("chown -R 65532:65532 /home/nonroot/%s", *c.values.BackupStore.Container)},
	VolumeMounts: getBackupRestoreVolumeMounts(c),
	SecurityContext: &corev1.SecurityContext{
	RunAsGroup: pointer.Int64(0),
	RunAsNonRoot: pointer.Bool(false),
	RunAsUser: pointer.Int64(0),
	},
	})
	}
	}

I would like these initContainers to be removed and their functionality replaced with the use of fsGroup: 65532 added here:

etcd-druid/pkg/component/etcd/statefulset/statefulset.go

Lines 526 to 530 in 4cb6e5c

	sts.Spec.Template.Spec.SecurityContext = &corev1.PodSecurityContext{
	RunAsGroup: pointer.Int64(65532),
	RunAsNonRoot: pointer.Bool(true),
	RunAsUser: pointer.Int64(65532),
	}

This will set group owner 65532 for mounted files and would make these files accessible for the etcd pod.

Why is this needed:
Remove unnecessary etcd containers and avoid changing owners on PV

[renormalize]

@unmarshall @shreyas-s-rao should we do this in #777 or should we avoid adding more changes to it?

[renormalize]

I've tested this out with AWS, and fsGroup is sufficient to eliminate the use of init containers as mentioned in the issue description with older volumes which run with etcd-custom-image in root:root.

Testing has to be performed with all providers to ensure that fsGroup is implemented by all CSI drivers. All access modes aren't supported for OpenStack, as written in kubernetes/cloud-provider-openstack#2075 (comment).

I'm currently unsure if fsGroup would work with the local provider.