Run as non-root Elasticsearch is outdated and not relevant when running on OpenShift

[barkbay]

Run as non-root Elasticsearch is outdated:

By default, the Elastisearch container is run as root and its entrypoint is responsible to run the Elasticsearch process with the elasticsearch user (defined with ID 1000). In the background, ECK uses an initContainer to make sure that the data volume is writable for the elasticsearch user.

This is not true since 8.0.0, Elastisearch container is run as user id 1000 by default:

docker run --rm docker.elastic.co/elasticsearch/elasticsearch:7.17.5 id
uid=0(root) gid=0(root) groups=0(root)

docker run --rm docker.elastic.co/elasticsearch/elasticsearch:8.3.3 id
uid=1000(elasticsearch) gid=0(root) groups=0(root)

We should also:

• State that this section is not relevant in the context of OpenShift as workloads are supposed to get assigned a random user id.
• Document the new default SecurityContext for Elasticsearch containers introduced with Hardened Security Context for Elasticsearch #6703

[gbschenkel]

Also when upgrading from 7.x to 8.x the container still use 0, I was needed to run a initContainer to chown the directory to user and group elasticsearch:elasticsearch which is 1000, but I didn't know when this stuck on my Openshift cluster.

This initContainer could be run as preStart of Elasticsearch nodes/pods, this could be like mentioned on #6239 but aimed for correct Elasticsearch pods permissions on upgrading from 7.x.

I still have a elk cluster on 7.17.9 and using eck operator 1.9.0 because that problem which the operator itself not upgrade to newer version.

[barkbay]

We should also document the new default SecurityContext for Elasticsearch containers introduced with #6703