Run a containerized code inside AiiDAlab via aiida-core

[csadorf]

Since we will soon have aiida-core support to running containerized code installed in the docker/Singularity/Sarus, it is a good idea to have aiidalab to also support this.
However, the aiidalab is running as the container, it is not naturally supported to run docker inside the docker container.
There are essentially three ways to do this https://devopscube.com/run-docker-in-docker/:

• using /var/run/docker.sock mapped to container
• using docker:dind base image.
• using Sysbox runtime VM support https://github.com/nestybox/sysbox

I think for the containerized code, running code on docker has many limitations such as no decent MPI support, and output files of the docker containerized code belong to root use which can not be deleted without sudo.
The better way would be to have Singularity installed and supported in the aiidalab container.

But it worth to have a look at the sysbox image which has many high level feature supported to make the container more like a VM. It can enable Docker-in-Docker and Kubernetes-in-Docker etc.
It is easy to installed on Linux hosts and then can be used by docker run --runtime=sysbox-runc -it <image> ...

[csadorf]

@unkcpz I converted this to an issue on aiidalab-docker-stack. Please feel free to transfer to another repository if you think it is more appropriate there. Thank you.

[unkcpz]

Some update and maybe it is worth to check this issue with AiiDA 2.1 supported. I have check to install and running image using Singularity, which looks all good. The only trick is the container start from localhost need to be started with --privileged option as instructed in https://stackoverflow.com/questions/61574793/how-to-use-singularity-in-docker-container.
Not sure how to set in in k8s, will give it a try.

EDIT: k8s + Jupyterhub elevated permission -> jupyterhub/zero-to-jupyterhub-k8s#379