Dockerized HEAT version

[cdrojasm]

Hi,

As @plasmapotential has said in #33, in the next weeks docker version of HEAT will be available. Is it possible to know the approach of the coming soon version.

I was capable to download and play some examples following the video tutorials. However, when I check image building the scripts, I note some things (Dockerfile and buildDocker) that could be considered to the next release.

1. I note that docker hub image is too large, 8.83GB. Big image is a problem in CI/CD due to slow pipelines. Also, it is so tedious to download this large images.
2. In base image Ubuntu:22.04 some features could not be required during execution of HEAT.
3. I see that many dependencies are installed in a handcraft way (mafot, openfoam, m3dc1, swak4foam, ....), this could be a problem with another developers which want to contribute in image or deployment scripts. At this point it is not possible to build the image, it is not replicable. It exists some paths that have absolute route based in the original host where HEAT was developed.
4. Package m3dc1 is not accessible using conventional repositories (images or code), it is required to get access from Princeton University staff in the GitHub repo.
5. handcrafted installed packages lack well-defined version, after check inside the files and debugging in the container I got these that seen to be package versions or proper branch names.

export IMAGE_NAME="HEAT" export IMAGE_VERSION="4.2" export MAOF_PACKAGE_BRANCH_TAG_VERSION="main" export PARAVIEW_PACKAGE_BRANCH_TAG_VERSION="v5.10.1" export OPENFOAM_PACKAGE_BRANCH_TAG_VERSION="20211218"

6. It not exists suitable scripts to execute installation, service start and cleaning.

Under the current development of Dockerized version of HEAT, probably these issues are being fixed. However, in the case that some of them are omitted, It is possible to work in order to achieve a better experience during development and sharing of the project image ?

I have certain recommendations over the last considerations.

1. We can change the base image for another lightweight one like alpine. However, installation of another packages will be tricky compared with Ubuntu base image.
2. Some of the dependencies could be installed using cloning, build stages. I checked maof, paraview and openfoam are dependencies that seen to be installable by mean of repositories code building.
3. We could define an image - container where image only get the required to build dependencies, and during container first initialization run the build and install each dependency.
4. An environment file could be used to store the dependencies and the environment variables required during image building and operation.
5. We could write files to handle build, start services and cleaning tasks related with the deployment.
6. In dockerfile we could condition the installation of developer dependencies according to certain environment variable.

I'm open to help in any of these tasks and other that you consider for the upcoming release, thanks a lot and sorry for this large text.

[tloobyCFS]

Hey @cdrojasm thanks for starting this dicussion. I think that all of these points you have made are good ones, and we should work to improve the developer experience. I did not plan to add any of these features in the v4.1 release next week (it is mainly just physics algorithm updates).

First let me respond to your observations:

1. I agree that we could work to make the image smaller. Currently, the size limitation has not been slowing me down so I haven't worked to shrink the image. Not that it isn't important, it is just of lesser importance than adding physics to the code.
2. This is probably true, but there are some advantages to using the ubuntu distro, especially when we are dealing with users that are not linux experts. The main reason I use ubuntu is because of the support and the package manager. I like Alpine (years ago i compiled my own gentoo kernel...painful lol), but it obviously does not have the huge repository of packages that ubuntu has.
3. I think cleaning this up would be a great improvement. Indeed some of these packages could be compiled directly into the container using clones. We should definitely strive to improve the build docker scripts
4. The M3DC1 bins/libs that are used in HEAT are not the actual M3DC1 code itself, but rather they are helper packages for M3DC1 that read the output, and they are open source. M3DC1 compatibility is a recent feature addition (v4.0).
5. The hand crafted packages definitely do not follow strict version control. We should improve this.
6. I agree that building new docker containers is not easy for 3rd party developers. I would be happy to clean up the code to make this possible. Over the years I have been the only one building the containers, and collaborators usually contribute to the physics of the code (in python). Having collaborators contribute to the workflow and CI is new to me so this will be a learning experience. I appreciate your patience.

Now let me respond to your recommendations:

1. Probably going to be difficult to switch to Alpine or another distro. I am not completely against it, but I think we should start with some of the other recommendations first
2. I completely agree. Creating clones and build stages would be a great addition to the development workflow.
3. I like this idea. Build on the first initialization would be good. I assume this would require a volume to be created so that the build is persistent?
4. I like this idea
5. This sounds like a good idea but I do not have much experience with this.
6. Would the environment variables be related to the architecture? Or were you thinking something else?

Overall, everything you outlined sounds good. My primary mission is physics (to make sure SPARC works), so I do not have a large amount of time to devote to these items. I could probably add some of these features slowly over time. If you wanted to help add some of these features more quickly, the help would be greatly appreciated. If you want to get started on some of these items, let me know what you would need from me.

Thanks again for your support.

[tloobyCFS]

after giving it some thought, I will try to add some of these features into the v4.1 release. I will update this thread as I add some of these features. I will probably start with recommendation 2