Loki transformations from config file

[mlange05]

This change updates Loki to the recently release version v0.2.6, updates the python module paths for the SCC-CUF family transformations and adds config entries for the three main SCC-class pipelines to the config. This effectively now triggers all Loki transformations from config file, thus allowing the eventual removal of some of the custom entry-point logic in loki-transform.py convert.

Note: I previously tried to piggyy-back this with the SCCRawStack transformation (PR #86), but that created weird crossfire for GCC builds. So one step at a time now...

Edit: Another update on this - the actual use of full-scale pipelines was actually trickier than I thought, but is now complete. This PR now switches all except the C-transpilation variants to use composed pipelines from config-files.

Integration note: The top-level commit repoints the Loki version to a dev branch, which needs removing or repointing to master once the requirements have been merged!

[mlange05]

@reuterbal This was trickier than originally envisaged, but it now seems to work for all custom, SCC and SCC-CUF pipelines (C-transpilation is left for another PR). Please have a look and see if this style is suitable for future developments.

There's also a weird recurring failure in the PyIface variant that seems unrelated, but odd. I'd appreciate any help or insights into potential CI gremlins if you have a minute to spare.

[reuterbal]

Apologies for the delay. This looks fantastic!

I would keep the Loki version in the bundle fixed (I realize that this was likely not possible at the time the PR was opened because there wasn't a release for that functionality, yet) to have CLOUDSC reference a stable version.

Otherwise it might make sense to add some more details in the config file simply to be able to point others to it as a reference.

[reuterbal]

We should fix a version here again, to make sure that CLOUDSC itself is stable - regression testing with latest main will be triggered from Loki side

[mlange05]

Yes, I finally got Loki v0.2.7 to work with this and pinned this version now. Anything before that will be able to compile the SCC-CUF transformations (enabled via --with-cuda), because of a Loki issue in v0.2.6.

[reuterbal]

Since this is more or less the public reference of how transformation pipelines can be defined in config files, we should maybe expend a somewhat detailed comment at the top of the file to explain the concept. Maybe something that at least touches on the fundamental building blocks:

• Definition of Scheduler behaviour ([default] - this should really be renamed :))
• Definition of entry points/item-specific config ([routines])
• Definition of transformation configs ([transformations]) that represent processing steps that will be used in one or multiple pipelines, and that each of them receives a name
• Definition of processing pipelines ([pipelines]) that combine multiple transformations into a processing sequence

[mlange05]

Yes, good point. I pushed a slightly more verbose explanation to the main loki config.