What variables and config files define the paths for the namelists and the executables?

[mandresm]

@tsemmler05 wrote

For some reason I can't find the config file from where it becomes clear from where the executable and namelists are drawn. Could you please give me a hint where to look for it?

NOTE: I received the email above, and I am working on a reply.

[denizural]

My recommendation is running the esm_versions in verbose mode (--verbose) and redirecting the output to a file. Eg.
esm_runscripts -e test my_runscript.yaml --check --verbose &>output.log

Searching for the executables and namelists in output.log shows which files esm_tools uses. Here is an example:

::: logging file category: recom
- source: /some/path/esm_tools/namelists/.//recom/awicm_recom//namelist.recom_co2_coupling
- target: /some/path/debug_4/run_18500101-18500131/work/namelist.recom

::: copying file category: recom
- source: /some/path/esm_tools/namelists/recom/awicm_recom/namelist.recom_co2_coupling
- target: /some/path/debug_4/run_18500101-18500131/config/recom/namelist.recom

Here namelist.recom_co2_coupling is taken, renamed to namelist.recom and copied to config and work directories.

[tsemmler05]

O.k., now I know the paths from where executables and namelists are copied but I would like to change one of these paths before the next job of the chain goes into the queue. In which config file can I do this?

[mandresm]

I think the answer bellow will answer your question.

[mandresm]

An additional note, for safety reasons the executables are copied only once, at the beginning of the experiment and never more updated (they are then copied into the work folder from the binary folder in the experiment in the legs 2, 3, 4...). If you want to update the binaries with an external folder in the middle of a simulation, in addition to the changes mentioned below in the runscript, you'll need to add --update-filetypes bin to the esm_runscripts call. You can see an explanation of that option in the esm_runscripts help (esm_runscripts -h)

[tsemmler05]

Thanks, this helped!

[mandresm]

Hi @tsemmler05,

Before we start

We are aware that this is difficult for the user. At the same time this is what allows ESM-Tools to be extremely modular and that component and machine files can be shared across different groups and coupled systems. To overcome the difficulty for the users, @pgierz has made great progresses on adding a feature to help the users to identify the provenance of variables. This is one of the top priorities for us in the coming months, so things will become more intuitive soon without the need of radically changing the ESM-Tools design. In the mean time you'll need to understand well the structure of the config files you are using so here is a detailed explanation:

Where the executable and namelist sources are set depends entirely on how the person/group that configured the particular model you are using decided to do so. For AWI-CM3 (and in general for every model supported in ESM-Tools as far as I'm aware) the two variables we are looking at are <model>.namelist_dir and general.model_dir or <setup>.model_dir. The first step is to understand that model_dir and namelist_dir mean nothing to ESM-Tools (they are not featured variables), the variables that actually mean something to ESM-Tools are actually bin_sources and config_sources. We will be looking how namelist_dir and model_dir end up in those other two featured variables and how to use them, in the following sections.

Another important concept to refresh is that variables, such as namelist_dir can be repeated across multiple configuration files (e.g. model_dir in the runscript, awicm3.yaml and fesom-2.0.yaml). The value of that variable is resolved with the following hierarchy:

For more information on the hierarchy, please read https://esm-tools.readthedocs.io/en/latest/yaml_hierarchy.html

model_dir for AWI-CM3

So let's start from the bottom of the hierarchy, first with FESOM. In fesom-2.0.yaml we see that the featured variable bin_source in charged of copying the executable from the source directory to the work directory is defined as follows:

esm_tools/configs/components/fesom/fesom-2.0.yaml

Lines 295 to 296 in ee4e403

	bin_sources:
	fesom: "${bin_dir}/${comp_executable}"

As you can see the path for the binary is composed by two variables: bin_dir and comp_executable. comp_executable is defined above in the file and I don't think is edited anywhere, but you could choose to change it in your runscript if necessary. bin_dir is defined in the same fesom-2.0.yaml here:

esm_tools/configs/components/fesom/fesom-2.0.yaml

Lines 93 to 94 in ee4e403

	setup_dir: "${model_dir}"
	bin_dir: "${setup_dir}/bin"

Now the question is whether these variables are edited somewhere else in a config file that is higher in the hierarchy: awicm3.yaml, a setup file. There, there is no bin_dir so we need to be looking for setup_dir. In the fesom section we can see:

esm_tools/configs/setups/awicm3/awicm3.yaml

Line 247 in ee4e403

setup_dir: ${general.model_dir}

So this wins over the setup_dir defined in the fesom-2.0.yaml and takes the model_dir from the general section of awicm3.yaml:

esm_tools/configs/setups/awicm3/awicm3.yaml

Lines 1 to 8 in ee4e403

	#########################################################################################
	######################### AWICM3 YAML CONFIGURATION FILE ###############################
	#########################################################################################
	general:
	model: awicm3
	model_dir: ${esm_master_dir}/awicm3-${version}

Users of AWI-CM3 always define their the directory where the coupled setup is built, under the awicm3 or general sections (they are equivalent). Because the runscript has the highest hierarchical level, general/<awicm3>.model_dir there will win over the one general.model_dir in the awicm3.yaml, which is what the users of AWI-CM3 are using all the time.

The same process can be followed for oifs.yaml.

If you want to check what is the actual source of the executables you can search for the bin_sources variable in the finished_config.yaml file (<experiment_path>/run_DATE/config/*_finished_config.yaml, if the run is not finished, or <experiment_path>/config/*_finished_config.yaml if the run is finished).

Advance example

Let's imagine you now to source the executables from different setup folders (we strongly advance against this, but it's just an example). As seen before, in awicm3.yaml the binaries are expected under the same folder controlled by general/<awicm3>.model_dir, however, you could modify from your runscript (top in hierarchy) how fesom.setup_dir and oifs.setup_dir are defined to source different paths (both end up in their respective fesom.bin_sources and oifs.bin_sources, in their component yaml files):
your_runscript.yaml

general:
    [ ... ]
fesom:
    setup_dir: "/a/path/here/"
oifs:
    setup_dir: "/a/different/path/here/"

namelist_changes in AWI-CM3

This is very similar to the previous case, but now the featured variable that controls the copying of the namelists is config_sources, that in fesom-2.0.yaml is defined as:

esm_tools/configs/components/fesom/fesom-2.0.yaml

Lines 307 to 313 in ee4e403

	config_sources:
	config: "${namelist_dir}/namelist.config"
	forcing: "${namelist_dir}/namelist.forcing"
	ice: "${namelist_dir}/namelist.ice"
	oce: "${namelist_dir}/namelist.oce"
	io: "${namelist_dir}/namelist.io"
	diag: "${namelist_dir}/namelist.diag"

We see here that the path of the namelist sources depends on namelist_dir variable, defined in the fesom-2.0.yaml

esm_tools/configs/components/fesom/fesom-2.0.yaml

Line 98 in ee4e403

namelist_dir: "${esm_namelist_dir}/fesom2/2.0/"

but this is overridden by the hierarchically higher awicm3.yaml:

esm_tools/configs/setups/awicm3/awicm3.yaml

Lines 101 to 108 in ee4e403

	choose_general.version:
	"v3.0":
	namelist_dir: ${general.esm_namelist_dir}/oifs/${version}/awicm3/v3.0
	"frontiers":
	namelist_dir: ${general.esm_namelist_dir}/oifs/${version}/awicm3/frontiers
	"frontiers-xios":
	with_xios: True
	namelist_dir: ${general.esm_namelist_dir}/oifs/${version}/awicm3/frontiers

where general.esm_namelist_dir is a special ESM-Tools variable that gives the path to the ESM-Tools installation.

Example

You could choose to change your namelist sources (not recommended, unless stricly needed), for example for FESOM only, by adding to your runscript:
your_runscript.yaml

fesom:
    namelist_dir: "/your/path/to/your/namelists/"

Bare in mind that ESM-Tools will still modify your namelists with the namelist_changes in the configuration files, the original namelists are treated by ESM-Tools as templates, while namelist_changes allow for automatisation of the configurations.

Please, let us know if that solve your questions.