setup_xml error

[benjamin-cash]

I've seen this error a few times now. The example below is from Hercules, with a version I recently updated to develop (commit ec63492)

[cash@hercules-login-4 workflow]$ ./setup_xml.py /work/noaa/nems/cash/RUNS/EXPDIR/SFS
Traceback (most recent call last):
  File "/work2/noaa/nems/cash/workflow_test/workflow/./setup_xml.py", line 79, in <module>
    main()
  File "/work2/noaa/nems/cash/workflow_test/workflow/./setup_xml.py", line 70, in main
    app_config = app_config_factory.create(f'{net}_{mode}', cfg)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/noaa/nems/cash/workflow_test/workflow/wxflow/factory.py", line 71, in create
    return self._builders[key](*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/noaa/nems/cash/workflow_test/workflow/applications/applications.py", line 21, in __call__
    obj = type.__call__(cls, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/noaa/nems/cash/workflow_test/workflow/applications/gefs.py", line 11, in __init__
    super().__init__(conf)
  File "/work2/noaa/nems/cash/workflow_test/workflow/applications/applications.py", line 72, in __init__
    if not AppConfig.is_monotonic(self.fcst_segments):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/noaa/nems/cash/workflow_test/workflow/applications/applications.py", line 236, in is_monotonic
    return all(x < y for x, y in zip(test_list, test_list[1:]))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/noaa/nems/cash/workflow_test/workflow/applications/applications.py", line 236, in <genexpr>
    return all(x < y for x, y in zip(test_list, test_list[1:]))
               ^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

The command to generate the experiments was:
./setup_expt.py gefs forecast-only --idate 1994050100 --edate 1994050100 --pslot SFS --app S2S --resdetatmos 96 --resensatmos 96 --resdetocean 1 --start cold --gfs_cyc 1 --nens 10 --comroot /work/noaa/nems/cash/RUNS/COMROOT --expdir /work/noaa/nems/cash/RUNS/EXPDIR --yaml /work2/noaa/nems/cash/workflow_test/RUN/SFS_options.yaml

and my currently loaded modules are:

[cash@hercules-login-4 workflow_test]$ module list

Currently Loaded Modules:
  1) contrib/0.1                       6) pigz/2.7          11) sqlite/3.43.2           16) libyaml/0.2.5         21) module_gwsetup.hercules
  2) rocoto/1.3.7                      7) zstd/1.5.2        12) util-linux-uuid/2.38.1  17) py-pyyaml/6.0
  3) intel-oneapi-compilers/2023.1.0   8) tar/1.34          13) python/3.11.6           18) openblas/0.3.24
  4) stack-intel/2021.9.0              9) gettext/0.21.1    14) py-markupsafe/2.1.3     19) py-setuptools/63.4.3
  5) zlib/1.2.13                      10) libxcrypt/4.4.35  15) py-jinja2/3.1.2         20) py-numpy/1.23.4

[aerorahul]

@benjamin-cash
I just cloned the hash noted above on Hercules and executed exactly the statements from this discussion (except, using my space for creating and writing the experiment, and using the gefs default CI yaml as I cannot see the SFS_options.yaml)

cd /work2/noaa/stmp/rmahajan/gwWork/debug

source workflow/gw_setup.sh

export HPC_ACCOUNT="da-cpu"
RUNTESTS="/work2/noaa/stmp/rmahajan/scratch/RUNTESTS"
yaml="/work2/noaa/nems/cash/workflow_test/RUN/SFS_options.yaml"
yaml="/work2/noaa/stmp/rmahajan/gwWork/debug/ci/cases/yamls/gefs_ci_defaults.yaml"

./workflow/setup_expt.py gefs forecast-only --idate 1994050100 --edate 1994050100 --pslot SFS --app S2S --resdetatmos 96 --resensatmos 96 --resdetocean 1 --start cold --gfs_cyc 1 --nens 10 --comroot "$RUNTESTS/COMROOT" --expdir "$RUNTESTS/EXPDIR" --yaml $yaml

./workflow/setup_xml.py $RUNTESTS/EXPDIR/SFS

Executing the above yields:

Running "module reset". Resetting modules to system default. The following $MODULEPATH directories have been removed: /work2/noaa/stmp/rmahajan/gwWork/debug/modulefiles
EDITED:  /work2/noaa/stmp/rmahajan/scratch/RUNTESTS/EXPDIR/SFS/config.base as per user input.
EDITED:  /work2/noaa/stmp/rmahajan/scratch/RUNTESTS/EXPDIR/SFS/config.stage_ic as per user input.
****************************************************************************************************
EXPDIR: /work2/noaa/stmp/rmahajan/scratch/RUNTESTS/EXPDIR/SFS
ROTDIR: /work2/noaa/stmp/rmahajan/scratch/RUNTESTS/COMROOT/SFS
****************************************************************************************************
Finalizing initialize
sourcing config.stage_ic
sourcing config.fcst
sourcing config.atmos_products
sourcing config.arch
sourcing config.cleanup
sourcing config.efcs
sourcing config.atmos_ensstat
sourcing config.oceanice_products

with an SFS.xml at /work2/noaa/stmp/rmahajan/scratch/RUNTESTS/EXPDIR/SFS/SFS.xml.

[aerorahul]

@benjamin-cash
Please share your SFS_options.yaml

[benjamin-cash]

Hi @aerorahul - Thanks for the test! I used your yaml file and it ran without issue, but threw the same error when I switched back. I think I have changed the permissions chain so that you can see the file now.
@NeilBarton-NOAA - tagging you in since this is the yaml file from the C96mx100 SFS_baseline and it seems like it may need some changes.

[aerorahul]

I'll take a look after Hercules returns from PM (9/25)

[aerorahul]

@benjamin-cash
There are unresolved templates in the yaml you had provided.
The best way to provide a yaml, is to include the defaults and then over-ride with user specific options.

Here is an edit of your yaml that works:

defaults:
  !INC {{ HOMEgfs }}/parm/config/gefs/yaml/defaults.yaml
base:
  DO_JEDIATMVAR: "NO"
  DO_JEDIATMENS: "NO"
  DO_JEDIOCNVAR: "NO"
  DO_JEDISNOWDA: "NO"
  DO_MERGENSST: "NO"
  FHMAX_GFS: 2976
  FHMAX_HF_GFS: 0
  KEEPDATA: "NO"
  REPLAY_ICS: "YES"
  DO_EXTRACTVARS: "NO"
  FLTFILEGFS: "postxconfig-NT-SFS.txt"
  FLTFILEGFSF00: "postxconfig-NT-SFS.txt"
  ACCOUNT: {{ 'HPC_ACCOUNT' | getenv }}
  HPSS_PROJECT: "emc-marine"
  BASE_CPLIC: {{ 'TOPICDIR' | getenv }}/REPLAY_ICs/C96mx100
fcst:
  TYPE: "hydro"
  MONO: "mono"

I just added the section defaults to your yaml.

[benjamin-cash]

Many thanks @aerorahul :). I'll close this now.