Oifs48r1 standalone

[JanStreffing]

Compiles and runs a one day long TCO95L91 standalone OpenIFS 48r1 simulation starting at 19900101. Caveats: For me! On levante! It's not beautiful.. Feedback on style welcome.

[joakimkjellsson]

Nice work! I think this will be a good guide for installing on other platforms, e.g. HLRN etc.
One question: Is there a reason you use GNU and OpenMPI, and not Intel? Since we compile XIOS, NEMO etc with Intel it would be preferrable to know we can run 48r1 also with Intel.

[joakimkjellsson]

Very nice work! I have no immediate plans to use this on HLRN soon, but it seems pretty straightforward to follow @JanStreffing work.

[JanStreffing]

Intel did not work on either juwels or levante. It compiles but runtime crashes with some library relinking error. Nothing fundamental I think, but I though it was not prio 1 to solve at that stage. I ended up copying the environments from DestinE project where they use GNU on levante. They also have the same error when they try intel. I agree, life for the coupled setup will be easier once intel works.

I tried to clean up a bit. Right now, I am replacing the juwels compiler_mpi for all versions. That's not good. I want to do that only for 48r1. I tried:

choose_version:
        48r1:
                mip: none
                o3_scheme: none
                cloudfixer: none
                massfixer: 3
                tl_o3_data_dir: "${pool_dir}/../climate.v020/"
                add_namelist_changes:
                        fort.4:
                                NAMRES:
                                        NFRRES: ${nfrres}
                                        NRESTS${nrest_ind}: ${nfrests}
                                NAMPAR0:
                                        NPROC: ${oifs.nproc}
                                NAMRIP:
                                        TSTEP: ${oifs.time_step}
                                        CSTOP: "t${oifs.next_step}"
                                NAMARG:
                                        CNMEXP: "${oifs.input_expid}"
                                NAMPPC:
                                        LRSACC: ".true." # reset accumulation of fluxes
                environment_changes:
                        add_export_vars:
                                OIFS_CYCLE: 48r1
                                OIFS_HOME: ${model_dir}
                                OIFS_LOGFILE: "$OIFS_HOME/oifs_test_log.txt"
                                OIFS_DATA_DIR: /work/ab0246/a270092/input/oifs-48r1/
                                DATA: "${general.work_dir}"
                                LD_LIBRARY_PATH: '$LD_LIBRARY_PATH:${model_dir}/build/lib'
                                GRIB_SAMPLES_PATH: '${model_dir}/build/share/eccodes/ifs_samples/grib1_mlgrib2'
                        # TODO: This can go into further reading oifs48.env.yaml, as soon as further reading works for esm_master
                        choose_computer.name:
                                juwels:
                                        compiler_mpi: gnu2023_ompi2023
                                        add_export_vars:
                                                OIFS_DATA_DIR: "/p/project/chhb19/streffing1/input/oifs-48r1/"
                                levante:
                                        compiler_mpi: gnu11_ompi2022
                                        iolibraries: system_gnu_libs
                                        add_export_vars:
                                                OIFS_DATA_DIR: "/work/ab0246/a270092/input/oifs-48r1/"

But I just get the default from levante.yaml. Another case of early resolve issue?

[JanStreffing]

the latest commit 0e47c09 breaks the selection of iolibraries: system_gnu_libs for levante. We need to get some form of this to work, because it's not okay to just change the iolibraries variable for all oifs versions.

[JanStreffing]

I changed the default behavior here, not by accident. I think we want testability and reproducibility from the standalone case. Perturbation experiments can be made, but are not the default.

[JanStreffing]

Changes reported by CI tests look ok to me. Ready for review

[JanStreffing]

Strange, I though we already solved the issue of further_reading not being evaluated by adding it to the early vars. In fact I'm positive I ran a test with that. Yet now I get the same behaviour again of not oifs48.env.yaml evaluation..

[mandresm]

I merge the polished fix for the further_reading that passed all the tests.

[JanStreffing]

@mandresm:
can you check what the runtime differences are for https://github.com/esm-tools/esm_tools/actions/runs/6826452745/job/18566413063?pr=1091 ?
Do they look ok, or is there something that needs to be changed still?

[mandresm]

Hi @JanStreffing,

Nice PR! The append_daily_icml_file is doing quite some nice magic. I think we hitted this wall once in DestinE, I guess you know about that. I hope this is now solved there too.

Other than that just some suggestions. I'll be adding a test for standalone OpenIFS-48r1 tomorrow.

[mandresm]

I'm having an issue during compilation:

[ 18%] Linking C shared library ../../lib/libeccodes.so
[ 18%] Built target eccodes
make: *** [Makefile:166: all] Error 2
ERROR: Command /work/ab0246/a270152/model_codes/oifs-48r1/build/build.sh --without-configure failed in directory                                                                                                   
/work/ab0246/a270152/model_codes/oifs-48r1

Time elapsed for failed build: 00:00:12

[JanStreffing]

I'm having an issue during compilation:

[ 18%] Linking C shared library ../../lib/libeccodes.so
[ 18%] Built target eccodes
make: *** [Makefile:166: all] Error 2
ERROR: Command /work/ab0246/a270152/model_codes/oifs-48r1/build/build.sh --without-configure failed in directory                                                                                                   
/work/ab0246/a270152/model_codes/oifs-48r1

Time elapsed for failed build: 00:00:12

Not an error I remember seeing. We did a good job on making sure no one can access OpenIFS code. So I can't look into your folder at your logfiles. :)

[mandresm]

#approve-changes

[mandresm]

#bump

[mandresm]

#bump #minor

[mandresm]

#bump #minor