Add active atmosphere applications with data ocn and data ice components (ATM_DS2S and ATM_DS2S-PCICE) and regression tests (atm_ds2s_docn_dice and atm_ds2s_docn_pcice)

[NickSzapiro-NOAA]

Commit Queue Requirements:

• Fill out all sections of this template.
• All sub component pull requests have been reviewed by their code managers.
• Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
• Commit 'test_changes.list' from previous step

---

Description:

This PR provides two initial configurations to run "atmosphere-only" experiments with external, varying ocean and sea ice boundary conditions within the coupled UFS framework. This may be useful for 1) sensitivity experiments to isolate coupling feedbacks, 2) spin-up the atmospheric component, or 3) computationally cheaper atmosphere-focused simulations.

For ATM_DS2S exercised by the atm_ds2s_docn_dice regression test, ocean and ice variables are input via CDEPS-->CMEPS-->FV3. Borrowing the "CPLHIST" datamode in CESM, the data components provide ocean and sea ice fields output from a separate coupled model run. In this preceding coupled model run, coupling variables may be output via component diagnostics or CMEPS mediator auxiliary or history files at desired frequency. For this, these are added to UFS model infrastructure:

• dice is enabled in CDEPS for UFS
• dice cplhist datamode is added to CDEPS
• ATM_DS2S application to configure FV3+docn+dice

For ATM_DS2S-PCICE exercised by the atm_ds2s_docn_pcice regression test, prescribed ice CICE mode is used for "data ice" instead. CDEPS-->CICE inputs sea ice concentration from file and CICE prescribes/calculates variables for coupling with the atmosphere CICE-->CMEPS-->FV3. For this, these are added to UFS model infrastructure:

• enabling CICE in UFS to call shared CDEPS code and use ice prescribed mode
• ATM_DS2S-PCICE application to configure FV3+docn+pcice

The two applications are complementary. For ATM_DS2S-PCICE, sea surface temperature and sea ice concentration are input and other coupling variables are prescribed/calculated. While the cplhist datamode avoids arbitrarily prescribing several sea ice fields as in dice_datamode_ssmi_mod.F90 or prescribed ice CICE, preceding model output is needed over the dates of interest. Note that, in the future, (1) multiple CDEPS streams can be combined to more observationally constrain simulations and (2) additional fields can be input to PCICE (e.g., sea ice thickness).

Both options permit developers to run ATM simulations within the coupled model framework with physical coupled fields. Example hourly timeseries of ice-->atm fields from 5d simulations illustrate these reasonable couplings:

at ~ -76N/176E, 79N/34E, and 88N/11E for (blue) cpld_control_gfsv17, (green) atm_ds2s_docn_dice, and (orange) atm_ds2s_docn_pcice. Note that gfsv17 and cplhist overlay each other. Sensitivities to initial sea ice temperature and snow thickness are evident for at least the first couple days wrt prescribed ice CICE.

New input data is required for the prescribed ice regression tests for input via CDEPS, available on Hera:
For pcice: /scratch1/NCEPDEV/nems/Nick.Szapiro/tasks/input_data/ERA5/*
The cplhist regression test is now a dependency of mediator history output from a coupled S2S test.

To use CMEPS to write auxiliary/histaux files for the cplhist mode, add the following to the MED_attributes of the ufs.configure file of the preceding coupled run. For smaller file sizes, one can use a colon-delimited listed of only the desired variables (rather than all) or less frequent history_n/history_option.

histaux_ice2med_file1_auxname = ice.ncpl.inst
histaux_ice2med_file1_doavg = .false.
histaux_ice2med_file1_enabled = .true.
histaux_ice2med_file1_flds = all
histaux_ice2med_file1_history_n = 1
histaux_ice2med_file1_history_option = nsteps
histaux_ice2med_file1_ntperfile = 1
histaux_ocn2med_file1_auxname = ocn.ncpl.inst
histaux_ocn2med_file1_doavg = .false.
histaux_ocn2med_file1_enabled = .true.
histaux_ocn2med_file1_flds = all
histaux_ocn2med_file1_history_n = 1
histaux_ocn2med_file1_history_option = nsteps
histaux_ocn2med_file1_ntperfile = 1

Commit Message:

* UFSWM - Add ATM_DS2S and ATM_DS2S-PCICE applications by adding CDEPS dice and enabling prescribed ice CICE. Add corresponding atm_ds2s_docn_dice and atm_ds2s_docn_dice-pcice regression tests. New gnv1_nested regression test with Water in the West domain at C96 resolution using templates compatible with global-workflow
  * CDEPS - Add dice cplhist datamode
  * CICE - Enable ice prescribed CICE mode

Priority:

• Normal

Git Tracking

UFSWM:

• Closes Create Global ATM-DOCN-Prescribed ICE configuration #1816
• Closes Create cplhist mode for dice NOAA-EMC/CDEPS#61

Sub component Pull Requests:

• CDEPS: Create cplhist mode for dice NOAA-EMC/CDEPS#62
• CICE: Enable prescribed ice for UFS NOAA-EMC/CICE#80

UFSWM Blocking Dependencies:

• None

---

Changes

Regression Test Changes (Please commit test_changes.list):

• PR Adds New Tests: atm_ds2s_docn_dice_intel atm_ds2s_docn_pcice_intel
• Baseline Changes: gnv1_nested_intel

Input data Changes:

• New input data.

Library Changes/Upgrades:

• No Updates

---

Testing Log:

• RDHPCS
  • Hera
  • Orion
  • Hercules
  • Jet
  • Gaea
  • Derecho
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
• opnReqTest (complete task if unnecessary)

[NickSzapiro-NOAA]

It would be nice to generate the input data and baselines with the merged UFS version. New input files currently referenced in tests/fv3_conf/cplhist_cdeps_docn_dice.IN can then be arranged on all platforms.

A preliminary RT case is available on Hera at /scratch1/NCEPDEV/stmp2/Nick.Szapiro/FV3_RT/rt_143382/atm_ds2s_docn_dice_intel

[NickSzapiro-NOAA]

Operational requirement tests pass on Hera with intel. Log is attached
OpnReqTests_atm_ds2s_docn_dice_hera.log

[DeniseWorthen]

@NickSzapiro-NOAA Have you done any verification that this is working correctly? For example, comparing the active FV3 atm output with that generated when running data ocean + data ice?

[NickSzapiro-NOAA]

The experiments are not expected to be bit identical as configured (e.g., CDEPS interpolating linear in time between 2 input files) but near-surface and other atmospheric variables are very similar across all tiles. Below are quick plots showing (left) ATM_DS2S and (right) original S2S RT after 24 hours:

@DeniseWorthen Is there anything in particular you would like to see?

[DeniseWorthen]

So the case starts at hour 6, but you don't provide input stream until hour=7 and you're interpolating between ocean and ice fields between hour=7 and hour 6 + 1 day?

Is there a reason you're not coupling the ice data stream at the fast coupling frequency and the ocean at the slow?

[NickSzapiro-NOAA]

tests/parm/ufs.configure.s2s.IN sets the run sequence. So, (data) ice does vary at the fast coupling frequency and (data) ocean varies at slow with CDEPS updating data component values every step.

Since taxmode01=extend and tInterpAlgo01=linear are set in tests/parm/global_d{ocn,ice}.streams.IN, data component values change from initial until hour=7, to linearly intermediate until hour=6+1day, to final values after. As is, the current 2 snapshots provide time varying boundary conditions to FV3.

The frequency of inputs into the data component is entirely specified in the stream_files_d{ocn,ice} lines and easy to change. We can add more snapshots by including more (times in the) input files. I'm not sure what frequency to set for this regression test...enough to provide a diurnal cycle?

[DeniseWorthen]

I understand that the run sequence is set by the config file. But you're only time-interpolating between two snapshots for both ice and ocean, correct?

Regarding what fields to look at, it might be interesting to see actual difference plots, esp between the ATM exports when coupled to active vs static components.

[NickSzapiro-NOAA]

Yes. Using all of the ice and ocean coupler history files from the coupled run works well too...an example is at:
/scratch1/NCEPDEV/stmp2/Nick.Szapiro/FV3_RT/rt_277562/

wrt differences from data vs. active components, mapalgo is currently set differently. CDEPS is currently set as the bilinear default.

[jkbk2004]

@BrianCurtis-NOAA input file updates are ready at

/scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/DOCN_DICE_ERA5
/scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/FV3_fix_tiled/C96-nested
/scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20240501/FV3_input_data/INPUT_L127_nested

[jkbk2004]

@zach1221 @FernandoAndrade-NOAA input files are ready at RDHPCS sides but Jet. Sounds like a trouble with both file systems Jet lfs4 and lfs5. We need to double check with Jet today.

[BrianCurtis-NOAA]

The atm_ds2s_docn_dice test is faiing to run on WCOSS2
first, loading nco in the .IN file is not working on WCOSS2, it wants netcdf 4.7.x but we load 4.9.x. Once I workaround it with a case statement, it later wants to use ncrcat which is broken on WCOSS2 with:

brian.curtis@clogin09:/lfs/h2/emc/nems/noscrub/brian.curtis/git/NickSzapiro-NOAA/ufs-weather-model/tests> ncrcat
ncrcat: error while loading shared libraries: libgsl.so.25: cannot open shared object file: No such file or directory

I'd recommend disabling that test on WCOSS2, but wanted more opinions first.

[NickSzapiro-NOAA]

The atm_ds2s_docn_dice test is faiing to run on WCOSS2 first, loading nco in the .IN file is not working on WCOSS2, it wants netcdf 4.7.x but we load 4.9.x. Once I workaround it with a case statement, it later wants to use ncrcat which is broken on WCOSS2 with:

brian.curtis@clogin09:/lfs/h2/emc/nems/noscrub/brian.curtis/git/NickSzapiro-NOAA/ufs-weather-model/tests> ncrcat
ncrcat: error while loading shared libraries: libgsl.so.25: cannot open shared object file: No such file or directory

I'd recommend disabling that test on WCOSS2, but wanted more opinions first.

Thanks for digging into it, @BrianCurtis-NOAA. I'm fine disabling this test on wcoss2. We can revisit if ncrcat is a problem for global-workflow too. If there is concern about the atm_ds2s_docn_dice test itself, we can run it not as a dependency by staging ice and ocn cplhist first:
https://github.com/NickSzapiro-NOAA/ufs-weather-model/blob/dice_cplhist/tests/fv3_conf/cpld_docn_dice.IN#L9

[FernandoAndrade-NOAA]

Jet and Acorn will be skipped for this PR, I'll leave a note in the sub PRs to merge in.

[NickSzapiro-NOAA]

@FernandoAndrade-NOAA Merged in subcomponents and updated commit message

[NickSzapiro-NOAA]

Should gnv1_nested_intel be added to the test_changes.list ?

[SamuelTrahanNOAA]

Should gnv1_nested_intel be added to the test_changes.list ?

Yes. If my PR was combined with this one correctly, then that test's output should have changed.