Variables in default_vars.sh for different machines (gaea, gaeaC5)?

[natalie-perlin]

Are there any descriptions of variables specified in default_vars.sh , and how do they need to be set up depending on the machines?

As I understand, TPN is "tasks per node". For Gaea, for example, current setting is TPN=24.

Current:

elif [[ $MACHINE_ID = gaea ]]; then

  TPN=24 

  INPES_dflt=3 ; JNPES_dflt=8
  INPES_thrd=3 ; JNPES_thrd=4
  INPES_c384=6 ; JNPES_c384=8  ; THRD_c384=1
  INPES_c768=8 ; JNPES_c768=16 ; THRD_c768=2

  THRD_cpl_atmw_gdas=3
  INPES_cpl_atmw_gdas=6; JNPES_cpl_atmw_gdas=8; WPG_cpl_atmw_gdas=24
  WAV_tasks_atmw_gdas=264

Gaea C4, has 36 cpus. Could the TPN be set to 36?
What things need to be considered for adjusting other variables?

Gaea C5 has 128 cores/node. Does TPN needs to be set to 128? How other variables need to be adjusted?

[DeniseWorthen]

The rt_utils.sh contains the routine to set the actual PE bounds needed within each test using the function compute_petbounds_and_tasks. It acccounts for the TPN defined for any platform.

In default_vars we set wcoss2's TPN to 128. I think you'd do the same for C5. It should generally be all you need. You might even be able to do (assuming we have a platform gaea_c5):

if [[ $MACHINE_ID = wcoss2 || $MACHINE_ID = acorn || $MACHINE_ID = gaea_c5 ]] ; then

  TPN=128

[natalie-perlin]

@DeniseWorthen - thank you, I'll use this setup for Gaea5!

My question was also general, it would be good to know and give suggestions to users in documentation or/and in the code comments (ideally, both) how these bounds are set, and what these variables represent. Is there anybody in particular who help to develop or determine these values?

[DeniseWorthen]

@natalie-perlin Certainly more documentation is always better. Some of this information had been on the wiki for ufs, but then starting trying to update the user guide instead and now someone outside of EIB (gillian) is working on updating that documentation. What we had in the wiki is here.

In terms of how many PEs each component gets, those are set somewhat arbitrarily for the ocean and wave. Generally the idea is to give them as few as possible and still get the RT to run in a reasonable time. For ice and atm, you need to consider the decomposition or layout and (for the atm) the write grid component.

[natalie-perlin]

@DeniseWorthen - thanks a lot for the link to https://github.com/ufs-community/ufs-weather-model/wiki/Advanced-Topics-for-developers , good to know what's available so far.
As to Gaea (C4), changing TPN=36 worked without additional modifications. GaeaC5 is set similar to Wcoss2. Closing the issue for now, but hoping more information will be included in comments and/or documentation.