Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotfix for RRTMGP using multiple threads #247

Merged
merged 10 commits into from
Feb 24, 2021

Conversation

dustinswales
Copy link
Contributor

This PR contains several changes to the initialization and setup of the RRTMGP radiation scheme to allow for use across multiple openMP threads.
*) Moved Interstitial RRTMGP DDTs (ty_rrtmgp_gas_optics, ty_cloud_optics) to static fields defined during initialization in module memory.
*) Add "$omp critical" statements around the calling type-bound procedures during initialization.
*) Move allocation of ty_gas_conc from Interstitial to GFS_typedefs. Initialize ty_gas_concs for all blocks during initialization.

This issue was brought to our attention by @yangfanglin and Qingfu Liu at EMC while testing RRTMGP in high-resolution cases, which require multiple threads.

The GP regression tests on hera using Intel were successful.
There are no changes to the baselines

Waiting on NCAR/ccpp-physics#568

@yangfanglin
Copy link
Collaborator

Please add Qinfu Liu as a reviewer

@Qingfu-Liu
Copy link
Collaborator

Qingfu-Liu commented Feb 23, 2021 via email

@@ -6438,6 +6437,14 @@ subroutine interstitial_create (Interstitial, IM, Model)
allocate (Interstitial%toa_src_sw (IM,Model%rrtmgp_nGptsSW))
allocate (Interstitial%toa_src_lw (IM,Model%rrtmgp_nGptsLW))
allocate (Interstitial%active_gases_array (Model%nGases))
! ty_gas_concs
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously %ncol and %nlay are fixed throughout the run.

What about gas_name, concs, conc? Do they depend on each grid, and if yes, do they need to be persistent? If not, then we should add them to the %rad_reset calls for the DDT (for clarity) - in the next PR. If they need to be persistent, they need to be moved to radtend or something like that (also in the next PR).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@climbfuji
I set the ncol and nlay here once in the DDT, type-bound procedures rely on it later.
gas_concentrations(ty_gas_concs) should be added to rad_reset. I will make sure to include this in my next PR.

@DusanJovic-NOAA DusanJovic-NOAA merged commit 4908898 into NOAA-EMC:develop Feb 24, 2021
@dustinswales dustinswales deleted the hotfix_GPthreading branch March 4, 2021 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants