sync up develop branch #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…mat=2 (#1287) TYPE: bug fix KEYWORDS: WRFDA, alpha_hydrometeors, ep_format=2 SOURCE: Jamie Bresch (NCAR) DESCRIPTION OF CHANGES: Problem: Only rootproc reads in the data, but the broadcasting from rootproc was missing for qice, qsnow and qgraup. Non-root processors will have qrain values for qice, qsnow and qgraup. Solution: add call wrf_dm_bcast_real(temp3d, ijk) LIST OF MODIFIED FILES: M var/da/da_setup_structures/da_setup_flow_predictors_ep_format2.inc TESTS CONDUCTED: RELEASE NOTE: Big fix for WRFDA incorrect ensemble perturbation values for qice, qsnow and qgraup when alpha_hydrometeors=true and ep_format=2.
TYPE: bug fix KEYWORDS: WRFDA, use_obs_errfac, max_ext_its > 1 SOURCE: Jamie Bresch (NCAR) DESCRIPTION OF CHANGES: Problem: CWB reported seeing obs errors keep decreasing with each outerloop. Solution: Add a check for outerloop number, so that when use_obs_errfac=true and errfac.dat exists in the working directory, observation error factors are applied only at the first outerloop. LIST OF MODIFIED FILES: M var/da/da_minimisation/da_get_innov_vector.inc TESTS CONDUCTED: RELEASE NOTE: Bug fix for WRFDA's incorrectly applying errfac.dat at each outerloop when use_obs_errfac=true and max_ext_its > 1.
TYPE: bug fix KEYWORDS: chem, drydep, wesely SOURCE: Lukas Pilz (Heidelberg University) DESCRIPTION OF CHANGES: Problem: A namelist option incompatibility between chem_opt (16) and gas_drydep_opt (1) leads to trying to access a variable that is not available in memory. This leads to unpredictable behaviors. When choosing a tracer-only chem_opt, numgas is 0. If gas_drydep_opt is 1 (the default), in the Wesely scheme initialization (dep_init), the field `dvj` is initialized with size numgas (here 0). Accessing the variable `dvj` is an error. Solution: A fatal error was added for when the Wesely scheme initialization is called with numgas = 0. This fix is in `chem/chemics_init.F` and not `share/module_check_a_mundo.F` because checking the namelist options would require some tricky-to-maintain hardcoding. LIST OF MODIFIED FILES: chem/chemics_init.F TESTS CONDUCTED: 1. Run with chem_opt = 16 and gas_drydep_opt = 1 fails appropriately 2. Run with chem_opt = 16 and gas_drydep_opt = 0 runs as it should 3. The jenkins testing is OK. RELEASE NOTE: For WRF-Chem, a namelist option incompatibility between chem_opt (16) and gas_drydep_opt (1) leads to trying to access a variable that is not available in memory. This leads to unpredictable behaviors. A fatal error was added for when the Wesely scheme initialization is called with numgas = 0.
TYPE: bug fix KEYWORDS: Deng shcu, resolved hydrometeors, non-active updrafts, precipitation units, variable initialization, limit NT SOURCE: Pedro A. Jimenez and Tim Juliano (NCAR/RAL), Eric Grimit and Mattew Wiley (Vaisala), and Xin-Zhong Liang (University of Maryland) DESCRIPTION OF CHANGES: Problem: The resolved hydrometeors are added to the unresolved hydrometeors inside Deng's parameterization. The radiation driver adds the resolved hydrometeors to the unresolved ones causing to double counting the effects of the resolved hydrometeors. The unresolved clouds are not properly removed when the updraft is not active Incorrect precipitation units Incorrect initialization of variable radiusc. The variable is initialized inside an #if construct for the 1D model If dt < 18 s NT is larger than 100 causing the code to look for values outside of array bounds for ainckfsa variable Solution: Do not add the resolved hydrometeors to the unresolved ones inside Deng's parameterization Set cloud fraction and the mixing ratio of the unresolved hydrometeors to 0 if the updraft is not active. Remove the conversion factor that was leading to the incorrect units. Initialize radiusc outside of the #if construct so radiusc is always initialized. Limit the maximum value of DT to 100 The following figure illustrates the impact of corrections 1 and 2. The time series of the mixing ratio now properly goes to 0 if the updraft is not active. image LIST OF MODIFIED FILES: M phys/module_shcu_deng.F TESTS CONDUCTED: We have run simulations with and without the fixes. The figure above illustrates that the clouds are now adequately removed if there are not active updrafts. Jenkins tests are all passing RELEASE NOTE: Several fixes introduced to the Deng's shallow cumulus parameterization: 1) Do not add the resolved hydrometeors to the unresolved ones to avoid double counting, 2) remove clouds if the updraft is not active, 3) correct conversion factor for precipitation, 4) correct initialization of radiusc variable, and 5) limit variable NT to avoid array out of bounds.
TYPE: bug fix KEYWORDS: ideal case, initialization SOURCE: Tim Raupach (UNSW CCRC) DESCRIPTION OF CHANGES: Problem: In ideal case initialization routines, incorrect indices were used in the calculation of ph_1 after a perturbation bubble was added. Impact: The calculation of perturbation geopotential ph_1 (perturbation geopotential) and Z_BASE (idealized base state height) were affected. 1. If hybrid coordinate option was selected, (hybrid_opt = 2), the entire ph_1 initialization is incorrect due to the offset in the indexing of c1h and c2h (half levels vs full levels). 2. If hybrid_opt = 0, since the 1d variables are initialized kms:kme, there is no impact. Solution: The vertical indices were corrected in the calculations of ph_1 in the idealized initialization routine. ISSUE: LIST OF MODIFIED FILES: M dyn_em/module_initialize_ideal.F TESTS CONDUCTED: 1. The jenkins testing is all pass. RELEASE NOTE: In the ideal case initialization routines, incorrect indices were used in the calculation of ph_1 after a perturbation bubble was added. For the idealized cases that do not use the hybrid vertical coordinate by default, there is no impact at all. The ideal cases with a hill had the correct hybrid formulation.
Co-authored-by: Arianna Valmassoi <valmass@cheyenne1.cheyenne.ucar.edu>
FARMS correctly detects unresolved clouds TYPE: bug fix KEYWORDS: FARMS, effective radius of unresolved hydrometeors, cloud fraction SOURCE: Ju-Hye Kim and Pedro A. Jimenez (NCAR/RAL), and Jimy Dudhia (NCAR/MMM) DESCRIPTION OF CHANGES: Problem: The previous version of FARMS did not correctly account for the radiative effects of unresolved clouds, because the FARMS calculates the surface irradiances using both cloud water contents and cloud effective radii. In previous version, cloud water contents came from both resolved and unresolved clouds, on the other hand, effective radii came only from resolved clouds. Solution: The climatological effective radii are assigned to unresolved clouds of liquid, ice and snow species, and are specified as the effective radii of clouds in microphysics schemes (e.g. mp=2) that do not provide any effective radii. The fix is also valid for Thompson MPs, WSMMPs, and WDMMPs by adjusting the background effective radii of these schemes. In addition to this, we improved calculations of surface irradiance by adding cloud fraction to the FARMS scheme. ISSUE: N/A LIST OF MODIFIED FILES: M phys/module_mp_thompson.F M phys/module_mp_wdm5.F M phys/module_mp_wdm6.F M phys/module_mp_wdm7.F M phys/module_mp_wsm3.F M phys/module_mp_wsm5.F M phys/module_mp_wsm6.F M phys/module_mp_wsm7.F M phys/module_physics_init.F M phys/module_ra_farms.F M phys/module_radiation_driver.F TESTS CONDUCTED: Do mods fix problem? How can that be demonstrated, and was that test conducted? Yes. The downward short wave flux (GHI) will increase in the cloud region with unresolved clouds by reduced scatterings due to increased cloud size. This was tested in the WRF-Solar model. Are the Jenkins tests all passing? We did't do the Jenkins tests. RELEASE NOTE: Bug fix to account for the effective radius of the unresolved hydrometeors by FARMS.
TYPE: bug fix KEYWORDS: mpi, real, bug SOURCE: Marc Honnorat (EXWEXs) DESCRIPTION OF CHANGES: Problem: When running real.exe on multiple processes with MPI, one or more process occasionally crashes in setup_physics_suite (in share/module_check_a_mundo.F). This has been linked to wrf_dm_initialize non-blocking MPI. The real.exe occasionally crashes in setup_physics_suite (in share/module_check_a_mundo.F#L2640) because the latter uses model_config_rec % physics_suite, which on some machines is not initialized. The behavior is as if the broadcast of model_config_rec performed just before in main/real_em.F#L124 had not been received by all processes. I had never seen this bug before, it has only happened on one machine (an Intel-based cluster using Intel-MPI and ifort). The current fix makes sure that all processes are well synced before proceeding with setup_physics_suite. It solves the issue on my machine. Since this is immediately after reading in the namelist, no performance issues are expected as this read and broadcast of the namelist occurs only once per real / WRF / ndown run. Solution: An MPI barrier is added at the end of wrf_dm_initialize to force all of the processes to be synchronized before checking the namelist consistency. This is a simplification of PR #1268, which had extra white space. ISSUE: Fixes #1267 LIST OF MODIFIED FILES: M external/RSL_LITE/module_dm.F TESTS CONDUCTED: 1. On the only machine were I have seen the bug occur, this change fixes the problem. No other test was conducted since I couldn't reproduce the bug on another setup. 2. Jenkins testing is all PASS. RELEASE NOTES: When running real.exe on multiple processes with MPI, one or more processes occasionally crash in setup_physics_suite (in share/module_check_a_mundo.F). This has been traced to the fact that wrf_dm_initialize is non-blocking from an MPI point of view. The problem is intermittent and has only happened on one machine (an Intel-based cluster using Intel-MPI and ifort). An MPI barrier has been added at the end of wrf_dm_initialize to force all processes to be synchronized before checking namelist consistency.
TYPE: bug fix KEYWORDS: HALO, h_mom_adv_order SOURCE: Ted Mansell (NOAA/NSSL) DESCRIPTION OF CHANGES: Problem: The halo exchange (used during distributed memory communication) had inconsistent use of `h_mom_adv_order`. There are locations when the namelist option to check should have `h_sca_adv_order`, and even a couple cases where the code should have tested both. Consequently, the MPI results did not match for different number of patches or processor counts if `h_mom_adv_order=3` and `h_sca_adv_order=5`. This problem only occurs when the advection order differs between the momentum and scalar variables. Solution: These changes (using the correct variable to test, or using both `h_mom_adv_order` and `h_sca_adv_order`) fix the problem of non-reproducibility on different core counts (where non-reproducibility means "wrong answers"). Also, a few of the halo exchanges in solve_em need to check both the momentum and scalar flags, since both types are communicated in the subroutine call. LIST OF MODIFIED FILES: dyn_em/module_first_rk_step_part1.F dyn_em/module_first_rk_step_part2.F dyn_em/solve_em.F TESTS CONDUCTED: 1. Jenkins testing is all PASS Tested ideal case (em_quarter_ss) on 4 vs 1 processes, with the following advection orders: - h_mom_adv_order=3 - h_sca_adv_order=5 2. Without changes, diffwrf found differences. 3. After changes the wrfout files match exactly. (Also tested for h_mom_adv_order=5, h_sca_adv_order=3) RELEASE NOTES: The halo exchange (used during distributed memory communication) had inconsistent use of h_mom_adv_order. There are locations in the ARW dynamics solver when the namelist option to check should have been h_sca_adv_order, and even a couple cases where the IF tests should have tested both options (both h_mom_adv_order and h_sca_adv_order). Consequently, the model results did not match bit-for-bit for differing numbers processors (when the advection order differed between the momentum and scalar variables). These IF-test changes (using the correct namelist option to compare, or possibly using both the h_mom_adv_order and h_sca_adv_order options) fix the problem of non-reproducibility due to the advection order. This problem only occurs when the advection order differs between the momentum and scalar variables.
TYPE: bug fix
KEYWORDS: gfortran/10
SOURCE: Srikanth Yalavarthi (Marvell Semiconductor)
DESCRIPTION OF CHANGES:
Problem:
Due to argument inconsistencies, the WRF code will not compile with gfortran/10.
Here are some examples of incorrect Fortran argument usages:
```
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
wrf_io.f:7605:71:
7605 | ,i1,i2,j1,j2,k1,k2 )
| 1
```
```
Error: Type mismatch in argument ‘field’ at (1); passed INTEGER(4) to REAL(8)
wrf_io.f:7744:49:
2538 | stat = NF_GET_ATT_INT (DH%NCID,NF_GLOBAL,Element,Buffer)
| 2
......
7744 | stat = NF_GET_ATT_INT(NCID,VarID,'FieldType',FType)
| 1
```
```
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-3)
field_routines.f:158:52:
110 | stat = NF_GET_VARA_INT(NCID,VarID,VStart,VCount,Data)
| 2
......
158 | stat = NF_GET_VARA_INT(NCID,VarID,VStart,VCount,Buffer)
| 1
```
```
Error: Rank mismatch in argument ‘fileindex’ at (1) (scalar and rank-1)
io_grib1.f90:685:27:
685 | CALL gr1_fill_eta_levels(fileinfo(DataHandle)%fileindex(:), FileFd(DataHandle), &
| 1
```
```
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(4)/REAL(4)).
module_internal_header_util.f:1592:39:
1592 | DataHandle, Data, Count, code )
| 1
......
1654 | DataHandle, Data, Count, code )
| 2
```
```
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
module_io_int_read.f:129:39:
129 | call mpi_file_read_at(ifd, offset, tmp, 1, &
| 1
......
573 | call mpi_file_read_at(ifd, offset, tmp, num, &
| 2
```
```
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(8)/COMPLEX(8)).
zmfm1b.f90:74:38:
72 | call zmf3kb ( lot, ido, l1, na, c, jump, inc, ch, 1, lot, wa(iw) )
| 2
73 | else if ( nbr == 4 ) then
74 | call zmf3kb ( lot, ido, l1, na, ch, 1, lot, c, jump, inc, wa(iw) )
| 1
```
The errors in the WRF infrastructure code can be eventually updated. However, the code from the FFTPACK library
will be left AS-IS. We are going to initially address all of the problems in a temporary fashion.
Temporary solution:
A compiler flag was added for the gfortran compiler to allow mismatched arguments, which enables the WRF system
to build. The new flag is only for versions of gfortran 10 and later. This change was introduced in the configure script,
which modifies all gfortran stanzas.
Permanent fix:
A complete fix requires changes in the Fortran sources for BOZ declarations and to fix mismatches in subroutine
arguments.
ISSUE:
Fixes #1250
LIST OF MODIFIED FILES:
M Makefile
M arch/configure.defaults
M configure
TESTS CONDUCTED:
1. Successfully built the master branch with GCC-10 on RPi4 Ubuntu-20.04 machine
2. Logic tested, where only GNU 10 and above are impacted.
3. Other compilers are not affected.
4. Jenkins testing is OK.
5. The DA variants have been tested also: I could build WRF / WRFPLUS / 3DVAR / 4DVAR
```
[00:01] [syalavarthi@sparrow WRF]$./configure wrfda && ./compile all_wrfvar && ls var/build/*.exe
var/build/da_advance_time.exe var/build/da_tune_obs_hollingsworth1.exe var/build/gen_be_addmean.exe var/build/gen_be_diags.exe var/build/gen_be_hist.exe var/build/gen_be_stage2_1dvar.exe var/build/gen_mbe_stage2.exe
var/build/da_bias_airmass.exe var/build/da_tune_obs_hollingsworth2.exe var/build/gen_be_cov2d.exe var/build/gen_be_diags_read.exe var/build/gen_be_stage0_gsi.exe var/build/gen_be_stage2_gsi.exe
var/build/da_bias_scan.exe var/build/da_update_bc.exe var/build/gen_be_cov2d3d_contrib.exe var/build/gen_be_ensmean.exe var/build/gen_be_stage0_wrf.exe var/build/gen_be_stage2a.exe
var/build/da_bias_sele.exe var/build/da_update_bc_ad.exe var/build/gen_be_cov3d.exe var/build/gen_be_ensrf.exe var/build/gen_be_stage1.exe var/build/gen_be_stage3.exe
var/build/da_bias_verif.exe var/build/da_verif_grid.exe var/build/gen_be_cov3d2d_contrib.exe var/build/gen_be_ep1.exe var/build/gen_be_stage1_1dvar.exe var/build/gen_be_stage4_global.exe
var/build/da_rad_diags.exe var/build/da_verif_obs.exe var/build/gen_be_cov3d3d_bin3d_contrib.exe var/build/gen_be_ep2.exe var/build/gen_be_stage1_gsi.exe var/build/gen_be_stage4_regional.exe
var/build/da_tune_obs_desroziers.exe var/build/da_wrfvar.exe var/build/gen_be_cov3d3d_contrib.exe var/build/gen_be_etkf.exe var/build/gen_be_stage2.exe var/build/gen_be_vertloc.exe
[00:10] [syalavarthi@sparrow WRF]$
[06:40] [syalavarthi@sparrow WRFPLUS]$./configure wrfplus && ./compile wrfplus && ls main/*.exe
main/wrfplus.exe
[07:16] [syalavarthi@sparrow WRF]$./configure 4dvar && ./compile all_wrfvar && ls var/build/*.exe
var/build/da_advance_time.exe var/build/da_tune_obs_hollingsworth1.exe var/build/gen_be_addmean.exe var/build/gen_be_diags.exe var/build/gen_be_hist.exe var/build/gen_be_stage2_1dvar.exe var/build/gen_mbe_stage2.exe
var/build/da_bias_airmass.exe var/build/da_tune_obs_hollingsworth2.exe var/build/gen_be_cov2d.exe var/build/gen_be_diags_read.exe var/build/gen_be_stage0_gsi.exe var/build/gen_be_stage2_gsi.exe
var/build/da_bias_scan.exe var/build/da_update_bc.exe var/build/gen_be_cov2d3d_contrib.exe var/build/gen_be_ensmean.exe var/build/gen_be_stage0_wrf.exe var/build/gen_be_stage2a.exe
var/build/da_bias_sele.exe var/build/da_update_bc_ad.exe var/build/gen_be_cov3d.exe var/build/gen_be_ensrf.exe var/build/gen_be_stage1.exe var/build/gen_be_stage3.exe
var/build/da_bias_verif.exe var/build/da_verif_grid.exe var/build/gen_be_cov3d2d_contrib.exe var/build/gen_be_ep1.exe var/build/gen_be_stage1_1dvar.exe var/build/gen_be_stage4_global.exe
var/build/da_rad_diags.exe var/build/da_verif_obs.exe var/build/gen_be_cov3d3d_bin3d_contrib.exe var/build/gen_be_ep2.exe var/build/gen_be_stage1_gsi.exe var/build/gen_be_stage4_regional.exe
var/build/da_tune_obs_desroziers.exe var/build/da_wrfvar.exe var/build/gen_be_cov3d3d_contrib.exe var/build/gen_be_etkf.exe var/build/gen_be_stage2.exe var/build/gen_be_vertloc.exe
```
RELEASE NOTE: Due to subroutine and function argument inconsistencies, the WRF code will not compile with gfortran/10. The fix introduces a new compiler flag for all gfortran stanzas (`-fallow-argument-mismatch -fallow-invalid-boz`).
TYPE: bug fix KEYWORDS: occasional instabilities, sfclayrev, look up tables SOURCE: Pedro A. Jimenez (NCAR/RAL) DESCRIPTION OF CHANGES: Problem: For some occasions the model turns unstable, and this was found to be originating due to the surface layer option 1, with the upper boundary values of the look up tables. Solution: Avoiding using the look up tables of the integrated similarity functions for the last tabulated value. LIST OF MODIFIED FILES: M phys/module_sf_sfclayrev.F TESTS CONDUCTED: After introducing the fix the model no longer turns unstable in the specific instances noted. The jenkins tests are all PASS. RELEASE NOTE: A minor fix (excluding use of the upper bounding value of the look-up table) was introduced to avoid occasional instabilities in the surface layer option 1.
TYPE: Enhancement KEYWORDS: Noah-MP-Crop, specific-crop, parameters, photosynthesis, stomata SOURCE: Zhe Zhang (University of Saskatchewan) DESCRIPTION OF CHANGES: The current release WRF-Crop option 1, (Liu Xing's model), uses generic-crop parameters in stomata subroutine, to calculate photosynthesis and stomata resistance, regardless of crop type. Also some parameters in the crop parameters in MPTABLE.TBL, the first and second columns for corn and soybean are not consistent with the values used in Xing's paper. This pull request fixes the parameter values for these two columns in the crop parameter section in MPTABLE.TBL, as well as adds specific-crop photosynthesis-stomata parameters in this section. LIST OF MODIFIED FILES: phys/module_sf_noahmplsm.F phys/module_sf_noahmpdrv.F run/MPTABLE.TBL TESTS CONDUCTED: Coupled WRF-Crop simulation is conducted on Cheyenne. The results seem reasonable in terms of crop LAI and biomass. Presented these results with the Noah-MP LSM group meeting with Dr. Fei Chen. Jenkins testing is all PASS. RELEASE NOTE: Incorporate specific-crop parameters for photosynthesis and stomata subroutine in Noah-MP-Crop. The current release WRF-Crop option 1, (Liu Xing's model), uses generic-crop parameters in stomata subroutine, to calculate photosynthesis and stomata resistance, regardless of crop type. Also some parameters in the crop parameters in MPTABLE.TBL, the first and second columns for corn and soybean are not consistent with the values used in Xing's paper. This update fixes the parameter values for these two columns in the crop parameter section in MPTABLE.TBL, as well as adds specific-crop photosynthesis-stomata parameters in this section. Zhang, Z., Barlage, M., Chen, F., Li, Y., Helgason, W., Xu, X., et al. (2020). Joint modeling of crop and irrigation in the Central United States using the Noah‐MP land surface model. Journal of Advances in Modeling Earth Systems. https://doi.org/10.1029/2020MS002159
TYPE: bug fix KEYWORDS: integer kind SOURCE: internal DESCRIPTION OF CHANGES: Problem: The nine calls to io_int_loc had inconsistent kinds of integer for the actual and dummy arguments. On every machine and compiler previously, the integer kinds for MPI_offset and c_int64_t were identical. On the raspberry pi (using the default apt-get openmpi and gnu fortran), these are not identical. Solution: As the integers are being shared through a C interface, that integer kind was selected. LIST OF MODIFIED FILES: modified: external/io_int/module_io_int_read.F90 TESTS CONDUCTED: 1. Without the mods, the WRF model did not build on a raspberry pi. With the mods, the code builds and runs. Here testing on a raspberry pi includes the distributed memory, so the MPI_offset variable would be in use. ``` pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 1 ./wrf.exe ; tail rsl.out.0000 starting wrf task 0 of 1 Timing for main: time 0001-01-01_00:00:24 on domain 1: 0.79946 elapsed seconds Timing for main: time 0001-01-01_00:00:36 on domain 1: 0.79981 elapsed seconds Timing for main: time 0001-01-01_00:00:48 on domain 1: 0.80066 elapsed seconds Timing for main: time 0001-01-01_00:01:00 on domain 1: 0.79946 elapsed seconds Timing for main: time 0001-01-01_00:01:12 on domain 1: 0.80392 elapsed seconds Timing for main: time 0001-01-01_00:01:24 on domain 1: 0.80236 elapsed seconds Timing for main: time 0001-01-01_00:01:36 on domain 1: 0.80006 elapsed seconds Timing for main: time 0001-01-01_00:01:48 on domain 1: 0.79996 elapsed seconds Timing for main: time 0001-01-01_00:02:00 on domain 1: 0.80133 elapsed seconds d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF ``` ``` pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 2 ./wrf.exe ; tail rsl.out.0000 starting wrf task 1 of 2 starting wrf task 0 of 2 Timing for main: time 0001-01-01_00:00:24 on domain 1: 0.53475 elapsed seconds Timing for main: time 0001-01-01_00:00:36 on domain 1: 0.53516 elapsed seconds Timing for main: time 0001-01-01_00:00:48 on domain 1: 0.53412 elapsed seconds Timing for main: time 0001-01-01_00:01:00 on domain 1: 0.53324 elapsed seconds Timing for main: time 0001-01-01_00:01:12 on domain 1: 0.53357 elapsed seconds Timing for main: time 0001-01-01_00:01:24 on domain 1: 0.53547 elapsed seconds Timing for main: time 0001-01-01_00:01:36 on domain 1: 0.53507 elapsed seconds Timing for main: time 0001-01-01_00:01:48 on domain 1: 0.53406 elapsed seconds Timing for main: time 0001-01-01_00:02:00 on domain 1: 0.53477 elapsed seconds d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF ``` ``` pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 3 ./wrf.exe ; tail rsl.out.0000 starting wrf task 0 of 3 starting wrf task 1 of 3 starting wrf task 2 of 3 Timing for main: time 0001-01-01_00:00:24 on domain 1: 0.48709 elapsed seconds Timing for main: time 0001-01-01_00:00:36 on domain 1: 0.48533 elapsed seconds Timing for main: time 0001-01-01_00:00:48 on domain 1: 0.48523 elapsed seconds Timing for main: time 0001-01-01_00:01:00 on domain 1: 0.48553 elapsed seconds Timing for main: time 0001-01-01_00:01:12 on domain 1: 0.48532 elapsed seconds Timing for main: time 0001-01-01_00:01:24 on domain 1: 0.48635 elapsed seconds Timing for main: time 0001-01-01_00:01:36 on domain 1: 0.48445 elapsed seconds Timing for main: time 0001-01-01_00:01:48 on domain 1: 0.48409 elapsed seconds Timing for main: time 0001-01-01_00:02:00 on domain 1: 0.48494 elapsed seconds d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF ``` ``` pi@raspberrypi:/media/pi/gill/WRF/test/em_quarter_ss $ mpirun -np 4 ./wrf.exe ; tail rsl.out.0000 starting wrf task 3 of 4 starting wrf task 0 of 4 starting wrf task 1 of 4 starting wrf task 2 of 4 Timing for main: time 0001-01-01_00:00:24 on domain 1: 0.65011 elapsed seconds Timing for main: time 0001-01-01_00:00:36 on domain 1: 0.64889 elapsed seconds Timing for main: time 0001-01-01_00:00:48 on domain 1: 0.64934 elapsed seconds Timing for main: time 0001-01-01_00:01:00 on domain 1: 0.64886 elapsed seconds Timing for main: time 0001-01-01_00:01:12 on domain 1: 0.65009 elapsed seconds Timing for main: time 0001-01-01_00:01:24 on domain 1: 0.64658 elapsed seconds Timing for main: time 0001-01-01_00:01:36 on domain 1: 0.64700 elapsed seconds Timing for main: time 0001-01-01_00:01:48 on domain 1: 0.64807 elapsed seconds Timing for main: time 0001-01-01_00:02:00 on domain 1: 0.64591 elapsed seconds d01 0001-01-01_00:02:00 wrf: SUCCESS COMPLETE WRF ``` 2. Jenkins testing is all PASS. RELEASE NOTES: An inconsistency in the declaration of integer kinds was discovered in the seldom-used routine that processes binary formatted data. In one routine the values were declared as an MPI type (and integer kind mpi_offset_type). That argument was passed to a routine that declared the integers as c_int64_t (which is part of the standard technique now to be used when working with C). These two integer kinds must have been the same 8-byte size for all of the x86_64 Linux machines that we have used. These integer kinds are however different for at least some gfortran / openmpi combinations on ARM processors (specifically found on a raspberry pi). There is no impact to existing users. If the code previously worked (built), this changes nothing.
Fix for high sulfate concentrations with biomass_burn_opt = 2 TYPE: bug fix KEYWORDS: chem,biomassburn,plumerise,sulfate SOURCE: Neeldip Barman (IIT Guwahati) DESCRIPTION OF CHANGES: Problem: Namelist option biomass_burn_opt = 2 when used with chem_opt=202 produces very high so4 concentration and so4 plumes reaching near the model top. This further leads to very large optical depths. Namelist option biomass_burn_opt = 2 does not take sulfate emissions as inputs in registry, but sulfate emission inputs have been defined in module_mosaic_addemiss.F Solution: Setting IF ( p_ebu_sulf .gt. 1) aem_so4 = bburn_mosaic_f(n)*ebu(i,k,j,p_ebu_sulf) inplace of aem_so4 = bburn_mosaic_f(n)*ebu(i,k,j,p_ebu_sulf) LIST OF MODIFIED FILES: module_mosaic_addemiss.F TESTS CONDUCTED: 1. chem_opt = 202 with the code modification RELEASE NOTE: sulf emissions are not defined for biomass_burn_opt = 2 (MOZART option) in the registry, but sulf emission inputs have been defined in module_mosaic_addemiss.F, which is used for any MOZART option coupled to MOSAIC
TYPE: new feature KEYWORDS: Chemical DA, Surface Observations, 3DVar, GENBE_2.0 SOURCE: Wei Sun (NCAR/MMM, CAS), Jake Liu (NCAR/MMM), Dan Chen (IUM, Beijing) DESCRIPTION OF CHANGES: 1. This PR adds the new capability for assimilating the surface chemical/aerosol observations (pm2.5, pm10, O3, NO2, SO2, CO) using 3DVar. 2. Several new namelist parameters are defined in registry.var_chem under wrfvarchem. The setting of chemicda_opt only works for MOSAIC scheme. The inflation factors of chemical background errors work for both aerosol schemes, but they are only tested under the MOSAIC scheme. &wrfvarchem chem_cv_options= 10/20: control variables are 4 gas + 15/32 GOCART/MOSAIC (4bin) aerosol variables chemicda_opt=1/2/3/4/5 # assimilate pm2.5/pm10/pm2.5+pm10/co+o3+so2+no2/6-species var_scaling12=36xmax_outer_iterations, # inflation factors of chemical background errors. len_scaling12=36xmax_outer_iterations, # inflation factors of chemical background errors. use_chemic_surfobs = .true., # read in ob_chemsfc.ascii / 3. To compile 3DVar with chemical DA capability, set the environment variable "WRF_CHEM" as 1, then ./configure wrfda; compile all_wrfvar as usual. The executable remains da_wrfvar.exe. 4. The chemical background error file be.dat needs to be generated using GENBE_2.0 on https://github.com/wrf-model/GENBE_2.0. LIST OF MODIFIED FILES: A Registry/Registry.wrfchemvar M Registry/registry.var_chem M compile M external/RSL_LITE/gen_comms.c M frame/module_domain.F M frame/module_driver_constants.F M share/mediation_integrate.F M share/output_wrf.F M tools/gen_allocs.c M var/build/da.make M var/build/depend.txt A var/da/da_chem_sfc/da_ao_stats_chem_sfc.inc A var/da/da_chem_sfc/da_calculate_grady_chem_sfc.inc A var/da/da_chem_sfc/da_check_max_iv_chem_sfc.inc A var/da/da_chem_sfc/da_chem_sfc.f90 A var/da/da_chem_sfc/da_get_innov_vector_chem_sfc.inc A var/da/da_chem_sfc/da_jo_and_grady_chem_sfc.inc A var/da/da_chem_sfc/da_jo_chem_sfc.inc A var/da/da_chem_sfc/da_oi_stats_chem_sfc.inc A var/da/da_chem_sfc/da_print_stats_chem_sfc.inc A var/da/da_chem_sfc/da_residual_chem_sfc.inc A var/da/da_chem_sfc/da_transform_xtoy_chem_sfc.inc A var/da/da_chem_sfc/da_transform_xtoy_chem_sfc_adj.inc M var/da/da_control/da_control.f90 A var/da/da_define_structures/da_allocate_observations_chem_sfc.inc A var/da/da_define_structures/da_allocate_y_chem_sfc.inc M var/da/da_define_structures/da_deallocate_background_errors.inc A var/da/da_define_structures/da_deallocate_y_chem_sfc.inc M var/da/da_define_structures/da_define_structures.f90 A var/da/da_define_structures/da_zero_xchem_type.inc M var/da/da_main/da_solve.inc M var/da/da_main/da_solve_init.inc M var/da/da_main/da_update_firstguess.inc M var/da/da_main/da_wrfvar_init2.inc M var/da/da_main/da_wrfvar_top.f90 M var/da/da_minimisation/da_amat_mul.inc M var/da/da_minimisation/da_calculate_gradj.inc M var/da/da_minimisation/da_calculate_grady.inc M var/da/da_minimisation/da_calculate_j.inc M var/da/da_minimisation/da_calculate_residual.inc M var/da/da_minimisation/da_dot_cv.inc M var/da/da_minimisation/da_get_innov_vector.inc M var/da/da_minimisation/da_get_var_diagnostics.inc M var/da/da_minimisation/da_jo_and_grady.inc M var/da/da_minimisation/da_minimisation.f90 M var/da/da_minimisation/da_minimise_cg.inc M var/da/da_minimisation/da_minimise_lz.inc M var/da/da_minimisation/da_sensitivity.inc M var/da/da_minimisation/da_transform_vtoy.inc M var/da/da_minimisation/da_transform_vtoy_adj.inc M var/da/da_minimisation/da_write_diagnostics.inc A var/da/da_obs/da_fill_obs_structures_chem_sfc.inc M var/da/da_obs/da_obs.f90 M var/da/da_obs/da_transform_xtoy.inc M var/da/da_obs/da_transform_xtoy_adj.inc A var/da/da_obs_io/da_final_write_obs_chem_sfc.inc A var/da/da_obs_io/da_final_write_obs_gas_sfc.inc M var/da/da_obs_io/da_obs_io.f90 A var/da/da_obs_io/da_read_obs_chem_sfc.inc M var/da/da_obs_io/da_read_omb_tmp.inc A var/da/da_obs_io/da_scan_obs_chem_sfc.inc A var/da/da_obs_io/da_write_obs_chem_sfc.inc M var/da/da_par_util/da_cv_to_vv.inc M var/da/da_par_util/da_par_util.f90 M var/da/da_par_util/da_vv_to_cv.inc M var/da/da_setup_structures/da_rescale_background_errors.inc M var/da/da_setup_structures/da_scale_background_errors.inc M var/da/da_setup_structures/da_setup_background_errors.inc M var/da/da_setup_structures/da_setup_be_regional.inc M var/da/da_setup_structures/da_setup_cv.inc M var/da/da_setup_structures/da_setup_firstguess.inc M var/da/da_setup_structures/da_setup_firstguess_wrf.inc M var/da/da_setup_structures/da_setup_obs_structures.inc A var/da/da_setup_structures/da_setup_obs_structures_chem_sfc.inc M var/da/da_setup_structures/da_setup_structures.f90 M var/da/da_statistics/da_analysis_stats.inc M var/da/da_statistics/da_statistics.f90 M var/da/da_test/da_check.inc A var/da/da_test/da_check_cvtovv_adjoint_chem.inc M var/da/da_test/da_check_gradient.inc A var/da/da_test/da_check_vchemtox_adjoint.inc A var/da/da_test/da_check_vtox_adjoint_chem.inc M var/da/da_test/da_test.f90 M var/da/da_transfer_model/da_transfer_model.f90 A var/da/da_transfer_model/da_transfer_wrftoxb_chem.inc M var/da/da_transfer_model/da_transfer_xatoanalysis.inc M var/da/da_transfer_model/da_transfer_xatowrf.inc A var/da/da_vtox_transforms/da_transform_vchemtox.inc A var/da/da_vtox_transforms/da_transform_vchemtox_adj.inc M var/da/da_vtox_transforms/da_transform_vtovv.inc M var/da/da_vtox_transforms/da_transform_vtovv_adj.inc M var/da/da_vtox_transforms/da_transform_vtox.inc M var/da/da_vtox_transforms/da_transform_vtox_adj.inc M var/da/da_vtox_transforms/da_vtox_transforms.f90 TESTS CONDUCTED: 1. WRFDA regression tests passed. 2. New chemical DA is tested with 6 types of surface obs and 4-bin MOSAIC aerosol scheme. RELEASE NOTE: Add new capability for assimilating surface pm2.5, pm10, O3, CO, NO2, SO2 using 3DVar. Sun, W., Liu, Z., Chen, D., Zhao, P., and Chen, M., 2020: Development and application of the WRFDA-Chem three-dimensional variational (3DVAR) system: aiming to improve air quality forecasting and diagnose model deficiencies, Atmos. Chem. Phys., 20, 9311-9329.
TYPE: bug fix KEYWORDS: FARMS, Microphysics, module_check_a_mundo.F SOURCE: Ju Hye Kim and Pedro Jimenez (NCAR/RAL) DESCRIPTION OF CHANGES: Problem: FARMS can work correctly with microphysics schemes that do not provide effective radii of clouds. But, current WRF model displays an ERROR message and stops when the 'swint_opt is 2' and 'use_mp_re is not 1'. Solution: We remove this constraint from ‘module_check_a_mundo.F’. LIST OF MODIFIED FILES: M share/module_check_a_mundo.F TESTS CONDUCTED: 1. When FARMS and a microphysics scheme that do not provide effective radii of clouds (ex: mp=2) are used together, the model does not stop anymore. 2. All jenkins tests are passing. RELEASE NOTE: Fixed an incorrect stop in the model when FARMS radiation is used with microphysics that do not provide effective cloud radii.
TYPE: bug fix KEYWORDS: saturation vapor pressure, Morrison double-moment scheme SOURCE: Hugh Morrison (NCAR) DESCRIPTION OF CHANGES: Problem: Current code gave unrealistically high saturation vapor pressures at T < -80 C. Solution: "function polysvp" in module_mp_morr_two_moment_aero.F and module_mp_morr_two_moment.F are modified to correct the problem. LIST OF MODIFIED FILES: M phys/module_mp_morr_two_moment_aero.F M phys/module_mp_morr_two_moment.F TESTS CONDUCTED: Idealized tests conduced by Morrison, the mods fix the problem. Jenkins tests have passed. RELEASE NOTE: Fix a problem with function polyqvp which gave unrealistically high saturation vapor pressures at T < -80 C.
TYPE: bug fix KEYWORDS: irrigation, time selection, day selection, bug fix SOURCE: Arianna Valmassoi (Uni-Bonn) DESCRIPTION OF CHANGES: Problem: The irrigation schemes weren't working when the date selection was across two different years: i.e. irr_end_julianday < irr_start_julianday. Same problem was found for active hours across the local 00 time: i.e. irr_start_hour+irr_num_hours>23 Solution: Fixed by adding/modifying the if statements LIST OF MODIFIED FILES: M phys/module_irrigation.F TESTS CONDUCTED: Do mods fix problem? yes Are the Jenkins tests all passing? Passed. RELEASE NOTE: Bug fixes for temporal selection that spans across different years and different local days.
…1313) TYPE: bug fix KEYWORDS: sea-ice fraction option, QSFC calculation, MYJSFC, QNSESFC SOURCE: Xin-Zhong Liang DESCRIPTION OF CHANGES: Problem: Incorrect parentheses in weighted average Solution: Fixed parentheses Effect: Test by Kevin Manning indicates small effect. ISSUE: For use when this PR closes an issue. Fixes #1297 (part) LIST OF MODIFIED FILES: M phys/module_surface_driver.F TESTS CONDUCTED: Test by Kevin Manning shows small impact. Jenkins test (TBD) RELEASE NOTE: Fix incorrect parentheses in weighted average for QSFC when sea-ice fraction is used with MYJSFC and QNSESFC surface layer physics.
#1259) TYPE: bug fix KEYWORDS: surface heat flux, use_theta_m, moist theta, non-local flux, 3D TKE SOURCE: Matthias Göbel (University of Innsbruck) DESCRIPTION OF CHANGES: The conversion of surface heat fluxes from dry to moist theta in WRF is inconsistent. This leads to an unclosed dry theta budget at the lowest grid point. The surface sensible heat flux in the routines `vertical_diffusion_2` and `vertical_diffusion_implicit` should be multiplied with <img src="https://latex.codecogs.com/svg.latex?1+\frac{R_{v}}{R_{d}}q" /> as is derived below. Using Reynold's decomposition and averaging we can show for the perturbation moist theta: <img src="https://latex.codecogs.com/svg.latex?\theta_{m}'=(\theta(1+q\alpha))'=\theta(1+q\alpha)-\overline{\theta(1+q\alpha)}=\theta-\overline{\theta}+\alpha(q\theta-\overline{q\theta})=\\=\theta'+\alpha\left[(\overline{\theta}+\theta')(\overline{q}+q')-\overline{\theta}\overline{q}-\overline{\theta'q'}\right]=\theta'(1+\alpha\overline{q})+\alpha\left[\overline{\theta}q'+\theta'q'-\overline{\theta'q'}\right]" /> with water vapor mixing ratio q and <img src="https://latex.codecogs.com/svg.latex?\alpha=\frac{R_{v}}{R_{d}}\approx1.61" /> This yields for the converted surface heat flux: <img src="https://latex.codecogs.com/svg.latex?\overline{w'\theta_{m}'}_0=(1+\alpha\overline{q}_0)\overline{w'\theta'}_0+\alpha\overline{\theta}_0\,\overline{w'q'}_0+\alpha\overline{w'\theta'q'}_0\approx(1+\alpha\overline{q}_0)\overline{w'\theta'}_0+\alpha\overline{\theta}_0\,\overline{w'q'}_0" /> The triple correlation in the equation above cannot be easily estimated in WRF, but judging from my tests it probably has a negligible impact. Instead of the given equation, WRF uses: <img src="https://latex.codecogs.com/svg.latex?\overline{w'\theta_{m}'}_0=\overline{w'\theta'}_0+\alpha\overline{\theta}_0\overline{w'q'}_0 " /> We therefore multiplied the surface heat flux with <img src="https://latex.codecogs.com/svg.latex?1+\alpha\overline{q}_0" />, when `use_theta_m=1`. For `km_opt=5`, also the contribution from the non-local heat flux is multiplied with a correction factor. LIST OF MODIFIED FILES: dyn_em/module_diffusion_em.F TESTS CONDUCTED: checked the dry theta budget of a simulation with use_theta_m=1: simple idealized LES once with model-computed surface heat flux and once with prescribed heat fluxes, RRTMG radiation, no microphysics, zero eddy diffusivities. Changed the code to output advection, SGS diffusion and radiation tendencies for dry theta (regardless of use_theta_m and without changing the tendencies used by the model). These forcing tendencies are averaged online over 30 minutes. The sum of the forcing terms is plotted against the actual tendency of dry theta (change of instantaneous dry theta between the half-hourly time stamps divided by 30 minutes) for all gridpoints and time stamps. All tendency terms are divided by the dry air mass to obtain more familiar units. The modified code fixes the issue of an unclosed dry theta budget at the lowest gridpoint (see attached figures). The simulations were repeated for km_opt=5. RELEASE NOTE: To close the dry theta budget at the lowest grid point when use_theta_m is enabled, the surface heat fluxes, and in the case of km_opt=5 also the non-local heat flux, are multiplied with 1+Rv/Rd*qv.   
TYPE: bug fix KEYWORDS: 3D TKE, km_opt, divide by zero SOURCE: Matthias Göbel (University of Innsbruck) DESCRIPTION OF CHANGES: Problem: When compiling WRF with the -D option, several divide-by-zero errors happen in the km_opt=5 scheme. This PR fixes the problems, and the changes are approved by the developer. LIST OF MODIFIED FILES: dyn_em/module_diffusion_em.F TESTS CONDUCTED: Jenkins testing is all PASS. RELEASE NOTE: Fixed divide by zero in the 3D-TKE option km_opt=5.
TYPE: bug fix KEYWORDS: RRTMG-K option (sw and lw option 14) SOURCE: Soonyoung Roh and Hwan-Jin Song (National Institute of Meteorological Science, Korea) DESCRIPTION OF CHANGES: The bugs include wrong surface downward diagnostic output for long- and short-wave fluxes (clear sky fluxes misplaced in cloudy sky flux arrays), a wrong data statement value for wavenum2 and a wrong value used for bound check for effective size of snow. Affects radiation driver call arguments for lw and sw scheme and internal numerical values in two places of the shortwave scheme (rrtmg_swk). This has been confirmed as a correct fix by the developers. ISSUE: Fixes #1312 LIST OF MODIFIED FILES: M phys/module_ra_rrtmg_swk.F M phys/module_radiation_driver.F TESTS CONDUCTED: Test reported by source (documented in Issue #1312) Jenkins testing all pass RELEASE NOTE: Fixed a bug in surface downward diagnostic output of long- and short-wave fluxes and two other bugs involving wrong numerical values used in the code (Thanks to Roh and Song of NIMS, Korea).
TYPE: bug fix KEYWORDS: tke, isfflx, tke_heat_flux, tke_drag_coefficient, tke_seed SOURCE: Matthias Göbel (University of Innsbruck) DESCRIPTION OF CHANGES: Problem: The tke seed value is used in the absence of surface drag or a surface heat flux, since there is no way to generate tke without pre-existing tke. So far the usage of the seed value is only tied to the namelist variables tke_drag_coefficient and tke_heat_flux being greater than 0. However, these namelist variables are only applied if no PBL scheme is used, diff_opt=2, and isfflx=0. If a PBL scheme is used or diff_opt=1, then isfflx=0 is a sufficient condition to get zero fluxes (see also technical notes). Solution: Fixed the if statements to enforce a correct setting of the tke seed value that fits to the purpose of the seed. LIST OF MODIFIED FILES: dyn_em/module_diffusion_em.F wrftladj/module_diffusion_em_ad.F wrftladj/module_diffusion_em_tl.F TESTS CONDUCTED: Jenkins testing is all pass. RELEASE NOTE: Corrected the tke seed value for km_opt 2 and 5 to be non-zero if and only if the surface heat and momentum fluxes are zero depending on isfflx, diff_opt, and bl_pbl_physics (Thanks to Matthias Göbel of University of Innsbruck).
…1301) TYPE: new feature KEYWORDS: raspberry pi SOURCE: Daniel Zurawski and internal DESCRIPTION OF CHANGES: Additional strings, `armv7l` and `aarch64`, have been added to the first gfortran configuration stanza so that these may be keywords for search from Raspberry Pi's `uname -a` output. No new stanza was added for the gfortran target, so no re-numbering is required for existing scripts that are reliant on such things as "32 = Gfortran for Linux with a serial build". Associated with this PR, #1300 (Inconsistent arg types (integer kind) across subroutine call) is also required to get the Raspberry Pi to build successfully. LIST OF MODIFIED FILES: modified: arch/configure.defaults TESTS CONDUCTED: 1. Previously, a Raspberry Pi build returned this: ``` ********************************************************* *** ERROR ERROR ERROR ERROR *** *** *** *** Configuration not found in configure.defaults *** ********************************************************* ``` 2. After this PR, the configure command is able to find a target (raspian buster): ``` ----------------------------------------------------------------------- Please select from among the following Linux armv7l options: 1. (serial) 2. (smpar) 3. (dmpar) 4. (dm+sm) GNU (gfortran/gcc) ``` 3. After this PR, the configure command is able to find a target (ubuntu mate): ``` ------------------------------------------------------------------------ Please select from among the following Linux aarch64 options: 1. (serial) 2. (smpar) 3. (dmpar) 4. (dm+sm) GNU (gfortran/gcc) ``` 4. Jenkins testing is all pass. RELEASE NOTES: Modifications to the WRF code now permit the model to be built on a Raspberry Pi with the GNU/8 compiler.
KEYWORDS: chemistry, emissions, aircraft, indexing SOURCE: Stacy Walters (formerly NCAR) DESCRIPTION OF CHANGES: Problem: An improper loop indexing that can cause addressing or floating point errors when aircraft emissions are used. Solution: Fix indexing (i - dimension) for the conversion term, from i to its:ite LIST OF MODIFIED FILES: M chem/module_emissions_anthropogenics.F TESTS CONDUCTED: 1. Jenkins test PASSED RELEASE NOTE: Improper indexing in a conversion factor when using aircraft emissions for WRF Chem previously caused addressing (seg fault) or floating point errors. This has been corrected by matching the indexing on the LHS with the RHS.
TYPE: bug fix
KEYWORDS: vertical, interpolation
SOURCE: internal
DESCRIPTION:
The stand-alone vertical interpolation program that is part of the module_initialize_real.F
file allows detailed testing of the existing vertical interpolation process used for ARW
in the real program. It is understood that practically no one except for me ever exercises
this utility.
1. The call to the main vertical interpolation routine, vert_interp, was out of date (missing
the grid id for some diagnostic error prints).
2. A suggestion for a build command is now provided.
3. Locations to use pressure-level or eta-level data are provided.
LIST OF MODIFIED FILES:
M module_initialize_real.F
TESTS CONDUCTED:
1. The stand-alone code builds and runs properly.
```
------------------------------------
UNIT TEST FOR VERTICAL INTERPOLATION
------------------------------------
------------------------------------
Lagrange Order = 1
------------------------------------
Level Pressure Field
(Pa) (generic)
------------------------------------
1 102000.000 102000.00
2 100000.000 100000.00
3 94722.219 94722.219
4 89444.445 89444.445
5 84166.664 84166.664
6 78888.891 78888.891
7 73611.109 73611.109
8 68333.336 68333.336
9 63055.555 63055.555
10 57777.777 57777.777
11 52500.000 52500.000
12 47222.223 47222.223
13 41944.445 41944.445
14 36666.668 36666.668
15 31388.889 31388.889
16 26111.111 26111.111
17 20833.334 20833.334
18 15555.556 15555.556
19 10277.777 10277.777
20 5000.000 5000.0000
Multi-Order Interpolator
------------------------------------
Level Pressure Field Field Field
(Pa) Calc Interp Diff
------------------------------------
1 100327.586 100327.6 100327.6 0.7812500E-02
2 96982.758 96982.76 96982.76 0.000000
3 93637.930 93637.93 93637.93 0.000000
4 90293.102 90293.10 90293.10 0.000000
5 86948.273 86948.27 86948.27 0.000000
6 83603.445 83603.45 83603.45 -0.7812500E-02
7 80258.617 80258.62 80258.62 0.000000
8 76913.797 76913.80 76913.80 0.000000
9 73568.969 73568.97 73568.97 0.000000
10 70224.141 70224.14 70224.14 0.000000
11 66879.312 66879.31 66879.31 0.000000
12 63534.484 63534.48 63534.48 0.000000
13 60189.656 60189.66 60189.66 0.000000
14 56844.828 56844.83 56844.83 0.000000
15 53500.000 53500.00 53500.00 0.000000
16 50155.172 50155.17 50155.17 0.000000
17 46810.344 46810.34 46810.35 -0.3906250E-02
18 43465.516 43465.52 43465.52 0.000000
19 40120.688 40120.69 40120.69 0.000000
20 36775.863 36775.86 36775.87 -0.3906250E-02
21 33431.035 33431.04 33431.04 0.000000
22 30086.207 30086.21 30086.21 0.000000
23 26741.379 26741.38 26741.38 0.000000
24 23396.551 23396.55 23396.55 0.000000
25 20051.725 20051.72 20051.73 -0.1953125E-02
26 16706.896 16706.90 16706.90 0.000000
27 13362.068 13362.07 13362.07 0.000000
28 10017.241 10017.24 10017.24 0.9765625E-03
29 6672.414 6672.414 6672.413 0.4882812E-03
```
2. Due to cpp ifdefs, the modifications have no impact on the traditionally used compilable code.
TYPE: bug fix KEYWORDS: km_opt=5, 3d tke scheme, use_theta_m=1 SOURCE: internal DESCRIPTION OF CHANGES: Problem: A previous surface flux fix (61029b1) applied a theta_m correction to all levels. However, this should only be used for qv at the surface. Solution: Switch k to kts in one place where this is applied, and pull the invariant assignment out of the k-index DO loop. LIST OF MODIFIED FILES: M dyn_em/module_diffusion.F TESTS CONDUCTED: Jenkins testing is all pass. RELEASE NOTE: Addendum to heat-flux theta_m fix (61029b1)* correct pwat function to use dry rho for integration
TYPE: bug fix KEYWORDS, real, int, integer, wrf_dm_max_int SOURCE: internal DESCRIPTION OF CHANGES: While looking to see if WRF had a function to return the maximum integer from among the max values of integers on each of the MPI ranks, it was noticed that the return value for the existing function was REAL. It should be an INTEGER. There is only a single use of this function in the entire model, in the file `share/module_trajectory.F`. ``` glb_traj_proc(trj) = wrf_dm_max_int( traj_proc(trj) ) ``` In that file, the input to the function, `traj_proc`, and the return value, `glb_traj_proc`, are both declared as integers. Therefore, there is no reason for a function to be given a list of integers, and when finding the max value to actually return that value as a real. ``` integer :: traj_proc(traj_max), glb_traj_proc(traj_max) ``` LIST OF MODIFIED FILES: M external/RSL_LITE/module_dm.F TESTS CONDUCTED: 1. Jenkins tests OK. 2. A small set of trajectories works as expected. 
TYPE: text only KEYWORDS: namelist, readme SOURCE: internal DESCRIPTION OF CHANGES: The trajectory examples.namelist was no longer providing sufficient info to run trajectories. 1. One of the helpful suggestions had a misspelled namelist record. It should have been `domains`, but it was referred to as `domain`. 2. There was no info on the additional options required: `dm_has_traj`. 3. There was no mention of the now mandatory additional text file `wrfinput_traj_d01` that has two namelist records. LIST OF MODIFIED FILES: M test/em_real/examples.namelist TESTS CONDUCTED: 1. This readme is not part of the build process, or part of compilable code. It will have no impact of results. Users likely also just skip past the incorrect namelist record identifier and grab the intended variable (num_traj). All in all, pretty small potatoes. 2. The jenksins tests pass, but no surprise - since we did not change anything that is part of the build.
TYPE: bug fix KEYWORDS: build, hydro, intel, stanza SOURCE: Ryan Cabell (NCAR) DESCRIPTION OF CHANGES: Problem: Building coupled WRF/WRF Hydro models fails when choosing configuration INTEL (ifort/icc): HSW/BDW. Solution: Add missing `ifort compiler with icc` text to ARCH line for the HSW/BDW configuration. For completeness, the KNL MIC configure stanza was also missing the same information, and has been updated. Neither of these stanzas are involved in the traditional WRF code build, so there is no impact on users of the default WRF atmosphere model. ISSUE: For use when this PR closes an issue. Fixes #1325 LIST OF MODIFIED FILES: M arch/configure.defaults TESTS CONDUCTED: 1. Modification causes WRF-Hydro build to complete successfully with the `INTEL (ifort/icc): HSW/BDW` configuration 2. Jenkins testing all positive
…1338) TYPE: enhancement KEYWORDS: geopotential,vertical advection SOURCE: Matthias Göbel (University of Innsbruck) DESCRIPTION OF CHANGES: The geopotential equation in WRF is formulated in the following (advective) form where staggering/destaggering is marked with overbars: <img src="https://latex.codecogs.com/svg.latex?\partial_{t}\phi^{\prime}+\mu_{d}^{-1}[m_{x}m_{y}\left(\overline{\overline{U}^\eta\:\delta_{x}\phi}^x+\overline{\overline{V}^\eta\:\delta_{y}\phi}^y\right)+m_{y}\overline{\overline{\Omega}^\eta\:\delta_{\eta}\phi}^\eta-m_{y}gW]=0" title="http://latex.codecogs.com/svg.latex?\partial_{t}\phi^{\prime}+\mu_{d}^{-1}[m_{x}m_{y}\left(\overline{\overline{U}^\eta\:\delta_{x}\phi}^x+\overline{\overline{V}^\eta\:\delta_{y}\phi}^y\right)+m_{y}\overline{\overline{\Omega}^\eta\:\delta_{\eta}\phi}^\eta-m_{y}gW]=0" /> The vertical advection term thus employs a double staggering of Omega. First, Omega is destaggered to mass levels and multiplied with the phi gradient. Then the product is staggered back to w-levels. This formulation is analogous to the horizontal terms, but the double averaging of Omega may introduce additional inaccuracies and is not necessary. The phi gradient can be first staggered to w-levels and then multiplied with the unaveraged Omega: <img src="https://latex.codecogs.com/svg.latex?\overline{\overline{\Omega}^\eta\:\delta_{\eta}\phi}^\eta\rightarrow\Omega\:\overline{\delta_{\eta}\phi}^\eta" /> For consistency and to avoid numerical instabilities, the same thing has to be done in the acoustic step. The new namelist option `phi_adv_z` allows switching between the two formulations: `phi_adv_z = 1`: old formulation (default) `phi_adv_z = 2`: new formulation LIST OF MODIFIED FILES: Registry/Registry.EM_COMMON dyn_em/module_big_step_utilities_em.F dyn_em/module_small_step_em.F run/README.namelist TESTS CONDUCTED: Idealized test cases em_les and em_hill2d_x. The vertical velocity is affected by the modification. Updrafts and downdrafts in the em_hill2d_x case become ~10% stronger with the new formulation:  In the LES case stronger updrafts and downdrafts occur more frequently:  I came across this issue when trying to transform advective components of temperature, humidity, and momentum into Cartesian form in a consistent way, using w for the vertical advection. Without the proposed modifications I could not close the budget, because the vertical advection of phi was not consistent with the vertical advection in the other budget equations. RELEASE NOTE: A new namelist option is included that allows a user to avoiding double stagger averaging of Omega in the vertical advection of geopotential. The new namelist option phi_adv_z allows switching between the two formulations: phi_adv_z = 1 (the old formulation, which remains the default), and phi_adv_z = 2 (the new formulation).
TYPE: Enhancement
KEYWORDS: BEP, BEP+BEM, YSU PBL scheme, vertical diffusion equations
SOURCE: Eric A. Hendricks (NCAR/RAL/NSAP)
DESCRIPTION OF CHANGES:
The multilayer BEP (Building Effects Parameterization) and BEP+BEM (BEP with the integration of a Building Energy Model) urban canopy models (UCMs) are added to the YSU planetary boundary layer (PBL) scheme. The implicit (A) and explicit (B) BEP and BEP+BEM source terms are added as forcing to the YSU implicit/explicit vertical diffusion equation solvers. Forcing is applied for zonal momentum, meridional momentum, potential temperature, and water vapor mixing ratio. The A sources are added to the tridiagonal matrix and the B sources are added to the explicit term vectors. Additionally, the finite differencing is modified using the BEP volume fraction and surface fraction not occupied by buildings. The BEP and BEP+BEM TKE tendency terms are added to the diagnostic TKE in YSU when topographic drag is turned on.
References:
* Hendricks, E. A., J. C. Knievel, and Y. Wang, 2020: Addition of multilayer urban canopy models to a nonlocal planetary boundary layer parameterization and evaluation using ideal and real cases, J. Appl. Met. Clim., 59, 1369-1392.
* Martilli et al. (2009), Description of the modifications made in WRF.3.1 and short user's manual of BEP, NCAR technical note.
Hendricks et al. (2020) demonstrated that the scheme works properly and produces expected behaviors. Results are qualitatively similar to MYJ and BOULAC with BEP and BEP+BEM. Minor differences are due to the different treatment of vertical mixing among the schemes.
LIST OF MODIFIED FILES:
phys/module_bl_ysu.F
phys/module_pbl_driver.F
phys/module_physics_init.F
run/README.namelist
TESTS CONDUCTED:
1. Real-case 48-h simulations of Houston urban area were conducted (10/05/2017-10/07/2017) with an innermost nest grid spacing of 1 km. The following test suite was set up:
Baseline tests (version before physics enhancement):
a. MYJ BEP
b. MYJ BEP+BEM
c. SU BULK
d. SU SLUCM
New code tests (new version with enhancement):
e. YSU BULK
f. YSU SLUCM
g. YSU BEP
h. YSU BEP+BEM
i. MYJ BEP
j. MYJ BEP+BEM
* Tests (e) and (f) were identical to tests (c) and (d), demonstrating that the new code did not affect the BULK and SLUCM existing code with the YSU PBL parameterization.
* Tests (g) and (h) were shown to produce the same behaviors as Hendricks et al. (2020).
* Tests (i) and (j) were identical to tests (a) and (b), demonstrating that the new code did not affect any existing BEP and BEP+BEM code with the MYJ PBL parameterization.
2. The real-case example test of the NMM dynamical core was also run for the YSU scheme without BEP and BEP+BEM prior to code modifications, and after the code modifications. The output was examined and shown to be the same. Therefore, the new YSU scheme with BEP and BEP+BEM functions properly in the NMM dynamical core when BEP and BEP+BEM are not active (note that the NMM dynamical core does not support BEP and BEP+BEM).
3. Since YSU is now functional with BEP and BEP+BEM, this line was added to replace the old lines in module_physics_init.F for all other PBL routines:
```
IF ((SF_URBAN_PHYSICS.eq.2).OR.(SF_URBAN_PHYSICS.EQ.3)) CALL wrf_error_fatal &
( 'module_physics_init: use ysu (option 1), myj (option 2), or boulac (option 8) with BEP/BEM urban scheme' )
```
This printout was shown to occur when attempting to run with PBL schemes that do not support the BEP and BEP+BEM functionality.
4. The restart capability was demonstrated to function properly with the new YSU BEP and YSU BEP+BEM code.
5. Jenkins testing is all PASS.
RELEASE NOTE: The multilayer BEP (Building Effects Parameterization) and BEP+BEM (BEP with the Building Energy Model) urban canopy models (UCMs) are added to the Yonsei University (YSU) planetary boundary layer parameterization. Reference: Hendricks, E. A., J. C. Knievel, and Y. Wang, 2020: Addition of multilayer urban canopy models to a nonlocal planetary boundary layer parameterization and evaluation using ideal and real cases, J. Appl. Met. Clim., 59, 1369-1392.
TYPE: enhancement KEYWORDS: ideal, run_me_first.csh, link, Makefile, SOURCE: Internal DESCRIPTION OF CHANGES: Problem: Several ideal cases (em_b_wave, em_convrad, em_quarter_ss, em_scm_xy, em_seabreeze2d_x, and em_tropical_cyclone) need additional linked files to run. Previously it was necessary for users to run a script called 'run_me_first.csh' to link-in the necessary files. Solution: Removed all 'run_me_first.csh' scripts from ideal case directories and modified the top-level Makefile so that now the files are linked during the compile. LIST OF MODIFIED FILES: M Makefile D test/em_b_wave/run_me_first.csh D test/em_convrad/run_me_first.csh D test/em_quarter_ss/run_me_first.csh D test/em_scm_xy/run_me_first.csh D test/em_seabreeze2d_x/run_me_first.csh D test/em_tropical_cyclone/run_me_first.csh TESTS CONDUCTED: 1. Ensured that code compiles and required files are linked for all modified ideal cases. 2. Jenkins is all pass. RELEASE NOTE: It is no longer necessary to execute a 'run_me_first.csh' script for any of the idealized cases. All necessary physics input files and look-up tables are linked to the corresponding directory during the compilation step.
TYPE: enhancement KEYWORDS: namelist fractional_seaice, default, replacing original PR#1128 SOURCE: Internal DESCRIPTION OF CHANGES: Since GFS, the most commonly used input data source, has been providing fractional seaice data for a while, it seems reasonable to make the default option for fractional_seaice to be 1. Both ERA5 and ERA-Interim provide fractional seaice data, too. When fractional seaice data is available, this option should be used in general. Turning on the fractional seaice flag broke many MPI vs OpenMP comparison tests. This led to the finding that the option is not thread safe for a number surface layer physics options: revised MM5, original MM5, MYJ, QNSE and PX (sf_sfclay_physics = 1, 91, 2, 4, and 7). This PR additionally provides the fix for these options. LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON M phys/module_surface_driver.F TESTS CONDUCTED: Jenkins test all pass. RELEASE NOTE: The default option for using fractional_seaice has been changed from 0 (off) to 1 (on) starting with release v4.3. In addition, several surface layer schemes are made thread safe if fractional seaice option is used. These are revised MM5, original MM5, MYJ, QNSE and PX surface layer scheme (sf_sfclay_physics = 1, 91, 2, 4, and 7).
TYPE: new feature
KEYWORDS: E-epsilon, EEPS, TKE, PBL
SOURCE: Chunxi Zhang: 1: (past) International Pacific Research Center, University of Hawai‘i at Mānoa
2: (past) Center for Analysis and Prediction of Storms, University of Oklahoma
3: (current) I. M. System Group, Inc. (IMSG); NOAA/EMC
Yuqing Wang International Pacific Research Center, and Department of Atmospheric Sciences, University of Hawai‘i at Mānoa
DESCRIPTION OF CHANGES:
A turbulence kinetic energy (TKE) and TKE dissipation rate (ε) based 1.5-order closure PBL parameterization is added:
1) the file module_bl_eepsilon.F includes the module for this scheme
2) standard procedures for a new scheme in WRF: changed init, addtendc and driver
3) three more variables are declared in registry.EM_COMMON. e.g.,
pep_pbl (TKE dissipation rate, state)
pek_adv (TKE advection, scalar)
pep_adv (PEP advection, scalar)
the files module_first_rd_step_part1.F and start_em.F are revised accordingly
this scheme is denfied as eepsscheme bl_pbl_physics == 16
4) add module_bl_eepsilon in Makefile
LIST OF MODIFIED FILES:
phys/module_bl_eepsilon.F
phys/module_pbl_driver.F
phys/module_physics_init.F
phys/module_physics_addtendc.F
phys/Makefile
Registry/Registry.EM_COMM
dyn_em/module_first_rk_step_part1.F
dyn_em/start_em.F
run/README.namelist
TESTS CONDUCTED:
1. Automated jenkins testing is all PASS.
2. The performances of the newly implemented EEPS scheme and the existing Yonsei University (YSU) scheme, the University of Washington (UW) scheme, and Mellor–Yamada–Nakanishi–Niino (MYNN) scheme are evaluated over the stratocumulus dominated southeast Pacific (SEP) and over the Southern Great Plains (SGP) where strong PBL diurnal variation is common. The simulations by these PBL parameterizations are compared with various observations from two field campaigns: the Variability of American Monsoon Systems Project (VAMOS) Ocean–Cloud–Atmosphere–Land Study (VOCALS) in 2008 over the SEP and the Land–Atmosphere Feedback Experiment (LAFE) in 2017 over the SGP. Results show that the EEPS and YSU schemes perform comparably over both regions, while the MYNN scheme performs differently in many aspects, especially over the SEP. The EEPS (MYNN) scheme slightly (significantly) underestimates liquid water path over the SEP. Compared with observations, the UW scheme produces the best PBL height over the SEP. The MYNN produces too high PBL height over the western part of the SEP while both the YSU and EEPS schemes produce too low PBL and cloud-top heights. The differences among the PBL schemes in simulating the PBL features over the SGP are relatively small.
3. See published paper:
Zhang, C., Y. Wang, and M. Xue, 2020: Evaluation of an E–ε and Three Other Boundary Layer Parameterization Schemes in the WRF Model over the Southeast Pacific and the Southern Great Plains. Mon. Wea. Rev., 148, 1121–1145, https://doi.org/10.1175/MWR-D-19-0084.1.
RELEASE NOTE: A turbulence kinetic energy (TKE) and TKE dissipation rate (ε) based 1.5-order closure PBL parameterization (E–ε, EEPS) is added (Zhang et al. 2020, MWR). Works with surface layer options, 1, 91, 2 and 5. (Thanks to Chunxi Zhang and Yuqing Wang of University of Hawaii.)
…NWP model. (#1329) TYPE: Enhancement KEYWORDS: Gravity wave drag, Turbulent orographic form drag SOURCE: Michael Toy (NOAA/GSL) DESCRIPTION OF CHANGES: Two subgrid-scale sources of orographic drag are added to the WRF gravity wave drag (GWD) parameterization. The enhanced GWD parameterization is activated by namelist option "gwd_opt = 3". The additional schemes are applicable at coarse horizontal grid resolution (~100s km) down to fine resolutions on the order of 1km, and they account for small-scale ~1km topographic variations. 1. The first scheme is the small-scale GWD parameterization of Tsiringakis et al. (QJRMS, 2017) and Steeneveld et al. (J. Appl. Meteor., 2008), which represents gravity wave vertical propagation and breaking in and above stable (typically nocturnal) planetary boundary layers. 2. The second scheme is the Turbulent Orographic Form Drag parameterization of Beljaars et al. (QJRMS, 2004). The original large-scale GWD and low-level flow blocking parameterizations of Kim and Doyle (QJRMS, 2005) are still incorporated in the drag scheme, but they have been modified to more properly adjust for the horizontal grid resolution. LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON M dyn_em/module_first_rk_step_part1.F M main/depend.common M phys/Makefile A phys/module_bl_gwdo_gsl.F M phys/module_pbl_driver.F M share/module_check_a_mundo.F M run/README.namelist TEST CONDUCTED: 1. Regression test all pass 2. Figure showing surface stress magnitude (N m-2) over Colorado, 13-km resolution for 2019 Feb 12 0100UTC. These plots were derived from the DUSFC_xx and DVSFC_cc output variables.  3. Figure showing the vertical profile of magnitude of the drag force, from the four drag components, averaged over Colorado. 13-km resolution for 2019 Feb 12 0100UTC. These curves were derived from the DTAUxxx output variables.  RELEASE NOTES: Two subgrid-scale sources of orographic drag are added to the WRF gravity wave drag (GWD) parameterization. The enhanced GWD parameterization is activated by namelist option "gwd_opt = 3". The additional schemes are applicable at coarse horizontal grid resolution (~100s km) down to fine resolutions on the order of 1km, and they account for small-scale ~1km topographic variations. Setting namelist option "gwd_diags = 1" (default is "0") causes diagnostic data to be output to the wrfout_d0x files. The data include: 1) 2D vertically integrated momentum fluxes, i.e., surface stress, DUSFC_xx (x-direction) and DVSFC_xx (y-direction), where xx={LS,BL,SS,FD}, which are "large-scale", "blocking", "small-scale" and "form drag" contributions, respectively; and 2) 3D drag forces DTAUX3D_xx (x-direction) and DTAUY3D_xx (y-direction), where xx={LS,BL,SS,FD}.
TYPE: bug fix KEYWORDS: fractional seaice, MM5 and revised MM5 surface layer schemes, uninitialized local arrays SOURCE: internal DESCRIPTION OF CHANGES: Problem: Intermittent model failure when fractional_seaice is turned with nesting, with errors like 'invalid memory reference'. Solution: Two local arrays, qsfc_sea (which causes the problem), and znt_sea are not initialized, and somehow the problem only shows up in nesting. This should fix one of the comparison test, 60NE, but may not fix test 07NE, due to use of urban physics option 2. LIST OF MODIFIED FILES: M phys/module_surface_driver.F TESTS CONDUCTED: 1. Do mods fix problem? Partially. It should fix model failure, and nesting bit-for-bit test with seaice flag on. 2. Jenkins tests are passing. RELEASE NOTE: Part of fix in order to turn on fractional_seaice flag by default.
TYPE: no impact KEYWORDS: remove SOURCE: internal DESCRIPTION OF CHANGES: There is a file in the test/em_real directory that does not belong in the repository: landFilenames. The `landFilenames` has a storied history: ``` commit 7629cd0 Author: John Michalakes <michalak@ucar.edu> Date: Fri Feb 18 18:55:16 2005 +0000 Temporary addition for hires topo input. Should remove later. JM git-svn-id: https://svn-wrf-model.cgd.ucar.edu/trunk@753 b0b5d27b-6f0f-0410-a2a3-cb1e977edc3d ``` John's advice was to "remove later". We waited 16 years, and now we are removing the file. LIST OF MODIFIED FILES: D test/em_real/landFilenames TESTS CONDUCTED: 1. Jenkins is OK.
TYPE: bug fix KEYWORDS: Thompson, MP, aerosol, constant SOURCE: Greg Thompson (JCSDA), found by Laura Fowler and Mary Barth (NCAR) DESCRIPTION OF CHANGES: A problem with a single constant was discovered a couple months ago, but did not get permanently updated in the repository. The fix is to reduce fake surface aerosol emissions by a factor of 4. LIST OF MODIFIED FILES: modified: phys/module_mp_thompson.F TESTS CONDUCTED: 1. Jenkins tests are all PASS. RELEASE NOTE: For the Thompson MP scheme, a problem with a single constant was discovered that caused approximately 4x too much fake surface aerosol emissions.
TYPE: New feature
KEYWORDS: MAD-WRF, MADCast, WRF-Solar, cloud initialization, cloud advection/diffusion, nudging
SOURCE: Pedro Jimenez, Jared Lee, and Jimy Dudhia (NCAR), and Greg Thompson (JCSDA)
DESCRIPTION OF CHANGES:
Problem: There is room for improvement in the short range shortwave irradiance predictions
Solution:
MAD-WRF is a solar irradiance nowcasting system that combines:
1. the benefits of data assimilation and the philosophy of advecting cloud properties of MADCast, and
2. the cloud physics of WRF-Solar.
The initial cloud field can be enhanced based on relative humidity and the model can impose the cloud mask and
the cloud top / base height from available earth observations. The hydrometeor content is advected and diffused
as tracers, and the resolved hydrometeors are nudged towards these tracers at the beginning of the simulation
to facilitate the cloud spin up. This results in improved solar irradiance prediction in the short-range (the first
hours). There is an option to run MAD-WRF with just advecting and diffusing the hydrometeors.
We extensively tested MAD-WRF performance using an implementation in an older version of WRF. Our results
show the improved performance at the beginning of the simulation.
The following technical description follows the README.madwrf file in the doc directory. There are two options to
run MAD-WRF:
1. madwrf_opt = 1: The initial hydrometeors are advected and diffused with the model dynamics without accounting
for any microphysical processes.
* The user should set mp_physics = 96 and use_mp_re = 0 in the physics block of the WRF namelist.
2. madwrf_opt = 2: There is a set of hydrometeor tracers that are advected and diffused with the model dynamics.
* At initial time the tracers are equal to the standard hydrometeors.
* During the simulation the standard hydrometeors are nudged towards the tracers.
* The namelist variable madwrf_dt_nudge sets the temporal period for hydrometeor nudging [min]
* The namelist madwrf_dt_relax sets the relaxation time for hydrometeor nudging [s]
MAD-WRF has an option to enhance the cloud initialization. To turn on (off) the cloud initialization set the namelist
variable madwrf_cldinit to 1 (0)
By default the model enhances the cloud analysis based on the analyzed relative humidity. The user can enhance
the cloud initialization by providing additional variables to Metgrid via the WPS intermediate format. These variables
are the cloud mask (CLDMASK), the cloud top height (CLDTOPZ [m asl]), the brightness temperature (BRTEMP,
e.g. GOES-R ABI channel 13), and the cloud base height (CLDBASEZ [m agl]). These are the combinations of
variables allowed in MAD-WRF (see the README.madwrf file in the doc subdirectory for further details).
1. Cloud mask (CLDMASK variable):
* Remove clouds if clear (cldmask = 0)
2. Cloud mask (CLDMASK variable) + brightness temperature (BRTEMP variable) sensitive to hydrometeor content
(e.g. GOES-R channel 13):
* Remove clouds if clear (cldmask = 0)
* Reduce / extend cloud top heights to match observations
* Add clouds over clear sky regions (cldmask = 1)
3. Cloud top height (CLDTOPZ variable) with 0 values over clear sky regions
* Remove clouds if clear (cldmask = 0)
* Reduce / extend cloud top heights to match observations
* Add clouds over clear sky regions (cldmask = 1)
4. Either 2 or 3 + the cloud base height (CLDBASEZ variable)
* Remove clouds if clear (cldmask = 0)
* Reduce / extend cloud top / base heights to match observations
The following lines should be added to the metgrid table:
```
========================================
name=CLDMASK
interp_option=nearest_neighbor
flag_in_output=FLAG_CLDMASK
========================================
name=CLDBASEZ
interp_option=nearest_neighbor
flag_in_output=FLAG_CLDBASEZ
========================================
name=CLDTOPZ
interp_option=nearest_neighbor
flag_in_output=FLAG_CLDTOPZ
========================================
name=BRTEMP
interp_option=nearest_neighbor
flag_in_output=FLAG_BRTEMP
========================================
```
LIST OF MODIFIED FILES:
M Registry/Registry.EM
M Registry/Registry.EM_COMMON
M Registry/Registry.NMM
M Registry/Registry.tladj
M Registry/registry.chem
A doc/README.madwrf
M dyn_em/depend.dyn_em
M dyn_em/module_big_step_utilities_em.F
M dyn_em/module_initialize_real.F
M dyn_em/solve_em.F
M main/depend.common
M phys/Makefile
A phys/module_madwrf.F
M phys/module_microphysics_driver.F
M phys/module_physics_init.F
M run/README.namelist
M share/module_check_a_mundo.F
M share/module_optional_input.F
TESTS CONDUCTED:
1. We are currently running a demonstration of the model (operations tab in https://ral.ucar.edu/projects/mad-wrf).
The following figure show the MAE as a function of the lead time calculated with MAD-WRF, WRF-Solar and
MAD-WRF run in MADCast mode. These results are from our MAD-WRF implementation in an older version of WRF
and will be updated with results using the code in this develop branch.
2. The jenkins tests are all passing.
RELEASE NOTE: Adding the MAD-WRF model to improve solar irradiance nowcasting (https://ral.ucar.edu/projects/mad-wrf).
…urban classes (31-33) (#1419) TYPE: new feature KEYWORDS: urban, local climate zone, SOURCE: Developer: Andrea Zonato (University of Trento, Italy, andrea.zonato@unitn.it) DESCRIPTION OF CHANGES: This update includes the extension of urban classes from 3 (31-33) to 11 WUDAPT Local Climate zones (31-41). See Stewart and Oke, 2012 for further details Briefly, a new urban parameter table (URBPARAM_LCZ.TBL) has been added for using the new LCZ urban category. The new LCZ definition has LU_INDEX=LCZ+30. If a user still wants to keep using the traditional 33 classes (31st-33rd for urban), please specify the namelist option "use_wudapt_lcz = 0". If a user wants to use the new LCZ, please specify the namelist option "use_wudapt_lcz = 1". If the number of urban category in the input files is inconsistent with the namelist option, error messages will occur. Only the LCZ modeling capability is provided here, not the dataset which is the responsibility of users. The method to create the LCZ data is described in the World Urban Database website: http://www.wudapt.org/ LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON M Registry/Registry.NMM M main/depend.common M dyn_em/module_initialize_real.F M main/depend.common M phys/Makefile A phys/module_bep_bem_helper.F M phys/module_physics_init.F M phys/module_sf_bep.F M phys/module_sf_bep_bem.F M phys/module_sf_clm.F M phys/module_sf_noahdrv.F M phys/module_sf_noahlsm.F M phys/module_sf_noahmpdrv.F M phys/module_sf_noahmplsm.F M phys/module_sf_urban.F M phys/module_surface_driver.F M run/LANDUSE.TBL M run/MPTABLE.TBL M run/URBPARM.TBL A run/URBPARM_LCZ.TBL M run/VEGPARM.TBL A share/module_check_a_mundo.F TESTS CONDUCTED: 1. Jenkins testing OK RELEASE NOTE: Stewart, I.D. and Oke, T.R. (2012) Local Climate Zones for Urban Temperature Studies. Bulletin of the American Meteorological Society, 93, 1879-1900. http://dx.doi.org/10.1175/BAMS-D-11-00019.1
TYPE: new feature
KEYWORDS: CLM surface scheme, MEGANv2.1 biogenic emissions
SOURCE: Brian Gaudet (PNNL)
DESCRIPTION OF CHANGES:
The CLM surface scheme and associated subroutines in the physics and chemistry packages were modified to be
made consistent with the MEGANv2.1 biogenic emission model, as opposed to earlier code which uses MEGANv2.0.
This upgrade to WRF-CLM parallels the use of MEGANv2.1 in CLM4.0, as described in Guenther et al. (2012, GMD).
Within module_sf_clm.F, the emissions for 150 species are computed, and then mapped to the chemistry packages
(e.g., SARPC99) in WRF-Chem. Emissions are a function of plant functional type (PFT) and emission factors that
depend on temperature, light, and LAI.
Among the key changes to module_sf_clm.F are:
* a MEGANv2.1 emissions factor file containing standard emissions as a function of PFT and 20 classes of compounds
is read in
* new modules for handling the MEGAN categories are created (and the use of these modules by other modules
necessitated the re-ordering of the subroutines in the source file)
* the environmental emissions factors are tracked through running averaged variables
* PFTs can now be determined from a geographical mapping in the wrfinput file rather than from an internal
vegetation type-based table - the code is looking for a variable of name 'PCT_PFT' in the wrfinput file; it is up to
the user to use external software to insert the variable containing the percentages for each CLM PFT
* mappings of MEGAN species to WRF-Chem species packages are included.
An option ('bio_emiss_opt = 4') to invoke MEGANv2.1 biogenic emissions is included in the `&chem` namelist section;
the user retains the option of using old MEGAN emissions (`bio_emiss_opt = 3`). When a user runs the new CLM but
without selecting `bio_emiss_opt = 4`, the old CLM method of mapping land use to PFTs is used, in order to reproduce
the meteorology of previous versions of the model.
LIST OF MODIFIED FILES:
M Registry/Registry.EM_COMMON
M Registry/Registry.NMM
M Registry/registry.chem
M Registry/registry.clm
M Registry/registry.dimspec
M chem/KPP/mechanisms/saprc99/saprc99.eqn
M chem/KPP/mechanisms/saprc99/saprc99.spc
M chem/chem_driver.F
M chem/chemics_init.F
M chem/emissions_driver.F
M chem/module_dep_simple.F
M chem/module_emissions_anthropogenics.F
M chem/module_ftuv_driver.F
M dyn_em/module_first_rk_step_part1.F
M dyn_em/start_em.F
M. dyn_nmm/module_PHYSICS_CALLS.F
M frame/module_driver_constants.F
M phys/module_physics_init.F
M phys/module_sf_clm.F
M phys/module_surface_driver.F
M share/module_check_a_mundo.F
TESTS CONDUCTED:
1. Jenkins testing is OK.
2. Tests with the changes merged into the develop branch reproduced the meteorology of the pure develop branch
when bio_emiss_opt = 4 _was not_ selected.
3. When bio_emiss_opt = 4 _was_ selected, the meteorology and emissions of PNNL reference simulations was
reproduced.
4. Tests of code merged with the up-to-moment version of develop are pending.
Examples of a meteorological tests and a chemistry test are attached.
RELEASE NOTE: The CLM surface scheme and associated subroutines in the physics and chemistry packages have been modified to be made consistent with the MEGANv2.1 biogenic emission model, as opposed to earlier code which used MEGANv2.0. This upgrade to WRF-CLM parallels the use of MEGANv2.1 in CLM4.0, as described in Guenther et al. (2012, GMD). Within module_sf_clm.F, the emissions for 150 species are computed, and then mapped to the chemistry packages
(e.g., SARPC99) in WRF-Chem. Emissions are a function of plant functional type (PFT) and emission factors that depend on temperature, light, and LAI.
) KEYWORDS: urban, green roof, solar panel, new building drag coefficient SOURCE: Andrea Zonato (University of Trento, Italy, andrea.zonato@unitn.it) DESCRIPTION OF CHANGES: This update includes new capabilities to account for urban solar panel, green roof, and new building drag coefficient. These three new capabilities are for BEP+BEM. The new building drag coefficient is based on Santiago and Martilli (2010) and Guti ́errez et al. (2015). More details are available in the WRF User's Guide. LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON M Registry/Registry.NMM M Registry/registry.dimspec M dyn_em/module_first_rk_step_part1.F M dyn_em/start_em.F M dyn_nmm/module_PHYSICS_CALLS.F M dyn_nmm/start_domain_nmm.F M phys/module_physics_init.F M phys/module_sf_bem.F M phys/module_sf_bep_bem.F M phys/module_sf_noahdrv.F M phys/module_sf_noahmpdrv.F M phys/module_sf_urban.F M phys/module_surface_driver.F M run/URBPARM.TBL M run/URBPARM_LCZ.TBL M share/module_check_a_mundo.F TESTS CONDUCTED: 1. The model compilation has been tested successfully (tested in Cheyenne by both the developer and Cenlin He, NCAR) 2. The model simulation has been tested successfully (tested in Cheyenne by the developer) 3. The model simulation has been tested successfully (tested in WRF Docker by Cenlin He, NCAR) 4. The jenkins testing is all PASS The original PR was split into two pieces. Here is a figure from the original PR (left) and the combination of the two PRs (right). We want to see nearly the exact same features. This is a figure of 2-meter temperature around the Denver area.  RELEASE NOTE: WRF-urban updates for green roof, solar panel, and new building drag coefficient for BEP+BEM. The paper describing the green roof update is under development. The new building drag coefficient is based on Santiago and Martilli (2010) and Guti ́errez et al. (2015). Santiago, J. L. and Martilli, A. (2010). A Dynamic Urban Canopy Parameterization for Mesoscale Models Based on Computational Fluid Dynamics Reynolds-Averaged Navier-Stokes Microscale Simulations. Boundary- Layer Meteorology, 137(3):417–439. Guti ́errez, E., Martilli, A., Santiago, J. L., and Gonz ́alez, J. E. (2015). A Mechanical Drag Coefficient Formulation and Urban Canopy Parameter Assimilation Technique for Complex Urban Environments. Boundary-Layer Meteorology, 157(2):333–341. A. Zonato, A. Martilli, E. Gutierrez, F. Chen, C. He, M. Barlage, D. Zardi, and L. Giovannini (2021): Exploring the effects of rooftop mitigation strategies on urban temperatures and energy consumption, Atmospheric Chemistry and Physics, under review.
TYPE: enhancement KEYWORDS: WRF-CTSM coupling, Community Terrestrial Systems Model (CTSM), Community Land Model (CLM), coupled land-atmosphere models SOURCE: Negin Sobhani <sup>1</sup> , Dave Lawrence <sup>1</sup> , Sam Levis <sup>2</sup> , Bill Sacks<sup>1</sup> <sup>1</sup> National Center for Atmospheric Research (NCAR) <sup>2</sup> SLevis Consulting DESCRIPTION OF CHANGES: This PR includes some modifications that improve WRF-CTSM coupling including the capability to handle lake points by CTSM instead of WRF. Problem: Previously lake points were not handled by the CTSM model; however, the CTSM community prefers lake points to be handled by CTSM lake model. This PR adds the capability to handle lake points by CTSM model. Solution: For solving this problem in `module_surface_driver.F`, we used ~~`xland(i,j) .gt. 1.5 .and. ht(i,j).gt.lake_min_elev`~~ `lakemask(i,j).eq.1.` logic. For CTSM cases we changed a local copy of `xland` (called `ctsm_xland`) for the lake points to 1, so that CTSM receives the lake points. LIST OF MODIFIED FILES: M phys/module_surface_driver.F M share/module_soil_pre.F TESTS CONDUCTED: 1. Following are a couple of figures, depicting a particular period's lakemask over CONUS, for both the input and theWRF model output. Specifically, the Great Lakes were part of the concern with this PR, and they are indeed represented as part of the lake identified cells. Model input:  Model output:  2. Figures of low-level temperature and moisture for a 1-week simulation, without spectral nudging. 2-meter temperature: Here is the plot showing ΔT2 for the first week of 2013-04. ΔT2 is old version-new version. The new version contains the local xland mods requested by the WRF committee.  2-meter RH. There is ~ a 5-10% difference in RH at 2m between the old and new versions over all the inland lakes. ΔRH2 is old version-new version. This means that over the lakes, the new version is 5-10% dryer than the old version.  3. Jenkins tests are passing.
TYPE: text only KEYWORDS: README.namelist, wudapt, rebalance, madwrf, P3, CTSM SOURCE: internal DESCRIPTION OF CHANGES: Problem: Several new namelist options have been added to the code for the V4.3 release, but a few weren't added to the README.namelist file. Solution: Added the missing namelist variables to the README.namelist file. LIST OF MODIFIED FILES: M run/README.namelist TESTS CONDUCTED: no tests necessary - text only Jenkins are all passing.
TYPE: no impact KEYWORDS: max_rot_angle_gwd, remove, Registry.EM_COMMON, non-applicable SOURCE: internal DESCRIPTION OF CHANGES: Problem: The variable 'max_rot_angle_gwd' is no longer used in the code since V4.0, but was still listed in the Registry.EM_COMMON file. Solution: Removed 'max_rot_angle_gwd' from Registry.EM_COMMON LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON TESTS CONDUCTED: No impact modification. No testing needed. This variable is not in any other code. Jenkins is OK
TYPE: bug fix KEYWORDS: RRTMG, shortwave radiation, solar source function SOURCE: Timothy W. Juliano and Pedro A. Jimenez (NCAR/RAL), Jimy Dudhia (NCAR/MMM) DESCRIPTION OF CHANGES: Problem: In subroutine `setcoef_sw` of `phys/module_ra_rrtmg_sw.F`, which is called within subroutine `rrtmg_sw`, the number of vertical levels (`nlayers`) and the tropopause level (`laytrop`) are calculated. If the domain top is sufficiently low, then the tropopause level is never "found" by the set pressure threshold, and `laytrop` = `nlayers`. In a typical real case, where the tropopause is below the model top, `laytrop < nlayers`. Thus, this scenario is likely to occur only in idealized cases. After `nlayers` and `laytrop` are set, subroutine `taumol_sw` is called within the radiation transfer model. Here, the different gas/molecular optical depths over the spectral bands are calculated, and there is a separate subroutine for each band (`taumol16`, `taumol17`, etc). One of the purposes of each of these subroutines is to fill the solar source function (`sfluxzen`). In each of the subroutines, there are two main loops: a lower atmosphere loop (extending from the lowest `k` level to the tropopause layer, `laytrop`) and an upper atmosphere loop (extending from `laytrop+1` to `nlayers`). Here is where the issue arises. In some of the `taumol*` subroutines, the indices of `sfluxzen` associated with that particular band are filled in the lower atmosphere loop, while in the other subroutines, the relevant indices of `sfluxzen` are filled in the upper atmosphere loop. In the subroutines where `sfluxzen` is intended to be filled in the upper atmosphere loop, it is never actually filled. This is because `laytrop = nlayers`, and the upper atmosphere loop is never executed. Thus, `sfluxzen` is filled with junk values (including `NaN`) for these indices. This is an issue because after `sfluxzen` is filled, it is then corrected by considering Earth/Sun distance and zenith angle and stored in `zincflx` in the `spcvmc_sw` subroutine. The spectral fluxes are then summed over the spectrum and stored in `pbbfu` (upwelling) and `pbbfd` (downwelling). The upwelling and downwelling SW fluxes are stored in variables `swuflx` and `swdflx` in subroutine `rrtmg_sw` and subsequently used to calculate `GSW` in subroutine `RRTMG_SWRAD`. The `NaN` values propagate through all of these variables, and when `SWDOWN` is calculated from `GSW` in the radiation driver, it is also `NaN`. The place where the code crashes is in the `diffuse_frac` loop, when it checks the value of `swdown(i,j)`. Solution: The simple and clean fix is to initialize `sfluxzen` to zero in `taumol_sw` before all of the `taumol*` subroutines are called. I implemented this one liner into the code and it integrates through as expected. LIST OF MODIFIED FILES: M phys/module_ra_rrtmg_sw.F TESTS CONDUCTED: 1. I have run an idealized LES at high latitude (setting `xlat` to 75N in `module_initialize_ideal.F`) starting at 06 UTC on March 28, 2020 with and without the bug fix. Without the bug fix, the simulation crashes as reported above. With the bug fix, the simulation integrates through without an issue. I'll also note that running the simulation at 75N except at start times of 12 and 18 UTC leads to failure, while starting at 00 UTC leads to successful integration. So, this issue appears to happen during daytime only. Moreover, this issue arises with only RRTMG SW radiation scheme; the code integrates properly using every other SW radiation scheme (with the exception of the other RRTMG flavors, which I have not tried), suggesting this issue is tied to RRTMG. 2. Jenkins tests are PASS. RELEASE NOTE: A fix is introduced to the RRTMG SW parameterization. When the model top is relatively low, and the threshold for tropopause pressure is not reached, `laytrop = nlayers`. In this case, some indices in `sfluxzen` can be filled with junk, ultimately leading to NaN value for `SWDOWN` and failure in the radiation driver. The modification is to initialize `sfluxzen` to zero before being filled.
) TYPE: bug fix KEYWORDS: vertical nesting, hybrid vertical coordinate, vert_refine_method, hybrid_opt SOURCE: Robert Arthur and Katie Lundquist (LLNL) DESCRIPTION OF CHANGES: Problem: When vertical nesting is used with the hybrid vertical coordinate option (hybrid_opt=2), large errors occur in the vertically-refined domain. Solution: An incorrect calculation of pb (base pressure) was fixed in subroutine force_domain_em_part2 (module_dm.F). The pb calculation was changed to match similar code in start_em. This fix is within a block of code that is only called if vert_refine_method .ne. 0, in order to rebalance on the "intermediate" nested grid. The error was likely a result of the switch to the new hybrid coordinate definition, which was not updated correctly in this particular subroutine. The check-a-mundo preventing vertical nesting from being used with the hybrid coordinate (see #1154) has also been removed. This PR is related to #901, which addressed vertical nesting with the hybrid coordinate (but did not fully fix the issue), as well as #1408, which addressed vertical nesting with use_theta_m=1. LIST OF MODIFIED FILES: external/RSL_LITE/module_dm.F share/module_check_a_mundo.F TESTS CONDUCTED: 1. The tests conducted here are similar to those in #1408. A 4-domain nested case was completed with vertical refinement on d04. All domains were initialized from a wrfinput file and eta levels were not provided. Pertinent namelist options are: ``` input_from_file = .true., .true., .true., .true., e_vert = 45, 45, 45, 55, vert_refine_method = 0, 0, 0, 2, rebalance = 1, hybrid_opt = 0 or 2, depending on the case use_theta_m = 0 ``` Results are summarized in the plot below, which shows U at the first grid point above the surface on d04 at various output times.  With the original code, vertical nesting works with hybrid_opt=0, but not hybrid_opt=2. With the updated code, results look reasonable for hybrid_opt=2 and are roughly the same as for hybrid_opt=0. Similar patterns are also seen for other variables, such as V and MU, although they are not shown here. 2. Several similar tests were also conducted, as above but with * d04 initialized via interpolation from d03 during runtime, rather than from wrfinput_d04 * eta levels specified for each domain in the namelist * use_theta_m=1 These tests provided qualitatively similar results, showing that the fix works for a range of possible setups. 3. Finally, the tests above were completed with hybrid_opt=0, but using the updated code. These showed bit-for-bit identical results (using diffwrf) to the same cases using the original code, confirming that previous vertical nesting functionality with hybrid_opt=0 is maintained. This was done for both use_theta_m options, 0 and 1. 4. Jenkins testing is OK RELEASE NOTE: Vertical nesting now works with the hybrid vertical coordinate.
Correct MU layer and PWAT, refine CAPE, CIN, LFC TYPE: bug fix, enhancement KEYWORDS: Most unstable layer, Precipitable water, CAPE, CIN, LFC SOURCE: Zhixiao Zhang (University of Utah), Adam Varble (PNNL and University of Utah), and Katelyn Barber (PNNL) DESCRIPTION OF CHANGES: Problem: The most unstable (MU) layer and precipitable water(PWAT) are inconsistent with the conventional definitions in meteorological communities. LFC, CAPE and CIN calculations do not work well while dealing with multiple inversion layers. Solution: 1. Correct the MU layer definition from the max water vapor mixing ratio (w) level to the max equivalent potential temperature (Theta-e) level. Because it is the theta-e rather than mixing ratio conserved during the dry/moist adiabatic lifting processes. 2. Correct PWAT definition from column accumulated total mass of water vapor and cloud water to only water vapor. This is based on the AMS glossary https://glossary.ametsoc.org/wiki/Precipitable_water. 3. Calculate MUCAPE, MUCIN and LFC by refined definitions: MUCAPE and MUCIN are defined as the vertically integrated positive and negative buoyant energy between LCL and EL with the air parcel lifted from MU layer. LFC is defined as the highest separation level when buoyancy turns from negative to positive. LIST OF MODIFIED FILES: phys/module_diag_functions.F TESTS CONDUCTED: 1. The modified code has been complied successfully and applied on regional climate simulations for CACTI field campaign. 2. In elevated deep convection regimes when the highest water vapor mixing ratios are still at low levels but the most unstable parcel is at a higher altitude, the differences can be large. An example of such a sounding is attached. The original diagnostic would choose a parcel starting at the surface where the highest water vapor mixing ratio exists. That parcel has no CAPE and obviously a lot of CIN. The most unstable parcel with the highest theta-e (you can find this using the highest dashed green moist adiabat value) is just above the 700-mb level which is a parcel with CAPE and very little CIN.  3. Jenkins testing is all PASS. OTHER NOTES: The modification is based on WRF 4.1.1. The relevant AFWA module, diagnostic driver and registry require to be consistently modified in the latest source code by maintainers. RELEASE NOTE: A few updates to the diagnostic schemes were added. The most unstable (MU) layer and precipitable water(PWAT) were inconsistent with the conventional definitions used by the meteorological communities. LFC, CAPE and CIN calculations previously did not work well while dealing with multiple inversion layers.
TYPE: new feature KEYWORDS: IEVA, vertical advection SOURCE: Louis Wicker (NOAA/NSSL) DESCRIPTION OF CHANGES: For grids with large aspect ratios (dx/dz >> 1) that permit explicit convection, the large time step is limited by the strongest updraft that occurs during integration. This results in time step often 20-30% smaller, or requires the use of w-filtering, such as latent-heat tendency limiting. Regions of large vertical velocities are also often very small relative to the domain. The Implicit-Explicit Vertical Advection (IEVA) scheme has been implemented (see Wicker, L. J., and W. C. Skamarock, 2020: An Implicit–Explicit Vertical Transport Scheme for Convection-Allowing Models. Mon. Wea. Rev., 148, 3893–3910) and that permits a larger time step by partitioning the vertical transport into an explicit piece, which uses the normal vertical schemes present in WRF, and a implicit piece which uses implicit transport (which is unconditionally stable). The combined scheme permits a larger time step than has been previously been used and reduced w-filtering. The scheme will be useful for CONUS scale CAM (convection allowing model) simulations (dx ~ 2-3 km) when the number of vertical levels > 50. Time steps can increase to as large as 25 s, depending on the problem. The Wicker and Skamarock paper demonstrates IEVA's advantages on the 27 April 2011 Alabama tornado outbreak by comparing it to the operational CAM (the High Resolution Rapid Refresh) configuration. Results are shown that the HRRR simulation is stable up to a dt=20 s and only with latent-heat limiting on. Using the IEVA scheme the dt can be increased to 24 s and no latent-heat limiting is needed. Overall integration efficiency increases ~ 15%, and the IEVA solutions are closer to a benchmark run using smaller dt (12 s) than the HRRR simulation. LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON M dyn_em/Makefile M dyn_em/module_advect_em.F M dyn_em/module_big_step_utilities_em.F M dyn_em/module_em.F A dyn_em/module_ieva_em.F M dyn_em/solve_em.F M run/README.namelist A test/em_real/namelist.input.IEVA.4km TESTS CONDUCTED: 1. This pull requested code has been tested repeatedly on 27 April for a 24 hour simulation with the parameters set as in the Wicker and Skamarock (2020) as well as a 10 step difference test with the em_quarter_ss case using a single node and 4 nodes. The differences in the results ~ 10^-5. 2. Jenkins tests are all pass. RELEASE NOTES: The Implicit-Explicit Vertical Advection (IEVA) scheme has been implemented (see Wicker, L. J., and W. C. Skamarock, 2020: An Implicit–Explicit Vertical Transport Scheme for Convection-Allowing Models. Mon. Wea. Rev., 148, 3893–3910) and that permits a larger time step by partitioning the vertical transport into an explicit piece, which uses the normal vertical schemes present in WRF, and a implicit piece which uses implicit transport (which is unconditionally stable). The combined scheme permits a larger time step than has been previously been used and reduced w-filtering. The scheme will be useful for CONUS-scale CAM (convection allowing model) simulations (dx ~ 2-3 km) when the number of vertical levels > 50. In these cases, time steps can increase to as large as 25 s, depending on the problem. Overall integration efficiency increases ~ 15%, and the IEVA solutions are closer to a benchmark run using smaller time step.
TYPE: bug fix KEYWORDS: idealized fire nested domain simulation SOURCE: Found by Masih Eghdami (NCAR), Pedro Jimenez (NCAR) DESCRIPTION OF CHANGES: Problem: The atmospheric coordinates (lat/lons) are initialized with spatial coordinates, this will create a fatal error when the model is run with nested domains. The error is because lat>90 in share/interp_fcn.F: ``` CALL wrf_error_fatal ( 'Nest over the pole, single input domain, longitudes will be wrong' ) ``` Solution: The lat/lon are initialized with zeros similar to LES idealized cases only if the model has 2 domains to prevent the error. If the model is run with 1 domain, the lat/lon are written with spatial coordinates allowing easier post-processing. ISSUE: This PR addresses the discussion in #1412 with @pedro-jm and @janmandel. LIST OF MODIFIED FILES: module_initialize_fire.F TESTS CONDUCTED: 1. Yes, the nested idealized fire case runs without the error described above. 2. They are passing.
TYPE: enhancement KEYWORDS: GOCART, dust emission, dust_opt, WRF-Chem SOURCE: Alexander Ukhov (KAUST) DESCRIPTION OF CHANGES: This PR affects dust emission schemes (dust_opt=1,3,4) coupled with GOCART aerosol module. Current implementation of dust_opt=1,3 calculates air mass in the grid box using cell area and height, and air density, however a more elegant solution is now implemented (by analogy with dust_opt=4). This solution only requires height of the lowest model layer. Another issue have been also resolved when using dust_opt=1. Due to incorrect indexing in the tile loop dust emissions diagnostics (variables EDUST1…5) were zero near the boundaries of the domain. Each emission scheme (dust_opt=1,3,4) reported instantaneous dust emissions diagnostics (variables EDUST1…5) in different units: dust_opt=1 [kg/cell/dt], dust_opt=3 [gram/m^2/s], dust_opt=4 [ug/m^2/s]. For user convenience this discrepancy also has been fixed. now EDUST1..5 variables store accumulated dust emissions [kg/m^2] for dust_opt=1,3,4. For example, this can be useful when is necessary to estimate dust mass balance. In this case accumulated dust emission needs to be multiplied by cell area (m^2), see Appendix of [1]. Code clean-up has been also done in dust_opt=1,3. LIST OF MODIFIED FILES: M Registry/registry.chem M chem/emissions_driver.F M chem/module_gocart_dust.F M chem/module_gocart_dust_afwa.F M chem/module_qf03.F M chem/module_uoc_dust.F TESTS CONDUCTED: To demonstrate correctness of the implemented changes a numerical simulations of the dust over the Middle East (chem_opt=301 with dust_opt=1,3,4, 100km resolution) with and without changes has been run.
TYPE: bug fix KEYWORDS: IEVA, OpenMP SOURCE: internal DESCRIPTION OF CHANGES: Problem: With IEVA activated, differences appear between OpenMP results with OMP_NUM_THREADS > 1 and any of the following: 1. serial results 2. MPI results 3. OpenMP results with a single thread Solution: Working backwards, the computation in WRF (pre-IEVA code) computed the full MU field only on the mass-point tile size: ``` DO j = jts, jte-1 DO i = its, ite-1 ``` We extend the computation one grid cell to the left and right: ``` DO j = jts-1, jte-1 DO i = its-1, ite-1 ``` Since WRF previously did not use those values, that is not a problem to have additional rows and columns of valid data inside of the halo region. This is a follow-on PR to 4412521 #1373 "Implicit Explicit Vertical Advection (IEVA)". LIST OF MODIFIED FILES: M dyn_em/module_big_step_utilities_em.F TESTS CONDUCTED: 1. I used a simple Jan 2000 case, with 60 levels, 30-km resolution, and a 20*dx time step. This caused calls to `advect_u_implicit` and `advect_v_implicit` in the first time step. Without the mods, the code generated different results depending on the number of OpenMP threads. With the mods, the results are bit-for-bit for OpenMP with the standard y-only decomposition and with a manual x-only decomposition. Below is a figure of the differences of the V field after the first time step (before the modification). This plot is the difference of the same executable using two different OMP_NUM_THREADS values. After the mod, the results are bit-for-bit. <img width="1152" alt="Screen Shot 2021-03-25 at 4 05 09 PM" src="https://user-images.githubusercontent.com/12666234/112549911-291e8e80-8d84-11eb-8b03-1e1ea50ef731.png"> Before the mods, during the first time step, the following diffs were apparent along the OpenMP boundaries: ``` Diffing np=1/wrfout_d01_2000-01-24_12:10:00 np=6/wrfout_d01_2000-01-24_12:10:00 Next Time 2000-01-24_12:10:00 Field Ndifs Dims RMS (1) RMS (2) DIGITS RMSE pntwise max U 49384 3 0.2158843112E+02 0.2158843113E+02 9 0.5344E-05 0.3589E-05 V 61738 3 0.1834835473E+02 0.1834835712E+02 6 0.1045E-03 0.2183E-03 W 139132 3 0.4977466348E-01 0.4977466098E-01 7 0.3382E-05 0.4809E-03 PH 66955 3 0.2327166773E+04 0.2327166753E+04 8 0.1078E-02 0.7572E-05 T 4838 3 0.7925254902E+02 0.7925254902E+02 12 0.9349E-05 0.2484E-05 THM 4812 3 0.7921679023E+02 0.7921679023E+02 12 0.9289E-05 0.2484E-05 MU 1286 2 0.1460135950E+04 0.1460135956E+04 8 0.1203E-02 0.5148E-05 P 6737 3 0.6512715435E+03 0.6512716390E+03 6 0.2086E-01 0.8162E-03 QVAPOR 26582 3 0.2913825518E-02 0.2913825518E-02 9 0.4536E-09 0.5671E-05 QCLOUD 429 3 0.6474288021E-05 0.6474289263E-05 6 0.3257E-09 0.3024E-03 QICE 715 3 0.4136477606E-05 0.4136463263E-05 5 0.1303E-09 0.1757E-03 QNICE 676 3 0.4164261806E+06 0.4164261805E+06 9 0.1341E+00 0.1125E-05 RAINNC 94 2 0.3158246772E-02 0.3158239178E-02 5 0.9447E-07 0.1558E-03 SNOWNC 94 2 0.3158246772E-02 0.3158239178E-02 5 0.9447E-07 0.1558E-03 SR 1 2 0.3353836226E+00 0.3353836226E+00 9 0.9006E-09 0.5960E-07 ``` 2. Wei successfully tested a separate case with 1x16 and 16x1 OpenMP decompositions, where there were bit-for-bit diffs without the mods. 3. Jenkins tests are all PASS.
TYPE: bug fix KEYWORDS: IEVA, tile or memory sized arrays SOURCE: internal DESCRIPTION OF CHANGES: Problem: Several local arrays used in the implicit advection routines are currently dimensioned by memory sizes. However this does not affect model results, whether it is serial, OMP or MPI. Solution: They should be dimensioned by tile sizes for proper memory scaling using OpenMP. LIST OF MODIFIED FILES: M dyn_em/module_ieva_em.F TESTS CONDUCTED: 1. It does not change any results. 2. Jenkins is OK
TYPE: Enhancement KEYWORDS: LSM, PBL, ACM2, P-X LSM, PBLH, Q2 SOURCE: Robert Gilliam and Jonathan Pleim (US EPA) DESCRIPTION OF CHANGES: The purpose of this change is to fix the ACM2 PBL scheme for rare crashes and add evaporation pathway to the Pleim-Xiu LSM. Problem: 1. Bug in the ACM2 PBL scheme caused very infrequent crash in extreme terrain of Himalayan mountains found by a collaborator who modeled Asia and found negative PBL heights being computed at the time of a crash. 2. No evaporation directly from ground in vegetation part of a grid cell. Solution: 1. Modified the PBL height algorithm for stable conditions so that the Richardson number is computed using windspeed in layer k rather than wind speed difference between layer k and ksrc. 2. Added new pathway for evaporation from the ground in the vegetated fraction of the grid cell. LIST OF MODIFIED FILES: M phys/module_bl_acm.F M phys/module_pbl_driver.F M phys/module_sf_pxlsm.F M phys/module_sf_pxsfclay.F TESTS CONDUCTED: 1. Collaborator confirmed the ACM2 fix resolved their crash issues. 2. We've ran full annual simulations over the CONUS at 12 km. We ran 4 and 1 km simulation centered over New York City. 3. Compared statistics and the ACM2 fix has no impact of statistics. 4. New pathway for evaporation from ground in Veg part of grid cells resulted in higher water vapor mixing ratio in a summer case using a 12 km CONUS domain. 5. Jenkins is all PASS RELEASE NOTE: Modifications to the ACM2 PBL scheme for rare crashes and also to add an evaporation pathway to the Pleim-Xiu LSM. The overall effects of these changes are generally small although the indexing fix will prevent model crashes in extreme terrain (e.g. Tibet). The biggest effects on surface statistics was the added evaporation from ground in vegetative fraction of the grid cell. This by nature increase near surface water vapor mixing ratio. In some of our tests this increased the bias in some areas.
TYPE: bug fix KEYWORDS: ntu_init, SNOWNC SOURCE: Tzu-Chin Tsai (National Taiwan University) DESCRIPTION OF CHANGES: Problem: 1. The calculation of air density (RHOA) is divided by zero (producing NaN) at the top full level (k=kte) in start_em.F. 2. Incompatible with the description of SNOWNC in the Registry (ACCUMULATED TOTAL GRID SCALE SNOW AND ICE). 3. Intel19 compiler error #1444 Solution: 1. The derivations of air density (RHOA), pressure (PRES), and height (z_at_q) are moved from the start_em.F to ntu_init through the passing of basic variables (PHB, PH, P, PB, ALT, and QV) from the WRF dynamics. Also, the ending indices of the vertical loop are changed to kte-1. 2. The sedimentation of pristine ice (ICENCV) is appended to the accumulated snow precipitation (SNOWNCV+ ICENCV). 3. The loop for DUST_IN0 has been removed since dust emission is not included. LIST OF MODIFIED FILES: dyn_em/start_em.F phys/module_mp_ntu.F phys/module_physics_init.F TESTS CONDUCTED: 1. Idealized squall_line_2d_x run The initial condition of background aerosols became a bit reduced (QDCN and QTCN were about -10% and QNIN was -5% on average) with the modification in this PR (red line, modify) compared to the results with the previous code (blue line, current). Also, another test made a new loop of kte-1 for RHOA and PRES in the start_em.F (green line, kte-1) for comparison. The below figures are the domain-mean vertical profiles in terms of different background aerosols at the initial time. This difference arose from the misplacement of the basic fields (thickness of the vertical layer, air density, and pressure) by one grid point because of the upside-down loop in ntu_init. In other words, the input fundamental fields were separately from kte to kts+1 and from kte-1 to kts using the previous and modified core. Note that this difference merely exists in the NTU scheme with the upside-down loop. At last, both the results of kte-1 and modify runs are almost identical to each other (the minor difference of 0.1% was from the z_at_w and z_at_q).  PS, The initial value of QDCN is identical to QTCN.
TYPE: text only KEYWORDS: ASCII SOURCE: internal DESCRIPTION OF CHANGES: Remove instances of non Fortran character set entries in the WRF source. All of these changes are comments. 1. The \textemdash becomes "-" 2. The non-American keyboard letters become a, e, i, o, u without the dots or accents 3. The superscript 2 is removed, and replaced by a regular 2: W/m2 4. Extra DOS carriage returns are removed (likely when the files come from windows machines) 5. The degree symbol was removed 6. An extra handy vim command was added LIST OF MODIFIED FILES: modified: chem/chemics_init.F modified: chem/module_mosaic_newnucb.F modified: hydro/Routing/module_gw_gw2d.F modified: phys/module_fr_fire_phys.F modified: phys/module_gocart_coupling.F modified: phys/module_mp_fast_sbm.F modified: phys/module_mp_gsfcgce_4ice_nuwrf.F modified: phys/module_mp_ntu.F modified: phys/module_ra_eclipse.F modified: phys/module_sf_bep_bem.F modified: phys/module_sf_noahmplsm.F modified: share/module_check_a_mundo.F modified: tools/non_ascii_finder.F TESTS CONDUCTED: 1. Text only, no changes to compiled code 2. Jenkins is all PASS
TYPE: bug fix KEYWORDS: IEVA, TLADJ, solve SOURCE: internal DESCRIPTION OF CHANGES: Problem: After the IEVA mods (commit 4412521, #1373 "Implicit Explicit Vertical Advection (IEVA)"), which changed the calls to rk_tendency and rk_scalar_tend, the WRFPlus code no longer compiled. Solution: The new arguments added to the calls to rk_tendency and rk_scalar_tend have been added inside the solve routine for WRFPlus. LIST OF MODIFIED FILES: wrftladj/solve_em_ad.F TESTS CONDUCTED: 1. Without mods, there are compiler errors from missing args. After the mods: ``` > ls -ls main/*.exe 94944 -rwxr-xr-x 1 gill p66770001 97217616 Mar 29 10:11 main/wrfplus.exe ``` 2. The WRFDA regtest is OK. 3. Jenkins is all PASS.
cenlinhe
pushed a commit
that referenced
this pull request
Mar 17, 2022
TYPE: bug fix KEYWORDS: unary operator, Noahmp SOURCE: Internal DESCRIPTION OF CHANGES: Problem: Found more occurrences of `a* -b` in Noahmp. A space is in between `*` and `-1`. OK even for Cray, but it is still non-standard. Solution: The same solution is applied: add parentheses: `a * (-b)`. ASSOCIATED REPOSITORY CHANGE: NCAR/noahmp#33 LIST OF MODIFIED FILES: modified: phys/noahmp TESTS CONDUCTED: 1. More code is fixed. 2. Jenkins tests are all passing.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The first line should be a single-line "purpose" for this change
TYPE: choose one of [bug fix, enhancement, new feature, feature removed, no impact, text only]
KEYWORDS: 5 to 10 words related to commit, separated by commas
SOURCE: Either "developer's name (affiliation)" .XOR. "internal" for a WRF Dev committee member
DESCRIPTION OF CHANGES:
Problem:
Generally or specifically, what was wrong and needed to be addressed?
Solution:
What was down algorithmically and in the source code to address the problem?
ISSUE: For use when this PR closes an issue.
Fixes wrf-model#123
LIST OF MODIFIED FILES: list of changed files (use
git diff --name-status masterto get formatted list)TESTS CONDUCTED:
RELEASE NOTE: Include a stand-alone message suitable for the inclusion in the minor and annual releases. A publication citation is appropriate.