US EPA PX LSM updates #733
Conversation
| @@ -893,6 +893,13 @@ state real T2OBS ij misc 1 - r "T | |||
| state real Q2OBS ij misc 1 - r "Q2OBS" "2-m mixing ratio from analysis " "kg/kg" | |||
| state real IMPERV ij misc 1 - i012rhd=(interp_mask_field:lu_index,iswater)u=(copy_fcnm) "IMPERV" "Impervious surface fraction NLCD" "percent" | |||
| state real CANFRA ij misc 1 - i012rhd=(interp_mask_field:lu_index,iswater)u=(copy_fcnm) "CANFRA" "Satellite canopy fraction" "percent" | |||
| state real LAI_PX ij misc 1 - rh "LAI_PX" "LAI used for PX LSM" "m2/m2" | |||
There was a problem hiding this comment.
@coastwx
I know that these are only 2d arrays, but can we package these? We can base it on pxlsm_modis_veg alone, but that would mean adding in a test in share/module_check_a_mundo.F to make sure that the PX scheme is active if pslsm_modis_veg =1.
| @@ -2245,6 +2252,7 @@ rconfig integer shcu_aerosols_opt namelist,physics max_domains 0 | |||
|
|
|||
| rconfig integer ICLOUD_CU derived max_domains 0 - "ICLOUD_CU" "" "" | |||
| rconfig integer pxlsm_smois_init namelist,physics max_domains 1 irh "PXLSM_SMOIS_INIT" "Soil moisture initialization option 0-From analysis 1-From MAVAIL" "" | |||
| rconfig integer pxlsm_modis_veg namelist,physics max_domains 0 irh "PXLSM_MODIS_VEG" "If 1 use MODIS VEGFRA from wrflowinp updates, 1=yes" "" | |||
There was a problem hiding this comment.
@coastwx
Have you tested a nested scenario where one domain used the data from the wrflowinp file and another domain did not? If this is not a recommended scenario, a test in check_a_mundo should be added. You could either make all of the domains the same as d01 (silently), or you could issue a fatal error.
| @@ -714,6 +714,11 @@ SUBROUTINE first_rk_step_part1 ( grid , config_flags & | |||
| & ,q2_ndg_new=grid%q2_ndg_new & | |||
| & ,sn_ndg_old=grid%sn_ndg_old & | |||
| & ,sn_ndg_new=grid%sn_ndg_new & | |||
| & ,pxlsm_modis_veg=config_flags%pxlsm_modis_veg & | |||
There was a problem hiding this comment.
@coastwx
Was there ever an expectation of NMM using this scheme? We can go several ways: ifdefs and optional fields and present statements, or mods to the NMM Registry and changes to the NMM module_PHYSICS_CALLS.F
|
@coastwx This option is compilable with NMM. Hence changes need to go into NMM registry and dyn_nmm/module_PHYSICS_CALLS.F. |
|
Hey Dave/Wei, I will address these today. We have a triple nested domain setup where we can do a real test of the nesting scenario. I'm quite confident it will work fine, but we neglected to include this in our test. I'll test of course all domains using the new modis veg option and a mix and match of one or two with and without. We have never used NMM and I'm not aware of any PX LSM runs using the NMM core, but if the current call to PX in NMM is in place like Wei says, I'll modify the driver. I'll also recompile a NMM version and do a quick test. This may take an extra day or two. I'm not as familiar with the packaging concept. Is this a condition where if a certain option is used, a set of variables that only applied to that option are output? If this is the case, I think basing a package on the PX LSM option would be better where VEGF_PX, LAI_PX, RA, RS, T2OBS, Q2OBS, IMPERV, CANFRA and maybe a couple more that I'm forgetting off the top of my head could be included. Will start on some of these tests right now. |
|
@coastwx The idea of packaging is that the memory as well as the output associated with packaged arrays won't be there if the option is not used. |
|
Ok, on the nesting scenario... I tested nesting and it works for all combinations of pxlsm_modis_veg. Both domains set to 0, both 1, and when the two domains are set differently. While in most cases, this modis setting should probably be the same, we do not want to force it. No check in check_a_mundo needed. On the packaging of variables. We would like to do packaging, but base it off the sf_surface_physics = 7 case. This would help CMAQ users by ensuring variables like RA, RS, LANDUSEF, etc are available in the model outputs. In some cases here a variable like LANDUSEF, ZNT, RA/RS were not in the output and that caused issues. I just need to know how to proceed. I'll start looking into this now, but as I type this, not sure how this is done. NMM. I looked into the NMM physics driver. It has never been updated for the PX LSM variables except very early on by someone other than the EPA. There are many variables that are not in the surface driver call that PX needs. Unless there is a new need that all physics work in both cores, we do not care if it's compatible. Since it has never worked with NMM, it would take some effort to test. So basically, all we need to resolve is how to package variables for cases where the PX LSM is employed. |
|
@coastwx There are some "derived" namelist variables that we employ if we need to actually have a variable be a function of two (or more) if tests based on namelist options. If you are running into such a scenario, let us know what you need and we can help you out. |
|
@coastwx Regarding NMM compile, have you tried it? It may not run with NMM, but we need it to compile with NMM. Give it a try. If it compiles, then that's great. If not, we need to do something so that it will compile. |
@weiwangncar Yes it does compile fine. Both real_nmm and wrf.exe. I'll consider this issue closed. |
@davegill I don't think we need tests. I have looked into this package idea. Looks like this is defined in the Registry file. I found this: package pxsfcscheme sf_sfclay_physics==7 Do we just add the variables that are unique in PX like the noahmpscheme package is done? Should the variable definitions in Registry.EM_COMMON for PX be moved into this Registry file where packages are define? Will be able to test this myself today and submit a PR. |
|
@coastwx You want to replace the last dash with something like: This is known as "packaging". The variables listed on this line are only and always available for input, output, communication, or computation when the sf_sfclay_physics option is set. The important piece for the rest of the model is that these variables do not contribute to the overall memory footprint if this particular namelist option is not selected. |
|
My apologies. I see this in the Registry.EM_COMMON file. Think your original question regarding packages was probably about adding these new variables (LAI_PX, WWLT_PX, etc) to this package definition. If this is correct just let me know asap and I'll fix and issue another pull request for the Registry file only (if that's the proper protocol). package pxlsmscheme sf_surface_physics==7 - state:t2_ndg_new,q2_ndg_new,t2_ndg_old,q2_ndg_old,vegf_px,imperv,canfra |
|
@coastwx For a package statement in the Registry, you may safely add any variable that is unique to a single particular scheme. If a variable is used by a few schemes, we really should include that variable in a package statement for each of those options. For example, this is what we do with fields like graupel, where several microphysics schemes require that field. This packaging is the part that is what causes troubles with nesting. However, none of your new fields have any nesting options, so there won't be a scenario where domain 2 tries to feed back a field to domain 1, and that field does not exist on domain 1. If the fields that you listed (t2_ndg_new, q2_ndg_new, t2_ndg_old, q2_ndg_old, vegf_px, imperv, canfra) are unique to the PX scheme, absolutely bundle those up in the package as well. |
|
No need for another PR. Simply modify your local sandbox copy, git add, git commit, and push it again. The new mods will appear. |
|
Perfect! I will do that. Just need these new variables. Thanks again for the help. The fog is lifting on this Git-Hub stuff.
|
Added new PX LSM variables to pxlsmscheme package definition
|
@coastwx Thanks for testing NMM compile. |
|
@weiwangncar |
|
I'm ok with this PR. |
|
@davegill @weiwangncar |
TYPE: bug fix KEYWORDS: bug, PX, RA, RS, package, Noah SOURCE: internal DESCRIPTION OF CHANGES: Problem: After commit sha eb79235 (PR wrf-model#733 US EPA PX LSM updates), most uses of sf_surface_physics=2 would fail in the first time step with messages that indicated some memory corruption: ``` wrf.exe(68190,0x7fff9be48380) malloc: *** error for object 0x7f8fdbc0db78: incorrect checksum for freed object - object was probably modified after being freed. ``` or a horrific number of CFLs that tend to not happen at the beginning of the WRF model simulation when given a reasonable IC: ``` 70631 points exceeded cfl=2 in domain d01 at time 2000-01-24_12:00:00 hours MAX AT i,j,k: 40 13 48 vert_cfl,w,d(eta)= 3.81348181E+18 -8.02897697E+15 4.13541310E-03 ``` Solution: A possible solution was to go ahead an include physics packages that permit super-light speeds, but that time consuming. Alternatively, it was noticed that the developers included RA and RS as part of the package for the sf_surfce_physics==7 (PXLSMSCHEME). These variables are not unique to the PX LSM scheme, and were part of the argument list for Noah LSM. Removing RA and RS from the PXLSMPSCHEME list is a correction that fits the symptomology of the reported errors. LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON TESTS CONDUCTED: 1. Serial, OpenMP, and MPI with sf_surface_physics=2 all fail in the first time step, without this mod. 2. All three build options with gnu/6.3.0 work when including this mod.
TYPE: bug fix KEYWORDS: bug, PX, RA, RS, package, Noah SOURCE: internal DESCRIPTION OF CHANGES: Problem: After commit sha eb79235 (PR #733 US EPA PX LSM updates), most uses of sf_surface_physics=2 would fail in the first time step with messages that indicated some memory corruption: ``` wrf.exe(68190,0x7fff9be48380) malloc: *** error for object 0x7f8fdbc0db78: incorrect checksum for freed object - object was probably modified after being freed. ``` or a horrific number of CFLs that tend to not usually happen at the beginning of the WRF model simulation when given a reasonable IC: ``` 70631 points exceeded cfl=2 in domain d01 at time 2000-01-24_12:00:00 hours MAX AT i,j,k: 40 13 48 vert_cfl,w,d(eta)= 3.81348181E+18 -8.02897697E+15 4.13541310E-03 ``` Solution: A possible solution was to go ahead and include physics packages that permitted super-light speeds, but that seemed time consuming and Star Fleet derivative. Alternatively, it was noticed that the developers included RA and RS as part of the package for the sf_surface_physics==7 (PXLSMSCHEME). These variables are not unique to the PX LSM scheme, and were part of the argument list for Noah LSM. Removing RA and RS from the PXLSMPSCHEME list of packaged variables is a correction that fits the symptomology of the reported errors. LIST OF MODIFIED FILES: M Registry/Registry.EM_COMMON TESTS CONDUCTED: 1. Serial, OpenMP, and MPI with sf_surface_physics=2 all fail in the first time step, without this mod. 2. All three build options with gnu/6.3.0 work when including this mod.
|
Thank you all for the help. We just got off furlough, so apologies for not responding to the "PR relate to Issue #123". Happy to see this PR was merged. |
TYPE: Enhancement (P-X LSM Option 7)
KEYWORDS: PX LSM, MODIS, VEGFRA, LAI, Soil
SOURCE: Jon Pleim, Limei Ran and Robert Gilliam, US EPA
DESCRIPTION OF CHANGES: Added vegetation and leaf-area index option for Pleim-Xiu land-surface runs. Until this version, the PX LSM uses VEGFRA and LAI computed from the module_sf_pxlsm_data.F PX data table. This uses fractional landuse and these lookup values to compute the LAI and VEGFRA for each grid cell. The new option (pxlsm_modis_veg = 1) is activated using this option in the physics section of the namelist.input file. It uses the time-varying VEGFRA and LAI from the wrflowinp_d01 file instead of the look-up values in the PX data table. This allows use of more accurate high resolution MODIS that is now available in WPS in WRFv4+. Alternatively, users can process their own MODIS data for specific years and put in this same input file.
Also, the soil calculation in the PX LSM were modified to use analytical functions from Noilhan and Mahfouf (1996) for field capacity, saturation and wilting point based on fractional soil data. Also, variables for fractional clay, fine and coarse sand were added in PX for output to the CMAQ air quality model. This is an important update because these data are used for dust emissions in the air quality model along with the new soil properties (wilting, saturation and field capacity). SOILTYP was also updated in PX LSM so soil classes are consistent with the standard 16 soil types in the WRF system. Prior, PX only had 12 classes and classes 4-12 were not the same as those classes used by other LSMs.
LIST OF MODIFIED FILES
dyn_em/module_first_rk_step_part1.F
phys/module_sf_pxlsm.F
phys/module_surface_driver.F
run/README.namelist
Registry/Registry.EM_COMMON
RELEASE NOTE: The new option (pxlsm_modis_veg = 1) was added in the physics section of the namelist file. When activated, this option uses the time-varying VEGFRA and LAI from the wrflowinp_d01 file instead of the look-up values in the PX data table. Users are encouraged to use this option if they are using WPS4+ that has the higher resolution MODIS greenfrac dataset. Also, the soil calculation in the PX LSM were modified to use analytical functions from Noilhan and Mahfouf (1996) for field capacity, saturation and wilting point based on fractional soil data. Soil categories were updated in PX to 16 class consistant with the WRF system and other LSMs.
TEST CONDUCTED: A supplemental document details the tests we done for this change. In brief, we ran an April-July simulations using updates dropped in WRFv4.0. Test shows the soil change had the most impact followed by the MODIS vegetation option. In both cases, the model's lower level temperature and moisture were improved. Much more detailed simulations were done using WRFv3.9.1 and published in several manuscripts.
Ran, L., R. Gilliam, F. S. Binkowski, A. Xiu, J. Pleim, and L. Band, 2015. Sensitivity of the WRF/CMAQ modeling system to MODIS LAI, FPAR, and albedo, J. Geophys. Res. Atmos., 120(16), 8491-8511, https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2015JD023424.
Ran, L., J. Pleim, R. Gilliam, F. S. Binkowski, C. Hogrefe, and L. Band, 2016. Improved meteorology from an updated WRF/CMAQ modeling system with MODIS vegetation and albedo, J. Geophys. Res. Atmos., 121, 2393–2415, https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2015JD024406.
Ran, L., R. Gilliam, D. Wong, H. Foroutan, J. Pleim, G. Pouliot, W. Appel, D. Kang, S. Roselle, B. Eder, E. Cooter (2017), Advanced Land Surface Processes in the Coupled WRF/CMAQ with MODIS Input, 16th Annual Community Modeling and Analysis (CMAS) Conference, Chapel Hill, NC, October 23-25 (Oral, https://www.cmascenter.org/conference//2017/slides/ran_advanced_land_2017.pptx).