Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix to allow use_theta_m=1 with vertical nesting and simplify rebalance options #1408

Merged
merged 5 commits into from
Feb 23, 2021

Conversation

rsarthur
Copy link
Contributor

@rsarthur rsarthur commented Feb 19, 2021

TYPE: bug fix

KEYWORDS: vertical nesting, vert_refine_method, use_theta_m, rebalance

SOURCE: Robert Arthur and Katie Lundquist, LLNL

DESCRIPTION OF CHANGES:
Problem:

  1. When vertical nesting is used with the moist theta option (use_theta_m=1), large errors occur in the vertically-
    refined domain. This is the main problem addressed in this PR.
  2. Another minor issue is that rebalance option 2 is redundant, and is therefore unnecessary.

Solution:

  1. Several use_theta_m flags were added to the rebalancing routines in start_em.F and module_dm.F, which are
    called when vertical nesting is used. Thus, the rebalanced pressure, inverse density, and geopotential fields are
    calculated correctly and consistently with the rest of the code, regardless of whether use_theta_m=0 or 1.
  2. Rebalance option 2 has been removed. Now, rebalance must be set to 1 when vertical nesting is turned on.

LIST OF MODIFIED FILES:
Registry/Registry.EM_COMMON
dyn_em/module_initialize_real.F
dyn_em/start_em.F
external/RSL_LITE/module_dm.F
share/module_check_a_mundo.F

TESTS CONDUCTED:

  1. A 4-domain nested case was completed with vertical refinement on d04. All domains were initialized from a
    wrfinput file and eta levels were not provided. Pertinent namelist options are:

input_from_file = .true., .true., .true., .true.,
e_vert = 45, 45, 45, 55,
vert_refine_method = 0, 0, 0, 2,
rebalance = 1,
hybrid_opt = 0,
use_theta_m = 0 or 1, depending on the case

Results are summarized in the plot below, which shows U at the first grid point above the surface on d04 at
various output times.
vnest_fix_use_theta_m
With the original code, vertical nesting works with use_theta_m=0, but not use_theta_m=1. With the updated code,
results look reasonable for use_theta_m=1 and are roughly the same as for use_theta_m=0. Similar patterns are also
seen for other variables, such as V and MU, although they are not shown here.

  1. Several similar tests were also conducted, as above but with
    • d04 initialized via interpolation from d03 during runtime, rather than from wrfinput_d04
    • eta levels specified for each domain in the namelist

These tests provided qualitatively similar results to the above, showing that the fix works for a range of possible
domain setups.

  1. Finally, a test was completed with use_theta_m=0, but using the updated code. This showed bit-for-bit identical
    results (using diffwrf) to the same case using the original code, confirming that previous vertical nesting functionality
    with use_theta_m=0 is maintained.

RELEASE NOTE: Vertical refinemnet now works with the moist theta option use_theta_m=1. The redundant "rebalance" option 2 was removed. The rebalance option must now be set to 1 when vertical refinement is turned on.

…ion 1 must be used with vertical nesting (affecting start_em, module_initialize_real, module_check_a_mundo, and Registry.EM_COMMON). Results match KAL tests for hybrid_opt 0 and 2 with use_theta_m=0.
…m and module_dm. This seems to work, but there is a spurious perturbation on vnest domain after 1 minute that goes away after 2 minutes and beyond. Still some discrepancies in rebalancing to look into.
…was not being recognized in the previous commit (needed to use grid% instead of config_flags%). This fixes the spurious perturbation and the hybrid_opt=0, use_theta_m=1 case looks good now.
@rsarthur rsarthur requested review from a team as code owners February 19, 2021 00:49
@davegill davegill changed the base branch from master to develop February 19, 2021 14:09
@davegill davegill merged commit ca6c977 into wrf-model:develop Feb 23, 2021
@weiwangncar
Copy link
Collaborator

@rsarthur How does vertical nesting relate to module_initialize_real.F? Is initialization of vertical nesting done in real.exe?

@rsarthur
Copy link
Contributor Author

@weiwangncar
Yes, if input_from_file=true for a domain with vertical nesting, then vert_refine_method is used in the specification of eta levels for that domain in module_initialize_real. Rebalancing is also performed in module_initialize_real.

If input_from_file=false and vertical nesting is turned on for a given domain, then the vertically-nested domain is interpolated from the parent during runtime. In this case, the eta levels are specified using the vert_cor_vertical_nesting_* subroutines in dyn_em/nest_init_utils.F.

@weiwangncar
Copy link
Collaborator

@rsarthur Thanks!

davegill pushed a commit that referenced this pull request Mar 22, 2021
)

TYPE: bug fix

KEYWORDS: vertical nesting, hybrid vertical coordinate, vert_refine_method, hybrid_opt

SOURCE: Robert Arthur and Katie Lundquist (LLNL)

DESCRIPTION OF CHANGES:
Problem:
When vertical nesting is used with the hybrid vertical coordinate option (hybrid_opt=2), large errors occur in the 
vertically-refined domain.

Solution:
An incorrect calculation of pb (base pressure) was fixed in subroutine force_domain_em_part2 (module_dm.F). The pb 
calculation was changed to match similar code in start_em. This fix is within a block of code that is only called if 
vert_refine_method .ne. 0, in order to rebalance on the "intermediate" nested grid. The error was likely a result of the 
switch to the new hybrid coordinate definition, which was not updated correctly in this particular subroutine.

The check-a-mundo preventing vertical nesting from being used with the hybrid coordinate (see #1154) has also 
been removed.

This PR is related to #901, which addressed vertical nesting with the hybrid coordinate (but did not fully fix the issue), 
as well as #1408, which addressed vertical nesting with use_theta_m=1. 

LIST OF MODIFIED FILES:
external/RSL_LITE/module_dm.F
share/module_check_a_mundo.F

TESTS CONDUCTED: 
1. The tests conducted here are similar to those in #1408. A 4-domain nested case was completed with vertical 
refinement on d04. All domains were initialized from a wrfinput file and eta levels were not provided. Pertinent 
namelist options are:
```
input_from_file = .true., .true., .true., .true.,
e_vert = 45, 45, 45, 55,
vert_refine_method = 0, 0, 0, 2,
rebalance = 1,
hybrid_opt = 0 or 2, depending on the case
use_theta_m = 0
```

Results are summarized in the plot below, which shows U at the first grid point above the surface on d04 at various 
output times.

![U](https://user-images.githubusercontent.com/46732079/111347582-958ae680-863c-11eb-9536-1862a1b650fe.png)

With the original code, vertical nesting works with hybrid_opt=0, but not hybrid_opt=2. With the updated code, results 
look reasonable for hybrid_opt=2 and are roughly the same as for hybrid_opt=0. Similar patterns are also seen for other 
variables, such as V and MU, although they are not shown here.

2. Several similar tests were also conducted, as above but with
   * d04 initialized via interpolation from d03 during runtime, rather than from wrfinput_d04
   * eta levels specified for each domain in the namelist
   * use_theta_m=1

These tests provided qualitatively similar results, showing that the fix works for a range of possible setups.

3. Finally, the tests above were completed with hybrid_opt=0, but using the updated code. These showed bit-for-bit 
identical results (using diffwrf) to the same cases using the original code, confirming that previous vertical nesting 
functionality with hybrid_opt=0 is maintained. This was done for both use_theta_m options, 0 and 1.

4. Jenkins testing is OK

RELEASE NOTE: Vertical nesting now works with the hybrid vertical coordinate.
vlakshmanan-scala pushed a commit to scala-computing/WRF that referenced this pull request Apr 4, 2024
…ce options (wrf-model#1408)

TYPE: bug fix

KEYWORDS: vertical nesting, vert_refine_method, use_theta_m, rebalance

SOURCE: Robert Arthur and Katie Lundquist, LLNL

DESCRIPTION OF CHANGES:
Problem:
1. When vertical nesting is used with the moist theta option (use_theta_m=1), large errors occur in the vertically-
refined domain. This is the main problem addressed in this PR.
2. Another minor issue is that rebalance option 2 is redundant, and is therefore unnecessary.

Solution:
1. Several use_theta_m flags were added to the rebalancing routines in start_em.F and module_dm.F, which are 
called when vertical nesting is used. Thus, the rebalanced pressure, inverse density, and geopotential fields are 
calculated correctly and consistently with the rest of the code, regardless of whether use_theta_m=0 or 1.
2. Rebalance option 2 has been removed. Now, rebalance must be set to 1 when vertical nesting is turned on.

LIST OF MODIFIED FILES: 
Registry/Registry.EM_COMMON
dyn_em/module_initialize_real.F
dyn_em/start_em.F
external/RSL_LITE/module_dm.F
share/module_check_a_mundo.F

TESTS CONDUCTED: 
1. A 4-domain nested case was completed with vertical refinement on d04. All domains were initialized from a 
wrfinput file and eta levels were not provided. Pertinent namelist options are:

input_from_file        = .true., .true., .true., .true.,
e_vert                      =  45,     45,     45,     55,
vert_refine_method = 0,        0,       0,       2,
rebalance                = 1,
hybrid_opt               = 0,
use_theta_m            = 0 or 1, depending on the case

Results are summarized in the plot below, which shows U at the first grid point above the surface on d04 at 
various output times.
![vnest_fix_use_theta_m](https://user-images.githubusercontent.com/46732079/108439600-7304d980-7206-11eb-99e5-269b06f04b1e.png)
With the original code, vertical nesting works with use_theta_m=0, but not use_theta_m=1. With the updated code, 
results look reasonable for use_theta_m=1 and are roughly the same as for use_theta_m=0. Similar patterns are also 
seen for other variables, such as V and MU, although they are not shown here.

2. Several similar tests were also conducted, as above but with
   * d04 initialized via interpolation from d03 during runtime, rather than from wrfinput_d04
   *  eta levels specified for each domain in the namelist

These tests provided qualitatively similar results to the above, showing that the fix works for a range of possible 
domain setups.

3. Finally, a test was completed with use_theta_m=0, but using the updated code. This showed bit-for-bit identical 
results (using diffwrf) to the same case using the original code, confirming that previous vertical nesting functionality 
with use_theta_m=0 is maintained.

RELEASE NOTE: Vertical refinemnet now works with the moist theta option use_theta_m=1. The redundant "rebalance" option 2 was removed. The rebalance option must now be set to 1 when vertical refinement is turned on.
vlakshmanan-scala pushed a commit to scala-computing/WRF that referenced this pull request Apr 4, 2024
…f-model#1435)

TYPE: bug fix

KEYWORDS: vertical nesting, hybrid vertical coordinate, vert_refine_method, hybrid_opt

SOURCE: Robert Arthur and Katie Lundquist (LLNL)

DESCRIPTION OF CHANGES:
Problem:
When vertical nesting is used with the hybrid vertical coordinate option (hybrid_opt=2), large errors occur in the 
vertically-refined domain.

Solution:
An incorrect calculation of pb (base pressure) was fixed in subroutine force_domain_em_part2 (module_dm.F). The pb 
calculation was changed to match similar code in start_em. This fix is within a block of code that is only called if 
vert_refine_method .ne. 0, in order to rebalance on the "intermediate" nested grid. The error was likely a result of the 
switch to the new hybrid coordinate definition, which was not updated correctly in this particular subroutine.

The check-a-mundo preventing vertical nesting from being used with the hybrid coordinate (see wrf-model#1154) has also 
been removed.

This PR is related to wrf-model#901, which addressed vertical nesting with the hybrid coordinate (but did not fully fix the issue), 
as well as wrf-model#1408, which addressed vertical nesting with use_theta_m=1. 

LIST OF MODIFIED FILES:
external/RSL_LITE/module_dm.F
share/module_check_a_mundo.F

TESTS CONDUCTED: 
1. The tests conducted here are similar to those in wrf-model#1408. A 4-domain nested case was completed with vertical 
refinement on d04. All domains were initialized from a wrfinput file and eta levels were not provided. Pertinent 
namelist options are:
```
input_from_file = .true., .true., .true., .true.,
e_vert = 45, 45, 45, 55,
vert_refine_method = 0, 0, 0, 2,
rebalance = 1,
hybrid_opt = 0 or 2, depending on the case
use_theta_m = 0
```

Results are summarized in the plot below, which shows U at the first grid point above the surface on d04 at various 
output times.

![U](https://user-images.githubusercontent.com/46732079/111347582-958ae680-863c-11eb-9536-1862a1b650fe.png)

With the original code, vertical nesting works with hybrid_opt=0, but not hybrid_opt=2. With the updated code, results 
look reasonable for hybrid_opt=2 and are roughly the same as for hybrid_opt=0. Similar patterns are also seen for other 
variables, such as V and MU, although they are not shown here.

2. Several similar tests were also conducted, as above but with
   * d04 initialized via interpolation from d03 during runtime, rather than from wrfinput_d04
   * eta levels specified for each domain in the namelist
   * use_theta_m=1

These tests provided qualitatively similar results, showing that the fix works for a range of possible setups.

3. Finally, the tests above were completed with hybrid_opt=0, but using the updated code. These showed bit-for-bit 
identical results (using diffwrf) to the same cases using the original code, confirming that previous vertical nesting 
functionality with hybrid_opt=0 is maintained. This was done for both use_theta_m options, 0 and 1.

4. Jenkins testing is OK

RELEASE NOTE: Vertical nesting now works with the hybrid vertical coordinate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants