Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing simulated radar reflectivity #1513

Closed
WalterKolczynski-NOAA opened this issue Apr 25, 2023 · 35 comments · Fixed by #2143
Closed

Missing simulated radar reflectivity #1513

WalterKolczynski-NOAA opened this issue Apr 25, 2023 · 35 comments · Fixed by #2143
Assignees
@WalterKolczynski-NOAA @RuiyuSun
Labels
bug Something isn't working

Comments

@WalterKolczynski-NOAA
Copy link
Contributor

Expected behavior
Output files should have valid simulated reflectivity (REFC and REFD), or it should not be a grib record.

Current behavior
The master file contains undefined fields for REFC and REFD.

Machines affected
All

To Reproduce
Run any experiment in workflow.

Context
The undefined variables were first identified by @TerrenceMcGuinness-NOAA during CI testing build-out. Additional undefined variables he found are resolved by #1509.

Detailed Description

Orion-03-09 master/>wgrib2 -stats -var gfs.t00z.master.grb2f024 | grep -P 'undef=73728'
7:194243:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD
8:203638:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD
9:213033:ndata=73728:undef=73728:mean=0:min=0:max=0:REFC
822:49446144:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD
823:49455539:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD

All master files (and grib files derived from it) have the same missing fields.

Possible Implementation
Either add the needed fields or, if GFS is not intended to produce simulated reflectivity, stop producing grib records for it.
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the bug Something isn't working label Apr 25, 2023
Merged
9 tasks
@WalterKolczynski-NOAA
Copy link
Contributor Author

Looks like we need this line in our diag tables to add radar reflectivity:
"gfs_phys", "refl_10cm" "refl_10cm" "fv3_history", "all", .false., "none", 2

However, this is not in any of the diag_tables in UFS for global except diag_table_thompson (not gfsv17 or any of the others). Can we get confirmation whether we should be producing simulated reflectivity? If we don't need to produce it, we need to stop listing the field in the grib files. @aerorahul

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA I added the following line to the diag_table, but the master files still have missing values for REFC and REFD:

"gfs_phys",    "refl_10cm"    "refl_10cm"  "fv3_history",    "all",  .false.,  "none",  2

Is there something I'm doing obviously wrong?

@WenMeng-NOAA
Copy link
Contributor

@WenMeng-NOAA I added the following line to the diag_table, but the master files still have missing values for REFC and REFD:

"gfs_phys",    "refl_10cm"    "refl_10cm"  "fv3_history",    "all",  .false.,  "none",  2

Is there something I'm doing obviously wrong?

I will look into my test and get back to you later. @WalterKolczynski-NOAA

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA Please check type of MP. The UPP read refl_10cm from model output when MP is Thompson. The 3D radar reflectivity is calculated in UPP for GFDL MP.

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA We're using Thompson.

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA Can you provide me with your model history files either on Hera or WCOSS2?

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA /scratch2/NCEPDEV/ensemble/noscrub/Walter.Kolczynski/for_wen

@WenMeng-NOAA
Copy link
Contributor

@WenMeng-NOAA /scratch2/NCEPDEV/ensemble/noscrub/Walter.Kolczynski/for_wen

@WalterKolczynski-NOAA Can you give me reading permission?

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA Done

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA With your model history files, my testing indicate radar reflectivity can be read and written out from UPP. Please see /scratch1/NCEPDEV/stmp2/Wen.Meng/post_gfs_2021122100-before.

[Wen.Meng@hfe05 post_gfs_2021122100-before]$ wgrib2 GFSPRS.GrbF06 -match REF -stats -s
7:186454:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:1 hybrid level:6 hour fcst:
8:186633:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:2 hybrid level:6 hour fcst:
9:186812:ndata=73728:undef=73728:mean=0:min=0:max=0:d=2021122100:REFC:entire atmosphere (considered as a single layer):6 hour fcst:
822:48649587:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:4000 m above ground:6 hour fcst:
823:48649766:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:1000 m above ground:6 hour fcst:

I will add a fix for outputting whole column radar reflectivity for GFS with Thompson MP in my upcoming UPP PR.

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA Will there be any changes needed in workflow beyond updating the UFS version?

@WalterKolczynski-NOAA WalterKolczynski-NOAA added the blocked Issue is currently being blocked by another issue label Jun 15, 2023
@WenMeng-NOAA
Copy link
Contributor

@WenMeng-NOAA Will there be any changes needed in workflow beyond updating the UFS version?

I don't think so as long as refl_10cm is available in model output. @WalterKolczynski-NOAA

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA Can you link the relevant PRs here so we can track the blockers?

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA Sure. I will do that when the PR is submitted.

@WenMeng-NOAA WenMeng-NOAA mentioned this issue Jun 23, 2023
Closed
@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA The UPP develop branch was updated with the PR #736 to output column radar reflectivity when GFS uses Thompson MP. This UPP revision will be implemented in ufs-weather-model side with my upcoming UFS PR #1794. Please stay tuned.

@WalterKolczynski-NOAA
Copy link
Contributor Author

Thanks for keeping us updated.

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA My UFS PR #1794 was merged in the ufs-weather-model develop branch at revision 506e532 on 07/27/2023. Please update the UPP revision in global-workflow and test for inline and offline post.

@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the blocked Issue is currently being blocked by another issue label Aug 8, 2023
@WalterKolczynski-NOAA
Copy link
Contributor Author

After updating the UFS version and the diag_table, I'm still seeing the same undefined variables:

Orion-18-02 master/>wgrib2 -stats -var gfs.t00z.master.grb2f024 | grep -P 'undef=73728'
7:204842:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD
8:214237:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD
9:223632:ndata=73728:undef=73728:mean=0:min=0:max=0:REFC
822:49910589:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD
823:49919984:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD

UFS is at 5aeeffd (current develop).

Saved the run directory: /work2/noaa/stmp/wkolczyn/RUNDIRS/refl_cyc/fcst.381922
Log: /work2/noaa/global/wkolczyn/noscrub/global-workflow/refl_cyc/logs/2021122100/gfsfcst.log

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA Please change access permission of the run directory for me.

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA should be open now

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA Still get permission denied.

[wmeng@Orion-login-4 ~]$ cd /work2/noaa/stmp/wkolczyn/
-bash: cd: /work2/noaa/stmp/wkolczyn/: Permission denied

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA Try again.

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA I can't check files at /work2/noaa/stmp/wkolczyn/RUNDIRS/refl_cyc/fcst.381922 since they are linked to /work2/noaa/global/wkolczyn/noscrub/global-workflow/refl_cyc/gfs.20211221/00//model_data/atmos/.

[wmeng@Orion-login-1 fcst.381922]$ ls -l  /work2/noaa/global/wkolczyn/noscrub/global-workflow/refl_cyc/gfs.20211221/00//model_data/atmos/history/gfs.t00z.atmf024.nc
ls: cannot access /work2/noaa/global/wkolczyn/noscrub/global-workflow/refl_cyc/gfs.20211221/00//model_data/atmos/history/gfs.t00z.atmf024.nc: Permission denied
[wmeng@Orion-login-1 fcst.381922]$ ls /work2/noaa/global/wkolczyn/noscrub/global-workflow/refl_cyc/gfs.20211221/00//model_data/atmos/master/gfs.t00z.master.grb2f024
ls: cannot access /work2/noaa/global/wkolczyn/noscrub/global-workflow/refl_cyc/gfs.20211221/00//model_data/atmos/master/gfs.t00z.master.grb2f024: Permission denied

@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA Try one more time. I've modified my .bashrc so hopefully this will cease being an issue in the future.

@WenMeng-NOAA
Copy link
Contributor

WenMeng-NOAA commented Aug 10, 2023 •
edited
Loading

@WalterKolczynski-NOAA I ran the UPP standalone test with model files from your run directory /work2/noaa/stmp/wkolczyn/RUNDIRS/refl_cyc/fcst.381922.
My test results are at /home/wmeng/stmp2/gfs.2021122100.

[wmeng@Orion-login-2 gfs.2021122100]$ wgrib2 gfs.t00z.master.grb2f006 -match REF -stats -s
7:195646:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:1 hybrid level:6 hour fcst:
8:195825:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:2 hybrid level:6 hour fcst:
9:196004:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFC:entire atmosphere (considered as a single layer):6 hour fcst:
822:48903639:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:4000 m above ground:6 hour fcst:
823:48903818:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:1000 m above ground:6 hour fcst:

It seems to me 3D radar reflectivity on model hybrid levels output constant value(-20) in your test. Can you check variable "refl_10c " in model raw dataset sfc*.nc?

@WalterKolczynski-NOAA
Copy link
Contributor Author

refl_10cm has a constant value of -35.0 in all the sfc history files.

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA I modified the inline post interface. Can you test my UFS version at /scratch1/NCEPDEV/stmp2/Wen.Meng/ufs_radar/ufs-weather-model with the glowbal-workflow on Hera?

WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Aug 16, 2023
@WalterKolczynski-NOAA
Interim commit to fix reflectivity
fbd68c2 
Refs NOAA-EMC#1513
@WalterKolczynski-NOAA
Copy link
Contributor Author

@WenMeng-NOAA I'm still getting the same result with constant fields. Let me know if you want a working directory; I didn't change the setting to keep them before I ran it.

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA Can you provide me the location of your test results? We would expect non-missing values in radar reflectivity at hybrid levels since my changes are for passing 3D radar from forecast component in the inline post interface and refl_10cm is set as a constant value from your model configuration.

@WalterKolczynski-NOAA
Copy link
Contributor Author 

/scratch2/NCEPDEV/ensemble/noscrub/Walter.Kolczynski/global-workflow/reflectivity/gfs.20211221/00/model_data/atmos/master

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA I saw radar reflectivity from your test as:

[Wen.Meng@hfe10 master]$ wgrib2 gfs.t00z.master.grb2f108 -match REF -stats -s
7:209504:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:1 hybrid level:108 hour fcst:
8:209683:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:2 hybrid level:108 hour fcst:
9:209862:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFC:entire atmosphere (considered as a single layer):108 hour fcst:
822:49647991:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:4000 m above ground:108 hour fcst:
823:49648170:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:1000 m above ground:108 hour fcst:

Your test sets refl_10cm as constant -35. In UPP, the lower bound of radar reflectivity is set -20.
BTW, you test indicate 3D radar is successfully read in. I will add the changes of inline post interface in the next UFS PR.

@WalterKolczynski-NOAA
Copy link
Contributor Author

But isn't the constant -35 the problem? How do we solve that?

@WenMeng-NOAA
Copy link
Contributor

@WalterKolczynski-NOAA The -35 is a fake value you set from the model side. From my debugging, this value can be correctly read in UPP and reset to -20 as the lower bound of radar reflectivity in UPP.
You last test on Orion indicated refl_10cm wasn't correctly read in from the model:
The REFD at hybrid levels have all missing values, 7:204842:ndata=73728:undef=73728:mean=0:min=0:max=0:REFD

Your latest test on Hera indicates the changes as:
7:209504:ndata=73728:undef=0:mean=-20:min=-20:max=-20:cos_wt_mean=-20:d=2021122100:REFD:1 hybrid level:108 hour fcst:

@RuiyuSun
Copy link

@WalterKolczynski-NOAA @WenMeng-NOAA I conducted an experiment with the latest workflow that @JessicaMeixner-NOAA provided. I added the following line to the diag_table and set the namelist variable lradar to .true.. The 3D reflectivity appears in the model surface history surface files. Reflectivity (composite of the column and a two levels) in the masterfiles also show reasonable values.

The model history files are at /scratch1/NCEPDEV/stmp2/Ruiyu.Sun/ROTDIRS/refl/gfs.20201203/00/model_data/atmos/history

The master files are at /scratch1/NCEPDEV/stmp2/Ruiyu.Sun/ROTDIRS/refl/gfs.20201203/00/model_data/atmos/master

@WalterKolczynski-NOAA Could you test it and make the change to the workflow?

Thanks!

@WenMeng-NOAA
Copy link
Contributor

@RuiyuSun I looked into mater files at /scratch1/NCEPDEV/stmp2/Ruiyu.Sun/ROTDIRS/refl/gfs.20201203/00/model_data/atmos/master and found all radar reflectivity fields with valid values.

[Wen.Meng@hfe06 master]$ wgrib2 gfs.t00z.master.grb2f090 -match REF -stats -s
7:7139757:ndata=4718592:undef=0:mean=-14.5043:min=-20:max=51.45:cos_wt_mean=-14.8049:d=2020120300:REFD:1 hybrid level:90 hour fcst:
8:9852482:ndata=4718592:undef=0:mean=-14.4844:min=-20:max=51.4:cos_wt_mean=-14.7867:d=2020120300:REFD:2 hybrid level:90 hour fcst:
9:12566246:ndata=4718592:undef=0:mean=-11.5809:min=-20:max=60.84:cos_wt_mean=-11.6809:d=2020120300:REFC:entire atmosphere (considered as a single layer):90 hour fcst:
822:2228770919:ndata=4718592:undef=0:mean=-17.905:min=-20:max=59.55:cos_wt_mean=-17.8416:d=2020120300:REFD:4000 m above ground:90 hour fcst:
823:2229921085:ndata=4718592:undef=0:mean=-15.3112:min=-20:max=52.47:cos_wt_mean=-15.56:d=2020120300:REFD:1000 m above ground:90 hour fcst:

You test looks good for me. Thanks!

WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Dec 12, 2023
@WalterKolczynski-NOAA
Interim commit to fix reflectivity
ce1181f 
Refs NOAA-EMC#1513
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@WalterKolczynski-NOAA WalterKolczynski-NOAA

@RuiyuSun RuiyuSun

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enable radar reflectivity output WalterKolczynski-NOAA/global-workflow
3 participants
@WalterKolczynski-NOAA @RuiyuSun @WenMeng-NOAA