Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incorporate global 30-sec MODIS vegetation data and STATSGO soil data. #702

Closed
GeorgeGayno-NOAA opened this issue Oct 12, 2022 · 11 comments · Fixed by #703
Closed

Incorporate global 30-sec MODIS vegetation data and STATSGO soil data. #702

GeorgeGayno-NOAA opened this issue Oct 12, 2022 · 11 comments · Fixed by #703
Assignees
@barlage @BenjaminBlake-NOAA @GeorgeGayno-NOAA

Comments

@GeorgeGayno-NOAA
Copy link
Collaborator

Issue #698 was open to add the 30-sec VIIRS vegetation data. The MODIS vegetation data and STATSGO soil data are also available to 30-sec. Currently, only coarser resolution versions are available. Update UFS_UTILS to use the 30-sec data.
@GeorgeGayno-NOAA GeorgeGayno-NOAA self-assigned this Oct 12, 2022
@GeorgeGayno-NOAA
Copy link
Collaborator Author

A program and run script to create this data was baselined on Hera: /scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/climo_fields_netcdf/sorc/fv3.soilt.vegt.hires.netcdf

GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Oct 12, 2022
@GeorgeGayno-NOAA
Update hera script comments for new data sources.
649aa8b 
Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Oct 13, 2022
@GeorgeGayno-NOAA
Update ./ush/sfc_climo_gen.sh to allow user to choose which
9e738fe 
soil type data to use. Update all driver script comments.

Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Oct 13, 2022
@GeorgeGayno-NOAA
Update comments in ./util/sfc_climo_gen/run.wcoss2.sh.
6234c36 
Add export of soil type data source variable.

Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Oct 13, 2022
@GeorgeGayno-NOAA
Update 'readthedocs' for new data sets.
7feafe8 
Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Oct 14, 2022
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/vegsoil30s
ff578bc 
Fixes ufs-community#702.
@GeorgeGayno-NOAA GeorgeGayno-NOAA mentioned this issue Oct 17, 2022
Closed
@GeorgeGayno-NOAA GeorgeGayno-NOAA mentioned this issue Oct 17, 2022
Merged
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Oct 19, 2022
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/vegsoil30s
70e7326 
Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Oct 20, 2022
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/vegsoil30s
a67cde3 
Fixes ufs-community#702.
@GeorgeGayno-NOAA
Copy link
Collaborator Author

First set of tests:

These updates should not affect the current grid generating capability. So, the branch at a67cd3d was compiled on Orion, Jet, Hera and Cactus, and the ops 'fix' directories were linked. The grid_gen consistency tests were then run. All tests passed on these machines as expected.

@GeorgeGayno-NOAA
Copy link
Collaborator Author

The 30 second MODIS vegetation type and STATSGO soil type were placed on Hera: /scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/UFS_UTILS/fix/sfc_climo.new

There are CONUS, Northern Hemisphere and global versions:

/scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/UFS_UTILS/fix/sfc_climo.new $ ls -l *30s*
-rw-r--r-- 1 George.Gayno stmp 12727109 Oct 12 15:15 soil_type.statsgo.30s.nc
-rw-r--r-- 1 George.Gayno stmp  2542939 Oct 12 15:20 soil_type.statsgo.conus.30s.nc
-rw-r--r-- 1 George.Gayno stmp  9081042 Oct 12 15:18 soil_type.statsgo.nh.30s.nc
-rw-r--r-- 1 George.Gayno stmp 51744793 Oct 12 15:12 vegetation_type.modis.igbp.30s.nc
-rw-r--r-- 1 George.Gayno stmp  7768557 Oct 12 15:07 vegetation_type.modis.igbp.conus.30s.nc
-rw-r--r-- 1 George.Gayno stmp 39221287 Oct 12 15:08 vegetation_type.modis.igbp.nh.30s.nc

@GeorgeGayno-NOAA
Copy link
Collaborator Author

@BenjaminBlake-NOAA Would you mind trying these new files? Check out my branch (a67cde3) on Hera (or test using your workflow). When you set the links in the 'fix' directory, set to the link to sfc_climo to this directory: /scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/UFS_UTILS/fix/sfc_climo.new

@BenjaminBlake-NOAA
Copy link

@GeorgeGayno-NOAA I tested the new files for the RRFS CONUS and North American grids, and both sets of fix files were created successfully.

@GeorgeGayno-NOAA
Copy link
Collaborator Author

GeorgeGayno-NOAA commented Oct 21, 2022 •
edited
Loading

I created a CONUS regional grid with all 30 sec data files (global, NH and CONUS). The output model vegetation and soil type files were identical as expected (because the NH and CONUS files are just sub-sections of the global file).

I then created the CONUS grid using the 0.05-degree vegetation and soil type data. Upon visual inspection, the vegetation and soil type files compared well to those created using the 30 second data.

@GeorgeGayno-NOAA
Copy link
Collaborator Author

Soil type on model grid using 0.05-degree input data:

soil 0 05

Soil type on model grid using the global 30 sec data:

soil 30s

@GeorgeGayno-NOAA
Copy link
Collaborator Author

Vegetation type on model grid using 0.05-degree input data.

vegt 0 05

Vegetation type on model grid using global 30 second input data.

vegt 30s

@GeorgeGayno-NOAA
Copy link
Collaborator Author

The 30 second MODIS vegetation type and STATSGO soil type were placed on Hera: /scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/UFS_UTILS/fix/sfc_climo.new

There are CONUS, Northern Hemisphere and global versions:

/scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/UFS_UTILS/fix/sfc_climo.new $ ls -l *30s*
-rw-r--r-- 1 George.Gayno stmp 12727109 Oct 12 15:15 soil_type.statsgo.30s.nc
-rw-r--r-- 1 George.Gayno stmp  2542939 Oct 12 15:20 soil_type.statsgo.conus.30s.nc
-rw-r--r-- 1 George.Gayno stmp  9081042 Oct 12 15:18 soil_type.statsgo.nh.30s.nc
-rw-r--r-- 1 George.Gayno stmp 51744793 Oct 12 15:12 vegetation_type.modis.igbp.30s.nc
-rw-r--r-- 1 George.Gayno stmp  7768557 Oct 12 15:07 vegetation_type.modis.igbp.conus.30s.nc
-rw-r--r-- 1 George.Gayno stmp 39221287 Oct 12 15:08 vegetation_type.modis.igbp.nh.30s.nc

@KateFriedman-NOAA Can you please add these files to the 'sfc_climo' directory on all machines. You can also remove the following files:

  • soil_type.statsgo.conus.0.01.nc
  • vegetation_type.modis.igbp.conus.0.01.nc

Many thanks!

@KateFriedman-NOAA
Copy link
Collaborator

@GeorgeGayno-NOAA Done on Hera, rsyncing to other platforms now:

[glopara@hfe08 20221017]$ pwd
/scratch1/NCEPDEV/global/glopara/fix/sfc_climo/20221017
[glopara@hfe08 20221017]$ ll
total 10827232
-rw-r--r-- 1 glopara global     279733 Dec 14  2018 facsf.1.0.nc
-rw-r--r-- 1 glopara global 4769464869 Dec 14  2018 leaf_area_index.0.05.nc
-rw-r--r-- 1 glopara global  103864687 Oct 17 16:09 maximum_snow_albedo.0.05.nc
-rw-r--r-- 1 glopara global     279733 Dec 13  2018 slope_type.1.0.nc
-rw-r--r-- 1 glopara global 4976826419 Oct 17 16:10 snowfree_albedo.4comp.0.05.nc
-rw-r--r-- 1 glopara global  288299994 Mar 11  2019 soil_type.statsgo.0.03.nc
-rw-r--r-- 1 glopara global  103864794 Dec 12  2018 soil_type.statsgo.0.05.nc
-rw-r--r-- 1 glopara global   12727109 Oct 12 15:15 soil_type.statsgo.30s.nc
-rw-r--r-- 1 glopara global    2542939 Oct 12 15:20 soil_type.statsgo.conus.30s.nc
-rw-r--r-- 1 glopara global    9081042 Oct 12 15:18 soil_type.statsgo.nh.30s.nc
-rw-r--r-- 1 glopara global     279733 Dec 14  2018 substrate_temperature.1.0.nc
-rw-r--r-- 1 glopara global      79995 Dec 14  2018 substrate_temperature.2.6x1.5.nc
-rw-r--r-- 1 glopara global  150071935 Dec 12  2018 vegetation_greenness.0.144.nc
lrwxrwxrwx 1 glopara global         34 Jul 29 12:35 vegetation_type.igbp.0.05.nc -> vegetation_type.modis.igbp.0.05.nc
-rw-r--r-- 1 glopara global  288299990 Mar 11  2019 vegetation_type.modis.igbp.0.03.nc
-rw-r--r-- 1 glopara global  103864790 Dec 12  2018 vegetation_type.modis.igbp.0.05.nc
-rw-r--r-- 1 glopara global   51744793 Oct 12 15:12 vegetation_type.modis.igbp.30s.nc
-rw-r--r-- 1 glopara global    7768557 Oct 12 15:07 vegetation_type.modis.igbp.conus.30s.nc
-rw-r--r-- 1 glopara global   39221287 Oct 12 15:08 vegetation_type.modis.igbp.nh.30s.nc
-rw-r--r-- 1 glopara global   72306138 Oct 17 16:10 vegetation_type.viirs.igbp.0.03.nc
-rw-r--r-- 1 glopara global   26110938 Dec  3  2020 vegetation_type.viirs.igbp.0.05.nc
-rw-r--r-- 1 glopara global    6584538 Dec  3  2020 vegetation_type.viirs.igbp.0.1.nc
-rw-r--r-- 1 glopara global   38111755 Oct 11 17:51 vegetation_type.viirs.igbp.30s.nc
-rw-r--r-- 1 glopara global    6032902 Oct 11 17:52 vegetation_type.viirs.igbp.conus.30s.nc
-rw-r--r-- 1 glopara global   29222645 Oct 11 17:47 vegetation_type.viirs.igbp.nh.30s.nc

@GeorgeGayno-NOAA
Copy link
Collaborator Author

@GeorgeGayno-NOAA Done on Hera, rsyncing to other platforms now:

[glopara@hfe08 20221017]$ pwd
/scratch1/NCEPDEV/global/glopara/fix/sfc_climo/20221017
[glopara@hfe08 20221017]$ ll
total 10827232
-rw-r--r-- 1 glopara global     279733 Dec 14  2018 facsf.1.0.nc
-rw-r--r-- 1 glopara global 4769464869 Dec 14  2018 leaf_area_index.0.05.nc
-rw-r--r-- 1 glopara global  103864687 Oct 17 16:09 maximum_snow_albedo.0.05.nc
-rw-r--r-- 1 glopara global     279733 Dec 13  2018 slope_type.1.0.nc
-rw-r--r-- 1 glopara global 4976826419 Oct 17 16:10 snowfree_albedo.4comp.0.05.nc
-rw-r--r-- 1 glopara global  288299994 Mar 11  2019 soil_type.statsgo.0.03.nc
-rw-r--r-- 1 glopara global  103864794 Dec 12  2018 soil_type.statsgo.0.05.nc
-rw-r--r-- 1 glopara global   12727109 Oct 12 15:15 soil_type.statsgo.30s.nc
-rw-r--r-- 1 glopara global    2542939 Oct 12 15:20 soil_type.statsgo.conus.30s.nc
-rw-r--r-- 1 glopara global    9081042 Oct 12 15:18 soil_type.statsgo.nh.30s.nc
-rw-r--r-- 1 glopara global     279733 Dec 14  2018 substrate_temperature.1.0.nc
-rw-r--r-- 1 glopara global      79995 Dec 14  2018 substrate_temperature.2.6x1.5.nc
-rw-r--r-- 1 glopara global  150071935 Dec 12  2018 vegetation_greenness.0.144.nc
lrwxrwxrwx 1 glopara global         34 Jul 29 12:35 vegetation_type.igbp.0.05.nc -> vegetation_type.modis.igbp.0.05.nc
-rw-r--r-- 1 glopara global  288299990 Mar 11  2019 vegetation_type.modis.igbp.0.03.nc
-rw-r--r-- 1 glopara global  103864790 Dec 12  2018 vegetation_type.modis.igbp.0.05.nc
-rw-r--r-- 1 glopara global   51744793 Oct 12 15:12 vegetation_type.modis.igbp.30s.nc
-rw-r--r-- 1 glopara global    7768557 Oct 12 15:07 vegetation_type.modis.igbp.conus.30s.nc
-rw-r--r-- 1 glopara global   39221287 Oct 12 15:08 vegetation_type.modis.igbp.nh.30s.nc
-rw-r--r-- 1 glopara global   72306138 Oct 17 16:10 vegetation_type.viirs.igbp.0.03.nc
-rw-r--r-- 1 glopara global   26110938 Dec  3  2020 vegetation_type.viirs.igbp.0.05.nc
-rw-r--r-- 1 glopara global    6584538 Dec  3  2020 vegetation_type.viirs.igbp.0.1.nc
-rw-r--r-- 1 glopara global   38111755 Oct 11 17:51 vegetation_type.viirs.igbp.30s.nc
-rw-r--r-- 1 glopara global    6032902 Oct 11 17:52 vegetation_type.viirs.igbp.conus.30s.nc
-rw-r--r-- 1 glopara global   29222645 Oct 11 17:47 vegetation_type.viirs.igbp.nh.30s.nc

I just checked all platforms, and the files are there. Thanks!

GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Nov 1, 2022
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/vegsoil30s
2fa176f 
Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Nov 18, 2022
@GeorgeGayno-NOAA
Update grid generation script comments.
65e1205 
Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Nov 22, 2022
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/vegsoil30s
2ffd997 
Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit to GeorgeGayno-NOAA/UFS_UTILS that referenced this issue Nov 29, 2022
@GeorgeGayno-NOAA
Merge branch 'develop' into feature/vegsoil30s
618567a 
Fixes ufs-community#702.
GeorgeGayno-NOAA added a commit that referenced this issue Nov 29, 2022
@GeorgeGayno-NOAA
High-resolution MODIS and STATSGO veg/soil data. (#703)
a0e3e56 
Updates to use the MODIS vegetation data and STATSGO soil data at its full 
global 30-second resolution.

Fixes #702.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@barlage barlage

@BenjaminBlake-NOAA BenjaminBlake-NOAA

@GeorgeGayno-NOAA GeorgeGayno-NOAA

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

New MODIS and STATSGO veg/soil data. GeorgeGayno-NOAA/UFS_UTILS
4 participants
@barlage @KateFriedman-NOAA @BenjaminBlake-NOAA @GeorgeGayno-NOAA