Merge remote-tracking branch 'upstream/develop' into merra2

* upstream/develop: remove flags that were not present in Makefiles update jasper to 2.x.25. remove compiler flag in fv3nc2nemsio Update EMC_verif-global tag to verif_global_v1.13.4 another use of _d where an _4 is needed Adjust how target is set for build_workflow_utils Add workflow_utils to build_all and link scripts copy/paste error from enkf_chgres_recenter.fd to enkf_chgres_recenter_nc.fd. When going from nemsio to netcdf, the linking of ip, sp and w3nco changed from _d to _4. update .gitignore to exclude build and install directories as well as compiled files. bugfix in build_workflow_utils.sh hack. nceplibs-ncio now creates the module ncio and not fv3gfs_ncio. add cmake build capability for workflow utilities Update Fit2Obs tag to newm.1.3 for bugfix Update Fit2Obs tag to newm.1.3 for bugfix Update EMC_verif-global tag for Hera bug Pull in nwpara/gfsv16.0.6 updates for parse-storm-type.pl HOTFIX: Update EMC_verif-global tag for Hera bug Issue NOAA-EMC#1 and issue NOAA-EMC#238 - update Externals.cfg to match checkout.sh updates for v16.0.6 move errchk definition from script to job for wave prdgen and gempak adding definition of errchk which was undefined in these scripts Issue NOAA-EMC#179 and issue NOAA-EMC#243 - update Fit2Obs to newm.1.2 tag and correct COMROOT path for Hera HOTFIX: Issue NOAA-EMC#241 - update EMC_verif-global tag to remove use of /tmp space Issue NOAA-EMC#1 and issue NOAA-EMC#233 - update GLDAS tag to gldas_gfsv16_release.v1.13.0 Issue NOAA-EMC#1 and issue NOAA-EMC#241 - update EMC_verif-global tag Issue NOAA-EMC#1 and issue NOAA-EMC#238 - remove ak_10m grid from config.wave and update checkout.sh tags for ufs-weather-model and EMC_gfs_wafs Issue NOAA-EMC#179 - update to config.vrfy for Fit2Obs tag which supports Orion Issue NOAA-EMC#1 and issue NOAA-EMC#226 - update UPP tag to upp_gfsv16_release.v1.1.3 in Externals.cfg Issue NOAA-EMC#1 and issue NOAA-EMC#226 - update UPP tag to upp_gfsv16_release.v1.1.3 Issue NOAA-EMC#233 - remove unnecessary extra space in tag line for gldas Issue NOAA-EMC#233 - update GLDAS tag to gldas_gfsv16_release.v1.13.0 Issue NOAA-EMC#1 and issue NOAA-EMC#226 - update exgfs_atmos_grib2_special_npoess.sh for dbn_alert modified: exglobal_forecast.sh The breakpoint restart only works for the first restart from a breakpoint. Restart files written in RERUN_RESTRAT after the first restart has a 3-hour time shift for DO_IAU=YES cases. Forecasts starting from the 2nd breakpoint and beyond will fail becasue of incorrect initial conditins. This commit fixes this bug. issue NOAA-EMC#227 reducing output bufr files to 64 levels per NCO request issue NOAA-EMC#227 reducing output bufr files to 64 levels per NCO request Issue NOAA-EMC#1: update ecflow to be consistent with NCO's gfs.v16.0.4 and update checkout to bring in new UPP tag (see issue NOAA-EMC#226) Upaded ush script scale_dec.sh Updated scripts gfs_v16.0 Issue NOAA-EMC#1 and issue NOAA-EMC#220 - set C192/C96/C48 npe_eobs back to dev values for develop Issue NOAA-EMC#1 - add grib_util module load to several analysis ecflow scripts Issue NOAA-EMC#189 - update ufs-weather-model hash Issue NOAA-EMC#201 - workaround for failing post000 job before hpc-stack solution Issue NOAA-EMC#1 - update WAFS tag to gfs_wafs.v6.0.17 for dbn_alert change Issue NOAA-EMC#1 - correct gridded wave parm files for v16.0.3 Issue NOAA-EMC#1 - changes from NCO for GFSv16.0.3 Issue NOAA-EMC#1 - update Externals.cfg with final tags for GFSv16.0.2 Fixes for issue NOAA-EMC#202 (FINDDATE) and issue NOAA-EMC#208 (postsnd.sh permissions) Issue NOAA-EMC#1 - correct permissions on jobs/rocoto/postsnd.sh Issue NOAA-EMC#1 - update gempak version to 7.3.3 in the ecflow gfs.ver file Issue NOAA-EMC#1 - update gempak and dumpjb versions to 7.3.3 and 5.1.0 respectively Script alert updates from NCO for wave downstream Add override for COMIN_WAV_RTOFS in emc mode for waveprep job Issue NOAA-EMC#197: place CDATE specific sections of config.anal and config.prep inside RUN_ENVIR=emc blocks. These sections are use for retrospective parallels and therefore do not need to be executed in operations (NCO). modified: JGLOBAL_FORECAST modified: JGLOBAL_FORECAST to make it work for both emc and nco running environments. modified: JGLOBAL_FORECAST A test showed that jobid is not defined in JGLOBAL_FORECAST running in the Rocoto environment. jobid is defined in ./env files. Defining DATA without sourcing ./env/$nachine.env caused the script to fail. Move the definitin of DATA after sourcing env parameters modified: checkout.sh to use WAFS tag gfs_wafs.v6.0.16 modified: checkout.sh to update UPP to upp_gfsv16_release.v1.1.1, a minor syntax bug fix modified: link_fv3gfs.sh to 1) use hard copies of external fix fields and executable for NCO installation 2) use soft links for all other files and directories for both NCO and EMC installations Compared local files in NCO implementation directory with release/gfs.v16.0.0 branch, changes made by NCO(Jen Yang) in the following files are either accepted or rejected. create a new branch release/gfsv16.0.0.nco to merge changes made by NCO in /gpfs/dell1/nco/ops/nwpara/gfs-v16/gfs.v16.0.1 back to EMC's repository update for the wave parm so that the wave model will look for the correct restart for when gfs is not run every cycle updates for checking if RTOFS files exist and only processing RTOFS files for needed fhr
AnningCheng-NOAA · Mar 8, 2021 · ac76613 · ac76613
2 parents 0f97ccd + f6aa693
commit ac76613
Show file tree

Hide file tree

Showing 826 changed files with 5,356 additions and 6,503 deletions.
diff --git a/.gitignore b/.gitignore
@@ -2,12 +2,14 @@
 #--------------------------
 __pycache__
 *.pyc
-*.o
+*.[aox]
 *.mod
 
-# Ignore exec folder
+# Ignore folders
 #-------------------
 exec/
+build*/
+install*/
 
 # Ignore fix directory symlinks
 #------------------------------

diff --git a/Externals.cfg b/Externals.cfg
@@ -1,7 +1,7 @@
 # External sub-modules of global-workflow
 
 [FV3GFS]
-tag = GFS.v16.0.14
+tag = GFS.v16.0.15
 local_path = sorc/fv3gfs.fd
 repo_url = https://github.com/ufs-community/ufs-weather-model.git
 protocol = git
@@ -15,14 +15,14 @@ protocol = git
 required = True
 
 [GLDAS]
-tag = gldas_gfsv16_release.v1.11.0
+tag = gldas_gfsv16_release.v1.13.0
 local_path = sorc/gldas.fd
 repo_url = https://github.com/NOAA-EMC/GLDAS.git
 protocol = git
 required = True
 
 [EMC_post]
-tag = upp_gfsv16_release.v1.1.0
+tag = upp_gfsv16_release.v1.1.3
 local_path = sorc/gfs_post.fd
 repo_url = https://github.com/NOAA-EMC/EMC_post.git
 protocol = git
@@ -36,14 +36,14 @@ protocol = git
 required = True
 
 [EMC_verif-global]
-tag = verif_global_v1.11.0
+tag = verif_global_v1.13.4
 local_path = sorc/verif-global.fd
 repo_url = https://github.com/NOAA-EMC/EMC_verif-global.git
 protocol = git
 required = True
 
 [EMC_gfs_wafs]
-tag = gfs_wafs.v6.0.14
+tag = gfs_wafs.v6.0.18
 local_path = sorc/gfs_wafs.fd
 repo_url = https://github.com/NOAA-EMC/EMC_gfs_wafs.git
 protocol = git

diff --git a/docs/Release_Notes.gfs_downstream.v16.0.0.txt b/docs/Release_Notes.gfs_downstream.v16.0.0.txt
@@ -64,7 +64,7 @@ PARM/wmo
    ###########################################
    # Now set up GEMPAK/NTRANS environment
    ###########################################
-   module load gempak/7.3.1
+   module load gempak/7.3.3
    module list
 
 FV3 GFS DRIVER:

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_00 b/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_00
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_06 b/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_06
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_12 b/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_12
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_18 b/driver/product/run_JGDAS_ATMOS_GEMPAK_META_NCDC_dell.sh_18
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_00 b/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_00
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_06 b/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_06
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_12 b/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_12
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_18 b/driver/product/run_JGDAS_ATMOS_GEMPAK_dell.sh_18
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_00 b/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_00
@@ -39,7 +39,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_06 b/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_06
@@ -39,7 +39,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_12 b/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_12
@@ -39,7 +39,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_18 b/driver/product/run_JGFS_ATMOS_GEMPAK_META_dell.sh_18
@@ -39,7 +39,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_00 b/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_00
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_06 b/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_06
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_12 b/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_12
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_18 b/driver/product/run_JGFS_ATMOS_GEMPAK_NCDC_UPAPGIF_dell.sh_18
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ##############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_00 b/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_00
@@ -37,7 +37,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_06 b/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_06
@@ -37,7 +37,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_12 b/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_12
@@ -37,7 +37,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_18 b/driver/product/run_JGFS_ATMOS_GEMPAK_PGRB2_SPEC_dell.sh_18
@@ -37,7 +37,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_00 b/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_00
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_06 b/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_06
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_12 b/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_12
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_18 b/driver/product/run_JGFS_ATMOS_GEMPAK_dell.sh_18
@@ -38,7 +38,7 @@ module load grib_util/1.1.0
 ###########################################
 # Now set up GEMPAK/NTRANS environment
 ###########################################
-module load gempak/7.3.1
+module load gempak/7.3.3
 module list
 
 ############################################

diff --git a/driver/product/run_postsnd.sh b/driver/product/run_postsnd.sh
@@ -29,7 +29,7 @@ module load CFP/2.0.1
 module load python/3.6.3
 
 module use -a /gpfs/dell1/nco/ops/nwpara/modulefiles/
-module load gempak/7.3.1
+module load gempak/7.3.3
 module use /usrx/local/nceplibs/dev/NCEPLIBS/modulefiles
 module load hdf5_parallel/1.10.6
 module load netcdf_parallel/4.7.4

diff --git a/driver/product/run_postsnd.sh.dell b/driver/product/run_postsnd.sh.dell
@@ -29,7 +29,7 @@ module load CFP/2.0.1
 module load python/3.6.3
 
 module use -a /gpfs/dell1/nco/ops/nwpara/modulefiles/
-module load gempak/7.3.1
+module load gempak/7.3.3
 module use /usrx/local/nceplibs/dev/NCEPLIBS/modulefiles
 module load hdf5_parallel/1.10.6
 module load netcdf_parallel/4.7.4

diff --git a/ecflow/ecf/scripts/gdas/atmos/analysis/jgdas_atmos_analysis.ecf b/ecflow/ecf/scripts/gdas/atmos/analysis/jgdas_atmos_analysis.ecf
@@ -1,15 +1,14 @@
-#! /bin/sh
-#BSUB -q %QUEUE%
-#BSUB -P %PROJ%
+#BSUB -L /bin/sh
 #BSUB -J %E%gdas_atmos_analysis_%CYC%
 #BSUB -o %COM%/output/%ENVIR%/today/gdas_atmos_analysis_%CYC%.o%J
-#BSUB -W 0:45
+#BSUB -q %QUEUE%
+#BSUB -P %PROJ%
+#BSUB -W 1:00
 #BSUB -n 1000
 #BSUB -R affinity[core(7)]
 #BSUB -R span[ptile=4]
 #BSUB -cwd /tmp
 
-module purge
 %include <head.h>
 %include <envir-p3.h>
 
@@ -32,14 +31,12 @@ export model=gfs
 ############################################################
 module load lsf/${lsf_ver}
 module load impi/${impi_ver}
-module load prod_util/${prod_util_ver}
-module load prod_envir/${prod_envir_ver}
 module load CFP/${CFP_ver}
 module load python/${python_ver}
 module load NetCDF-parallel/${netcdf_parallel_ver}
 module load HDF5-parallel/${hdf5_parallel_ver}
-module use -a /gpfs/dell1/nco/ops/nwprod/modulefiles/
 module load crtm/${crtm_ver}
+module load grib_util/${grib_util_ver}
 
 module list
 
@@ -52,12 +49,8 @@ export FORT_BUFFERED=true
 export OMP_NUM_THREADS_CY=28
 export NTHREADS=$OMP_NUM_THREADS_CY
 export NTHREADS_GSI=$threads
-export COMPONENT=${COMPONENT:-atmos}
-export ROTDIR=%COM%/$NET/$envir
-export CDUMP=gdas
 export cyc=%CYC%
 export cycle=t%CYC%z
-export CDATE=$PDY$cyc
 export USE_CFP=YES
 export PREINP=''
 

diff --git a/ecflow/ecf/scripts/gdas/atmos/analysis/jgdas_atmos_analysis_calc.ecf b/ecflow/ecf/scripts/gdas/atmos/analysis/jgdas_atmos_analysis_calc.ecf
@@ -1,15 +1,14 @@
-#! /bin/sh
-#BSUB -q %QUEUE%
-#BSUB -P %PROJ%
+#BSUB -L /bin/sh
 #BSUB -J %E%gdas_atmos_analysis_calc_%CYC%
 #BSUB -o %COM%/output/%ENVIR%/today/gdas_atmos_analysis_calc_%CYC%.o%J
+#BSUB -q %QUEUE%
+#BSUB -P %PROJ%
 #BSUB -W 0:10
 #BSUB -n 140
 #BSUB -R affinity[core(1)]
 #BSUB -R span[ptile=28]
 #BSUB -cwd /tmp
 
-module purge
 %include <head.h>
 %include <envir-p3.h>
 
@@ -32,8 +31,6 @@ model=gfs
 ############################################################
 module load lsf/${lsf_ver}
 module load impi/${impi_ver}
-module load prod_util/${prod_util_ver}
-module load prod_envir/${prod_envir_ver}
 module load python/${python_ver}
 module load NetCDF-parallel/${netcdf_parallel_ver}
 module load HDF5-parallel/${hdf5_parallel_ver}
@@ -43,12 +40,8 @@ module list
 #############################################################
 # WCOSS environment settings
 #############################################################
-export COMPONENT=${COMPONENT:-atmos}
-export ROTDIR=%COM%/$NET/$envir
-export CDUMP=gdas
 export cyc=%CYC%
 export cycle=t%CYC%z
-export CDATE=$PDY$cyc
 export USE_CFP=YES
 
 ############################################################