Support Rocky8 on CSPs #982

weihuang-jedi · 2024-09-10T22:45:57Z

As Rocky 8 will become the only supported OS after 1/1/2025 on CSPs (AWS/Azure/Google), we need to test earlier.

Spack-stack is available with Rocky 8 on AWS/Azure (and soon on Google).

We start update modulefiles/build.noaacould.interl.lua on CSPs.

But after update the above file, we got error:

[ 18%] Linking C static library libshared_lib.a
[ 18%] Built target shared_lib
[ 19%] Building Fortran object sorc/grid_tools.fd/regional_esg_grid.fd/CMakeFiles/regional_esg_grid.dir/pmat2.f90.o
[ 19%] Building Fortran object sorc/grid_tools.fd/regional_esg_grid.fd/CMakeFiles/regional_esg_grid.dir/pmat4.f90.o
[ 19%] Building Fortran object sorc/grid_tools.fd/regional_esg_grid.fd/CMakeFiles/regional_esg_grid.dir/pmat5.f90.o
[ 20%] Building Fortran object sorc/grid_tools.fd/regional_esg_grid.fd/CMakeFiles/regional_esg_grid.dir/pesg.f90.o
[ 21%] Building Fortran object sorc/grid_tools.fd/regional_esg_grid.fd/CMakeFiles/regional_esg_grid.dir/regional_esg_grid.f90.o
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/grid_tools.fd/regional_esg_grid.fd/regional_esg_grid.f90(23): error #7012: The module file cannot be read. Its format requires a more recent F90 compiler. [NETCDF]
use netcdf
------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/grid_tools.fd/regional_esg_grid.fd/regional_esg_grid.f90(28): error #6683: A kind type parameter must be a compile-time constant. [DP]
real(dp) :: plat,plon,pazi=0.0
-------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/grid_tools.fd/regional_esg_grid.fd/regional_esg_grid.f90(29): error #6683: A kind type parameter must be a compile-time constant. [DP]
real(dp) :: delx,dely
-------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/grid_tools.fd/regional_esg_grid.fd/regional_esg_grid.f90(33): error #6683: A kind type parameter must be a compile-time constant. [DP]
real(dp),parameter :: re=6371200.0
-------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/grid_tools.fd/regional_esg_grid.fd/regional_esg_grid.f90(34): error #6683: A kind type parameter must be a compile-time constant. [DP]
real(dp),parameter :: lam=0.8
-------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/grid_tools.fd/regional_esg_grid.fd/regional_esg_grid.f90(39): error #6683: A kind type parameter must be a compile-time constant. [DP]
real(dp),dimension(:,:),allocatable:: glat,glon
-------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/grid_tools.fd/regional_esg_grid.fd/regional_esg_grid.f90(40): error #6683: A kind type parameter must be a compile-time constant. [DP]
real(dp),dimension(:,:),allocatable:: garea

We have modules:

[Wei.Huang@awsrocky8epicwei-14 ufs-utils-cloud]$ module list netcdf

Currently Loaded Modules Matching: netcdf

netcdf-c/4.9.2 2) netcdf-fortran/4.6.1 3) parallel-netcdf/1.12.2

[Wei.Huang@awsrocky8epicwei-14 ufs-utils-cloud]$ module show netcdf-fortran/4.6.1

/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/intel-oneapi-mpi/2021.10.0/intel/2021.10.0/netcdf-fortran/4.6.1.lua:

whatis("Name : netcdf-fortran")
whatis("Version : 4.6.1")
whatis("Target : skylake_avx512")
whatis("Short description : NetCDF (network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. This is the Fortran distribution.")
whatis("Configure options : --enable-static --enable-shared --disable-doxygen --disable-parallel-tests")
help([[Name : netcdf-fortran]])
help([[Version: 4.6.1]])
help([[Target : skylake_avx512]])
]])
help([[NetCDF (network Common Data Form) is a set of software libraries and
machine-independent data formats that support the creation, access, and
sharing of array-oriented scientific data. This is the Fortran
distribution.]])
depends_on("netcdf-c/4.9.2")
prepend_path("PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/bin")
prepend_path("MANPATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/share/man")
prepend_path("LD_LIBRARY_PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/lib")
prepend_path("DYLD_LIBRARY_PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/lib")
prepend_path("CPATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/include")
prepend_path("PKG_CONFIG_PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/lib/pkgconfig")
prepend_path("CMAKE_PREFIX_PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/.")
prepend_path("PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/bin")
prepend_path("MANPATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/share/man")
prepend_path("PKG_CONFIG_PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/lib/pkgconfig")
prepend_path("CMAKE_PREFIX_PATH","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/.")
setenv("netcdf_fortran_ROOT","/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni")
append_path("MANPATH","")

[Wei.Huang@awsrocky8epicwei-14 ufs-utils-cloud]$ ls /contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/intel/2021.10.0/netcdf-fortran-4.6.1-bke33ni/include
netcdf4_f03.mod netcdf4_nf_interfaces.mod netcdf_fortv2_c_interfaces.mod netcdf.mod netcdf_nc_interfaces.mod netcdf_nf_interfaces.mod
netcdf4_nc_interfaces.mod netcdf_f03.mod netcdf.inc netcdf_nc_data.mod netcdf_nf_data.mod typesizes.mod

natalie-perlin · 2024-09-11T12:55:03Z

Hi Wei,
What code are you trying to build, and what are your modulefiles ?
Current SRW builds with no issues when modules are updated appropriately, but the repository yet needs to be updated

natalie-perlin · 2024-09-11T12:57:10Z

What are the steps to reproduce the error?

weihuang-jedi · 2024-09-11T13:41:53Z

Hello Natalie, I am trying on AWS, at dir: /contrib/Wei.Huang/dev/ufs-utils-cloud The code I use is git check-out from: Warning: Permanently added 'github.com,140.82.113.3' (ECDSA) to the list of known hosts. * remote origin Fetch URL: ***@***.***:NOAA-EPIC/UFS_UTILS-cloud.git Push URL: ***@***.***:NOAA-EPIC/UFS_UTILS-cloud.git HEAD branch: develop Remote branches: develop tracked wei-epic-gcp tracked Local branches configured for 'git pull': csps-rocky8 merges with remote develop develop merges with remote develop wei-epic-gcp merges with remote wei-epic-gcp Local refs configured for 'git push': develop pushes to develop (up to date) wei-epic-gcp pushes to wei-epic-gcp (local out of date) ***@***.*** ufs-utils-cloud]$ git br * csps-rocky8 develop wei-epic-gcp Modulefile: ***@***.*** ufs-utils-cloud]$ more modulefiles/build.noaacloud.intel.lua help([[ Load environment to compile UFS_UTILS on NOAA CSPs using Intel ]]) prepend_path("MODULEPATH", "/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core") prepend_path("MODULEPATH", "/apps/modules/modulefiles") prepend_path("PATH", "/contrib/EPIC/bin") load("gnu") load("stack-intel") load("stack-intel-oneapi-mpi") stack_intel_ver=os.getenv("stack_intel_ver") or "2021.10.0" load(pathJoin("stack-intel", stack_intel_ver)) stack_impi_ver=os.getenv("stack_impi_ver") or "2021.10.0" load(pathJoin("stack-intel-oneapi-mpi", stack_impi_ver)) unload("gnu") load("common4noaacloud") setenv("CC", "mpiicc") setenv("CXX", "mpiicpc") setenv("FC", "mpiifort") whatis("Description: UFS_UTILS build environment on NOAA Cloud")

…

On Wed, Sep 11, 2024 at 6:57 AM Natalie Perlin ***@***.***> wrote: What are the steps to reproduce the error? — Reply to this email directly, view it on GitHub <#982 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASMCH62OPMWSEPBY3EOUYMLZWA43XAVCNFSM6AAAAABN7W5EJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBTGU4TQMJTHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

weihuang-jedi · 2024-09-12T14:12:19Z

More error msg:

/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/emcsfc_ice_blend.fd/emcsfc_ice_blend.f90(95): error #7012: The module file cannot be read. Its format requires a more recent F90 compiler. [GRIB_MOD]
use grib_mod ! grib 2 libraries
-----^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/emcsfc_ice_blend.fd/emcsfc_ice_blend.f90(99): error #6457: This derived type name has not been declared. [GRIBFIELD]
type(gribfield) :: ims !< ims is Ice Mapping System 5-min grid date
------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/emcsfc_ice_blend.fd/emcsfc_ice_blend.f90(100): error #6457: This derived type name has not been declared. [GRIBFIELD]
type(gribfield) :: mask !< data mask
------^
/contrib/Wei.Huang/dev/ufs-utils-cloud/sorc/emcsfc_ice_blend.fd/emcsfc_ice_blend.f90(101): error #6457: This derived type name has not been declared. [GRIBFIELD]
type(gribfield) :: mmab !< NCEP/MMAB global 5-minute ice concentration data

natalie-perlin · 2024-10-01T21:01:30Z

Checking out new NOAA Azure Rocky8 cluster from the Marketplace on Parallel works and setting a modulefile (from git@github.com:NOAA-EPIC/gfs-utils.git) to the one shown below seems to resolve the module loading issue.

help([[
Build environment for GFS utilities on NOAA Cloud
]])

prepend_path("MODULEPATH", "/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/gsi-addon-env/install/modulefiles/Core")
prepend_path("MODULEPATH", "/apps/modules/modulefiles")
load("gnu")
load("stack-intel")
load("stack-intel-oneapi-mpi")
unload("gnu")
load("cmake/3.23.1")

load("gfsutils_common")

whatis("Description: GFS utilities environment on NOAA Cloud with Intel Compilers")

natalie-perlin · 2024-10-01T21:30:32Z

To load the modulefile for the repository git@github.com:NOAA-EPIC/UFS_UTILS-cloud.git, a modulefile common4noaacloud.lua modulefile needs to have the following paths set and modules loaded:

prepend_path("MODULEPATH", "/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/ue-intel/install/modulefiles/Core")
prepend_path("MODULEPATH", "/apps/modules/modulefiles")
load("gnu")
load("stack-intel")
load("stack-intel-oneapi-mpi")
unload("gnu")
load("cmake/3.23.1")

load("common4noaacloud")

A modulefile common4noaacloud.lua needs to have only the following modules loaded:

bacio_ver=os.getenv("bacio_ver") or "2.4.1"
load(pathJoin("bacio", bacio_ver))

g2_ver=os.getenv("g2_ver") or "3.4.5"
load(pathJoin("g2", g2_ver))

ip_ver=os.getenv("ip_ver") or "4.3.0"
load(pathJoin("ip", ip_ver))

nemsio_ver=os.getenv("nemsio_ver") or "2.5.4"
load(pathJoin("nemsio", nemsio_ver))

sp_ver=os.getenv("sp_ver") or "2.5.0"
load(pathJoin("sp", sp_ver))

w3emc_ver=os.getenv("w3emc_ver") or "2.10.0"
load(pathJoin("w3emc", w3emc_ver))

-- Uncomment when CHGRES_ALL is ON
--sfcio_ver=os.getenv("sfcio_ver") or "1.4.1"
--load(pathJoin("sfcio", sfcio_ver))

sigio_ver=os.getenv("sigio_ver") or "2.3.2"
load(pathJoin("sigio", sigio_ver))

zlib_ver=os.getenv("zlib_ver") or "1.2.13"
load(pathJoin("zlib", zlib_ver))
png_ver=os.getenv("png_ver") or "1.6.37"
load(pathJoin("libpng", png_ver))

hdf5_ver=os.getenv("hdf5_ver") or "1.10.6"
load(pathJoin("hdf5", hdf5_ver))

netcdf_ver=os.getenv("netcdf_ver") or "4.6.1"
load(pathJoin("netcdf", netcdf_ver))

nccmp_ver=os.getenv("nccmp_ver") or "1.9.0.1"
load(pathJoin("nccmp", nccmp_ver))

esmf_ver=os.getenv("esmf_ver") or "8.6.0"
load(pathJoin("esmf", esmf_ver))

nco_ver=os.getenv("nco_ver") or "4.9.1"
load(pathJoin("nco", nco_ver))

natalie-perlin · 2024-10-01T21:40:58Z

For the UPP from the repository git@github.com:NOAA-EPIC/UPP-cloud.git, the following changes are needed in ./modulefiles/noaacloud.lua to load the modules:

prepend_path("MODULEPATH", "/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core")
prepend_path("MODULEPATH", "/apps/modules/modulefiles")
load("gnu")
load("stack-intel")
load("stack-intel-oneapi-mpi")
unload("gnu")
load("cmake/3.23.1")

natalie-perlin · 2024-10-01T21:45:13Z

@weihuang-jedi - please let us know if there are still issues with loading modules and building packages on NOAA CSPs with Rocky8!

weihuang-jedi · 2024-10-01T22:44:50Z

With the above changes, UFS-UTILS compiles fine both on AWS and Azure.
Thanks Natalie, a nice job!

Fixes #982.

weihuang-jedi closed this as completed Oct 1, 2024

weihuang-jedi mentioned this issue Oct 2, 2024

Support UFS_UTILS on CSPs under Rocky 8 #989

Merged

2 tasks

GeorgeGayno-NOAA added this to Version 1.14.0 Oct 9, 2024

GeorgeGayno-NOAA moved this to Done in Version 1.14.0 Oct 9, 2024

GeorgeGayno-NOAA assigned weihuang-jedi Oct 9, 2024

GeorgeGayno-NOAA pushed a commit that referenced this issue Oct 9, 2024

Support UFS_UTILS on CSPs under Rocky 8 (#989)

2323761

Fixes #982.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Rocky8 on CSPs #982

Support Rocky8 on CSPs #982

weihuang-jedi commented Sep 10, 2024

natalie-perlin commented Sep 11, 2024 •

edited

Loading

natalie-perlin commented Sep 11, 2024

weihuang-jedi commented Sep 11, 2024 via email

weihuang-jedi commented Sep 12, 2024

natalie-perlin commented Oct 1, 2024

natalie-perlin commented Oct 1, 2024

natalie-perlin commented Oct 1, 2024

natalie-perlin commented Oct 1, 2024

weihuang-jedi commented Oct 1, 2024

Support Rocky8 on CSPs #982

Support Rocky8 on CSPs #982

Comments

weihuang-jedi commented Sep 10, 2024

[Wei.Huang@awsrocky8epicwei-14 ufs-utils-cloud]$ module show netcdf-fortran/4.6.1

/contrib/spack-stack-rocky8/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/intel-oneapi-mpi/2021.10.0/intel/2021.10.0/netcdf-fortran/4.6.1.lua:

natalie-perlin commented Sep 11, 2024 • edited Loading

natalie-perlin commented Sep 11, 2024

weihuang-jedi commented Sep 11, 2024 via email

weihuang-jedi commented Sep 12, 2024

natalie-perlin commented Oct 1, 2024

natalie-perlin commented Oct 1, 2024

natalie-perlin commented Oct 1, 2024

natalie-perlin commented Oct 1, 2024

weihuang-jedi commented Oct 1, 2024

natalie-perlin commented Sep 11, 2024 •

edited

Loading