Upgrade modules to spack-stack/1.5.1 on non-production machines.

[DavidHuber-NOAA]

This upgrades the modules on non-productions machines to use the local spack-stack installations. This has been build-tested on Hera (Intel and GNU), Orion, S4, Jet, and WCOSS2. It has been run-tested on Orion and Hera within global workflow experiments.

In the process, the Cheyenne modulefiles have been removed as that system is being decomissioned and spack-stack/1.5.1 is not available there.

Resolves #18

[aerorahul]

looks good to me

[RussTreadon-NOAA]

We have no CI testing for GSI-utils. This is problematic.

@DavidHuber-NOAA tested the updated GSI-utils in g-w on Hera and Orion. The build was tested on WCOSS2 (along with other machines). Since Dave did not report any anomalies, I assume GSI-utils behaves acceptably when built from spack-stack/1.5.1.

`src/netcdf_io/interp_inc.fd/driver.F90 was updated. The change is related to ip. An ip-lib subject matter expert could check this update if we want confirmation that the change is OK.

[DavidHuber-NOAA]

@RussTreadon-NOAA I am working on updating the CI for gsi-utils for this PR. I will submit it shortly.

[DavidHuber-NOAA]

Hmm, for some reason, sudo apt install cmake is installing version 3.16, which is too old for our purposes. I will need to read up on how to force a newer version.

[DavidHuber-NOAA]

@aerorahul I'm not sure how to get cmake/hdf5/MPI to work together here. As is, cmake, during the hdf5 build, is unable to find MPI_C. UFS_utils, which I have been using as a template, uses ubuntu-20.04, which builds cmake/3.16, but that version is too old for at least one of the library builds, so I am currently working with ubuntu-latest and this version is unable to find MPI. Any ideas on how to reconcile this?

[aerorahul]

@aerorahul I'm not sure how to get cmake/hdf5/MPI to work together here. As is, cmake, during the hdf5 build, is unable to find MPI_C. UFS_utils, which I have been using as a template, uses ubuntu-20.04, which builds cmake/3.16, but that version is too old for at least one of the library builds, so I am currently working with ubuntu-latest and this version is unable to find MPI. Any ideas on how to reconcile this?

can you install cmake from spack in the spack.yaml?

[aerorahul]

LGTM.
Thanks for updating the CI, updating the utilities to newer versions of the libraries and porting to spack-stack.

[DavidHuber-NOAA]

@souopgui-redline identified a bug causing a hang on S4. The netcdf file read in interp_inc is opened in parallel mode, but only read serially. Changing this to open it in serial mode solved the hang. This has only been tested on S4 so far, but I could run a quick test on Orion as well.

[DavidHuber-NOAA]

This also worked on Orion. I think this is now ready to merge once CI tests complete.

[RussTreadon-NOAA]

Trust your testing @DavidHuber-NOAA and trust @aerorahul 's review.

Approve.

[DavidHuber-NOAA]

All CI tests passed. I don't think there are any further hurdles and that this is ready to merge.