Error installing with compile_MET_all.sh

[benjamin-cash]

I attempted installing MET via compile_MET_all.sh, and it made it to the make step before dying with the following:

numpbmsg.o: In function numpbmsg_': numpbmsg.f:(.text+0x2a): undefined reference to ufbtab_'
numpbmsg.o: In function numpbmsg_new_': numpbmsg.f:(.text+0xd0): undefined reference to ufbtab_'
collect2: error: ld returned 1 exit status
make[5]: *** [Makefile:475: pb2nc] Error 1
make[5]: Leaving directory '/groups/BCASH/MET-11.0.1/src/tools/other/pb2nc'
make[4]: *** [Makefile:395: all-recursive] Error 1
make[4]: Leaving directory '/groups/BCASH/MET-11.0.1/src/tools/other'
make[3]: *** [Makefile:374: all-recursive] Error 1
make[3]: Leaving directory '/groups/BCASH/MET-11.0.1/src/tools'
make[2]: *** [Makefile:371: all-recursive] Error 1
make[2]: Leaving directory '/groups/BCASH/MET-11.0.1/src'
make[1]: *** [Makefile:429: all-recursive] Error 1
make[1]: Leaving directory '/groups/BCASH/MET-11.0.1'
make: *** [Makefile:370: all] Error 2

[jprestop]

Hi @benjamin-cash. I suspect the BUFR library did not compile successfully. If I can take a look at some of the files, I will likely be able to figure out what the problem is. Can you please attach the make.log file for the BURF library to this Discussion? If you are using the compile_MET_all.sh script to install BUFR lib, the make.log file should be in external_libs/bufrlib/BUFRLIB_v11.3.0/make.log. Also could you please attach the MET config.log file and the configuration file that you are passing to compile_MET_all.sh?

[benjamin-cash]

Hi @jprestop - This leads to a different error, because the script then looks for a module gnu/10.3.0, which does not exist on my system. But it looks like the script does not do a module purge, so I will try loading the gnu/hpc-openmpi module manually before running the script and then setting the compiler as you indicate here.

[jprestop]

@benjamin-cash That is interesting. We have not encountered this issue previously. Hopefully, it will work out with the gnu/hpc-openmpi module loaded manually.

[benjamin-cash]

The BUFRLIB make.log now shows:
ufbtab.f:306:31:

288 | DO WHILE(IREADMG(-LUNIT,SUBSET,IDATE).GE.0)
| 2
......
306 | 10 IF(IREADMG(-LUNIT,SUBSET,JDATE).LT.0) GOTO 25
| 1
Warning: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
ufbtab.f:534:37:

288 | DO WHILE(IREADMG(-LUNIT,SUBSET,IDATE).GE.0)
| 2
......
534 | DO WHILE(IREADMG(-LUNIT,SUBSET,JDATE).GE.0)
| 1
Warning: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)

It appears to have compile BUFRLIB successfully, although it still appears to be making the other libraries so I don't know if the build as a whole works yet.

[benjamin-cash]

@benjamin-cash That is interesting. We have not encountered this issue previously. Hopefully, it will work out with the gnu/hpc-openmpi module loaded manually.

The modules on hopper (the GMU cluster) seem to have somewhat idiosyncratic naming convention. The gnu modules include the version number in the name, e.g. gnu9/9.3.0. However, if you put gnu9_9.3.0 for COMPILER, the script will error out at line 298 because gnu9 is not in the list of valid compilers.

[benjamin-cash]

As a bit of follow-up, the issue with the script arises because the script assumes the module is going to be named $COMPILER_FAMILY/$COMPILER_VERSION. Any other naming convention is going to cause an error. Given that the script only load the compiler and python modules it might be good to allow the user to override this default module and specify these.

[benjamin-cash]

I've run into a different issue now:
/usr/bin/ld: cannot find -lintl
collect2: error: ld returned 1 exit status
make[5]: *** [Makefile:433: ensemble_stat] Error 1
make[5]: Leaving directory '/groups/BCASH/MET-11.0.1/src/tools/core/ensemble_stat'
make[4]: *** [Makefile:380: all-recursive] Error 1
make[4]: Leaving directory '/groups/BCASH/MET-11.0.1/src/tools/core'
make[3]: *** [Makefile:374: all-recursive] Error 1
make[3]: Leaving directory '/groups/BCASH/MET-11.0.1/src/tools'
make[2]: *** [Makefile:371: all-recursive] Error 1
make[2]: Leaving directory '/groups/BCASH/MET-11.0.1/src'
make[1]: *** [Makefile:429: all-recursive] Error 1
make[1]: Leaving directory '/groups/BCASH/MET-11.0.1'
make: *** [Makefile:370: all] Error 2

[jprestop]

Hi @benjamin-cash. I recently experienced this same problem on one of the WCOSS2 HPC machines.

Your value for "MET_PYTHON_LD":

export MET_PYTHON_LD="$(python3-config --ldflags --embed)"

is likely including "-lintl" and the linker cannot seem to find this file, so you will need to find it and tell the linker where to get the library file it needs. To find the file, please run the following command:

find /opt/sw/spack -name "libintl*" -type f

Once you find the file, place the directory only (do not include the file name) in a variable called ADDTL_DIR in your install_met_env.generic.txt. You can see an example of using that variable in this file:
https://github.com/dtcenter/MET/blob/develop/internal/scripts/installation/config/install_met_env.acorn_py3.10

Please let us know how it goes. With regard to:

As a bit of follow-up, the issue with the script arises because the script assumes the module is going to be named $COMPILER_FAMILY/$COMPILER_VERSION. Any other naming convention is going to cause an error. Given that the script only load the compiler and python modules it might be good to allow the user to override this default module and specify these.

In light of this problem, it does seem like a good idea to allow the user to override those values. I'll create an issue to resolve this problem.

[benjamin-cash]

Thanks @jprestop - Do I need to go through the entirety of compile_MET_all.sh again once I do this?

[jprestop]

You only need to compile MET. You can set the other variables so that the libraries do not recompile.

[benjamin-cash]

Thanks. :) A different question, looking at that install file. There are a few lib64 directories, but also export SET_D64BIT=FALSE. Should that be set TRUE for a 64bit machine?

[jprestop]

That should NOT be set TRUE for a 64bit machine. The D64BIT flag used to be used in compiling MET because the g2c library used it. If the g2c library used it, MET also needed to be compiled with it. However, compiling with it (for both g2clib and MET) has shown problems with GRIB/GRIB2 files over 2GB in size. It it was completely removed with version 1.6.4 of the g2c library - NOAA-EMC/NCEPLIBS-g2c#30, but on some machines, the g2c library is still being built with it, which is why that option exists the configuration file.

[benjamin-cash]

Success! Using compile_MET_all.sh and install_met_env.generic I was able to install MET and run through all of the tests. Modifications required to make this happen were:

1. Adding --embed to the export MET_PYTHON_LD statement
2. Adding export ADDTL_DIR to point to location of -lintl
3. Specifying COMPILER in the expected _ format and then manually loading the compiler module.

@jprestop

[benjamin-cash]

Thanks for all of the help in getting through this. :)

[jprestop]

@benjamin-cash That is great news! Thank you for the nice summary you have provided here and for marking an answer. I'll go ahead and lock this Discussion. I'm glad you got it working!