Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WRFv4.5 not compatible with intel llvm compilers #1884

Closed
HathewayWill opened this issue Jun 17, 2023 · 7 comments · Fixed by #1946
Closed

WRFv4.5 not compatible with intel llvm compilers #1884

HathewayWill opened this issue Jun 17, 2023 · 7 comments · Fixed by #1946
Assignees
@mgduda @islas

Comments

@HathewayWill
Copy link

HathewayWill commented Jun 17, 2023 •
edited by mgduda
Loading

Describe the bug
In the later half of 2023, intel will be mothballing their intel compilers (icc,ifort,icpc) for new intel compilers that are built on the llvm architecture.

These new commands are:

export CC=icx
export CXX=icpx
export FC=ifx
export F77=ifx
export F90=ifx
export MPIFC="mpiifort -fc=ifx"
export MPIF77="mpiifort -fc=ifx"
export MPIF90="mpiifort -fc=ifx"
export MPICC="mpiicc -cc=icx"
export MPICXX="mpiicpc -cxx=icpx"

wrf v4.5 is not compatibile with these new llvm compilers and will make the intel compilers for new users useless. Anyone who has intel compilers and update their intel compiler packages will also lose the ability to use them. This is a MAJOR problem for the WRF community.

To Reproduce
Steps to reproduce the behavior:

Attached is a couple of scripts that show how to install the required libraries needed to run WRF using the intel llvm compilers.
All system tests pass so the new intel llvm compilers

However, when you change the old intel compilers listed in the configure.wrf for the new llvm compilers the wrf will not install.

Expected behavior
wrf v4.5 expected to install with new llvm installers without having to modify any major files.

Attachments
wrf_4.5_new_intel_commands.sh: This is a script that installs all the required libraries for WRF. It works when using the old intel compilers as it is currently being used on github.
wrf.new.inte.compile.log: Build log using the new intel compilers.
configure.wrf: this is the configure file built using the new intel llvm compilers.

WRF_v4.5_Intel_LLVM_Compilers.tar.gz
Edit: link removed
@mgduda mgduda changed the title WRFv4.5 not compatible with intel llvm compilers (MAJOR PROBLEM) WRFv4.5 not compatible with intel llvm compilers Jun 19, 2023
@mgduda
Copy link
Collaborator

mgduda commented Jun 19, 2023

@HathewayWill I've modified the title of this Issue, as it's atypical to provide an assessment of the severity of an issue in the title.

Can you provide more details about the exact nature of the incompatibility?

@mgduda
Copy link
Collaborator

mgduda commented Jun 19, 2023

I've removed the download link, as the script in the download is system-specific and contains sudo commands.

@HathewayWill
Copy link
Author

I've removed the download link, as the script in the download is system-specific and contains sudo commands.

@mgduda I will upload the compile logs and configure.wrf for reference material.

@HathewayWill
Copy link
Author

HathewayWill commented Jun 19, 2023 •
edited
Loading

@HathewayWill I've modified the title of this Issue, as it's atypical to provide an assessment of the severity of an issue in the title.

Can you provide more details about the exact nature of the incompatibility?

@mgduda

so the first issue is that the intel LLVM compilers require specific warning commands to remove errors that the old jasper 1.900.1

next is that you have to use automake and autoconfigure in the process to install the libraries properly with the new intel llvm.

These commands are here:

############################# ZLib ############################

  cd $WRF_FOLDER/Downloads
  tar -xvzf v1.2.13.tar.gz
  cd zlib-1.2.13/
  

  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS  ./configure --prefix=$DIR/grib2
  

  make -j $CPU_HALF_EVEN 2>&1 | tee zlib.make.log
  # make check | tee zlib.makecheck.log
  make -j $CPU_HALF_EVEN install 2>&1 | tee make.install.log | tee zlib.makeinstall.log

  echo " "
  ############################# LibPNG ############################

  cd $WRF_FOLDER/Downloads

  # other libraries below need these variables to be set
  export LDFLAGS=-L$DIR/grib2/lib
  export CPPFLAGS=-I$DIR/grib2/include

  tar -xvzf libpng-1.6.39.tar.gz
  cd libpng-1.6.39/
  autoreconf -i -f
  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS  ./configure --prefix=$DIR/grib2
  
  automake -a -f
  make -j $CPU_HALF_EVEN 2>&1 | tee libpng.make.log
  #make -j $CPU_HALF_EVEN check | tee libpng.makecheck.log
  make -j $CPU_HALF_EVEN install 2>&1 | tee libpng.makeinstall.log

  echo " "
  ############################# JasPer ############################

  cd $WRF_FOLDER/Downloads
  unzip jasper-1.900.1.zip
  cd jasper-1.900.1/
  autoreconf -i -f
  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS  ./configure --prefix=$DIR/grib2
  
  automake -a -f
  make -j $CPU_HALF_EVEN 2>&1 | tee jasper.make.log
  make  -j $CPU_HALF_EVEN install 2>&1 | tee jasper.makeinstall.log

  # other libraries below need these variables to be set
  export JASPERLIB=$DIR/grib2/lib
  export JASPERINC=$DIR/grib2/include

  echo " "
  ############################# HDF5 library for NetCDF4 & parallel functionality ############################

  cd $WRF_FOLDER/Downloads
  tar -xvzf hdf5-1_14_1-2.tar.gz
  cd hdf5-hdf5-1_14_1-2
  autoreconf -i -f
  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS ./configure --prefix=$DIR/grib2 --with-zlib=$DIR/grib2 --enable-hl --enable-fortran --enable-parallel

  automake -a -f
  make -j $CPU_HALF_EVEN 2>&1 | tee hdf5.make.log
  make -j $CPU_HALF_EVEN install 2>&1 | tee hdf5.makeinstall.log

  # other libraries below need these variables to be set
  export HDF5=$DIR/grib2
  export LD_LIBRARY_PATH=$DIR/grib2/lib:$LD_LIBRARY_PATH
  export PATH=$HDF5/bin:$PATH
  export PHDF5=$DIR/grib2

  echo " "


  #############################Install Parallel-netCDF##############################
  #Make file created with half of available cpu cores
  #Hard path for MPI added
  ##################################################################################
  cd $WRF_FOLDER/Downloads
  tar -xzvf pnetcdf-1.12.3.tar.gz
  cd pnetcdf-1.12.3
  autoreconf -i -f
  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS ./configure --prefix=$DIR/grib2  --enable-shared --enable-static 2>&1 | tee configure_pnetcdf.log

  automake -a -f
  make -j $CPU_HALF_EVEN 2>&1 | tee make.log
  make -j $CPU_HALF_EVEN install 2>&1 | tee make.install.log
  #make check

  export PNETCDF=$DIR/grib2


  ############################## Install NETCDF-C Library ############################

  cd $WRF_FOLDER/Downloads
  tar -xzvf v4.9.2.tar.gz
  cd netcdf-c-4.9.2/

  # these variables need to be set for the NetCDF-C install to work
  export CPPFLAGS=-I$DIR/grib2/include
  export LDFLAGS=-L$DIR/grib2/lib
  export LIBS="-lhdf5_hl -lhdf5 -lz -lcurl -lgfortran -lgcc -lm -ldl -lpnetcdf"
  
  autoreconf -i -f
  
  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS  ./configure --prefix=$DIR/NETCDF --disable-dap --enable-netcdf-4 --enable-netcdf4 --enable-shared --enable-pnetcdf --enable-cdf5 --enable-parallel-tests | tee netcdf.configure.log

  automake -a -f
  make -j $CPU_HALF_EVEN 2>&1 | tee netcdf.make.log
  make -j $CPU_HALF_EVEN install 2>&1 | tee make.install.log | tee netcdf.makeinstall.log

  # other libraries below need these variables to be set
  export PATH=$DIR/NETCDF/bin:$PATH
  export NETCDF=$DIR/NETCDF

  echo " "
  ############################## NetCDF-Fortran library ############################

  cd $WRF_FOLDER/Downloads
  tar -xvzf v4.6.0.tar.gz
  cd netcdf-fortran-4.6.0/

  # these variables need to be set for the NetCDF-Fortran install to work
  export LD_LIBRARY_PATH=$DIR/NETCDF/lib:$LD_LIBRARY_PATH
  export CPPFLAGS="-I$DIR/NETCDF/include -I$DIR/grib2/include"
  export LDFLAGS="-L$DIR/NETCDF/lib -L$DIR/grib2/lib"
  export LIBS="-lnetcdf -lpnetcdf -lcurl -lhdf5_hl -lhdf5 -lz -lm -ldl -lgcc -lgfortran"
  
  autoreconf -i -f
  CC=$MPICC FC=$MPIFC CXX=$MPICXX F90=$MPIF90 F77=$MPIF77 CFLAGS=$CFLAGS  ./configure --prefix=$DIR/NETCDF --enable-netcdf-4 --enable-netcdf4 --enable-parallel-tests --enable-hdf5
  
  automake -a -f
  make -j $CPU_HALF_EVEN 2>&1 | tee netcdf-f.make.log
  make  -j $CPU_HALF_EVEN install 2>&1 | tee netcdf-f.makeinstall.log


  echo " "

Here is some links from intel about their new llvm compiler porting:

https://www.intel.com/content/www/us/en/developer/articles/guide/porting-guide-for-ifort-to-ifx.html

https://www.intel.com/content/www/us/en/developer/articles/guide/porting-guide-for-icc-users-to-dpcpp-or-icx.html

As for the WRF problems, I'm not even sure where to begin. There are so many errors in the compile log that I will just have to attach them.

wrf.intel.llvm.zip

One thing worth nothing is those commands above for installation of the libraries do pass all the system environment and compatibility tests.

@mgduda
Copy link
Collaborator

mgduda commented Jun 30, 2023

In your wrf.compile.log, one of the first errors that I see suggests a problem with your MPI compiler wrappers not accepting -fc=...:

make[3]: Entering directory '/home/workhorse/WRF_Intel/WRFV4.5/external/io_pnetcdf'
make[3]: c=ifx -real-size 32 -i4  -O3 -ip -fp-model precise -w -ftz -align all -fno-alias -FR -convert big_endian  : No such file or directory
make[3]: *** No rule to make target 'c=ifx -real-size 32 -i4  -O3 -ip -fp-model precise -w -ftz -align all -fno-alias -FR -convert big_endian  '.  Stop.
make[3]: Leaving directory '/home/workhorse/WRF_Intel/WRFV4.5/external/io_pnetcdf'
make[2]: [../configure.wrf:313: wrfio_pnf] Error 2 (ignored)

For what it's worth, initial testing on our end with the Intel oneAPI compilers hasn't turned up any problems in the WRF source code.

I'm closing this issue, but if you can point to a specific problem in the WRF code or WRF configure/compile/configure.defaults, feel free to re-open it an add more details.

@mgduda mgduda closed this as completed Jun 30, 2023
@HathewayWill
Copy link
Author

In your wrf.compile.log, one of the first errors that I see suggests a problem with your MPI compiler wrappers not accepting -fc=...:

make[3]: Entering directory '/home/workhorse/WRF_Intel/WRFV4.5/external/io_pnetcdf'
make[3]: c=ifx -real-size 32 -i4  -O3 -ip -fp-model precise -w -ftz -align all -fno-alias -FR -convert big_endian  : No such file or directory
make[3]: *** No rule to make target 'c=ifx -real-size 32 -i4  -O3 -ip -fp-model precise -w -ftz -align all -fno-alias -FR -convert big_endian  '.  Stop.
make[3]: Leaving directory '/home/workhorse/WRF_Intel/WRFV4.5/external/io_pnetcdf'
make[2]: [../configure.wrf:313: wrfio_pnf] Error 2 (ignored)

For what it's worth, initial testing on our end with the Intel oneAPI compilers hasn't turned up any problems in the WRF source code.

I'm closing this issue, but if you can point to a specific problem in the WRF code or WRF configure/compile/configure.defaults, feel free to re-open it an add more details.

Were you using the new llvm intel compilers? I'll try to see what's going on on my side.

@HathewayWill
Copy link
Author

In your wrf.compile.log, one of the first errors that I see suggests a problem with your MPI compiler wrappers not accepting -fc=...:

make[3]: Entering directory '/home/workhorse/WRF_Intel/WRFV4.5/external/io_pnetcdf'
make[3]: c=ifx -real-size 32 -i4  -O3 -ip -fp-model precise -w -ftz -align all -fno-alias -FR -convert big_endian  : No such file or directory
make[3]: *** No rule to make target 'c=ifx -real-size 32 -i4  -O3 -ip -fp-model precise -w -ftz -align all -fno-alias -FR -convert big_endian  '.  Stop.
make[3]: Leaving directory '/home/workhorse/WRF_Intel/WRFV4.5/external/io_pnetcdf'
make[2]: [../configure.wrf:313: wrfio_pnf] Error 2 (ignored)

For what it's worth, initial testing on our end with the Intel oneAPI compilers hasn't turned up any problems in the WRF source code.
I'm closing this issue, but if you can point to a specific problem in the WRF code or WRF configure/compile/configure.defaults, feel free to re-open it an add more details.

Were you using the new llvm intel compilers? I'll try to see what's going on on my side.

Was able to get the the libraries to install properly but the WRF doesn't recognize the llvm compilers. I'm not sure what to do here.

@islas islas mentioned this issue Jul 3, 2023
Merged
weiwangncar pushed a commit that referenced this issue Jul 19, 2023
@islas
Intel oneAPI LLVM Compilers Configuration (#1893)
5047064 
Addition of compile configuration using the new Intel LLVM compilers ifx and icx for Fortran and C code, respectively

TYPE: enhancement

KEYWORDS: Intel, LLVM, oneAPI, compilation

SOURCE: internal

DESCRIPTION OF CHANGES:
Problem:
Addressing issue #1884 for Intel oneAPI ifx/icx builds

Solution:
Add configuration to configure.defaults mirroring original Intel ifort/icc build with minor tweaking

ISSUE: For use when this PR closes an issue.
Fixes #1884

LIST OF MODIFIED FILES:
M arch/configure.defaults

TESTS CONDUCTED:
1. WRF Core em_real compiles with ifx/icx (see attached log).
2. It does not impact other parts of the code.

RELEASE NOTE:
Added build configuration for new Intel oneAPI LLVM ifx/icx compilers, which will be available on NCAR's new computer, Derecho.
@weiwangncar weiwangncar mentioned this issue Dec 7, 2023
Merged
weiwangncar added a commit that referenced this issue Dec 9, 2023
@weiwangncar
add compilation stanza for Intel oneAPI for 4.5.2 (#1946)
45215e0 
TYPE: enhancement

KEYWORDS: Intel, LLVM, oneAPI, compilation

SOURCE: internal

DESCRIPTION OF CHANGES:
Problem:
Addressing issue #1884 for Intel oneAPI ifx/icx builds

Solution:
Add configuration to configure.defaults mirroring original Intel ifort/icc build with minor tweaking.

ISSUE: For use when this PR closes an issue.
Fixes #1884

LIST OF MODIFIED FILES:
M arch/configure.defaults

TESTS CONDUCTED:

1. WRF core em_real compiles with ifx/icx in combination with PR-1823.
2. The Jenkins tests are all passing.
Note that this is the same PR as in #1893, but committed to release-v4.5.2 branch (instead of develop)

RELEASE NOTE: Added build configuration for new Intel oneAPI LLVM ifx/icx compilers, which will be available on NCAR's new computer, Derecho.
@HathewayWill HathewayWill mentioned this issue Dec 22, 2023
Closed
vlakshmanan-scala pushed a commit to scala-computing/WRF that referenced this issue Apr 4, 2024
@weiwangncar @vlakshmanan-scala
add compilation stanza for Intel oneAPI for 4.5.2 (wrf-model#1946)
f71c461 
TYPE: enhancement

KEYWORDS: Intel, LLVM, oneAPI, compilation

SOURCE: internal

DESCRIPTION OF CHANGES:
Problem:
Addressing issue wrf-model#1884 for Intel oneAPI ifx/icx builds

Solution:
Add configuration to configure.defaults mirroring original Intel ifort/icc build with minor tweaking.

ISSUE: For use when this PR closes an issue.
Fixes wrf-model#1884

LIST OF MODIFIED FILES:
M arch/configure.defaults

TESTS CONDUCTED:

1. WRF core em_real compiles with ifx/icx in combination with PR-1823.
2. The Jenkins tests are all passing.
Note that this is the same PR as in wrf-model#1893, but committed to release-v4.5.2 branch (instead of develop)

RELEASE NOTE: Added build configuration for new Intel oneAPI LLVM ifx/icx compilers, which will be available on NCAR's new computer, Derecho.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mgduda mgduda

@islas islas

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add compilation stanza for Intel oneAPI for 4.5.2 weiwangncar/WRF
4 participants
@mgduda @davegill @HathewayWill @islas