Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v4.6 build "make petsc-slepc" fails with "C compiler you provided with -with-cc= cannot be found or does not work." on Ubuntu 20.04 #141

Closed
cirosantilli opened this issue Jun 9, 2020 · 23 comments
Closed

v4.6 build "make petsc-slepc" fails with "C compiler you provided with -with-cc= cannot be found or does not work." on Ubuntu 20.04 #141

cirosantilli opened this issue Jun 9, 2020 · 23 comments

Comments

@cirosantilli
Copy link

cirosantilli commented Jun 9, 2020 •
edited
Loading

Just trying it out for fun, I ran the build commands from the README:

sudo apt build-dep freefem
git clone https://github.com/FreeFem/FreeFem-sources
cd FreeFem-sources
git checkout v4.6
autoreconf -i
./configure --enable-download --enable-optim --prefix="$(pwd)/FreeFem-install"
./3rdparty/getall -a
cd 3rdparty/ff-petsc
make petsc-slepc

and the last command fails with:

dirname: missing operand
Try 'dirname --help' for more information.
mkdir -p /home/ciro/test/FreeFem-sources/FreeFem-install/ff-petsc//r
chown ciro /home/ciro/test/FreeFem-sources/FreeFem-install/ff-petsc//r
mkdir -p /home/ciro/test/FreeFem-sources/FreeFem-install/ff-petsc//c
chown ciro /home/ciro/test/FreeFem-sources/FreeFem-install/ff-petsc//c
no need of sudo
test -e "/home/ciro/test/FreeFem-sources/FreeFem-install/ff-petsc//r" -a -e "/home/ciro/test/FreeFem-sources/FreeFem-install/ff-petsc//c"
cd petsc-3.13.0 && ./configure MAKEFLAGS='' \
        --prefix=/home/ciro/test/FreeFem-sources/FreeFem-install/ff-petsc//r \
        --with-debugging=0 COPTFLAGS='-O3 -mtune=native' CXXOPTFLAGS='-O3 -mtune=native' FOPTFLAGS='-O3 -mtune=native' --with-cxx-dialect=C++11 --with-ssl=0 --with-x=0 --with-fortran-bindings=0 --with-cc='' --with-cxx='' --with-fc='' --with-scalar-type=real --with-blaslapack-include='' --with-blaslapack-lib='-llapack -lblas' --download-scalapack --download-metis --download-ptscotch --download-mumps --download-hypre --download-parmetis --download-superlu --download-suitesparse --download-tetgen --download-slepc --download-hpddm --download-hpddm-commit=e8639ff PETSC_ARCH=fr
===============================================================================
             Configuring PETSc to compile on your system
===============================================================================
TESTING: checkCCompiler from config.setCompilers(config/BuildSystem/config/setCompilers.py:604)                                                                                                                                                                                 *******************************************************************************
         UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log for details):
-------------------------------------------------------------------------------
C compiler you provided with -with-cc= cannot be found or does not work.
*******************************************************************************

make: *** [Makefile:210: petsc-3.13.0/tag-conf-real] Error 1

This error message is mentioned at:

but I couldn't find easily in those threads the solution/if we share the same problem.

Edit: working procedure

Maybe you need to uninstall openmpi before starting. Then this runs the hello world examples from the repo:

sudo apt build-dep freefem
git clone https://github.com/FreeFem/FreeFem-sources
cd FreeFem-sources
# Post v4.6 with some fixes.
git checkout 3df0e2370d9752801ac744b11307b14e16743a44

# Won't apply automatically due to tab hell.
# https://superuser.com/questions/607410/how-to-copy-paste-tab-characters-via-the-clipboard-into-terminal-session-on-gnom
git apply <<'EOS'
diff --git a/3rdparty/ff-petsc/Makefile b/3rdparty/ff-petsc/Makefile
index dc62ab06..13cd3253 100644
--- a/3rdparty/ff-petsc/Makefile
+++ b/3rdparty/ff-petsc/Makefile
@@ -204,7 +204,7 @@ $(SRCDIR)/tag-make-real:$(SRCDIR)/tag-conf-real
 $(SRCDIR)/tag-install-real :$(SRCDIR)/tag-make-real
     cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr install
     -test -x "`type -p otool`" && make changer
-    cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr check
+    #cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr check
     test -e $(DIR_INSTALL_REAL)/include/petsc.h
     test -e $(DIR_INSTALL_REAL)/lib/petsc/conf/petscvariables
     touch $@
@@ -293,7 +293,6 @@ $(SRCDIR)/tag-tar:$(PACKAGE)
     -tar xzf $(PACKAGE)
     patch -p1 < petsc-hpddm.patch
 ifeq ($(WIN32DLLTARGET),)
-    patch -p1 < petsc-metis.patch
 endif
     touch $@
 $(PACKAGE):
EOS

autoreconf -i
./configure --enable-download --enable-optim --prefix="$(pwd)/../FreeFem-install"
./3rdparty/getall -a
cd 3rdparty/ff-petsc
make petsc-slepc
cd -
./reconfigure
make -j`nproc`
make install
cd ../FreeFem-install
PATH="${PATH}:$(pwd)/bin" ./bin/FreeFem++ ../FreeFem-sources/examples/tutorial/
@prj-
Copy link
Member

prj- commented Jun 9, 2020 •
edited
Loading

You do not have MPI installed on your machine (or at least FreeFEM can't detect it).
Two options:

  1. install MPI using your favorite package manager, then reconfigure, and if this still fails, send us your FreeFEM config.log;
  2. switch to the develop branch, which now lets PETSc install MPICH for you in case FreeFEM can't detect MPI. Then PETSc-installed MPICH will be used by FreeFem++-mpi. This will be available in the next FreeFEM release (4.6.1 or 4.7).

Please let us know if this fixes your issue.

@cirosantilli
Copy link
Author

Hmmm, I think I already had the package libopenmpi-dev installed. Now I have it for sure, and also installed sudo apt install libmpich-dev` and I restarted the process, but got the same error.

The problem seems to come from:

-with-cc=''

and then if I do:

make petsc-slepc MPICC=/usr/bin/gcc MPICXX=/usr/bin/g++ MPIFC=/usr/bin/gfortran

it progresses much further and blows up at:

C++ error! MPI_Finalize() could not be located!

From logs, it failed to compile:

Executing: /usr/bin/g++ -c -o /tmp/petsc-tlf4tu2b/config.libraries/conftest.o -I/tmp/petsc-tlf4tu2b/config.setCompilers -I/tmp/petsc-tlf4tu2b/config.compilers -I/tmp/petsc-tlf4tu2b/config.types -I/tmp/petsc-tlf4tu2b/config.functions -I/tmp/petsc-tlf4tu2b/config.utilities.missing -I/tmp/petsc-tlf4tu2b/config.libraries  -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -O3 -mtune=native  -fPIC    /tmp/petsc-tlf4tu2b/config.libraries/conftest.cc 
Possible ERROR while running compiler: exit code 1
stderr:
/tmp/petsc-tlf4tu2b/config.libraries/conftest.cc:4:10: fatal error: mpi.h: No such file or directory
    4 | #include <mpi.h>
      |          ^~~~~~~
compilation terminated.
Source:
#include "confdefs.h"
#include "conffix.h"
/* Override any gcc2 internal prototype to avoid an error. */
#include <mpi.h>
static void _check_MPI_Finalize() { int ierr;
ierr = MPI_Finalize();; }

int main() {
_check_MPI_Finalize();;
  return 0;
}

because mpi.h is not found, because it needs pkg-config --cflags to provide a missing

-I/usr/lib/x86_64-linux-gnu/openmpi/include

I am able to compile that test program with:

gcc -pthread -I/usr/lib/x86_64-linux-gnu/openmpi/include -I/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi main.c /usr/lib/x86_64-linux-gnu/libmpi.so.40

The pkg-config --libs mpi is broken BTW, the package forgot to provide a symlink from libmpi.so.40 to libmpi.so.

The mpich version pkg-config does work directly however.

I'll try develop next.

@prj-
Copy link
Member

prj- commented Jun 9, 2020 •
edited
Loading

This is not a PETSc problem. This is a FreeFEM problem. Please send your FreeFEM config.log. If you had OpenMPI properly installed, then it should have been passed to PETSc, and you shouldn't have empty wrappers passed.

Also, this shouldn't be make petsc-slepc MPICC=/usr/bin/gcc MPICXX=/usr/bin/g++ MPIFC=/usr/bin/gfortran but make petsc-slepc MPICC=mpicc MPICXX=mpic++ MPIFC=mpif90. You are using sequential compiler, you have to pass MPI wrappers. But again, it should be fixed in FreeFEM ./configure instead.

Also, please do not have both MPICH and OpenMPI installed, this is a terrible idea, unless you exactly know what you are doing.

@cirosantilli
Copy link
Author

config.log: https://gist.github.com/cirosantilli/601c5b1b58827516f0e66c600d552890

3rdparty/ff-petsc/config.log


## ---------------------- ##
## Running config.status. ##
## ---------------------- ##

This file was extended by FreeFEM config.status 4.6, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  CONFIG_FILES    = 
  CONFIG_HEADERS  = 
  CONFIG_LINKS    = 
  CONFIG_COMMANDS = 
  $ ../../config.status --file=Makefile.inc:Makefile-PETSc.inc

on ciro-p51

config.status:1218: creating Makefile.inc

@prj-
Copy link
Member

prj- commented Jun 9, 2020 •
edited
Loading

This is not FreeFEM config.log... this is 3rdparty/ff-petsc/config.log. Look in your config.log, you'll see that MPI is not detected, and please post it here as well.

Thanks, I'll look into that.

@cirosantilli
Copy link
Author

I also posted config.log at https://gist.github.com/cirosantilli/601c5b1b58827516f0e66c600d552890 it's not that one?

@prj-
Copy link
Member

prj- commented Jun 9, 2020

Yup, there is something messed up with your MPI wrappers.

configure:12788: eval /usr/bin/mpic++ -o conftest -g  -DNDEBUG -O3 -mmmx -mavx -std=c++11 -DBAMG_LONG_LONG  -DNCHECKPTR   conftest.cpp -lrt   >&5
/usr/bin/ld: cannot find -lmpi

And later on.

configure:23475:   --  with MPI:     no

You are not the only one having difficulties with OpenMPI/MPICH from Ubuntu 20.04, cf. this post from the forum. I'd suggest removing all MPI packages, and letting PETSc install MPICH itself.

@frederichecht
Copy link
Contributor

frederichecht commented Jun 9, 2020 via email

@cirosantilli
Copy link
Author

cirosantilli commented Jun 9, 2020 •
edited
Loading

OK, Ubuntu mpic++ is completely broken :-)

I now tried on develop at 9e8f0fa

It progressed much further, but it failed later on with:

*******************************************************************************
         UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log for details):
-------------------------------------------------------------------------------
Error configuring METIS with cmake                        
*******************************************************************************

Full config.log https://gist.github.com/cirosantilli/601c5b1b58827516f0e66c600d552890#file-config2-log

3rdparty/ff-petsc/petsc-3.13.2/configure.log: https://gist.github.com/cirosantilli/601c5b1b58827516f0e66c600d552890#file-ff-petsc-config-log

/home/ciro/test/FreeFem-install/ff-petsc/r/bin/mpicc is present, maybe it does not work with ./configure --prefix?

@cirosantilli
Copy link
Author

I wasn't very familiar how gist handles large files, github added a message to me saying I can view it in full at: https://gist.githubusercontent.com/cirosantilli/601c5b1b58827516f0e66c600d552890/raw/f66dc09cb7b5728c15e1ec427ca77892c3cbe796/ff-petsc-config.log and that appears to be the full file. Let me know if you can't see the full thing there.

cmake version 3.16.3.

@prj-
Copy link
Member

prj- commented Jun 9, 2020 •
edited
Loading

OK, I see the issue, I'll get in touch with the PETSc developers. In the meantime, could you please comment/remove this line and start over? Thanks for your patience :)

By start over, I mean delete the folder petsc-3.13.2 and relaunch make petsc-slepc.

@cirosantilli
Copy link
Author

With that change PETSc build appears to have worked, but then it starts running some tests to see if build was correct, and is stuck at:

C/C++ example src/snes/tutorials/ex19 run successfully with mumps

for one hour thirty. ps shows a process running all along:

./ex19 -da_refine 3 -snes_monitor_short -pc_type lu -pc_factor_mat_solver_type umfpack

Is this normal? How long does it take for you?

If it were possible to use a prebuilt PETSc from the package system, that would be awesome :-) related #128 Ubuntu 20.04 is at PETSc 3.12 which is not old.

@prj-
Copy link
Member

prj- commented Jun 9, 2020 •
edited
Loading

No, that's not normal, but you can safely skip that. If you need complex PETSc, do make petsc-3.13.2/tag-install-complex COMPLEX_CASE=1, and then in the top directoy ./reconfigure and you should be good to go.

You can of course use a prebuilt PETSc, cf. the instructions from the issue you mentioned.

It seems your previous issue comes from the fact that you have the nvcc compiler in your path, this should be fixed soon, cf. this PETSc GitLab MR.

prj- added a commit that referenced this issue Jun 9, 2020
@prj-
Disables CUDA in PETSc for now.
1450c2f 
See issue #141.
@prj-
Copy link
Member

prj- commented Jun 9, 2020

I've disabled CUDA in PETSc for now, it should be much easier for you to get through the compilation process.

@cirosantilli
Copy link
Author

Thanks for looking further into this.

I tried make clean;make petsc-3.13.2/tag-install-complex COMPLEX_CASE=1 but it still got stuck on that test.

BTW, I ctrl+c kill the run and then try to proceed with the freefem build, I got this (but might be due to failed previous step?):

SuperLu.cpp:29:10: fatal error: slu_ddefs.h: No such file or directory                                                                  
   29 | #include "slu_ddefs.h"                          
      |          ^~~~~~~~~~~~~                                      
compilation terminated.                                      
make[4]: *** [Makefile:894: SuperLu.so] Error 1

The file /home/ciro/test/FreeFem-install/ff-petsc/r/include/slu_ddefs.h exists.

@prj-
Copy link
Member

prj- commented Jun 10, 2020

Could you please post your FreeFEM config.log after having ./reconfigure? I'm now realizing that before making complex PETSc, FreeFEM does a make check, which in your case fails. So you need to comment this line first. I've explicitly deactivated CUDA in 1450c2f, so I hope these problems won't appear if you restart from scratch.

@cirosantilli
Copy link
Author

This is the config.log: https://gist.githubusercontent.com/cirosantilli/601c5b1b58827516f0e66c600d552890/raw/5abcb5db3dcb98f63e8a9047e5fa529ef1779d46/reconfigure

@prj-
Copy link
Member

prj- commented Jun 10, 2020 •
edited
Loading

You still have /usr/bin/mpic++ in your path. Mixing MPI implementations is kind of a bad idea, as highlighted before. Any chance you could completely remove your previous MPI installation?

configure:13798: eval /usr/bin/mpic++ -o conftest -g  -DNDEBUG -O3 -mmmx -mavx -std=c++11 -DBAMG_LONG_LONG  -DNCHECKPTR   conftest.cpp -lrt   >&5
conftest.cpp:31:10: fatal error: mpi.h: No such file or directory
   31 | #include <mpi.h>
      |          ^~~~~~~

@frederichecht
Copy link
Contributor

frederichecht commented Jun 10, 2020 via email

prj- added a commit that referenced this issue Jun 10, 2020
@prj-
Removes MPIF77 definition.
dffba71 
MUMPS and ScaLAPACK use MPIFC. See issue #141.
@prj-
Copy link
Member

prj- commented Jun 10, 2020

You actually don't need to set the PATH variable since dffba71. I was previously only getting the MPIRUN variable from PETSc, and your MPICXX variable was pointing to the faulty /usr/bin/mpic++ wrapper. See the config.log:

configure:13442: checking for MPIRUN
configure:13501: result: /home/ciro/test/FreeFem-install/ff-petsc/r/bin/mpiexec <-- good
[...]
configure:13775: checking for MPICXX
configure:13798: eval /usr/bin/mpic++ -o conftest -g  -DNDEBUG -O3 -mmmx -mavx -std=c++11 -DBAMG_LONG_LONG  -DNCHECKPTR   conftest.cpp -lrt   >&5 <-- bad

So, now, you don't need to uninstall your system MPI. You can git pull the develop branch, autoreconf -i && ./reconfigure. If this still fails, please do upload a newly generated config.log.

Thanks for all the troubleshooting!

@frederichecht
Copy link
Contributor

frederichecht commented Jun 10, 2020 via email

@cirosantilli
Copy link
Author

I nuked Ubuntu openmpi, pulled FreeFem to 3df0e23 applied:

diff --git a/3rdparty/ff-petsc/Makefile b/3rdparty/ff-petsc/Makefile
index dc62ab06..13cd3253 100644
--- a/3rdparty/ff-petsc/Makefile
+++ b/3rdparty/ff-petsc/Makefile
@@ -204,7 +204,7 @@ $(SRCDIR)/tag-make-real:$(SRCDIR)/tag-conf-real
 $(SRCDIR)/tag-install-real :$(SRCDIR)/tag-make-real
 	cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr install
 	-test -x "`type -p otool`" && make changer
-	cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr check
+	#cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr check
 	test -e $(DIR_INSTALL_REAL)/include/petsc.h
 	test -e $(DIR_INSTALL_REAL)/lib/petsc/conf/petscvariables
 	touch $@
@@ -293,7 +293,6 @@ $(SRCDIR)/tag-tar:$(PACKAGE)
 	-tar xzf $(PACKAGE)
 	patch -p1 < petsc-hpddm.patch
 ifeq ($(WIN32DLLTARGET),)
-	patch -p1 < petsc-metis.patch
 endif
 	touch $@
 $(PACKAGE):

and now it worked!!!

cd ../FreeFem-install
PATH="${PATH}:$(pwd)/bin" ./bin/FreeFem++ tmp.pde

Thanks so much for your help, feel free to close if you feel that all the problems found have been addressed.

If possible, I'd highly recommend adding instructions to build with Ubuntu packaged PETSc to README :-)

@prj-
Copy link
Member

prj- commented Jun 11, 2020

That's good news :)
With the commit you fetched, I actually don't believe there is a need to remove the petsc-metis.patch command, and I'm guessing the SuiteSparse test will not fail anymore as well. Deactivating CUDA is the proper way to go (for now), I think, and that's what I did by adding the --with-cudac=0 flag. But please do let me know if this is not the case.
Anyway, I'll close the issue for now, do not hesitate re-opening it if you face new problems.

@prj- prj- closed this as completed Jun 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cirosantilli @prj- @frederichecht