Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADIOS Spack package broken on Summit #209

Open
NAThompson opened this issue Aug 12, 2020 · 5 comments
Open

ADIOS Spack package broken on Summit #209

NAThompson opened this issue Aug 12, 2020 · 5 comments

Comments

@NAThompson
Copy link

To reproduce:

[nthompson@summitdev-login1]~% ml pgi/19.9
[nthompson@summitdev-login1]~% spack install adios
...
==> Installing openmpi
==> No binary for openmpi found: installing from source
==> openmpi: Executing phase: 'autoreconf'
==> openmpi: Executing phase: 'configure'
==> openmpi: Executing phase: 'build'
==> openmpi: Executing phase: 'install'
[+] /autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/openmpi-3.1.6-z36ridnac6bgsinx677u5oooviyrpnd4
==> Installing adios
==> No binary for adios found: installing from source
==> adios: Executing phase: 'autoreconf'
==> adios: Executing phase: 'configure'
==> Error: ProcessError: Command exited with status 1:
    '/tmp/nthompson/spack-stage/spack-stage-adios-1.13.1-xchp66cjc3oafb2g67pdumtcwbcbqolk/spack-src/configure' '--prefix=/autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/adios-1.13.1-xchp66cjc3oafb2g67pdumtcwbcbqolk' 'CFLAGS=-fPIC' '--enable-shared' '--disable-fortran' '--with-mpi=/autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/openmpi-3.1.6-z36ridnac6bgsinx677u5oooviyrpnd4' '--without-infiniband' '--with-blosc=/autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/c-blosc-1.17.0-efj77ikklysbx5fhah5tajm7uqaeetcr' '--with-zlib=/autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/zlib-1.2.11-tjbynt2kcsdo5ehwlcrxoqhbb7snlqdm' '--without-bzip2' '--without-szip' '--with-zfp=/autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/zfp-0.5.5-oyzh7mkqqnjbwg2dlwxavejrsaeblyty' '--with-sz=/autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/sz-1.4.12.3-apzgwhhqybemdkv6sjoeaozdsjr3blor' '--with-lz4=/autofs/nccs-svm1_home1/nthompson/spack/opt/spack/linux-rhel7-power8le/gcc-4.8.5/lz4-1.9.2-tpkrjicqiew4wkc5t7f6hzibyf4bvm5s' '--without-phdf5' '--without-netcdf' '--without-flexpath' '--without-dataspaces'

1 error found in build log:
     47    checking whether /autofs/nccs-svm1_home1/nthompson/spack/lib/spack/e
           nv/gcc/g++ accepts -g... yes
     48    checking dependency style of /autofs/nccs-svm1_home1/nthompson/spack
           /lib/spack/env/gcc/g++... gcc3
     49    checking for mpicc... /autofs/nccs-svm1_home1/nthompson/spack/opt/sp
           ack/linux-rhel7-power8le/gcc-4.8.5/openmpi-3.1.6-z36ridnac6bgsinx677
           u5oooviyrpnd4/bin/mpicc
     50    checking for MPI_Init... no
     51    checking for MPI_Init in -lmpi... no
     52    checking for MPI_Init in -lmpich... no
  >> 53    configure: error: could not find mpi library for C

See build log for details:
  /tmp/nthompson/spack-stage/spack-stage-adios-1.13.1-xchp66cjc3oafb2g67pdumtcwbcbqolk/spack-build-out.txt

I would attach the build log, but it gives the same information as the console.

@pnorbert
Copy link
Contributor

I am not sure if you are supposed to build your own MPI compiler on summit.
Can you build adios if you force spack to use the system's MPI compiler?

@NAThompson
Copy link
Author

@pnorbert : I'm confused; I didn't tell spack to build openmpi; in addition I had an MPI loaded:

ml

Currently Loaded Modules:
  1) hsi/5.0.2.p5   2) xalt/1.1.3   3) lsf-tools/2.0   4) DefApps   5) pgi/19.9   6) spectrum-mpi/10.3.0.1-20190611

Do you mean that I should add a compiler to spack?

Here's what I currently have available:

spack compilers
==> Available compilers
-- gcc rhel7-ppc64le --------------------------------------------
gcc@4.8.5

-- pgi rhel7-ppc64le --------------------------------------------
pgi@19.9

@pnorbert
Copy link
Contributor

You need to point spack to a system MPI and disable building its own. By default ALL dependencies are built from scratch. You only pointed spack to the system compiler.

Here is my setting on Summit (not summitdev):

$ cat /ccs/proj/csc303/adios2-test-suite/summit/spack/etc/spack/packages.yaml
packages:

  # concretization preferences
  all:
    compiler: [gcc/8.1.1]
    providers:
      mpi: [spectrum-mpi]

  # system modules externals. 
  spectrum-mpi:
    modules:
      spectrum-mpi@10.3.1.2-20200121%gcc@8.1.1: spectrum-mpi/10.3.1.2-20200121
    buildable: False
 ...

@NAThompson
Copy link
Author

NAThompson commented Aug 12, 2020

@pnorbert : I get the following when I tried your suggestion:

==> Warning: the attribute "modules" in the "packages" section of the configuration has been deprecated [entry=CommentedMap([('spectrum-mpi@10.3.0.1-20190611%pgi@19.9', 'spectrum-mpi/10.3.0.1-20190611')])]

However, with the following packages.yaml I was able to succeed in getting a successful compilation:

[nthompson@summitdev-login1]~% cat ~/.spack/packages.yaml
packages:
    all:
        compiler: [gcc/4.8.5]
        providers:
            mpi: [spectrum-mpi]
    spectrum-mpi:
        modules:
            spectrum-mpi@10.3.0.1-20190611%gcc@4.8.5: spectrum-mpi/10.3.0.1-20190611
        buildable: False

Strangely, the spack package for adios2 succeeds without this file, even though I was unable to see any significant differences in the spack info log.

@williamfgc
Copy link

williamfgc commented Aug 12, 2020

@NAThompson system MPI libraries are required, as @pnorbert suggested, as your package might get out of sync when they do system updates on Summit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants