compile error when using dagmc from conda #739

shimwell · 2021-05-13T16:57:39Z

Describe the Bug

compile error due to dagmc pyne meta data class

To Reproduce

Build the following Dockerfile
docker build -t openmc_using_dagmc_from_conda .


FROM continuumio/miniconda3:4.9.2

RUN apt-get --yes update && apt-get --yes upgrade

RUN apt-get --yes install libeigen3-dev

RUN conda install -c conda-forge dagmc


RUN git clone --single-branch --branch v0.12.1 --depth 1 https://github.com/openmc-dev/openmc.git

RUN conda install cmake
RUN conda install cython
RUN conda install h5py
RUN conda install numpy
RUN conda install pip
RUN conda install setuptools
RUN conda install hdf5
RUN apt-get install -y cmake
RUN apt-get install -y g++

RUN cd openmc && \
    mkdir build && \
    cd build && \
    cmake -Doptimize=on \
          -Ddagmc=ON \
        #   -DDAGMC_ROOT=/DAGMC \
          -DHDF5_PREFER_PARALLEL=off ..  && \
    make -j && \
    make -j install && \ 
    cd openmc/ && \
    pip install .

Expected Behavior

I was hoping this would compile without error

Screenshots or Code Snippets

Please complete the following information regarding your system:

OS: ubuntu
Physics codes versions installed with: openmc 12.1.0

The text was updated successfully, but these errors were encountered:

pshriwise · 2021-05-13T18:03:09Z

OpenMC v0.12.1 requires DAGMC >= 3.2.0 as of this PR

It looks like the version being installed conda is labeled on conda-forge as 3.2.0 but the DAGMC version header that actually gets installed indicates that it's DAGMC 3.1. From /opt/conda/include/DagMCVersion.hpp in the resulting docker image:

#define DAGMC_VERSION 3.1
#define DAGMC_VERSION_STRING "3.1"
#define DAGMC_INTERFACE_REVISION "$Rev$"

Seems like those version numbers may have gotten out of sync?

At the same time, we have a check in OpenMC v0.12.1 that should fail in configuration with an invalid DAGMC version. Not sure why it isn't getting triggered here. I'll look into that.

makeclean · 2021-05-13T18:03:24Z

This looks to be an OpenMC bug?

pshriwise · 2021-05-13T18:18:54Z

Ah, yeah it looks like the DAGMC version check was accidentally removed. I'll submit a PR shortly that adds it back in so this problem is caught earlier and produces a clearer error message.

pshriwise · 2021-05-13T18:33:39Z

Just opened a PR for the DAGMC check: openmc-dev/openmc#1838.

Not sure what's going on with the DAGMC conda package though.

bam241 · 2021-05-13T19:00:49Z

for me looks like a conflict between the old uwuw.materiallibrary and the new one.

the previous one used a map<string, material> and the new uses a material_library

pshriwise · 2021-05-13T21:24:52Z

Yes, these issues were addressed in the OpenMC PR mentioned above.

openmc-dev/openmc#1747

But the version of DAGMC being installed with conda in the Dockerfile isn't compatible with those updates. It's too old.

shimwell · 2021-05-13T23:13:17Z

If I understand correctly DAGMC v3.2.0 should work but the latest Conda distribution is somehow getting DAGMC v3.1 when it should be getting v3.2.0

So shall we increment the build number to 1 to trigger a rebuild the Conda distribution of DAGMC 3.2.0

Or alternatively make another DAGMC release via the GitHub releases, perhaps v3.2.1

pshriwise · 2021-05-13T23:49:00Z

If I understand correctly DAGMC v3.2.0 should work but the latest Conda distribution is somehow getting DAGMC v3.1 when it should be getting v3.2.0

Yeah, that sums it up well.

So shall we increment the build number to 1 to trigger a rebuild the Conda distribution of DAGMC 3.2.0

Or alternatively make another DAGMC release via the GitHub releases, perhaps v3.2.1

I'm not as well-versed in Conda distributions as you are @shimwell, so I'm happy to go with whichever is better. Do we have an idea of how to determine why these versions are out of sync?

shimwell · 2021-05-16T08:36:25Z

Searching conda for dagmc shows 4 versions of dagmc are available

(base) jshim@jshim-desktop:~$ conda search dagmc --channel conda-forge
Loading channels: done
# Name                       Version           Build  Channel             
dagmc                            3.2      h6a4b6ed_2  conda-forge         
dagmc                            3.2      h6d06269_0  conda-forge         
dagmc                          3.2.0      hcc255d8_0  conda-forge         
dagmc                            3.2      hfc221f0_2  conda-forge

It appears that when I do conda install dagmc that I get this version

>>conda list
dagmc                     3.2                  hfc221f0_2

I have also tried installing a specific version using conda install -c conda-forge dagmc=3.2.0 but this also resulted in the same compile error reported in this issue

I am wondering if it might be best to release another version of dagmc with a bumped patch number as there have been a few improvements since January when 3.2.0 was released such as #722 which might help openmc check the version and looks like there were also a couple of pyne related improvements #734 #731

pshriwise · 2021-05-17T17:28:03Z

To clarify, you're getting this error whether you use 3.2 or 3.2.0?

shimwell · 2021-05-17T19:49:40Z

To clarify, you're getting this error whether you use 3.2 or 3.2.0?

Both

pshriwise · 2021-05-18T16:57:05Z

I am wondering if it might be best to release another version of dagmc with a bumped patch number as there have been a few improvements since January when 3.2.0 was released such as #722 which might help openmc check the version and looks like there were also a couple of pyne related improvements #734 #731

Would bumping the build number also trigger and update by chance?

Asking based on this in the dagmc-feedstock README:

In order to produce a uniquely identifiable distribution:

  - If the version of a package is not being increased, please add or increase the build/number.
  - If the version of a package is being increased, please remember to return the build/number back to 0.

gonuke · 2021-05-18T17:40:54Z

We are somewhat stuck waiting for a MOAB update, I think. After that, we can trickle a variety of releases through the whole stack.

gonuke · 2021-05-18T17:42:53Z

Or at least waiting for your race condition patch because none of our DAGMC tests are passing right now

pshriwise · 2021-05-18T18:30:22Z

True. Just gave them another nudge! Hopefully, we'll be able to move forward soon!

shimwell · 2021-06-23T14:27:16Z

Just reporting that it looks like Moab was updated recently
https://anaconda.org/conda-forge/moab

I'm just trying the Dockerfile again with a minor adjustment to the OpenMC version as well

FROM continuumio/miniconda3:4.9.2

RUN apt-get --yes update && apt-get --yes upgrade

RUN apt-get --yes install libeigen3-dev

RUN conda install -c conda-forge dagmc


RUN git clone --single-branch --branch v0.12.2 --depth 1 https://github.com/openmc-dev/openmc.git

RUN conda install cmake
RUN conda install cython
RUN conda install h5py
RUN conda install numpy
RUN conda install pip
RUN conda install setuptools
RUN conda install hdf5
RUN apt-get install -y cmake
RUN apt-get install -y g++

RUN cd openmc && \
    mkdir build && \
    cd build && \
    cmake -Doptimize=on \
          -Ddagmc=ON \
        #   -DDAGMC_ROOT=/DAGMC \
          -DHDF5_PREFER_PARALLEL=off ..  && \
    make -j && \
    make -j install && \ 
    cd openmc/ && \
    pip install .

shimwell · 2021-06-23T14:48:23Z

Using the latest MOAB v5.2.1 and OpenMC v12.2 and rebuilding the Docker failed with the discovery of DAGMC v 3.1.0 by the OpenMC compiling step. This is due to the recent PR that checks the DAGMC version.

I wonder if we are now at the point where the DAGMC conda install can be updated.

shimwell · 2021-08-23T22:02:34Z

For anyone looking into this I've had to update the dockerfile I've been using to test this as OpenMC has moved to version 0.12.2 and allow-releaseinfo-change update is need as debian was also updated

FROM continuumio/miniconda3:4.9.2

RUN apt-get --allow-releaseinfo-change update
RUN apt-get --yes update && apt-get --yes upgrade

RUN apt-get --yes install libeigen3-dev

RUN conda install -c conda-forge dagmc


RUN git clone --single-branch --branch v0.12.2 --depth 1 https://github.com/openmc-dev/openmc.git

RUN conda install cmake
RUN conda install cython
RUN conda install h5py
RUN conda install numpy
RUN conda install pip
RUN conda install setuptools
RUN conda install hdf5
RUN apt-get install -y cmake
RUN apt-get install -y g++

RUN cd openmc && \
    mkdir build && \
    cd build && \
    cmake -Doptimize=on \
          -Ddagmc=ON \
        #   -DDAGMC_ROOT=/DAGMC \
          -DHDF5_PREFER_PARALLEL=off ..  && \
    make -j && \
    make -j install && \ 
    cd openmc/ && \
    pip install .

pshriwise · 2022-02-19T14:39:52Z

@shimwell, is it correct that once this PR is in that this issue can be closed?

pshriwise · 2022-02-19T14:40:02Z

Oops, didn't mean to close.

shimwell · 2022-02-19T19:35:02Z

Jus to mention I am retrying this Docker build and currently getting a PackagesNotFoundError

I guess it might just take a while for the package to move through the cf-staging to conda-forge
https://anaconda.org/conda-forge/dagmc/files

in the meantime here is an updated Dockerfile pinning v0.13.0 of openmc and v3.2.1. of dagmc

FROM continuumio/miniconda3:4.9.2

RUN apt-get --allow-releaseinfo-change update
RUN apt-get --yes update && apt-get --yes upgrade

RUN apt-get --yes install libeigen3-dev

RUN git clone --single-branch --branch v0.13.0 --depth 1 https://github.com/openmc-dev/openmc.git

RUN conda install cmake
RUN conda install cython
RUN conda install h5py
RUN conda install numpy
RUN conda install pip
RUN conda install setuptools
RUN conda install hdf5
RUN apt-get install -y cmake
RUN apt-get install -y g++

RUN conda install -c conda-forge dagmc=3.2.1
RUN cd openmc && \
    mkdir build && \
    cd build && \
    cmake -Doptimize=on \
          -Ddagmc=ON \
        #   -DDAGMC_ROOT=/DAGMC \
          -DHDF5_PREFER_PARALLEL=off ..  && \
    make -j && \
    make -j install && \ 
    cd openmc/ && \
    pip install .

which can be built with

docker build -t openmc_using_dagmc_from_conda .

shimwell · 2022-02-19T21:39:34Z

It looks like dagmc 3.2.1 is now being found by conda when i do a conda install -c conda-forge dagmc=3.2.1 within the dockerfile 🎉

However I am not sure it is working correctly

Here are some screen shots for the terminal output during the dockerfile build

gonuke · 2022-02-19T21:46:33Z

🤦‍♂️after all that, we forgot to update the version numbers in CMakeLists.txt. Or maybe that got overwritten when we merged in the citations file?

pshriwise · 2022-02-19T21:48:06Z

Looks like that line was last written in 2018 😞 Missed it in review. Sigh. Sorry.

gonuke · 2022-02-19T21:50:39Z

It's such an annoying thing to correct, as well!!

gonuke · 2022-02-19T22:07:58Z

I'm going to circumvent the release procedure for this, if folks agree:

patch CMakeLists.txt
update stable tag
update Release
update conda-forge recipe
rebuild pyne docker ci images (not in critical path, but for consistency)

gonuke · 2022-02-19T22:08:47Z

In most cases, PyNE's dependency on DAGMC is unidirectional.

gonuke · 2022-02-19T22:09:30Z

It's primarily changes in materialLibrary that can cause the quasi-circular dependency

shimwell · 2022-02-20T13:33:41Z

The DAGMC version in the conda install is now 3.2.1 🎉

However Eigan3 is still not being found

I wonder if we need to include eigan in the run section of the meta.yml for the DAGMC conda recipe as it is currently only in the host section.

Or perhaps this is more of an issue in the way I am compiling OpenMC

-- Found xtl v0.6.13
-- Project 'gsl_lite', package 'gsl-lite' version: '0.36.1'
-- Configuring done
CMake Error in CMakeLists.txt:
  Imported target "dagmc-shared" includes non-existent path

    "/opt/conda/include/eigen3"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

gonuke · 2022-02-20T14:10:08Z

Does OpenMC depend on eigen3 directly? It is a source code header only library that is compiled into DAGMC/MOAB where necessary, so shouldn't be necessary to use those unless the downstream code requires it directly

pshriwise · 2022-02-20T16:16:28Z

OpenMC doesn't rely on Eigen3 directly, no. I think CMake is just unhappy that an include directory of DAGMC doesn't exist, whether the header files in that directory are necessary or not. As to why that directory doesn't exist, I'm not sure.

gonuke · 2022-02-20T18:17:58Z

It might be because DagMC.hpp imports so many Moab headers and one of those may depend on eigen3.... two fixes:

short term: include eigen in the run environment for DAGMC
long term: clean up DagMC.hpp

gonuke · 2022-02-20T21:21:43Z

Yikes! Just explored DAGMC & MOAB on this.... there is so much header pollution! In summary, we have this hierarchy of includes in header files:
DagMC.hpp -> moab/GeomQueryTool.hpp -> OrientedBoxTreeTool.hpp -> OrientedBox.hpp -> Matrix3.hpp -> Eigen/dense.

It's probably true that we don't need those to be included in headers throughout, but we can let MOAB clean up their own mess and probably fix what we include from DagMC ourselves.

shimwell · 2022-02-20T21:22:58Z

Thanks for looking into this, I've just made a PR to the DAGMC conda-forge repo to include eigen in the run environment
conda-forge/dagmc-feedstock#10

pshriwise · 2022-02-21T00:10:16Z

Thanks for looking into this, I've just made a PR to the DAGMC conda-forge repo to include eigen in the run environment conda-forge/dagmc-feedstock#10

Just tried this now that the PR has been merged and had success! 🎉 Can you confirm @shimwell?

Noting that I did have to set the HDF5_ROOT to avoid a conflicting version of HDF5.

shimwell · 2022-02-21T01:13:20Z

Making use of the build 2 of the dagmc conda build for v3.2.1 in the docker file gets further than before

Building the dockerfile gives an error right at the end of the openmc compile for me

usr/bin/ld: /opt/conda/lib/libdagmc.so: undefined reference to `std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream()@GLIBCXX_3.4.26'

@pshriwise could you let us know what you set for the hdf5 root dir so I can try that please

pshriwise · 2022-02-21T03:55:10Z

I set the HDF5_ROOT to the root of my conda installation ($HOME/miniconda3) by default, but this was solving problem with redefinitions caused by different versions of HDF5 being included -- both the system and conda installations were being used. I'd be surprised if that fixed the problem shown here.

shimwell · 2022-02-21T10:54:20Z

Just in case this is handy

I attach the dockerfile and build log.

This build log can be reporduced by running docker build -t test_docker .

latest_docker_file_and_build_log.zip

gonuke · 2022-02-21T13:07:17Z

It looks like a compiler version mismatch??

makeclean · 2022-02-21T16:00:11Z

looks like hdf5 is pulling in gcc-7, but you're building openmc with gcc-8, IIRC gcc-7 was an absolute horrorshow of complications, can you request hdf5 with a higher gcc version?

makeclean · 2022-02-21T16:04:34Z

Or at least, compile OpenMC with gcc-7 - but thats a pretty old compiler

shimwell · 2022-02-21T16:21:21Z

Ah thanks for all these tips

I think the dockerfile is just installing old versions of a few packages from the default conda channel

this command installs version 1.10.6 of hdf5
RUN conda install hdf5

this command installs version 1.12.1 of hdf5
RUN conda install -c conda-forge hdf5

So I shall change the dockerfile to install from the conda-forge channel and rebuild

gonuke · 2022-02-21T16:38:20Z

Yeah - I think it becomes important to prefer conda-forge to get all the dependencies right

shimwell · 2022-02-21T16:52:21Z

Unfortunately the Dockerfile with conda-forge specified also fails
output.zip

pshriwise · 2022-02-21T17:03:06Z

I've been playing around with this too. I think that @gonuke's suggestion of sticking to installation from the conda-forge channel for consistency is important. I think we'll also need to install gxx from conda to make sure the compiler used for the conda packages and the one used to compile OpenMC are compatible.

Doing both of these things got me a successfully built OpenMC executable, but I ran into problems installing the python module due to issues finding the pthread libraries. I think I've run into that problem before with conda and can't remember how I resovled it in the past.

Here's the current version of the Dockerfile I'm using:

# build with
# docker build -t openmc_dagmc_conda_test .

FROM continuumio/miniconda3:4.7.12
# FROM condaforge/mambaforge-pypy3:4.9.2-7

RUN apt-get --allow-releaseinfo-change update
RUN apt-get --yes update && apt-get --yes upgrade

# check conda-forge for packages first
RUN conda config --add channels conda-forge
RUN conda config --set channel_priority strict

RUN git clone --single-branch --branch v0.13.0 --depth 1 https://github.com/openmc-dev/openmc.git

RUN conda install gxx
RUN conda install cmake
RUN conda install make
RUN conda install binutils
RUN conda install dagmc=3.2.1
RUN cd openmc && \
    mkdir build && \
    cd build && \
    cmake -Doptimize=on \
          -Ddagmc=ON \
          -DHDF5_PREFER_PARALLEL=off ..  && \
    make -j 2 && \
    make -j 2 install && \
    cd /openmc/

gonuke · 2022-02-21T17:05:28Z

Related: is there appetite for expanding OpenMC's conda-forge build to have both DAGMC and non-DAGMC options? Or would this become a separate conda-forge package? The former is definitely possible and can probably streamline things if the OpenMC team is on board.

shimwell · 2022-02-21T17:06:48Z

I am trying to build openmc with dagmc over on the conda-forge/openmc-feedstock repo conda-forge/openmc-feedstock#42

It would be great (in my option) to have dagmc as an option for the conda install of openmc

pshriwise · 2022-02-21T18:12:36Z

Related: is there appetite for expanding OpenMC's conda-forge build to have both DAGMC and non-DAGMC options? Or would this become a separate conda-forge package? The former is definitely possible and can probably streamline things if the OpenMC team is on board.

I'm on board! Bumping this discussion thread might be a good start toward getting confirmation from others too https://openmc.discourse.group/t/conda-installation-of-openmc-with-dagmc-on/1389.

shimwell · 2022-08-10T11:55:29Z

This dockerfile builds 🎉 , happy to close this issue if others are

# build with
# docker build -t openmc_dagmc_conda_test .

FROM continuumio/miniconda3:4.12.0
# FROM condaforge/mambaforge-pypy3:4.9.2-7

RUN apt-get --allow-releaseinfo-change update
RUN apt-get --yes update && apt-get --yes upgrade

# check conda-forge for packages first
RUN conda config --add channels conda-forge
RUN conda config --set channel_priority strict

RUN git clone --single-branch --branch develop --depth 1 https://github.com/openmc-dev/openmc.git

# using mamba to avoid tempest error in conda install dagmc
RUN conda install mamba -y

RUN mamba install gxx -y

RUN mamba install cmake -y

RUN mamba install make -y

RUN mamba install binutils -y

RUN mamba install -c conda-forge dagmc=3.2.2 -y

RUN cd openmc && \
    mkdir build && \
    cd build && \
    cmake -Doptimize=on \
          -Ddagmc=ON \
          -DHDF5_PREFER_PARALLEL=off ..  && \
    make -j && \
    make -j install && \
    cd /openmc/ && \
    pip install .

shimwell · 2022-08-10T15:39:51Z

I might have celebrated too soon. While the DAGMC from conda does now allow openmc to compile it looks like something is still wrong.

The resulting openmc doesn't realize it has DAGMC within

See this CI where I've installed the same stack of software in the above docker image and then I've performed a openmc simulation on a DAGMC geometry but get the error message

RuntimeError: DAGMC Universes are present but OpenMC was not configuredwith DAGMC

However perhaps this is a different issue

shimwell · 2022-08-10T16:31:42Z

I totally forgot that the CMAKE lists have recently been updated for OpenMC.

I've correct the cmake command to use OPENMC_USE_DAGMC and everything works fine.

Closing issue for now, feel free to reopen if this problem comes back

Extra thanks to @pshriwise for slack help

shimwell added the Type: Defect label May 13, 2021

gonuke added the Next Release label Jul 23, 2021

shimwell mentioned this issue Oct 18, 2021

Adding DAGMC to the openmc conda package. conda-forge/openmc-feedstock#38

Closed

pshriwise closed this as completed Feb 19, 2022

pshriwise reopened this Feb 19, 2022

shimwell mentioned this issue Feb 20, 2022

added eigen to the run env conda-forge/dagmc-feedstock#10

Merged

5 tasks

shimwell mentioned this issue Feb 20, 2022

feature request: add a few more tests to the conda test section in meta conda-forge/dagmc-feedstock#11

Open

1 task

gonuke mentioned this issue Feb 20, 2022

Remove need for downstream codes to have Eigen3 installed #799

Open

shimwell closed this as completed Aug 10, 2022

compile error when using dagmc from conda #739

compile error when using dagmc from conda #739

Comments

shimwell commented May 13, 2021

pshriwise commented May 13, 2021 • edited Loading

makeclean commented May 13, 2021

pshriwise commented May 13, 2021 • edited Loading

pshriwise commented May 13, 2021

bam241 commented May 13, 2021 • edited Loading

pshriwise commented May 13, 2021 • edited Loading

shimwell commented May 13, 2021

pshriwise commented May 13, 2021 • edited Loading

shimwell commented May 16, 2021

pshriwise commented May 17, 2021

shimwell commented May 17, 2021

pshriwise commented May 18, 2021

gonuke commented May 18, 2021

gonuke commented May 18, 2021

pshriwise commented May 18, 2021

shimwell commented Jun 23, 2021

shimwell commented Jun 23, 2021

shimwell commented Aug 23, 2021

pshriwise commented Feb 19, 2022

pshriwise commented Feb 19, 2022

shimwell commented Feb 19, 2022

shimwell commented Feb 19, 2022 • edited Loading

gonuke commented Feb 19, 2022

pshriwise commented Feb 19, 2022

gonuke commented Feb 19, 2022

gonuke commented Feb 19, 2022

gonuke commented Feb 19, 2022

gonuke commented Feb 19, 2022

shimwell commented Feb 20, 2022

gonuke commented Feb 20, 2022

pshriwise commented Feb 20, 2022

gonuke commented Feb 20, 2022

gonuke commented Feb 20, 2022

shimwell commented Feb 20, 2022

pshriwise commented Feb 21, 2022 • edited Loading

shimwell commented Feb 21, 2022 • edited Loading

pshriwise commented Feb 21, 2022 • edited Loading

shimwell commented Feb 21, 2022

gonuke commented Feb 21, 2022

makeclean commented Feb 21, 2022

makeclean commented Feb 21, 2022

shimwell commented Feb 21, 2022

gonuke commented Feb 21, 2022

shimwell commented Feb 21, 2022 • edited Loading

pshriwise commented Feb 21, 2022

gonuke commented Feb 21, 2022

shimwell commented Feb 21, 2022 • edited Loading

pshriwise commented Feb 21, 2022 • edited Loading

shimwell commented Aug 10, 2022 • edited Loading

shimwell commented Aug 10, 2022 • edited Loading

shimwell commented Aug 10, 2022

pshriwise commented May 13, 2021 •

edited

Loading

pshriwise commented May 13, 2021 •

edited

Loading

bam241 commented May 13, 2021 •

edited

Loading

pshriwise commented May 13, 2021 •

edited

Loading

pshriwise commented May 13, 2021 •

edited

Loading

shimwell commented Feb 19, 2022 •

edited

Loading

pshriwise commented Feb 21, 2022 •

edited

Loading

shimwell commented Feb 21, 2022 •

edited

Loading

pshriwise commented Feb 21, 2022 •

edited

Loading

shimwell commented Feb 21, 2022 •

edited

Loading

shimwell commented Feb 21, 2022 •

edited

Loading

pshriwise commented Feb 21, 2022 •

edited

Loading

shimwell commented Aug 10, 2022 •

edited

Loading

shimwell commented Aug 10, 2022 •

edited

Loading