Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installation challenges #196

Open
takluyver opened this issue May 5, 2020 · 9 comments
Open

Installation challenges #196

takluyver opened this issue May 5, 2020 · 9 comments

Comments

@takluyver
Copy link
Contributor

What's the recommended way to install SimEx? I've been looking at various things without much success:

  • The conda environment file seems to be missing several dependencies even to start building, and I can't get far even by adding things. Eventually, with some pointers from @ejcjason I was able to get it to work by disabling almost every component in install.sh and commenting out various imports in the Python code, but this doesn't feel like a good answer.
  • The install docs refer to a .deb package, but it's from a few years ago, and my target isn't Debian-based anyway.
  • They also refer to a docker image yakser/simex, which was last updated in 2017.
  • I tried building a docker image from Docker/simex within the repository, but it fell down pretty quickly. I haven't investigated this much, but I see the docker folder hasn't been touched for 2 years, so I doubt you're using that frequently.
@CFGrote
Copy link
Collaborator

CFGrote commented May 6, 2020

Thanks, this is very important information.

The recommended way to install simex is using cmake. install.sh serves as a guideline. More hints may be found in .travis.yml and in scripts under Travis/ .

Installation has successfully been tested on various linux distributions including debian7+ ubuntu 14.04+, centos7.

What is your target? c compiler? hdf version? cmake version? mpi ? mkl?

where does it fail if you enable more components? which ones?

@takluyver
Copy link
Contributor Author

Thanks Carsten!

In the first instance I want to install it on Maxwell (I know there's already a module, but I want to test a branch @ejcjason is working on). So CentOS 7. I'll focus on the conda installation for now. Once I've got a direct installation working, I might take an interest in using Spack to install it, and/or containerising it.

So, I've recreated a conda environment using conda-requirements.yml. I've modified install.sh to disable everything except SingFEL & WPG. Then I get CMake 3.12 or higher is required. OK, conda install cmake to get a newer version.

This time it worked. Previously I tried for longer with other modules enabled, and something I installed for one of the other modules seemed to break WPG. I'll experiment further and get back to you.

@takluyver
Copy link
Contributor Author

This is a bit long, but I figure it's worth recording the details of what errors I'm getting and what I'm trying. I should also have mentioned that I turned DEVELOPER_MODE off on @ejcjason's suggestion.

Disable WPG (for now), enable CrystFEL.

/home/kluyvert/.conda/envs/simex/lib/libpng16.so.16: undefined reference to `inflateValidate@ZLIB_1.2.9'

I didn't see this one before. conda install zlib ? No, that doesn't fix it. conda install libpng ? Replaces a conda-forge package with one from the default channel. No, still the same error.

I recall there was a change in the way conda works at some point, and now the recommendation is to install a compiler from conda, so you're not using something outside the environment. Maybe I'm running into some symptom of that. conda install gcc_linux-64 .

src/utils.h:41:10: fatal error: gsl/gsl_matrix.h: No such file or directory

Now we're using a compiler in conda, it no longer uses libraries from the system, if I've understood correctly. This is the first of a series of problems building CrystFEL which I ran into before, so let's install several packages I already found it needs: conda install gsl cairo pango gdk-pixbuf .

Building CrystFEL now works. Re-enable WPG.

x86_64-conda_cos6-linux-gnu-cc: error trying to exec 'cc1plus': execvp: No such file or directory

Missing a C++ compiler? Trying conda install gxx_linux-64 .

/home/kluyvert/.conda/envs/simex/bin/../lib/gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../x86_64-conda_cos6-linux-gnu/bin/ld: cannot find -lgomp

This is where I got stuck last time around. conda install libgomp . A package gets installed, but I still get the same error. This appears to come while building srwlpy for XFELPhotonPropagator.

Summary: Relying on compilers & libraries on the system, I can build WPG, but building CrystFEL fails. By installing compilers in the conda environment, I can get CrystFEL to build, but then WPG fails, and I've not yet worked out how to solve that.

@CFGrote
Copy link
Collaborator

CFGrote commented May 9, 2020 via email

@CFGrote
Copy link
Collaborator

CFGrote commented May 9, 2020 via email

@belfhi
Copy link

belfhi commented May 9, 2020

@CFGrote I can talk to Frank, we don't support Python 2.7 any more, so maybe it makes sense to modify the module to not set the python path...

@takluyver
Copy link
Contributor Author

Thanks Carsten. I found the module, but I was trying to test a branch that @ejcjason was working on, so I want to do the installation myself.

I'll try building it with the intel module loaded.

@takluyver
Copy link
Contributor Author

I tried with module load intel (it seems to have picked intel/2020 as the default version), but I still see the same error about -lgomp.

@CFGrote
Copy link
Collaborator

CFGrote commented May 15, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@takluyver @belfhi @CFGrote