Astropy 3.0.x and numpy and CentOS-5 options

[taldcroft]

We'd like to continue to use a set of packages that are CentOS5 compatible. Presently that means that we've locked down conda to and old version and on the Linux side we are only fetching patches from older channels (newer default channels have linux packages built on CentOS6). However, there is no astropy 3.0.x available that way.

We have three options:

1. (First thought from @taldcroft is that this is the worst option).

• (numba and llvmlite=0.18.0)
• Numpy <= 1.12 from conda
• Astropy 2.0.x (LTS) from conda
• CentOS-5 support

2. OK but not many core packages are going to float high.

• (numba and llvmlite=0.18.0)
• Numpy <= 1.12 from conda
• Astropy 3.0.x compiled and locally built package
• CentOS-5 support

• (numba and llvmlite any)
• Numpy any from conda
• Astropy any from conda
• No CentOS-5 support

We need to decide which of these three. Waiting on input from MB about whether chimchim-w is CentOS-5 or 6+.

Question: do we want starcheck to be able to run on CentOS-5 (assuming chimchim-w is 5).

[taldcroft]

MB has informed that chimchim will be on CentOS-5 through relo. So I think option 2 is the best for now. That means less pressure to upgrade any packages since most won't float too much, which implies we don't need to do #34 right away.

Note that we will get stuck at astropy 3.0.x because 3.1 requires a newer numpy (1.13 I think).

[jeanconn]

Actually, it looks like I had some details wrong in our discussion. I think numpy 1.12.1 is fine if we get the package from the reduced set of "default" channels that only include older packages. Newer "builds" of Numpy 1.12.1 seem to break. That's a little scary (our version "pins" won't necessarily protect us). But if we can avoid accidentally updating until we get rid of CentOS 5 this should be fine. I think this also means that we need to use a custom .condarc when installing on CentOS5, which is fine. Other "users" will likely get the newer non-CentOS5 build of Numpy 1.12.1 . Hopefully that'll still be fine.

Also, the newer (3.0.x) astropy builds are only in the newer channels that bring in packages that break things (I think... this isn't easy to see.. and I think only comes with newer versions of conda itself), but may not have an explicit dependence on newer-than-numpy-1.12. I'll have to check that.

So I expect that we can either build astropy or go and find one of those packages manually to install.

[taldcroft]

I think numpy 1.12.1 is fine if we get the package from the reduced set of "default" channels that only include older packages.

Can you be sure this will be the case for users installing standalone?

Even 1.12.1 won't be enough for astropy 3.1. It uses new features in numpy 1.13 and I think astropy will break badly without 1.13. From the top-level release notes I don't see anything in numpy 1.12 that I'm jumping for.

[jeanconn]

Can you be sure this will be the case for users installing standalone?

That's what I meant by "Other "users" will likely get the newer non-CentOS5 build of Numpy 1.12.1". I can't imagine we have standalone users installing on CentOS5 without knowing what they are doing. (I still think that we are the users, but if you have other users in mind, let me know). Of course we'll need to test both ways, and I'm disturbed that requiring a specific version of numpy can get you different built code... though I suppose we already knew that for the OSX vs Linux vs Windows versions.

And yes, by "fine" I mean that we should be able to build or find an astropy 3.0.x for numpy 1.12.1. We'll have to build numpy if we want astropy 3.1 .

[taldcroft]

OK, well as long as whatever we end up with is foolproof and documented. You can reliably assume that I will have completely forgotten this thread and any restrictions about channels in a few weeks.

[jeanconn]

OK, so I think outstanding actions in this thread are

• Include directions for setting up .condarc on linux build system on wiki page
• Build an astropy 3.0.x package

[jzuhone]

@taldcroft @jeanconn sorry I'm coming late to this party (was unexpectedly out of town for a few days). Can I get a read on the reason for the NumPy version restriction? I am aware of the bigger changes in 1.13 but I wasn't sure how that affected us.

[jeanconn]

Right now the restriction is completely based on the fact we are still trying to support some GRETA CentOS5 machines. There are no official conda 'defaults' packages, in the CentOS5 supporting channels, for numpy 1.13 or astropy >= 3.0x. So we could roll our own (building on CentOS 5) or wait or work with some compromises. For ska3 dev work, you could use astropy > 3.0.3 and numpy > 1.12, but for our "production" environment we'd like to keep it CentOS5 compatible. Does that make some sense?

I'd rather not make a custom numpy >= 1.13 built on CentOS5, but if we have a driver, we could try that too.

[jzuhone]

@jeanconn thanks for the explanation--totally makes sense.

[taldcroft]

Sigh, astropy 3.2 will be out soon, numpy is at 1.16. We can live with this for now but I just wanted to complain. 🙍‍♂️