implemented setup.py for install easybuild-easyconfigs

[boegel]

No description provided.

[boegel]

Tested both with python setup install --prefix=/tmp, and with easy_install --prefix=/tmp https://github.com/boegel/easybuild-easyconfigs/tarball/easy_install_support, works perfectly.

[JensTimmerman]

Can you also update the README.md to reflect this installation method?

[boegel]

Turns out this doesn't work as expected with setuptools v0.6c5 and Python 2.4, which is what we have installed on our gengar cluster.

It's installed a zipped egg with that setup, and thus the easyconfigs are not easy to use. You can install an unzipped egg with easy_installs --always-unzip option, but then the easyconfigs reside in a share directory inside the egg, which is not what we want.

[boegel]

I just looked into fixing the setup.py script such that the easyconfigs are also installed in $PREFIX/share when easy_i stall is used.
It turns out that the version of setuptools has nothing to do with it. The hack that's in place now in setup.py which works when the easyconfigs are installed with "python setup.py install [--prefix /tmp]" just doesn't work with easy_install.

After over two hours of fiddling about and figuring out the easy_install source code, my conclusion is that there is no decent way to get around the fact that setuptools want to install everything inside an egg, even data files that are basically just text files.
The hack that works for the setup.py installation mode is pretty ugly already, and I haven't found a way to extend it so that also easy_install is tricked into installing the easyconfigs somewhere outside of an egg.

Thus, I feel we just have to live with the fact that the easyconfigs will be installed in a somewhat fishy (imho) directory deep inside the install dir that was specified.
We can slightly relieve this issue by making EasyBuild look for the easyconfig file there in case a non-absolute path was specified, and add an option to list all available easyconfigs. It would probably also make sense to make this the default path for --robot. I plan to implement exactly that and include it in the setup.py pull request in the easybuild-framework repo.

Thoughts, @JensTimmerman, @stdweird?

[stdweird]

since easy_install installs inside a containerised environment, you can't put anything out of that container (i tried hard for VSC-tools).
what we could do is ship an easyconfig installer (ie wrapper around regular distutils setup.py) with the framework; or add such an option to the eb script (--update-configs ?).
for the easyblocks it's a similar issue imho, on one side they actaully do belong in an egg (being modules and all that), on the other side, people also want to modify/study them.

apologies for raising more issues @boegel @JensTimmerman

[boegel]

I disagree that this is a similar issue for easyblocks. If people want to see the code, they can browse the repository on GitHub.
For easyconfigs however, you need to specify their path to EasyBuild. I'd hate to run EasyBuild each time with a command line like eb /usr/local/lib/python2.4/site-packages/easybuild_easyconfigs-0.9.0dev-py2.7.egg/easybuild/easyconfigs/g/GCC/GCC-4.5.3.eb . That's just insane.

We could resolve this by making EasyBuild look in this path for relative easyconfig paths that can not be resolved in the regular way, thus making eb g/GCC/GCC-4.5.3.eb or even eb GCC-4.5.3.eb equivalent to the above, no matter where the easybuild-easyconfigs package is installed.

[stdweird]

to make handling the easyconfig more user-friendly, eb should indeed look relative to the install path if it can't resolve the easyconfig file argument as a file (abolute path is not the issue, but reltive to current working directory).
the only missing feature then will be tab-completion. but even that cna be solved, see http://aplawrence.com/Unix/customtab.html, it's bash specific and requires to add something to .bashrc, but i think we can live with that.

[JensTimmerman]

look into generate_wiki script for code that lists all easyconf files
(but it's basically just a os.walk and parse files that look like eb's)
If you do add a function that does this, the generate wiki script can use this instead of doing the os.walk itself
However, try to keep in mind that it we would like to add a method to list all easyconfigs inside a github repository aswell. (using the github api)

[boegel]

@JensTimmerman: It's already there, see easybuilders/easybuild-framework@64a5252 .

This may be hoisted into a function that the wiki generation script can use as well, let's look into that later. I really want to move forward now, get this easy_install show on the road so we can test the whole toolchain business.

[boegel]

Fixing this also results in a much nicer quick demo for the impatient. It boils down to a single short line now:

eb HPL-2.0-goalf-1.1.0-no-OFED.eb --robot

Nice!

[boegel]

@JensTimmerman, please review and test via e.g. easy_install --user easybuild

[JensTimmerman]

works here.