A simplified version of minimal toolchains that does the toolchain re… #1614

ocaisa · 2016-02-15T15:04:25Z

…solution as the easyconfig deps are being parsed.

…solution as the easyconfig deps are being parsed

hpcugentbot · 2016-02-15T15:12:29Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2683/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

boegel · 2016-02-15T15:16:29Z

easybuild/framework/easyconfig/tools.py

-from easybuild.framework.easyconfig.easyconfig import ActiveMNS, create_paths, process_easyconfig
-from easybuild.framework.easyconfig.easyconfig import robot_find_easyconfig
+from easybuild.framework.easyconfig.easyconfig import ActiveMNS, create_paths, process_easyconfig, module_is_available
+from easybuild.framework.easyconfig.easyconfig import robot_find_easyconfig, get_toolchain_hierarchy


are these actually still used here?

hpcugentbot · 2016-02-15T15:30:17Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2684/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-15T16:01:33Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2685/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-15T16:04:15Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2686/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-15T16:18:50Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2687/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-15T16:30:18Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2688/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-15T18:35:11Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2691/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

…rchy

hpcugentbot · 2016-02-16T09:24:10Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2693/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

ocaisa · 2016-02-16T09:29:51Z

Includes the fix of #1576 and satisfies #1579

boegel · 2016-02-16T09:44:52Z

@ocaisa: so both #1576 and #1579 should be closed?

boegel · 2016-02-16T09:45:24Z

easybuild/framework/easyconfig/easyconfig.py

+
+        dep_tcs = []
+        for dep in parsed_ec['dependencies']:
+            # dep == icc


remove this comment?

fix mocking + clear cache to fix broken test

hpcugentbot · 2016-02-16T16:48:12Z

EasyBuild framework unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2701/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2016-02-16T17:28:04Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2702/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-02-17T08:02:30Z

easybuild/framework/easyconfig/easyconfig.py

 from easybuild.framework.easyconfig.parser import DEPRECATED_PARAMETERS, REPLACED_PARAMETERS
 from easybuild.framework.easyconfig.parser import EasyConfigParser, fetch_parameters_from_easyconfig
 from easybuild.framework.easyconfig.templates import TEMPLATE_CONSTANTS, template_constant_dict
-
+from easybuild.toolchains.gcccore import GCCcore


please keep imports alphabetically sorted

You say that, but I'm pretty sure f comes before t in the alphabet...

boegel · 2016-02-17T09:33:30Z

@ocaisa: some style cleanup/minor refactoring in ocaisa#30

There's one issue though... With this approach, the easyconfig file that is dumped in the install dir will no longer have the minimal toolchains that were used for the dependencies.

We should probably get that fixed before merging this PR?

some refactoring and style fixes

hpcugentbot · 2016-02-17T10:29:03Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2705/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

… the repo

hpcugentbot · 2016-02-17T12:17:23Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2706/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

ocaisa · 2016-02-17T13:32:45Z

Ok, this method now creates a new subdir in the easybuild directory of the installation for a dump of the parsed easyconfig (which describes exactly which toolchains were used for dependencies). The new subdirectory is called reprod and is intended to host all information required to reproduce the build (so it is currently incomplete since it doesn't house easyblock and toolchain classes used, but it's a start).

This dump is also the one that is submitted to the repository so that the build can be reproduced (without using --minimal-toolchains).

ocaisa · 2016-02-17T13:35:45Z

lgtm?

refactor dumping easyconfig in reprod

hpcugentbot · 2016-02-17T17:01:44Z

EasyBuild framework unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-framework-pr-builder/2708/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2016-02-17T18:59:11Z

Good to go, thanks @ocaisa!

A simplified version of minimal toolchains that does the toolchain re…

boegel · 2016-02-19T17:17:35Z

easybuild/framework/easyblock.py

@@ -2333,7 +2349,7 @@ def build_and_install_one(ecdict, init_env):
        move_logs(app.logfile, application_log)

        try:
-            newspec = os.path.join(new_log_dir, "%s-%s.eb" % (app.name, det_full_ec_version(app.cfg)))
+            newspec = os.path.join(new_log_dir, ec_filename)


ec_filename is out of scope here... >_<

fixed in #1625

A simplified version of minimal toolchains that does the toolchain re…

f454c22

…solution as the easyconfig deps are being parsed

boegel reviewed Feb 15, 2016
View reviewed changes

Remove old implementation

a224028

Tidy up a bit

dc5c739

Remove test

5fcc121

Update tests

5ea7c77

boegel added this to the v2.7.0 milestone Feb 15, 2016

Update main

c3c276e

Add easybuilders#1576 to be more thorough when constructing the hiera…

d38a984

…rchy

boegel reviewed Feb 16, 2016
View reviewed changes

easybuild/framework/easyconfig/easyconfig.py

dep_tcs = []

for dep in parsed_ec['dependencies']:

# dep == icc

Copy link

Member

boegel Feb 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this comment?

boegel mentioned this pull request Feb 16, 2016

fix get_toolchain_hierarchy w.r.t. finding subtoolchain version (WIP) #1576

Closed

ocaisa mentioned this pull request Feb 16, 2016

Search for available modules in reverse order... #1579

Closed

fix mocking + clear cache to fix broken test

dc2312f

boegel mentioned this pull request Feb 16, 2016

fix mocking + clear cache to fix broken test ocaisa/easybuild-framework#29

Merged

Merge pull request #29 from boegel/improved_minimal_toolchains

eb8d8ee

fix mocking + clear cache to fix broken test

ocaisa mentioned this pull request Feb 16, 2016

--minimal-toolchains seems to don't work with ToolchainMNS #1606

Closed

boegel reviewed Feb 17, 2016
View reviewed changes

some refactoring and style fixes

1a4e188

boegel mentioned this pull request Feb 17, 2016

some refactoring and style fixes ocaisa/easybuild-framework#30

Merged

boegel mentioned this pull request Feb 17, 2016

EasyBuild 2.7.0 (roadmap) easybuilders/easybuild#192

Closed

17 tasks

Merge pull request #30 from boegel/improved_minimal_toolchains

861c88a

some refactoring and style fixes

Dump the parsed easyconfig to a reproducability directory and also to…

ac7f8b7

… the repo

refactor dumping easyconfig in reprod

eb26475

boegel mentioned this pull request Feb 17, 2016

refactor dumping easyconfig in reprod ocaisa/easybuild-framework#31

Merged

Merge pull request #31 from boegel/improved_minimal_toolchains

e6d2cd5

refactor dumping easyconfig in reprod

boegel mentioned this pull request Feb 17, 2016

add end-to-end test for --minimal-toolchains #1619

Merged

boegel added a commit that referenced this pull request Feb 17, 2016

Merge pull request #1614 from ocaisa/improved_minimal_toolchains

5c91585

A simplified version of minimal toolchains that does the toolchain re…

boegel merged commit 5c91585 into easybuilders:develop Feb 17, 2016

ocaisa deleted the improved_minimal_toolchains branch February 18, 2016 08:50

boegel mentioned this pull request Feb 19, 2016

fix bug triggered with use of --stop + add test that triggers it #1625

Merged

boegel reviewed Feb 19, 2016
View reviewed changes

ocaisa mentioned this pull request Feb 23, 2016

--try-toolchain not very robust #1627

Open

boegel mentioned this pull request Nov 27, 2016

only dump easyconfig with modified deps due to --minimal-toolchains to 'reprod' subdir of install dir #2028

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

A simplified version of minimal toolchains that does the toolchain re… #1614

A simplified version of minimal toolchains that does the toolchain re… #1614

ocaisa commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

boegel Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 16, 2016

ocaisa commented Feb 16, 2016

boegel commented Feb 16, 2016

boegel Feb 16, 2016

hpcugentbot commented Feb 16, 2016

hpcugentbot commented Feb 16, 2016

boegel Feb 17, 2016

ocaisa Feb 17, 2016

boegel Feb 17, 2016

boegel commented Feb 17, 2016

hpcugentbot commented Feb 17, 2016

hpcugentbot commented Feb 17, 2016

ocaisa commented Feb 17, 2016

ocaisa commented Feb 17, 2016

hpcugentbot commented Feb 17, 2016

boegel commented Feb 17, 2016

boegel Feb 19, 2016

A simplified version of minimal toolchains that does the toolchain re… #1614

A simplified version of minimal toolchains that does the toolchain re… #1614

Conversation

ocaisa commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

boegel Feb 15, 2016

Choose a reason for hiding this comment

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 15, 2016

hpcugentbot commented Feb 16, 2016

ocaisa commented Feb 16, 2016

boegel commented Feb 16, 2016

boegel Feb 16, 2016

Choose a reason for hiding this comment

hpcugentbot commented Feb 16, 2016

hpcugentbot commented Feb 16, 2016

boegel Feb 17, 2016

Choose a reason for hiding this comment

ocaisa Feb 17, 2016

Choose a reason for hiding this comment

boegel Feb 17, 2016

Choose a reason for hiding this comment

boegel commented Feb 17, 2016

hpcugentbot commented Feb 17, 2016

hpcugentbot commented Feb 17, 2016

ocaisa commented Feb 17, 2016

ocaisa commented Feb 17, 2016

hpcugentbot commented Feb 17, 2016

boegel commented Feb 17, 2016

boegel Feb 19, 2016

Choose a reason for hiding this comment