Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix extension filter for Perl packages #2699

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from

Conversation

boegel
Copy link
Member

@boegel boegel commented Mar 23, 2022

(created using eb --new-pr)

The perldoc command we were using doesn't verify that all required dependenices of a Perl module are also available.

This should not get merged straight away, a couple of minor fixes will need to be added to the Perl easyconfigs, including:

  • adding the missing Pod::Parser extension;
  • changing Term::ReadLine::Gnu to Term::ReadLine (because require Term::ReadLine::Gnu triggers an error stating that it should not be used directly);
  • using 'modulename': False for the if extension (which doesn't provide a Perl module in the traditional sense, but adds support for using use if, cfr. https://metacpan.org/pod/if)

@boegel boegel added the bug fix label Mar 23, 2022
@boegel boegel added this to the 4.x milestone Mar 23, 2022
@branfosj
Copy link
Member

Replaces #1449?

Also, the comment on the line above will need to be updated.

easybuild/easyblocks/p/perl.py Outdated Show resolved Hide resolved
@boegel
Copy link
Member Author

boegel commented Sep 9, 2022

Spent a little bit more time on this, made some progress but not out of the woods yet with necessary changes to Perl easyconfigs before this can be merged.

Changes to Perl-5.34.1-GCCcore-11.3.0.eb that make fewer checks fail for extensions:

--- a/easybuild/easyconfigs/p/Perl/Perl-5.34.1-GCCcore-11.3.0.eb
+++ b/easybuild/easyconfigs/p/Perl/Perl-5.34.1-GCCcore-11.3.0.eb
@@ -989,6 +989,11 @@ exts_list = [
         'source_urls': ['https://cpan.metacpan.org/authors/id/A/AS/ASPINELLI'],
         'checksums': ['409a8e0e4b1025c8e80f628f65a9778aa77ab285161406ca4a6c097b13656d0d'],
     }),
+    ('Pod::Parser', '1.65', {
+        'source_tmpl': 'Pod-Parser-%(version)s.tar.gz',
+        'source_urls': ['https://cpan.metacpan.org/authors/id/M/MA/MAREKR/'],
+        'checksums': ['3ba7bdec659416a51fe2a7e59f0883e9c6a3b21bc9d001042c1d6a32d401b28a'],
+    }),
     ('Pod::LaTeX', '0.61', {
         'source_tmpl': 'Pod-LaTeX-%(version)s.tar.gz',
         'source_urls': ['https://cpan.metacpan.org/authors/id/T/TJ/TJENNESS'],
@@ -1590,6 +1595,7 @@ exts_list = [
         'checksums': ['9841be5587bfb7cd1f2fe267b5e5ac04ce25e79d5cc77e5ef9a9c5abd101d7b1'],
     }),
     ('Term::ReadLine::Gnu', '1.42', {
+        'modulename': 'Term::ReadLine',
         'source_tmpl': 'Term-ReadLine-Gnu-%(version)s.tar.gz',
         'source_urls': ['https://cpan.metacpan.org/authors/id/H/HA/HAYASHI'],
         'checksums': ['3c5f1281da2666777af0f34de0289564e6faa823aea54f3945c74c98e95a5e73'],
@@ -1640,6 +1646,7 @@ exts_list = [
         'checksums': ['dbf7c827984951fb248907f940fd8f19f2696bc5545c0a15287e0fbe56a52308'],
     }),
     ('if', '0.0608', {
+        'modulename': False,
         'source_tmpl': 'if-%(version)s.tar.gz',
         'source_urls': ['https://cpan.metacpan.org/authors/id/X/XS/XSAWYERX'],
         'checksums': ['37206e10919c4d99273020008a3581bf0947d364e859b8966521c3145b4b3700'],

This is not enough though, since two checks are still failing:

  • PDF::API2:
    failing sanity check for 'PDF::API2' extension: command "perl -e 'require PDF::API2'" failed; output:
    Attempt to reload Compress/Zlib.pm aborted.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Content.pm line 11.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Content.pm line 11.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Page.pm line 15.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2/Page.pm line 15.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2.pm line 18.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/PDF/API2.pm line 18.
    Compilation failed in require at -e line 1.
    
  • IO::Compress::Bzip2
    failing sanity check for 'IO::Compress::Bzip2' extension: command "perl -e 'require IO::Compress::Bzip2'" failed; output:
    Compress::Raw::Bzip2 version 2.103 required--this is only version 2.101 at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Adapter/Bzip2.pm line 9.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Adapter/Bzip2.pm line 9.
    Compilation failed in require at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Bzip2.pm line 11.
    BEGIN failed--compilation aborted at /software/Perl/5.34.1-GCCcore-11.3.0/lib/perl5/site_perl/5.34.1/IO/Compress/Bzip2.pm line 11.
    Compilation failed in require at -e line 1.
    

@akesandgren
Copy link
Contributor

akesandgren commented Sep 14, 2023

As for the Bzip2 module it needs to be downgraded to 2.102, at the highest, to get it to require Compress::Raw::Bzip2 and Compress::Zlib versions 2.101 which are what comes internally in that Perl version.

Testing locally with that now...

And it actually solves the PDF::API2 problem too.

@boegel
Copy link
Member Author

boegel commented Oct 11, 2023

@boegelbot please test @ generoso
EB_ARGS="Perl-5.36.1-GCCcore-12.3.0.eb Perl-5.36.1-GCCcore-13.1.0.eb Perl-5.38.0-GCCcore-13.2.0.eb Perl-bundle-CPAN-5.36.1-GCCcore-12.3.0.eb"

@boegelbot
Copy link

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=2699 EB_ARGS="Perl-5.36.1-GCCcore-12.3.0.eb Perl-5.36.1-GCCcore-13.1.0.eb Perl-5.38.0-GCCcore-13.2.0.eb Perl-bundle-CPAN-5.36.1-GCCcore-12.3.0.eb" EB_CONTAINER= EB_REPO=easybuild-easyblocks /opt/software/slurm/bin/sbatch --job-name test_PR_2699 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 11921

Test results coming soon (I hope)...

- notification for comment with ID 1758312397 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS Perl-5.36.1-GCCcore-12.3.0.eb
  • SUCCESS Perl-5.36.1-GCCcore-13.1.0.eb
  • SUCCESS Perl-5.38.0-GCCcore-13.2.0.eb
  • SUCCESS Perl-bundle-CPAN-5.36.1-GCCcore-12.3.0.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/506f0ab448869230e58c1817484eeba9 for a full test report.

@boegel
Copy link
Member Author

boegel commented Oct 27, 2023

easybuilders/easybuild-easyconfigs#18789 helps a lot with making this viable to merge, but I just realized that fixing the Perl easyconfigs may not be sufficient, since this is also used in PerlModule which means we may have to fix additional easyconfigs like XML-LibXML, worker, GD, etc which also PerlModule...

I'm starting to think we should get this included with EasyBuild 5.0, since it will no doubt result in some breakage left and right (in easyconfigs we don't control) because we're stricter when testing the "import" of installed Perl modules.

@boegel boegel modified the milestones: next release (4.9.0), 5.0 Dec 28, 2023
@boegel boegel added the EasyBuild-5.0 EasyBuild 5.0 label Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants