Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GAP 4.13.1 #38804

Merged
merged 17 commits into from
Oct 26, 2024
Merged

GAP 4.13.1 #38804

merged 17 commits into from
Oct 26, 2024

Conversation

orlitzky
Copy link
Contributor

Follow-up to:

With the four additional work items I mentioned in a comment on the latter:

  1. Everything has been rebased
  2. There's a new feature to detect the polenta GAP package
  3. The three failing simplicial sets tests have been marked # needs gap_package_polenta
  4. I backported Fixes to hash functions for lists gap-system/gap#5796 to the GAP spkg so that the optional GRAPE tests will pass (untested).

Let's see what the CI has to say...

tornaria and others added 14 commits October 12, 2024 16:40
In gap 4.13 there are some improvements, e.g. converting fp groups to
permutation groups, computing abelianization of fp groups, which lead to
different generators.

This commit fixes doctests so they pass using gap 4.13.
…ced to dummy from list of packages whose test suite is disabled
There are a few simplicial set tests that require the polenta package
with gap-4.13.x. We declare a new feature to support the corresponding
"needs" tag.
…ackage

Three tests in this file need the GAP package "polenta" to be
installed; otherwise, the output (although perhaps not _wrong_) is not
quite what we expect.
Fixes gap-system/gap#5796

This is the patch I'm using on Gentoo, it combines the upstream fix
and one of its dependencies (without which the desired fix does not
apply). Without it, you'll get some test failures with gap-4.13.x
when GRAPE happens to be installed.
I've left the upper bound in there, but set to "5.0.0" which should
have no real effect. But if 4.14.0 comes out and breaks somes tests,
it's a lot easier to tweak the existing upper bound than it is to add
one (again) and reindent everything.
@orlitzky
Copy link
Contributor Author

Locally, I've still got the element.pyx segfault from #37026, but everything else is passing.

The CI is unhappy about the upgrade path, but I'm not sure why exactly. This is the failure... it looks like sagelib wasn't rebuilt after GAP was upgraded. The Makefile should handle that IIRC.

2024-10-12T21:10:34.0893216Z Running doctests with ID 2024-10-12-21-10-34-b8b2f1f4.
2024-10-12T21:10:34.0894767Z Running with SAGE_LOCAL='/sage/local' and SAGE_VENV='/sage/local/var/lib/sage/venv-python3.10'
2024-10-12T21:10:34.0895510Z Using --optional=debian,pip,sage,sage_spkg
2024-10-12T21:10:34.0906368Z Features to be detected: 4ti2,SAGE_SRC,benzene,bliss,buckygen,conway_polynomials,coxeter3,csdp,cvxopt,cvxopt,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_cubic_hecke,database_ellcurves,database_graphs,database_jones_numfield,database_knotinfo,dvipng,ecm,fpylll,fricas,gap_package_atlasrep,gap_package_design,gap_package_grape,gap_package_guava,gap_package_hap,gap_package_polenta,gap_package_polycyclic,gap_package_qpa,gap_package_quagroup,gfan,giac,glucose,graphviz,imagemagick,ipython,jmol,jupymake,jupyter_sphinx,kenzo,kissat,latte_int,lrcalc_python,lrslib,mathics,matroid_database,mcqd,meataxe,mpmath,msolve,nauty,networkx,numpy,palp,pandoc,pdf2svg,pdftocairo,pexpect,phitigra,pillow,plantri,polytopes_db,polytopes_db_4d,pplpy,primecountpy,ptyprocess,pycosat,pycryptosat,pynormaliz,pyparsing,python_igraph,requests,rpy2,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.braiding,sage.libs.ecl,sage.libs.flint,sage.libs.gap,sage.libs.giac,sage.libs.homfly,sage.libs.linbox,sage.libs.m4ri,sage.libs.ntl,sage.libs.pari,sage.libs.singular,sage.misc.cython,sage.modular,sage.modules,sage.numerical.mip,sage.plot,sage.rings.complex_double,sage.rings.finite_rings,sage.rings.function_field,sage.rings.number_field,sage.rings.padics,sage.rings.polynomial.pbori,sage.rings.real_double,sage.rings.real_mpfr,sage.sat,sage.schemes,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,scipy,singular,sirocco,sphinx,symengine_py,sympy,tdlib,threejs,topcom
2024-10-12T21:10:34.0914912Z Sorting sources by runtime so that slower doctests are run first....
2024-10-12T21:10:34.0915388Z Doctesting 7 files using 4 threads.
2024-10-12T21:10:34.7054133Z Traceback (most recent call last):
2024-10-12T21:10:34.7054986Z   File "/sage/src/bin/sage-runtests", line 9, in <module>
2024-10-12T21:10:34.7055891Z     sys.exit(main())
2024-10-12T21:10:34.7056369Z   File "/sage/src/sage/doctest/__main__.py", line 192, in main
2024-10-12T21:10:34.7056896Z     err = DC.run()
2024-10-12T21:10:34.7057321Z   File "/sage/src/sage/doctest/control.py", line 1602, in run
2024-10-12T21:10:34.7057833Z     self.run_doctests()
2024-10-12T21:10:34.7058344Z   File "/sage/src/sage/doctest/control.py", line 1186, in run_doctests
2024-10-12T21:10:34.7058972Z     self.dispatcher = DocTestDispatcher(self)
2024-10-12T21:10:34.7059543Z   File "/sage/src/sage/doctest/forker.py", line 1717, in __init__
2024-10-12T21:10:34.7060084Z     init_sage(controller)
2024-10-12T21:10:34.7060569Z   File "/sage/src/sage/doctest/forker.py", line 207, in init_sage
2024-10-12T21:10:34.7061119Z     controller.load_environment()
2024-10-12T21:10:34.7061693Z   File "/sage/src/sage/doctest/control.py", line 686, in load_environment
2024-10-12T21:10:34.7062385Z     return import_module(self.options.environment)
2024-10-12T21:10:34.7063079Z   File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
2024-10-12T21:10:34.7063842Z     return _bootstrap._gcd_import(name[level:], package, level)
2024-10-12T21:10:34.7064586Z   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
2024-10-12T21:10:34.7065115Z   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
2024-10-12T21:10:34.7065674Z   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
2024-10-12T21:10:34.7066233Z   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
2024-10-12T21:10:34.7066768Z   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
2024-10-12T21:10:34.7067350Z   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2024-10-12T21:10:34.7067983Z   File "/sage/src/sage/repl/ipython_kernel/all_jupyter.py", line 6, in <module>
2024-10-12T21:10:34.7068494Z     from sage.all_cmdline import *
2024-10-12T21:10:34.7068886Z   File "/sage/src/sage/all_cmdline.py", line 17, in <module>
2024-10-12T21:10:34.7069483Z     from sage.all import *
2024-10-12T21:10:34.7069807Z   File "/sage/src/sage/all.py", line 98, in <module>
2024-10-12T21:10:34.7070171Z     from sage.groups.all import *
2024-10-12T21:10:34.7070543Z   File "/sage/src/sage/groups/all.py", line 5, in <module>
2024-10-12T21:10:34.7070966Z     from sage.groups.matrix_gps.all import *
2024-10-12T21:10:34.7071437Z   File "/sage/src/sage/groups/matrix_gps/all.py", line 13, in <module>
2024-10-12T21:10:34.7071963Z     import sage.groups.matrix_gps.pickling_overrides
2024-10-12T21:10:34.7072523Z   File "/sage/src/sage/groups/matrix_gps/pickling_overrides.py", line 7, in <module>
2024-10-12T21:10:34.7073273Z     from sage.groups.matrix_gps.finitely_generated_gap import FinitelyGeneratedMatrixGroup_gap
2024-10-12T21:10:34.7074066Z   File "/sage/src/sage/groups/matrix_gps/finitely_generated_gap.py", line 30, in <module>
2024-10-12T21:10:34.7074750Z     from sage.groups.matrix_gps.matrix_group_gap import MatrixGroup_gap
2024-10-12T21:10:34.7075392Z   File "/sage/src/sage/groups/matrix_gps/matrix_group_gap.py", line 24, in <module>
2024-10-12T21:10:34.7076006Z     from sage.groups.libgap_mixin import GroupMixinLibGAP
2024-10-12T21:10:34.7076516Z   File "/sage/src/sage/groups/libgap_mixin.py", line 15, in <module>
2024-10-12T21:10:34.7076983Z     from sage.libs.gap.libgap import libgap
2024-10-12T21:10:34.7077509Z ImportError: libgap.so.8: cannot open shared object file: No such file or directory

@orlitzky
Copy link
Contributor Author

I tried to remove $HOME/.sage/gap before running the tests in the CI but that didn't help. My interest in a proprietary test suite that I can't actually run only goes so far.

Tomorrow I'll test the SPKG upgrade path myself.

@dimpase
Copy link
Member

dimpase commented Oct 13, 2024

proprietary test suite that I can't actually run only goes so far.

It's not proprietary: you can either attach your runner to GitHub CI, or run tests fully locally: https://nektosact.com/ (you'd need Docker and go installed, though)

@enriqueartal
Copy link
Contributor

For what it is worth in a test installation of Fedora 40, with this branch using spkg gap and installing gap_packages and sirocco, no gap-related test fails.

@orlitzky
Copy link
Contributor Author

make doesn't rebuild the extension modules in sage/libs/gap after the gap upgrade. I can force it to rebuild locally by touching the three pyx files, but I guess that doesn't help the CI.

@enriqueartal
Copy link
Contributor

If I understand correctly, in several local installation the branch is successful, but not in CI, because some files are not rebuilt. What can be done to make possible a positive review?

@tobiasdiez
Copy link
Contributor

tobiasdiez commented Oct 20, 2024

It's indeed not ideal but the PR is already in Volker's release branch, so I'll try to prepare the necessary conda updates and mark them as ci-fix as soon as the new beta is out.

@tobiasdiez tobiasdiez mentioned this pull request Oct 22, 2024
5 tasks
vbraun pushed a commit to vbraun/sage that referenced this pull request Oct 23, 2024
    
Follow-up to:

* sagemath#37884
* sagemath#38169

With the four additional work items I mentioned in a comment on the
latter:

1. Everything has been rebased
2. There's a new feature to detect the polenta GAP package
3. The three failing simplicial sets tests have been marked `# needs
gap_package_polenta`
4. I backported gap-system/gap#5796 to the GAP
spkg so that the optional GRAPE tests will pass (untested).

Let's see what the CI has to say...
    
URL: sagemath#38804
Reported by: Michael Orlitzky
Reviewer(s): Dima Pasechnik, Enrique Manuel Artal Bartolo
@vbraun vbraun merged commit d69ef01 into sagemath:develop Oct 26, 2024
26 of 45 checks passed
@dimpase
Copy link
Member

dimpase commented Oct 28, 2024

This has completely missed the needed for gap_packages bump to libsemigroups - they should be done in lockstep. @orlitzky

@orlitzky
Copy link
Contributor Author

This has completely missed the needed for gap_packages bump to libsemigroups - they should be done in lockstep. @orlitzky

Sorry, it simply never occurred to me.

@dimpase
Copy link
Member

dimpase commented Oct 28, 2024

I've opened a PR to upgrade - see #38875

@dimpase dimpase mentioned this pull request Oct 28, 2024
2 tasks
@culler
Copy link
Contributor

culler commented Oct 30, 2024

Unfortunately, the gap_packages-4.13.1 spkg does not build with XCode 16 on Intel macOS Sequoia 15.0 using the libsemigroups package added in PR #38875. I tried with both XCode 16.0 and 16.1 (released 2 days ago). The build works with XCode 16.1 on macOS 15.1 with the M1 arm cpu. The log file is attached. The relevant part is:

[gap_packages-4.13.1] [spkg-install] g++ -std=gnu++11 -pthread -O2 -mmacosx-version-min=10.9 -mno-avx -mno-avx2 -mno-bmi2 -stdlib=libc++ -fno-common -MQ gen/src/cong.o -MMD -MP -MF gen/src/cong.d -std=gnu++14 -O3 -mavx -flax-vector-conversions -Igapbind14/include/ -DHPCOMBI_CONSTEXPR_FUN_ARGS -DFMT_HEADER_ONLY -DNDEBUG -O2 -mmacosx-version-min=10.9 -mno-avx -mno-avx2 -mno-bmi2 -stdlib=libc++ -o gen/src/cong.o -I/private/var/tmp/sage-10.5-current/local/include/gap -I/private/var/tmp/sage-10.5-current/local/include -DUSE_GASMAN=1 -c src/cong.cpp
[gap_packages-4.13.1] [spkg-install] In file included from src/cong.cpp:29:
[gap_packages-4.13.1] [spkg-install] In file included from src/to_cpp.hpp:59:
[gap_packages-4.13.1] [spkg-install] In file included from /private/var/tmp/sage-10.5-current/local/include/libsemigroups/transf.hpp:49:
[gap_packages-4.13.1] [spkg-install] In file included from /private/var/tmp/sage-10.5-current/local/include/libsemigroups/hpcombi.hpp:35:
[gap_packages-4.13.1] [spkg-install] In file included from /private/var/tmp/sage-10.5-current/local/include/libsemigroups/HPCombi/include/hpcombi.hpp:19:
[gap_packages-4.13.1] [spkg-install] /private/var/tmp/sage-10.5-current/local/include/libsemigroups/HPCombi/include/epu.hpp:64:15: error: static assertion failed due to requirement 'alignof(__attribute__((__vector_size__(32 * sizeof(unsigned char)))) unsigned char) == 32': xpu8 type is not properly aligned by the compiler !
[gap_packages-4.13.1] [spkg-install]    64 | static_assert(alignof(xpu8) == 32,
[gap_packages-4.13.1] [spkg-install]       |               ^~~~~~~~~~~~~~~~~~~
[gap_packages-4.13.1] [spkg-install] /private/var/tmp/sage-10.5-current/local/include/libsemigroups/HPCombi/include/epu.hpp:64:29: note: expression evaluates to '16 == 32'
[gap_packages-4.13.1] [spkg-install]    64 | static_assert(alignof(xpu8) == 32,
[gap_packages-4.13.1] [spkg-install]       |               ~~~~~~~~~~~~~~^~~~~
[gap_packages-4.13.1] [spkg-install] 1 error generated.
[gap_packages-4.13.1] [spkg-install] make[5]: *** [gen/src/cong.o] Error 1
[gap_packages-4.13.1] [spkg-install] ********************************************************************************
[gap_packages-4.13.1] [spkg-install] Error building gap_packages-4.13.1

gap_packages-4.13.1.log

@dimpase
Copy link
Member

dimpase commented Oct 30, 2024

This is certainly an upstream issue. Please report it to
https://github.com/semigroups/Semigroups

By the wat, gap 4.14 is due to be released soon.
Perhaps it's already dealt with there

@culler
Copy link
Contributor

culler commented Oct 30, 2024

I did some experiments to see what is going on here. It turns out that the -no-avx flag causes the static assertion to fail. If that flag is removed from the g++ command then the static assertion succeeds. The flag is there because we were trying to support all Intel CPUs that are compatible with macOS 10.12. Essentially we are targeting the Core-2 Duo chip, which did not support avx. Maybe I will declare that SageMath-10-5.app requires at least a Sandy Bridge CPU. I doubt that libsemigroups wants to do any extra work in order to support older CPUs than Sandy Bridge. That means Mac models released before 2010.

@dimpase
Copy link
Member

dimpase commented Oct 30, 2024

My old core2 duo macAir stopped being useful 5 or more years ago. I suppose we can drop supporting these configurations in Sage in general

@dimpase
Copy link
Member

dimpase commented Oct 30, 2024

after all, macOS 12 has reached EOL last month.
Let's drop support for it.
https://endoflife.date/macos

@culler
Copy link
Contributor

culler commented Oct 30, 2024 via email

@dimpase
Copy link
Member

dimpase commented Oct 30, 2024

I am talking about macOS 10.12, which is 9 years old, not macOS 12 which is
only 3 years old. I am certainly not dropping support for 3 year old
operating systems.

OK, sorry, I messed up with digits here...
How about macOS 10?
And 11 too? @vbraun - any opinion on this?

https://endoflife.date/macos

@culler
Copy link
Contributor

culler commented Oct 30, 2024 via email

@vbraun
Copy link
Member

vbraun commented Oct 31, 2024

Afaik the oldest hardware that Apple supports is a 2017 iMac (running Ventura). Thats 7-th Gen Intel (Kaby Lake). That definitely runs AVX. We shouldn't try to build new versions for hardware that the manufacturer abandoned.

@culler
Copy link
Contributor

culler commented Oct 31, 2024 via email

@dimpase
Copy link
Member

dimpase commented Oct 31, 2024

This discussion comes from the fact you stumbled upon a macOS specific issue, which originated from trying to support museum-grade hardware. Which I would classify as a Sage bug. Thanks for digging it up.

If Sage didn't support the aforementioned in the first place, we would not be having this discussion in the first place. Now we have uncovered this bug, and we shall proceed to fix it. Right, @vbraun ?

@culler
Copy link
Contributor

culler commented Oct 31, 2024 via email

vbraun pushed a commit to vbraun/sage that referenced this pull request Nov 5, 2024
    
necessary for GAP 4.13.1

This was forgotten in sagemath#38804


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38875
Reported by: Dima Pasechnik
Reviewer(s): Marc Culler
vbraun pushed a commit to vbraun/sage that referenced this pull request Nov 6, 2024
    
necessary for GAP 4.13.1

This was forgotten in sagemath#38804


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38875
Reported by: Dima Pasechnik
Reviewer(s): Marc Culler
vbraun pushed a commit to vbraun/sage that referenced this pull request Nov 7, 2024
    
necessary for GAP 4.13.1

This was forgotten in sagemath#38804


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38875
Reported by: Dima Pasechnik
Reviewer(s): Marc Culler
vbraun pushed a commit to vbraun/sage that referenced this pull request Nov 8, 2024
    
necessary for GAP 4.13.1

This was forgotten in sagemath#38804


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38875
Reported by: Dima Pasechnik
Reviewer(s): Marc Culler
vbraun pushed a commit to vbraun/sage that referenced this pull request Nov 9, 2024
    
necessary for GAP 4.13.1

This was forgotten in sagemath#38804


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38875
Reported by: Dima Pasechnik
Reviewer(s): Marc Culler
vbraun pushed a commit to vbraun/sage that referenced this pull request Nov 13, 2024
    
necessary for GAP 4.13.1

This was forgotten in sagemath#38804


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38875
Reported by: Dima Pasechnik
Reviewer(s): Marc Culler
vbraun pushed a commit to vbraun/sage that referenced this pull request Nov 14, 2024
    
necessary for GAP 4.13.1

This was forgotten in sagemath#38804


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38875
Reported by: Dima Pasechnik
Reviewer(s): Marc Culler
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants