Merge pynac sources as src/sage/symbolic/ginac

[mkoeppe]

Pynac has a compile-time dependency on the Python library but is not installed using Python package infrastructure. This is problematic because Python users cannot install it using standard Python tools - for example for testing different Python versions.

Pynac has no other uses than as the core of the symbolic expressions facility of Sage; and cannot even be tested without Sage.

We merge Pynac (i.e., the directory https://github.com/pynac/pynac/tree/master/ginac) into the Sage library as src/sage/symbolic/ginac. It is not a very big package - one flat directory with 100 *.h and *.cpp files, a total of 50kLOC. (The existing !Python/Cython symbolics code in sage.symbolic, sage.calculus, sage.functions is about 60kLOC.)

The Pynac sources are compiled like other C++ sources that are already in the Sage source tree. They are linked into a single Python extension module, sage.symbolic.expression. All other extension modules that used to link to pynac (sage.libs.pynac.pynac, sage.symbolic.function, sage.symbolic.series etc.) are

• either refactored so that they only import helper functions provided by sage.symbolic.expression instead of having to cimport Pynac functions directly (see sage.symbolic.ring: Remove direct use of cimports from pynac #32391, sage.symbolic.function: Remove direct use of cimports from pynac #32407)
• or merged into the extension module sage.symbolic.expression (via Cython include statements, not by copy-paste of source code, to keep the diff small)

This solves the same issues that #30534 tried to address, which ran into unresolved technical difficulties.

It will also make it easier for Sage developers to make changes to Pynac and the related symbolic implementation.

Follow-up: #32387 Remove pynac spkg

Depends on #32391
Depends on #32407
Depends on #32461

CC: @kliem @kiwifb @antonio-rojas @DaveWitteMorris @slel @dimpase @tscrim @egourgoulhon @nbruin

Component: symbolics

Author: Matthias Koeppe, Jonathan Kliem

Branch: 0d0b58f

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/32386

[mkoeppe]

Branch: u/mkoeppe/merge_pynac_as_src_sage_symbolic_ginac

[mkoeppe]

comment:3

I ran git filter-branch -f --tree-filter 'rm -f Makefile.am; mkdir -p src/sage/symbolic/ginac; mv *.* src/sage/symbolic/ginac/; :' -- --all on pynac 0.7.29 and merged it here, thus preserving the relevant history

---

Last 10 new commits:

f3bb79f	Merge pull request #374 from slel/fix-exp-inv-hyp
a99791b	apply patches from #31479 and #31554
91cb6fa	apply trac 31869 fix pynac integer_content
d96393c	apply trac 30688 patch infinite loop in pynac
80269c5	apply trac 31645 handling of constant term in series
4caca2e	apply trac 31530 patch pynac power::subs
cba9038	trac 31585 32-bit overflow
8acb548	also fix /=
f2040c8	ginac/numeric.cpp: Only use builtin for smull_overflow if supported by the compiler
e43830d	Merge remote-tracking branch 'pynac-filtered/master' into t/32386/merge_pynac_as_src_sage_symbolic_ginac

[mkoeppe]

Commit: e43830d

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2bbf412

build/pkgs/pynac: Remove

[sagetrac-git]

Changed commit from e43830d to 2bbf412

[mkoeppe]

comment:5

Next: Move stuff from sage.libs.pynac to sage.symbolic.ginac, leaving some deprecated reimports behind.

Help from Cython experts very welcome...

[sagetrac-git]

Changed commit from 2bbf412 to e693827

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

e693827

Move stuff from sage.libs.pynac to sage.symbolic.ginac, leaving some deprecated reimports behind

[sagetrac-git]

Changed commit from e693827 to 0581bac

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

0bfc24f	src/sage/symbolic/ginac/version.h.in: Remove
0581bac	src/sage/symbolic/ginac/pynac-config.h: New

[mkoeppe]

comment:8

This version compiles OK

[mkoeppe]

Author: Matthias Koeppe

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

496102d	src/sage/symbolic/ginac/utils.h, pynac-config.h: Eliminate use of PYNAC_SIZEOF_LONG, PYNAC_SIZEOF_LONG_LONG
fc29c1d	src/sage/symbolic/pynac.pyx, pynac.pxd: Fixup distutils: sources

[sagetrac-git]

Changed commit from 0581bac to fc29c1d

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

8223127	src/sage/symbolic/ginac/utils.h: Fixup
add5e89	src/sage/symbolic/ginac/utils.cpp: Fixup removal of version.h

[sagetrac-git]

Changed commit from fc29c1d to add5e89

[mkoeppe]

comment:14

This mostly works but I'm getting a few timeouts/crashes that I'm not sure about

sage -t --random-seed=0 src/sage/schemes/elliptic_curves/ell_number_field.py  # Timed out
sage -t --random-seed=0 src/sage/manifolds/differentiable/degenerate_submanifold.py  # 1 doctest failed
sage -t --random-seed=0 src/sage/manifolds/differentiable/tensorfield.py  # Timed out
sage -t --random-seed=0 src/sage/plot/plot3d/parametric_plot3d.py  # Timed out
sage -t --random-seed=0 src/sage/plot/plot3d/plot3d.py  # Timed out
sage -t --random-seed=0 src/sage/plot/plot3d/base.pyx  # Timed out
sage -t --random-seed=0 src/sage/plot/plot3d/implicit_plot3d.py  # Timed out
sage -t --random-seed=0 src/sage/symbolic/relation.py  # Timed out
sage -t --random-seed=0 src/sage/symbolic/expression.pyx  # Killed due to illegal instruction
sage -t --random-seed=0 src/sage/symbolic/pynac_constant.pyx  # 18 doctests failed

[sagetrac-git]

Changed commit from add5e89 to d9a1420

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

d9a1420

src/sage/symbolic/pynac_constant.pyx: Fix imports in doctests

[mkoeppe]

comment:16

The crash is from this doctest:

>>> m = var('m')
>>> assume(m, 'integer')
>>> (I**m).real_part()

which seems to lead to an infinite recursion through GiNaC::power::real_part()

[mkoeppe]

comment:17

Oh, this is #28357 and I forgot to apply realpartloop.patch

[dimpase]

Changed reviewer from https://github.com/mkoeppe/sage/actions/runs/1168763794, https://github.com/mkoeppe/sage/actions/runs/1168763799, ... to Dima Pasechnik

[dimpase]

comment:107

lgtm

[mkoeppe]

comment:108

Thank you!

[dimpase]

comment:109

should #32461 be a dependency here?

[mkoeppe]

Changed dependencies from #32391, #32407 to #32391, #32407, #32461

[sagetrac-git]

Changed commit from 327ca5f to 0d0b58f

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

0d0b58f

Merge tag '9.5.beta1' into t/32386/merge_pynac_as_src_sage_symbolic_ginac

[vbraun]

Changed branch from u/mkoeppe/merge_pynac_as_src_sage_symbolic_ginac to 0d0b58f

[egourgoulhon]

Changed commit from 0d0b58f to none

[egourgoulhon]

comment:114

Replying to @mkoeppe:

It will also need to be checked that the deferred imports (such as those introduced in my commit 3153781) do not lead to speed regressions

FWIW, the change performed here did not introduced any speed regression in the rather demanding Kerr spacetime notebook, which involves parallelized (8 threads) heavy symbolic computations. On the contrary, it seems there is some little speed up :-) On my i7-8665U laptop:

• Sage 9.4: 461 s
• Sage 9.5.beta2: 449 s

[mkoeppe]

comment:115

Thanks for testing!