Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sagemath_environment must respect --enable-system-site-packages #38190

Closed
2 tasks done
dimpase opened this issue Jun 10, 2024 · 8 comments · Fixed by #38201
Closed
2 tasks done

sagemath_environment must respect --enable-system-site-packages #38190

dimpase opened this issue Jun 10, 2024 · 8 comments · Fixed by #38201

Comments

@dimpase
Copy link
Member

dimpase commented Jun 10, 2024

Steps To Reproduce

Trigger building sagemath_environment, e.g. (after applying #38163) by make sagemath_tdlib, or from scratch, by

./bootstrap && ./configure --enable-system-site-packages --enable-sagemath_tdlib && make build

then, assuming that either setuptools or wheel packages come from the system, it will fail, as observed on #38159 (comment)

Expected Behavior

should work

Actual Behavior

in does not.

Additional Information

There is a lookup happening for wheels for setuptools and wheel - something that is almost sure to fail if the latter come from the system.

Environment

- **OS**: Gentoo Linux
- **10.4.beta9+#38163**:

Checklist

  • I have searched the existing issues for a bug report that matches the one I want to file, without success.
  • I have read the documentation and troubleshoot guide
@dimpase
Copy link
Member Author

dimpase commented Jun 10, 2024

@orlitzky @kiwifb

@saraedum
Copy link
Member

@dimpase am I correct that this is only a blocker if #38163 gets merged?

@dimpase
Copy link
Member Author

dimpase commented Jun 11, 2024

it is a blocker if any use is made of sagemath_environment in its current state.

@dimpase
Copy link
Member Author

dimpase commented Jun 11, 2024

more precisely it could be used with another experimental package, but not with optional or standard.

@dimpase
Copy link
Member Author

dimpase commented Jun 11, 2024

As you see, this bug (two bugs in one - dependency on an exterimental package + breaking --enable-system-side-packages)
has proliferated itself quite a bit

$ git grep sagemath_environment build/pkgs/
build/pkgs/sagemath_bliss/dependencies: bliss cysignals | $(PYTHON_TOOLCHAIN) sage_setup sage_conf sagemath_environment cython pkgconfig $(PYTHON)
build/pkgs/sagemath_categories/dependencies: sagemath_objects | $(PYTHON_TOOLCHAIN) sagemath_environment sage_setup cython pkgconfig python_build $(PYTHON)
build/pkgs/sagemath_coxeter3/dependencies: coxeter3 | $(PYTHON_TOOLCHAIN) sage_setup sagemath_environment cython pkgconfig $(PYTHON)
build/pkgs/sagemath_environment/spkg-src:cd build/pkgs/sagemath_environment
build/pkgs/sagemath_meataxe/dependencies: meataxe cysignals | $(PYTHON_TOOLCHAIN) sage_setup sagemath_environment cython pkgconfig $(PYTHON)
build/pkgs/sagemath_objects/dependencies:FORCE  cysignals gmpy2 | $(PYTHON_TOOLCHAIN) sagemath_environment sage_setup cython pkgconfig python_build $(PYTHON)
build/pkgs/sagemath_repl/dependencies: sagemath_objects sagemath_environment ipython ipywidgets | $(PYTHON_TOOLCHAIN) python_build $(PYTHON)
build/pkgs/sagemath_sirocco/dependencies: sirocco cypari cysignals mpfr | $(PYTHON_TOOLCHAIN) sage_setup sagemath_environment cython pkgconfig $(PYTHON)
build/pkgs/sagemath_tdlib/dependencies: tdlib cysignals boost_cropped | $(PYTHON_TOOLCHAIN) sage_setup sagemath_environment cython pkgconfig $(PYTHON)

of these, sagemath_objects, sagemath_repl, sagemath_categories are experimental. So these don't violate anything,
they are just broken, wrt to the 2nd bug.

For the other ones, something needs to be done. The most natural is to fix sagemath_environment so that
the 2nd bug is fixed, and promote it to optional, removing the 1st bug.

@dimpase
Copy link
Member Author

dimpase commented Jun 11, 2024

I misread the diff for #38163 - the breaking change is much less recent.
This certainly does not make the situation any better, just shifts the blame from one PR to another PR, #35661

@dimpase
Copy link
Member Author

dimpase commented Jun 11, 2024

@dimpase am I correct that this is only a blocker if #38163 gets merged?

no, this is not correct, apologies. The breaking changes came from #35661 - @kiwifb : you didn't notice there that it made a number of optional packages dependent upon an experimental package, sagemath_environment.

And it has broken --enable-system-side-packages

vbraun pushed a commit to vbraun/sage that referenced this issue Jun 16, 2024
…,repl}: Change to optional

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

... from "experimental".

As suggested in
sagemath#38190 (comment)

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38200
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
vbraun pushed a commit to vbraun/sage that referenced this issue Jun 16, 2024
…,repl}: Change to optional

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

... from "experimental".

As suggested in
sagemath#38190 (comment)

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38200
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
vbraun pushed a commit to vbraun/sage that referenced this issue Jun 17, 2024
…,repl}: Change to optional

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

... from "experimental".

As suggested in
sagemath#38190 (comment)

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38200
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
@dimpase
Copy link
Member Author

dimpase commented Jun 28, 2024

with 10.4.rc1, the issue is still here:

[sagemath_environment-10.4.rc1] Setting up build directory /mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/var/tmp/sage/build/sagemath_environment-10.4.rc1
[sagemath_environment-10.4.rc1] Host system: Linux hilbert 6.0.8-gentoo-x86_64 #2 SMP PREEMPT_DYNAMIC Sun Dec 18 22:31:12 GMT 2022 x86_64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz GenuineIntel GNU/Linux
[sagemath_environment-10.4.rc1] C compiler: gcc, Using built-in specs., COLLECT_GCC=gcc, COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/14/lto-wrapper, Target: x86_64-pc-linux-gnu, Configured with: /var/tmp/portage/sys-devel/gcc-14.1.1_p20240615/work/gcc-14-20240615/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/14 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/14/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/14 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/14/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/14/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/14/include/g++-v14 --disable-silent-rules --disable-dependency-tracking --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/14/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 14.1.1_p20240615 p2' --with-gcc-major-version-only --enable-libstdcxx-time --enable-lto --disable-libstdcxx-pch --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --with-multilib-list=m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --enable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --with-zstd --without-isl --enable-default-pie --enable-host-pie --enable-host-bind-now --enable-default-ssp --disable-fixincludes --with-build-config='bootstrap-O3 bootstrap-cet', Thread model: posix, Supported LTO compression algorithms: zlib zstd, gcc version 14.1.1 20240615 (Gentoo 14.1.1_p20240615 p2)
[sagemath_environment-10.4.rc1] No stamp file for package 'sagemath_environment' in /mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/var/lib/sage/installed
[sagemath_environment-10.4.rc1] No spkg-legacy-uninstall script; nothing to do
[sagemath_environment-10.4.rc1] [spkg-install] * Creating isolated environment: venv+pip...
[sagemath_environment-10.4.rc1] [spkg-install] * Installing packages in isolated environment:
[sagemath_environment-10.4.rc1] [spkg-install]   - setuptools >= 68.1.1
[sagemath_environment-10.4.rc1] [spkg-install]   - wheel >=0.36.2
[sagemath_environment-10.4.rc1] [spkg-install] > /mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/bin/python3 -m pip
[sagemath_environment-10.4.rc1] [spkg-install]   --python /tmp/build-env-wch4ns_y/bin/python install --use-pep517 --no-warn-
[sagemath_environment-10.4.rc1] [spkg-install]   script-location --no-compile -r /tmp/build-reqs-h7n4fa42.txt
[sagemath_environment-10.4.rc1] [spkg-install] < Looking in links: file:///mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-
[sagemath_environment-10.4.rc1] [spkg-install]   python3.12/var/lib/sage/wheels
[sagemath_environment-10.4.rc1] [spkg-install] < ERROR: Could not find a version that satisfies the requirement
[sagemath_environment-10.4.rc1] [spkg-install]   setuptools>=68.1.1 (from versions: none)
[sagemath_environment-10.4.rc1] [spkg-install] < ERROR: No matching distribution found for setuptools>=68.1.1
[sagemath_environment-10.4.rc1] [spkg-install] 
[sagemath_environment-10.4.rc1] [spkg-install] Traceback (most recent call last):
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/__main__.py", line 178, in _handle_build_error
[sagemath_environment-10.4.rc1] [spkg-install]     yield
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/__main__.py", line 429, in main
[sagemath_environment-10.4.rc1] [spkg-install]     built = build_call(
[sagemath_environment-10.4.rc1] [spkg-install]             ^^^^^^^^^^^
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/__main__.py", line 268, in build_package_via_sdist
[sagemath_environment-10.4.rc1] [spkg-install]     sdist = _build(isolation, srcdir, outdir, 'sdist', config_settings, skip_dependency_check, installer)
[sagemath_environment-10.4.rc1] [spkg-install]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/__main__.py", line 170, in _build
[sagemath_environment-10.4.rc1] [spkg-install]     return _build_in_isolated_env(srcdir, outdir, distribution, config_settings, installer)
[sagemath_environment-10.4.rc1] [spkg-install]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/__main__.py", line 135, in _build_in_isolated_env
[sagemath_environment-10.4.rc1] [spkg-install]     env.install(builder.build_system_requires)
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/env.py", line 136, in install
[sagemath_environment-10.4.rc1] [spkg-install]     self._env_backend.install_requirements(requirements)
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/env.py", line 265, in install_requirements
[sagemath_environment-10.4.rc1] [spkg-install]     run_subprocess(cmd)
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/site-packages/build/_ctx.py", line 71, in run_subprocess
[sagemath_environment-10.4.rc1] [spkg-install]     subprocess.run(cmd, capture_output=True, check=True, env=env)
[sagemath_environment-10.4.rc1] [spkg-install]   File "/usr/lib/python3.12/subprocess.py", line 571, in run
[sagemath_environment-10.4.rc1] [spkg-install]     raise CalledProcessError(retcode, process.args,
[sagemath_environment-10.4.rc1] [spkg-install] subprocess.CalledProcessError: Command '['/mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/bin/python3', '-m', 'pip', '--python', '/tmp/build-env-wch4ns_y/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-h7n4fa42.txt']' returned non-zero exit status 1.
[sagemath_environment-10.4.rc1] [spkg-install] 
[sagemath_environment-10.4.rc1] [spkg-install] ERROR Command '['/mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/bin/python3', '-m', 'pip', '--python', '/tmp/build-env-wch4ns_y/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-h7n4fa42.txt']' returned non-zero exit status 1.
[sagemath_environment-10.4.rc1] [spkg-install] ************************************************************************************************************************************************************************************************************************************************************
[sagemath_environment-10.4.rc1] [spkg-install] Failure building sdist and wheel
[sagemath_environment-10.4.rc1] [spkg-install] ************************************************************************************************************************************************************************************************************************************************************
[sagemath_environment-10.4.rc1] ************************************************************************
[sagemath_environment-10.4.rc1] Error installing package sagemath_environment-10.4.rc1
[sagemath_environment-10.4.rc1] ************************************************************************
[sagemath_environment-10.4.rc1] Please email sage-devel (http://groups.google.com/group/sage-devel)
[sagemath_environment-10.4.rc1] explaining the problem and including the log files
[sagemath_environment-10.4.rc1]   /mnt/opt/Sage/sage-dev/logs/pkgs/sagemath_environment-10.4.rc1.log
[sagemath_environment-10.4.rc1] and
[sagemath_environment-10.4.rc1]   /mnt/opt/Sage/sage-dev/config.log
[sagemath_environment-10.4.rc1] Describe your computer, operating system, etc.
[sagemath_environment-10.4.rc1] If you want to try to fix the problem yourself, *don't* just cd to
[sagemath_environment-10.4.rc1] /mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/var/tmp/sage/build/sagemath_environment-10.4.rc1 and type 'make' or whatever is appropriate.
[sagemath_environment-10.4.rc1] Instead, the following commands setup all environment variables
[sagemath_environment-10.4.rc1] correctly and load a subshell for you to debug the error:
[sagemath_environment-10.4.rc1]   (cd '/mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/var/tmp/sage/build/sagemath_environment-10.4.rc1' && '/mnt/opt/Sage/sage-dev/sage' --buildsh)
[sagemath_environment-10.4.rc1] When you are done debugging, you can type "exit" to leave the subshell.
[sagemath_environment-10.4.rc1] ************************************************************************
make[4]: *** [Makefile:3606: sagemath_environment-SAGE_VENV-no-deps] Error 1
make[3]: *** [Makefile:3606: /mnt/opt/Sage/sage-dev/local/var/lib/sage/venv-python3.12/var/lib/sage/installed/sagemath_environment-10.4.rc1] Error 2
make[2]: *** [Makefile:3062: all-build] Error 2
make[2]: Leaving directory '/mnt/opt/Sage/sage-dev/build/make'
***************************************************************
Error building Sage.

So an optional package (sagemath_environment got promoted to optional) fails to build.

@vbraun vbraun closed this as completed in 2a67457 Jul 4, 2024
@mkoeppe mkoeppe added this to the sage-10.4 milestone Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants