Add install instructions for editable Python installation #721

jorgensd · 2023-10-27T09:54:57Z

Tested with:

FROM ubuntu:22.04

ENV DEBIAN_FRONTEND=noninteractive
ARG TARGET_PLATFORM
RUN apt-get update && \
    apt-get install -y software-properties-common
RUN apt-get install -y git \
    cmake \
    gcc \
    g++ \
    ninja-build \
    libopenblas-dev \ 
    liblapack-dev
ENV DEB_PYTHON_INSTALL_LAYOUT=deb_system
RUN apt-get install -y python3-pip
RUN python3 -m pip install -U setuptools pip
RUN python3 -m pip install git+https://github.com/FEniCS/ufl.git

ENV CMAKE_BUILD_PARALLEL_LEVEL 2
WORKDIR /src
RUN git clone --branch=dokken/basix-install-instructions --single-branch https://github.com/FEniCS/basix.git
WORKDIR /src/basix
RUN cmake -G Ninja -B build-dir -DCMAKE_BUILD_TYPE=Debug -S cpp/ && \
    cmake --build build-dir && \ 
    cmake --install build-dir

WORKDIR /src/basix/python
RUN python3 -m pip -v install -r build-requirements.txt
RUN python3 -m pip -v install --config-settings=build-dir="build" --config-settings=cmake.build-type="Debug" --no-build-isolation -e .

WORKDIR /tmp
RUN python3 -c "import basix"

and

FROM ubuntu:22.04

ENV DEBIAN_FRONTEND=noninteractive
ARG TARGET_PLATFORM
RUN apt-get update && \
    apt-get install -y software-properties-common
RUN apt-get install -y git \
    cmake \
    gcc \
    g++ \
    ninja-build \
    libopenblas-dev \ 
    liblapack-dev
ENV DEB_PYTHON_INSTALL_LAYOUT=deb_system
RUN apt-get install -y python3-pip
RUN python3 -m pip install -U setuptools pip
RUN python3 -m pip install git+https://github.com/FEniCS/ufl.git

ENV CMAKE_BUILD_PARALLEL_LEVEL 2
WORKDIR /src
RUN git clone --branch=dokken/basix-install-instructions --single-branch https://github.com/FEniCS/basix.git
WORKDIR /src/basix
RUN cmake -G Ninja -B build-dir -DCMAKE_BUILD_TYPE=Debug -S cpp/ && \
    cmake --build build-dir && \ 
    cmake --install build-dir

WORKDIR /src/basix
RUN python3 -m pip -v install -r build-requirements.txt
RUN python3 -m pip -v install --config-settings=build-dir="build" --config-settings=cmake.build-type="Debug" --no-build-isolation -e .

WORKDIR /tmp
RUN python3 -c "import basix"

Resolves #719

…stall of C++)

python/README.rst

garth-wells

See earlier PR - #720.

Co-authored-by: Jørgen Schartum Dokken <dokken@simula.no>

jhale · 2023-10-27T12:21:48Z

I think this PR is a little more complete than #720

jorgensd · 2023-10-27T12:52:14Z

See earlier PR - #720.

720 is not complete as it doesn't specify the build requirements.

francesco-ballarin · 2023-10-27T12:58:19Z

As user, I struggle to understand* why build isolation is required with editable installs, but not with standard ones. Maybe the reason for this is worth mentioning in the README?

* I do understand FEniCS/dolfinx#2707 instead, in the sense that with build isolation even the standard install would fail to locate mpi4py and petsc4py.

garth-wells · 2023-10-27T13:41:28Z

I think this PR is a little more complete than #720

It has too much text. Too much to read and too much that can go out of date. The instructions are for developers, who can do a bit of work themselves.

jorgensd · 2023-10-27T14:01:12Z

I think this PR is a little more complete than #720

It has too much text. Too much to read and too much that can go out of date. The instructions are for developers, who can do a bit of work themselves.

I dont this this is fair. It is far from trivial to get this to work. The other PR doesn’t even have valid instructions, as it is missing pins and build dependencies

jorgensd · 2023-10-27T14:19:58Z

As user, I struggle to understand* why build isolation is required with editable installs, but not with standard ones. Maybe the reason for this is worth mentioning in the README?

* I do understand FEniCS/dolfinx#2707 instead, in the sense that with build isolation even the standard install would fail to locate mpi4py and petsc4py.

I'm not sure what the cause of the issue is, but if you don't use build-isolation, you get the following error when import basix:
``bash

[14/14] RUN python3 -c "import basix":
0.408 :241: RuntimeWarning: nanobind: type 'LatticeType' was already registered!
0.408
0.409 :241: RuntimeWarning: nanobind: type 'LatticeSimplexMethod' was already registered!
0.409
0.410 :241: RuntimeWarning: nanobind: type 'PolynomialType' was already registered!
0.410
0.410 :241: RuntimeWarning: nanobind: type 'MapType' was already registered!
0.410
0.410 :241: RuntimeWarning: nanobind: type 'SobolevSpace' was already registered!
0.410
0.410 :241: RuntimeWarning: nanobind: type 'QuadratureType' was already registered!
0.410
0.411 :241: RuntimeWarning: nanobind: type 'CellType' was already registered!
0.411
0.411 :241: RuntimeWarning: nanobind: type 'ElementFamily' was already registered!
0.411
0.411 :241: RuntimeWarning: nanobind: type 'FiniteElement' was already registered!
0.411
0.412 :241: RuntimeWarning: nanobind: type 'LagrangeVariant' was already registered!
0.412
0.412 :241: RuntimeWarning: nanobind: type 'DPCVariant' was already registered!
0.412
0.412 :241: RuntimeWarning: nanobind: type 'PolysetType' was already registered!
0.412
0.413 Traceback (most recent call last):
0.413 File "", line 1, in
0.414 File "/src/basix/python/basix/init.py", line 7, in
0.414 from . import cell, finite_element, lattice, polynomials, quadrature, sobolev_spaces, variants
0.414 File "/src/basix/python/basix/finite_element.py", line 3, in
0.415 from ._basixcpp import ElementFamily as _EF
0.415 ImportError: cannot import name 'ElementFamily' from 'basix.finite_element._basixcpp' (/usr/lib/python3/dist-packages/basix/_basixcpp.cpython-310-x86_64-linux-gnu.so)
0.419 free(): invalid pointer
0.507 Aborted (core dumped)

francesco-ballarin · 2023-10-27T14:24:18Z

I'm not sure what the cause of the issue is, but if you don't use build-isolation, you get the following error when import basix:

I guess that my question is more: how come that pip install . (without -e) works correctly?

jhale · 2023-10-27T19:53:17Z

@francesco-ballarin It is a bug in scikit-build-core latest version for editable builds that will be fixed upstream at some point.

jorgensd · 2023-10-27T20:02:52Z

@francesco-ballarin It is a bug in scikit-build-core latest version for editable builds that will be fixed upstream at some point.

Note that even pining scikit-build-core doesn’t help wrt removing build isolation. So there are two scikit bugs here:

Need build isolation for editable install
Import issues with relative paths and editable install scikit-build/scikit-build-core#532

jorgensd · 2023-10-28T10:58:49Z

A compromise would be to use split install in one of the CIs, as it would then be easily maintainable and easily accessed by developers.

mscroggs · 2023-10-30T11:22:15Z

split install

I think that this run on the CI is trying to do this, but perhaps needs updating? Or do you mean something else by split install?

jorgensd · 2023-10-30T11:57:17Z

split install

I think that this run on the CI is trying to do this, but perhaps needs updating? Or do you mean something else by split install?

It does a split install, but not a split editable installation. Most developers would probably use the editable install option (which has the requirements described above).

mscroggs · 2023-10-30T11:58:56Z

split install

I think that this run on the CI is trying to do this, but perhaps needs updating? Or do you mean something else by split install?

It does a split install, but not a split editable installation. Most developers would probably use the editable install option (which has the requirements described above).

We change it to editable. Or include both (is there any benefit to having both?)

jorgensd · 2023-10-30T12:08:17Z

split install

I think that this run on the CI is trying to do this, but perhaps needs updating? Or do you mean something else by split install?

It does a split install, but not a split editable installation. Most developers would probably use the editable install option (which has the requirements described above).

We change it to editable. Or include both (is there any benefit to having both?)

Well, it would give better coverage. In any case, we would need the build_requirements.txt to be Added to Basix.

garth-wells · 2023-10-30T15:12:20Z

INSTALL.md

@@ -5,13 +5,20 @@
 Basix can be installed using

 ```console
-pip install .
+python3 -m pip install .


The former was fine. Less is more.

garth-wells · 2023-10-30T15:12:49Z

INSTALL.md

+For Python development, an editable install can be done with
+
+```console
+python3 -m pip -v install -r build-requirements.txt


-v not required

garth-wells · 2023-10-30T15:12:58Z

INSTALL.md

+
+```console
+python3 -m pip -v install -r build-requirements.txt
+python3 -m pip -v install --config-settings=build-dir="build" --config-settings=cmake.build-type="Debug" --no-build-isolation -e .


-v not required

garth-wells · 2023-10-30T15:13:17Z

INSTALL.md

 ### Python interface

 After installing the C++ library, install the Python interface by running in
 the directory `python/`:

 ```console
-pip install .
+python3 -m pip install .


Use short version.

garth-wells · 2023-10-30T15:16:11Z

build-requirements.txt

@@ -0,0 +1,2 @@
+scikit-build-core[pyproject]==0.5.1 # Pinned due to https://github.com/scikit-build/scikit-build-core/issues/532


We shouldn't pin scikit-build-core; we support it, but it has a specific issue with editable install. spack, for example, could use a higher version without problems.

Add a note on editable builds and the scikit-build-core version.

garth-wells · 2023-10-30T15:18:28Z

python/README.rst

Use markdown. We avoid rst where possible.

garth-wells · 2023-10-30T15:18:50Z

python/README.rst

+Basix Python interface
+========================
+
+This document explains how to install the Basix Python interface, given that the C++ interface has been built.


Shorten text.

garth-wells · 2023-10-30T15:19:14Z

python/build-requirements.txt

See earlier version comments.

garth-wells · 2023-10-30T15:19:44Z

python/README.rst

+
+2. Ensure the build time requirements are installed::
+
+     python3 -m pip -v install -r build-requirements.txt


Use pip (shorter) and no need for -v.

garth-wells · 2023-10-30T15:19:51Z

python/README.rst

+
+3. Build Basix Python interface::
+
+    python3 -m pip -v install --no-build-isolation .


See above comments.

garth-wells · 2023-11-07T18:15:11Z

Closing this as the editable issue was fixed by #727. Can be re-opened with any refinements of the docs.

jorgensd added 2 commits October 27, 2023 11:45

Add install instructions

157ea6a

Add install instruction for pure python installation (not separate in…

5fb7c06

…stall of C++)

jorgensd requested review from jhale and garth-wells October 27, 2023 09:54

jhale reviewed Oct 27, 2023

View reviewed changes

python/README.rst Outdated Show resolved Hide resolved

jorgensd mentioned this pull request Oct 27, 2023

Update install docs #720

Merged

jorgensd commented Oct 27, 2023

View reviewed changes

python/README.rst Outdated Show resolved Hide resolved

garth-wells reviewed Oct 27, 2023

View reviewed changes

Update python/README.rst

ac1dd10

Co-authored-by: Jørgen Schartum Dokken <dokken@simula.no>

garth-wells requested changes Oct 30, 2023

View reviewed changes

garth-wells closed this Nov 7, 2023

jorgensd deleted the dokken/basix-install-instructions branch May 22, 2024 06:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add install instructions for editable Python installation #721

Add install instructions for editable Python installation #721

jorgensd commented Oct 27, 2023 •

edited

Loading

garth-wells left a comment

jhale commented Oct 27, 2023

jorgensd commented Oct 27, 2023

francesco-ballarin commented Oct 27, 2023 •

edited

Loading

garth-wells commented Oct 27, 2023

jorgensd commented Oct 27, 2023

jorgensd commented Oct 27, 2023

francesco-ballarin commented Oct 27, 2023

jhale commented Oct 27, 2023

jorgensd commented Oct 27, 2023

jorgensd commented Oct 28, 2023

mscroggs commented Oct 30, 2023

jorgensd commented Oct 30, 2023

mscroggs commented Oct 30, 2023

jorgensd commented Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells Oct 30, 2023

garth-wells commented Nov 7, 2023

		@@ -0,0 +1,2 @@
		scikit-build-core[pyproject]==0.5.1 # Pinned due to https://github.com/scikit-build/scikit-build-core/issues/532


		2. Ensure the build time requirements are installed::

		python3 -m pip -v install -r build-requirements.txt


		3. Build Basix Python interface::

		python3 -m pip -v install --no-build-isolation .

Add install instructions for editable Python installation #721

Add install instructions for editable Python installation #721

Conversation

jorgensd commented Oct 27, 2023 • edited Loading

garth-wells left a comment

Choose a reason for hiding this comment

jhale commented Oct 27, 2023

jorgensd commented Oct 27, 2023

francesco-ballarin commented Oct 27, 2023 • edited Loading

garth-wells commented Oct 27, 2023

jorgensd commented Oct 27, 2023

jorgensd commented Oct 27, 2023

francesco-ballarin commented Oct 27, 2023

jhale commented Oct 27, 2023

jorgensd commented Oct 27, 2023

jorgensd commented Oct 28, 2023

mscroggs commented Oct 30, 2023

jorgensd commented Oct 30, 2023

mscroggs commented Oct 30, 2023

jorgensd commented Oct 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

garth-wells commented Nov 7, 2023

jorgensd commented Oct 27, 2023 •

edited

Loading

francesco-ballarin commented Oct 27, 2023 •

edited

Loading