pip install displaycal error

[carlosvigil]

Edit: Addressed, keeping open for others to see. Might want to update README instructions. Feel free to close it of-course.

m1 Mini 13.2.1 (22D68)
pip 23.0.1 from /opt/homebrew/lib/python3.11/site-packages/pip (python 3.11)

  Using cached wxPython-4.2.0.tar.gz (71.0 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/bv/xspgbdnx1v55sgcg_h009ztm0000gn/T/pip-install-npcwj2ei/wxpython_0acf9a31d985429287cbeac96075ff5f/setup.py", line 27, in <module>
          from buildtools.config import Config, msg, opj, runcmd, canGetSOName, getSOName
        File "/private/var/folders/bv/xspgbdnx1v55sgcg_h009ztm0000gn/T/pip-install-npcwj2ei/wxpython_0acf9a31d985429287cbeac96075ff5f/buildtools/config.py", line 30, in <module>
          from attrdict import AttrDict
      ModuleNotFoundError: No module named 'attrdict'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.```

[carlosvigil]

Relevant & Duplicate #213

Hey @tgurr can you try to build with the dev-branch? I had to do some minor tweaks to get displaycal running with python 3.11.
The main branch will not run with python 3.11 because we set some limitations in the early porting phase.

Originally posted by @p5k369 in #218 (comment)

Hey @Sergij-SApcPro, this issue has already been fixed 2 weeks ago with these commits: 3762e7b and 97f0c91
As you can also read here: #218

If you want to install via pip you have to apply those changes by yourself because I am not able to release and publish a pypi package nor will I push to the main branch. Or you just follow our install instructions. Make sure to checkout the develop branch.

Originally posted by @p5k369 in #213 (comment)

So this issue is fixed and we just have to wait for a new PyPi release (or build it ourselves)? Thank you! :) I'll mark as fixed then.

Originally posted by @starsforeveryone in #213 (comment)

[JacekRuzyczka]

I get exactly the same error /* and a big bunch of other errors, which might me caused by this error */ when trying to install the dev beanch of DisplayCAL on Python 3.6, which comes with my OpenSuSE 15.4 system. Is Python 3.6 already too old for DisplayCAL to compile?

[p5k369]

I get exactly the same error /* and a big bunch of other errors, which might me caused by this error */ when trying to install the dev beanch of DisplayCAL on Python 3.6, which comes with my OpenSuSE 15.4 system. Is Python 3.6 already too old for DisplayCAL to compile?

Python 3.6 already reached its EOL over a year ago. To run DisplayCal try 3.8+.

[JacekRuzyczka]

That's the problem: OpenSuSE 15.4 comes with the Python 3.6, and 15.5 will not appear before late June. And the community package Python 3.8 raises an error when installing.

[hadrabap]

@JacekRuzyczka Try installing attrdict manually before proceeding to DisplayCal stuff. Look at #246 (comment) That is based on my Linux experience (Oracle Linux 8). If you want, I can share bits and pieces of my spec file for RPM.

[JacekRuzyczka]

Thank you for your hint, @hadrabap ! The install process now works without any errors, but the build does not because of a lacking module:

╭─jacek@epica ~/bin/displaycal-py3  ‹develop› 
╰─➤  python -m build
/usr/bin/python: No module named build

Why?

[hadrabap]

@JacekRuzyczka There is some kind of build harness, which is necessary. Next, DisplayCal uses wxPython (or something like that), which requires additional -devel RPM packages from the Xorg family. Plus GCC (or Clang, whichever is native in your distro). Finally, it requires ArgulCMS.

Try pip install build inside your virtualenv. Usually, the Python module name corresponds with the parameter for pip.

Unfortunately, all my machines are down for good night right now. Tomorrow I can give you more details. Or complete working steps for Oracle Linux 8 with X11 (Wayland switched off), including building ArgylCMS from sources.

[hadrabap]

Here is the SPEC file for my RPM.

Fun facts:

• The application is installed into /mnt/nvme/opt/displaycal/ and desktop integration stuff goes to /mnt/nvme/opt/share/. All binaries are linked into /mnt/nvme/opt/bin/ which is on $PATH. This is my customization! You need to change it.
• I'm using Python 3.9.
• I have my own RPM build infrastructure which prepares referenced displaycal.tar.gz as an export of 3.9.10 tag.

Source package creation:

mkdir displaycal
pushd displaycal
git clone https://………
cd displaycal-py3
git checkout tags/3.9.10
cd ..
tar -cvf ../displaycal.tar displaycal-py3
popd
gzip -9 displaycal.tar
rm -rf displaycal

Do not delete the .git/ directory as it is required for the build to recognize the version.

And here is the SPEC file itself:

%global debug_package %{nil}
%define _build_id_links none

Name:           displaycal
Version:        3.9.10
Release:        %{?dist}
Summary:        DisplayCal
Provides:       displaycal
License:        GPLv2
URL:            https://github.com/eoyilmaz/displaycal-py3
Source0:        displaycal.tar.gz
Requires:       argyllcms
Requires:       python39

AutoReqProv: no

%description
DisplayCal

%prep

%setup -Tcn displaycal
tar -xvf %{SOURCE0}

%build
cd displaycal-py3

# Create virtualenv
virtualenv-3 -p python3.9 displaycal
source displaycal/bin/activate
# install missing dependencies
pip install -U virtualenv attrdict

# Install requirements
pip install -r requirements.txt
# Build displaycal
python -m build
# Install displaycal
pip install dist/DisplayCAL-*.whl

# Virtualenv clean-up
virtualenv-3 -p python3.9 --relocatable --distribute displaycal
find displaycal/ -name __pycache__ -type d -prune -exec rm -rfv '{}' +
find displaycal/ -name '*.py[oc]' -delete
find displaycal/bin \( -name 'pip*' -o -name 'easy_install*' -o -name 'python-config*' \) -exec rm -fv '{}' ';' || :
find displaycal/lib/python3.9/site-packages \( -name 'pip*' -o -name 'easy_install*' \) -exec rm -rfv '{}' ';' || :
rm -f \
        displaycal/bin/activate      \
        displaycal/bin/activate.csh  \
        displaycal/bin/activate.fish

# Additional fixes
sed -i 's|#! /usr/bin/env python|#!/usr/bin/env python3.9|' displaycal/lib/python3.9/site-packages/google/protobuf/internal/_parameterized.py

%install
cd displaycal-py3
install -d $RPM_BUILD_ROOT/mnt/nvme/opt
cp -a displaycal $RPM_BUILD_ROOT/mnt/nvme/opt/
install -d $RPM_BUILD_ROOT/mnt/nvme/opt/share
mv \
        $RPM_BUILD_ROOT/mnt/nvme/opt/displaycal/share/applications \
        $RPM_BUILD_ROOT/mnt/nvme/opt/displaycal/share/icons        \
        $RPM_BUILD_ROOT/mnt/nvme/opt/displaycal/share/man          \
        $RPM_BUILD_ROOT/mnt/nvme/opt/share/
mv $RPM_BUILD_ROOT/mnt/nvme/opt/displaycal/share $RPM_BUILD_ROOT/mnt/nvme/opt/displaycal/lib/python3.9/site-packages/
install -d $RPM_BUILD_ROOT/mnt/nvme/opt/bin
pushd $RPM_BUILD_ROOT/mnt/nvme/opt/bin
for bin in $( find ../displaycal/bin -name 'displaycal*' ); do
    ln -s "$bin" "$( basename $bin )"
done

%clean
rm -rf %{buildroot}

%files
%defattr(-,root,root,-)
/mnt/nvme/opt/displaycal/*
/mnt/nvme/opt/share/*
/mnt/nvme/opt/bin/displaycal*

%changelog
* Fri Jan 27 2023 Petr Hadraba <hadrabap@gmail.com> - 3.9.10
- Initial SPEC file

Target environment:

• Oracle Linux 8.6+
• X11 instead of Wayland
• Standard GCC toolchain 8.5
• Distribution packages only (AppStream, EPEL, Code Ready Builder, Distro Builder, see Oracle YUM server

[hadrabap]

And here is ArgyllCMS.

The only issue is that ArgyllCMS uses Jam/MR build system. As I saw that tool for the first in my life, I decided not to package it separately – but rather to build it as part of ArgyllCMS build itself. I'm using the 2.6.1 version. The source ZIP archive can be downloaded from the files page.

The following SPEC file:

• requires Jam/MR v2.6.1 in a jam.src.zip
• requires Argyll_V2.3.1_src.zip downloaded from official download page
• installs everything into standard locations (/etc/… and /usr/…)

%global debug_package %{nil}

Name:           argyllcms
Version:        2.3.1
Release:        %{?dist}
Summary:        Argyll CMS
Provides:       argyllcms
License:        GPLv2
URL:            http://www.argyllcms.com/index.html
Source0:        jam.src.zip
Source1:        Argyll_V2.3.1_src.zip
BuildRequires:  libXxf86vm-devel
BuildRequires:  libXinerama-devel
BuildRequires:  libXScrnSaver-devel
BuildRequires:  libXdmcp-devel
BuildRequires:  libtiff-devel

%description
Argyll CMS

%prep

%setup -Tcn argyllcms
mkdir jam
mkdir argyllcms
cd jam
unzip %{SOURCE0}
mv */* .
cd ../argyllcms
unzip %{SOURCE1}
mv */* .

%build
cd jam
make -j10
cd ../argyllcms
PATH=../jam/bin.linuxx86_64:$PATH make

%install
cd argyllcms
PATH=../jam/bin.linuxx86_64:$PATH make install
# Fix permissions
find doc -type f -exec chmod -x '{}' ';'
find ref -type f -exec chmod -x '{}' ';'
# installation
install -d $RPM_BUILD_ROOT%{_bindir}
install -p -m 0755 bin/* $RPM_BUILD_ROOT%{_bindir}/
rm $RPM_BUILD_ROOT%{_bindir}/License*
install -d $RPM_BUILD_ROOT%{_datarootdir}/argyllcms/doc
cp -R doc/* $RPM_BUILD_ROOT%{_datarootdir}/argyllcms/doc
install -d $RPM_BUILD_ROOT%{_datarootdir}/argyllcms/ref
cp -R ref/* $RPM_BUILD_ROOT%{_datarootdir}/argyllcms/ref
mv $RPM_BUILD_ROOT%{_datarootdir}/argyllcms/doc/License* $RPM_BUILD_ROOT%{_datarootdir}/argyllcms/
rm $RPM_BUILD_ROOT%{_datarootdir}/argyllcms/ref/License*
install -d $RPM_BUILD_ROOT%{_sysconfdir}/hotplug/usb
install -p -m 0755 usb/Argyll $RPM_BUILD_ROOT%{_sysconfdir}/hotplug/usb/
install -p -m 0644 usb/Argyll.usermap $RPM_BUILD_ROOT%{_sysconfdir}/hotplug/usb/
install -d $RPM_BUILD_ROOT%{_sysconfdir}/udev/rules.d
install -p -m 0644 usb/45-Argyll.rules usb/55-Argyll.rules $RPM_BUILD_ROOT%{_sysconfdir}/udev/rules.d/

%clean
rm -rf %{buildroot}

%files
%defattr(-,root,root,-)
%{_sysconfdir}/*
%{_bindir}/*
%{_datarootdir}/*

%changelog
* Fri Dec 16 2022 Petr Hadraba <hadrabap@gmail.com> - 2.3.1
- Initial SPEC file

Target environment:

• Oracle Linux 8.6+
• X11 instead of Wayland
• Standard GCC toolchain 8.5
• Distribution packages only (AppStream, EPEL, Code Ready Builder, Distro Builder, see Oracle YUM server

[hadrabap]

@JacekRuzyczka My infrastructure is up-and-running so I provided my SPEC files. Hope it will help you with your issues.

[JacekRuzyczka]

@hadrabap I've already got the python3-wxpython package on my machine. A separate wxpython3-devel package seems not to exist at all. GCC and ArgyllCMS have just been upgraded when migrating to OpenSuSE 15.4. And: pip install build did NOT help.

And with OpenSuSE 15.5 I've got the problem that it will ship with Python 3.10, but the public beta is stuck at 3.6 (why??), so that I'll have to wait till late June in any case. :-(

[hadrabap]

@JacekRuzyczka Well, in mean time, while waiting for new openSUSE, you can temporarily use your own build of Python 3.9.

Let me try:

[opc@sws ~]$ cd Desktop/Projects/
[opc@sws Projects]$ mkdir tmp
[opc@sws Projects]$ cd tmp/
[opc@sws tmp]$ wget https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tgz
[opc@sws tmp]$ tar -xvf Python-3.9.0.tgz
………
[opc@sws tmp]$ cd Python-3.9.0/
[opc@sws Python-3.9.0]$ mkdir build dist
[opc@sws Python-3.9.0]$ cd build
[opc@sws build]$ ../configure --prefix=$( pwd )/../dist
………
[opc@sws build]$ make -j100
………
[opc@sws build]$ make install
………
[opc@sws build]$ cd ../dist/bin/
[opc@sws bin]$ ./python3.9 -m pip install --upgrade pip
………
[opc@sws bin]$ ./pip3 install virtualenv
………
[opc@sws bin]$ cd ../
[opc@sws dist]$ bin/virtualenv -p bin/python3.9 displaycal
………
[opc@sws dist]$ cd displaycal/bin/
[opc@sws bin]$ source activate
(displaycal) [opc@sws bin]$ 

And voilà, we have virtualenv linked to Python3.9 ready for DisplayCal.

Now, you should download source code of DisplayCal and run the following commands from the SPEC file above; namely:

pip install -U virtualenv attrdict

# Install requirements
pip install -r requirements.txt
# Build displaycal
python -m build
# Install displaycal
pip install dist/DisplayCAL-*.whl

Could you please give it a try?

You can also follow another build instructions #246 (comment) which are for macOS, but that does not matter.

[JacekRuzyczka]

OK, I'm gonna give Python 3.9 a try and tell you whether DisplayCAL now builds. Unfortunately, the pip upgrade isn's working because of an unknown dep named msvcrt:

╭─jacek@epica ~/bin/tmp/Python-3.9.0/dist/bin  
╰─➤  ./python3.9 -m pip install --upgrade pip
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
  File "/home/jacek/bin/tmp/Python-3.9.0/dist/lib/python3.9/subprocess.py", line 73, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

A Microsoft Visual C++ runtime? On my Linux machine? Are you kidding me??

On the package front, something's going on, as one of the guys at SuSE has accepted a request ro reintegrate DisplayCAL into OpenSuSE (Leap 15.5?): https://build.opensuse.org/package/show/multimedia:color_management/DisplayCAL

[hadrabap]

@JacekRuzyczka :-D LOL Now, I'm convinced you fully understands why I'm avoiding Python as much as possible. DisplayCal is the only exception till I find non-Python solution.

Anyway. It seems there is some kind of issue in the configure script. Did you try looking for other users' hints how to install Python 3.9 on SUSE? Unfortunately I'm not familiar with SUSE ecosystem.

[JacekRuzyczka]

@hadrabap I have already done that: https://superuser.com/questions/1783660/how-to-overcome-a-missing-msvcrt-module-when-installing-python-3-from-source-on?noredirect=1#comment2786754_1783660

This issue has been known for 4 years, but the suggested workaround doesn't work for me, unfortunately: https://stackoverflow.com/questions/51201459/python-3-7-install-not-working-on-opensuse-leap-42-3?rq=1

The configure script does not mention msvcrt at all, BTW.

[JacekRuzyczka]

OK, building Python (I've now got the newest 3.10.11) now works well after numerous changes:

1. The trouble with msvcrt can be solved by adding a symlink: ln -s /usr/local/lib64/python3.10.11/lib-dynload/ /usr/local/lib/python3.10.11/lib-dynload
2. I had to uncomment a line in the Python-3.10.11/Modules/Setup file configuring SSL support and copy the file to Setup.dist in the same directory.
3. I had to install openssl-devel in order to enable pip to download packages via HTTPS and rebuild Python with the --with-ssl option.

But: I still can't build DisplayCAL yet! pip install -r requirements.txt does not fetch the necessary metadata:

  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'

…due to a whole broadside of undefined references, which seem to stem from a file named hashopenssl.c. Here are the last two of them:

      /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /home/jacek/bin/tmp/Python-3.10.11/build/../Modules/_hashopenssl.c:441: undefined reference to `EVP_DigestUpdate'
      /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /home/jacek/bin/tmp/Python-3.10.11/build/../Modules/_hashopenssl.c:441: undefined reference to `EVP_DigestUpdate'
      collect2: error: ld returned 1 exit status
      ninja: build stopped: subcommand failed.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

After installing all packages listed in requirements.txt, it turned out that dbus-python; sys_platform=='linux' was the culprit. wxPython>=4.1.1 didn't get installed either: https://superuser.com/questions/1784005/cant-install-wxpython-by-using-pip-on-a-python-3-10-11-installation

What's wrong now? Please note that the instructions your gave me above partially contradict your installing guide in the README file. My version of the GCC, if this is relevant, is: gcc (SUSE Linux) 7.5.0 from 2017.

[Wedge009]

I ran into the original issue with attrdict being absent. Installing it manually worked around that but then it looks like attrdict is trying to import Mapping from collections instead of collections.abc - is that distribution-specific? I'm on Ubuntu 22.04.

  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-5mu4xxg2/wxpython_d1c483fb645747f0ab88b0c3c7d148b6/setup.py", line 27, in <module>
          from buildtools.config import Config, msg, opj, runcmd, canGetSOName, getSOName
        File "/tmp/pip-install-5mu4xxg2/wxpython_d1c483fb645747f0ab88b0c3c7d148b6/buildtools/config.py", line 30, in <module>
          from attrdict import AttrDict
        File "/home/[redacted]/.local/lib/python3.10/site-packages/attrdict/__init__.py", line 5, in <module>
          from attrdict.mapping import AttrMap
        File "/home/[redacted]/.local/lib/python3.10/site-packages/attrdict/mapping.py", line 4, in <module>
          from collections import Mapping
      ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

[p5k369]

Hey @Wedge009,
you are experiencing this issue -> #242 (comment)
This is probably because you installed attrdict. You will have to install attrdict3 if you want to use wxPython-4.2.0 and dont want to make changes to the wxpython package.

Note that neither of the two is really required to build wxpython wheels on linux. Its just that the dependencies got messed up.

[Wedge009]

Thanks for that - yes, that's the one. I didn't know about the different Python versions for attrdict - normally I see when a package is available for both Python 2 and 3, I see the packages named 'name' for Python 3 and 'name2' for Python 2, not the other way around as it seems to be for attrdict.

Some more progress, but now running into #251...

[JacekRuzyczka]

FYI: OpenSuSE Leap 15.5, which was released last week, still comes with the age-old Python 3.6, and the Python 3.10 from a community package comes without pip.

UPDATE: There's apparently a bunch of python311 packages available for Leap 15.5, which do come with pip. python3-wxpython is a community package you can also install.

BUT: When trying to install DisplayCAL with pip install displaycal, I get two errors: One stems from wxPython pip tries to install although it has already been imported from the python3-wxpython package mentioned above. See also: https://github.com/wxWidgets/Phoenix/issues/2422

The second one, however, seems to be a mistake in DisplayCAL itself:

     error: can't copy '/tmp/pip-install-wdsu6vec/displaycal_d6e819087b4946e3ad1de77f9f7d5167/DisplayCAL/../dist/net.displaycal.DisplayCAL.appdata.xml': doesn't exist or not a regular file
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for displaycal

I've found such an XML file on my machine (can't upload it here as XML is blocked in this forum for whatever reason) and implanted it into the downloaded archive like this:

What still remains, is the issue with wxPython, though: Even if I install it before building DisplayCAL and then run pip list to check whether it has been installed correctly:

╭─jacek@epica ~  
╰─➤  pip list
Package            Version
------------------ --------
attrdict3          2.0.2
build              0.10.0
casttube           0.2.1
certifi            2023.5.7
charset-normalizer 3.1.0
dbus-python        1.3.2
distro             1.8.0
idna               3.4
ifaddr             0.2.0
numpy              1.25.0
packaging          23.1
Pillow             10.0.0
pip                23.1.2
protobuf           4.23.3
PyChromecast       13.0.7
pyproject_hooks    1.0.0
requests           2.31.0
Send2Trash         1.8.2
setuptools         68.0.0
six                1.16.0
urllib3            2.0.3
wheel              0.40.0
wxPython           4.2.1
zeroconf           0.69.0

…the build process of DisplayCAL will still start a sub-process, which will then attempt to build wxPython for a second time — and fail.

[eoyilmaz]

This has been fixed with a0356db in #306