Pillow wheel compiled on aarch64 missing binary libraries

[jonaswinkler]

Hello.

I've got a project that uses Pillow v8.1.0 and build docker images for multiple architectures (amd64, armhf, aarch64). Users have reported that they encounter Pillow errors on aarch64:

cannot import name '_imagingcms' from 'PIL' (/usr/local/lib/python3.7/site-packages/PIL/__init__.py)

On amd64, Pillow is installed from PyPi. On armhf, Pillow is installed from https://www.piwheels.org/. On aarch64, the wheel is compiled during the image build due to the lack of precompiled wheels. I've inspected the resulting image and found that Pillow is in fact missing the _imagingcms library on aarch64:

$ ls /usr/local/lib/python3.7/site-packages/PIL
...
-rwxr-xr-x 1 jonas jonas 506992 Jan 11 01:45 _imaging.cpython-37m-aarch64-linux-gnu.so
-rwxr-xr-x 1 jonas jonas  26736 Jan 11 01:45 _imagingmath.cpython-37m-aarch64-linux-gnu.so
-rwxr-xr-x 1 jonas jonas  10400 Jan 11 01:45 _imagingmorph.cpython-37m-aarch64-linux-gnu.so
-rwxr-xr-x 1 jonas jonas  10408 Jan 11 01:45 _imagingtk.cpython-37m-aarch64-linux-gnu.so
...

On amd64:

...
-rwxr-xr-x 1 root root 665776 Jan 10 19:05 _imaging.cpython-37m-x86_64-linux-gnu.so
-rwxr-xr-x 1 root root  38968 Jan 10 19:05 _imagingcms.cpython-37m-x86_64-linux-gnu.so
-rwxr-xr-x 1 root root  42992 Jan 10 19:05 _imagingft.cpython-37m-x86_64-linux-gnu.so
-rwxr-xr-x 1 root root  25016 Jan 10 19:05 _imagingmath.cpython-37m-x86_64-linux-gnu.so
-rwxr-xr-x 1 root root   8312 Jan 10 19:05 _imagingmorph.cpython-37m-x86_64-linux-gnu.so
-rwxr-xr-x 1 root root   9592 Jan 10 19:05 _imagingtk.cpython-37m-x86_64-linux-gnu.so
...

No errors or warnings when compiling the wheel (as part of pip install -r requirements.txt).

Are any additional libraries required for compiling this? How do I provide more useful information so we can figure out what's going on? I don't have aarch64 hardware.

Thank you.

[hugovk]

You can get the binary wheels from:

• Release Pillow 8.1.0 on January 2, 2021 #4997 (comment) -> https://github.com/python-pillow/pillow-wheels/releases/tag/untagged-16ee6820e47366c4ca5b

Please could you check one works for you? We can then upload them to PyPI.

[jonaswinkler]

As said above, I don't have aarch64 hardware, I simply observed that my wheels appear to be incomplete for some reason. The wheels from the link you posted do have the missing binary files and I'd assume that they'll work. Interestingly enough, up until 8.1.0, PyPi had wheels for aarch64.

[hugovk]

Yeah, wheels are built on Travis CI, and they appear to have given up on supporting open source (#5028) and we have very few trial credits left for building aarch64 wheels.

Anyway, I've now uploaded those to PyPI:

https://pypi.org/project/Pillow/8.1.0/#files

[jonaswinkler]

Yeah, wheels are built on Travis CI, and they appear to have given up on supporting open source (#5028) and we have very few trial credits left for building aarch64 wheels.

I know. I wondered whether I should bother contacting them, but went with github actions as well. Their new model for acquiring OSS credits does not sound like something you can rely on.

Thanks for uploading. This should resolve the issue (although I'm still curious about what's wrong over here), and will close as soon as I get positive feedback from users.

[hugovk]

Yup, so we might have enough credits for aarch64 wheels for the next release, and don't yet have a replacement. So it's possible aarch64 wheels will be missing for future releases, so you may need to build from source.

Docs for that is here, which includes the necessary dependencies:

https://pillow.readthedocs.io/en/stable/installation.html#building-from-source

[rhenwood-arm]

@hugovk: I would be sad to see a lack of credits reduce the quality of the AArch64 release - let me see if I can ask for more from Travis-CI.

[hugovk]

I asked Travis for credits for Pillow on 2 December but got nothing but a ticket autoreply (#5028), and asked for an update on 3 January, but again only an autoreply.

And here's the struggle with my own account: https://twitter.com/hugovk/status/1323520355403780096. I also haven't got much further with requests for other projects and have pretty much given up on Travis and we should find a replacement.

But thank you for asking as well! 🤞

[kennedyjosh]

Hi @hugovk – is there any update on this issue? I am using Pillow in a Docker container running Debian (Buster) amd32v7, and am getting the error:

Traceback (most recent call last):
  File "/app/main.py", line 107, in <module>
    img_dr.convert_to_srgb(img)    # avoids a warning
  File "/app/img_dr.py", line 12, in convert_to_srgb
    src_profile = ImageCms.ImageCmsProfile(io_handle)
  File "/usr/local/lib/python3.7/dist-packages/PIL/ImageCms.py", line 172, in __init__
    self._set(core.profile_frombytes(profile.read()))
  File "/usr/local/lib/python3.7/dist-packages/PIL/_util.py", line 19, in __getattr__
    raise self.ex
  File "/usr/local/lib/python3.7/dist-packages/PIL/ImageCms.py", line 23, in <module>
    from PIL import _imagingcms
ImportError: cannot import name '_imagingcms' from 'PIL' (/usr/local/lib/python3.7/dist-packages/PIL/__init__.py)

I believe my problem is related to this issue. Should I attempt a permanent workaround, or disable this functionality temporarily while I wait for a fix? Thanks in advance for your response.

[radarhere]

What version of Pillow are you using? Pillow 8.1.1 was just released today, and has aarch64 wheels.

[kennedyjosh]

@radarhere 8.1.1

Step 5/6 : RUN python3 -m pip install --upgrade Pillow
 ---> Running in da9459f0b6f4
Collecting Pillow
  Downloading Pillow-8.1.1.tar.gz (45.0 MB)
Building wheels for collected packages: Pillow
  Building wheel for Pillow (setup.py): started
  Building wheel for Pillow (setup.py): still running...
  Building wheel for Pillow (setup.py): finished with status 'done'
  Created wheel for Pillow: filename=Pillow-8.1.1-cp37-cp37m-linux_armv7l.whl size=1152677 sha256=9c8487d7026871b4ce3764aa9bf7d01c8c39f8ce8a43f70d9c26916f41775c7e
  Stored in directory: /root/.cache/pip/wheels/b5/05/f8/fdc6f0ac6f0a52c5d93a76cac6324af2224228aa0c44e64d1d
Successfully built Pillow
Installing collected packages: Pillow
Successfully installed Pillow-8.1.1
Removing intermediate container da9459f0b6f4

[radarhere]

Ok, so this issue has been about aarch64, but it looks like you're after armv7l. Would you mind creating a new issue with your request?