Unable to build debian packages for debian buster with Pillow 7.1.0

[anoadragon453]

Description

When trying to build debian packages for Synapse v1.12.1, we ran into a snag with the following error on debian:buster:

find debian/`dh_listpackages`/ -path '*/site-packages/psycopg2/*.so' | \
    xargs dpkg-shlibdeps -Tdebian/`dh_listpackages`.substvars \
        -pshlibs1 -dRecommends
dh_shlibdeps -X site-packages/PIL/.libs -X site-packages/psycopg2
dpkg-shlibdeps: error: cannot find library libz-a147dcb0.so.1.2.3 needed by debian/matrix-synapse-py3/opt/venvs/matrix-synapse/lib/python3.7/site-packages/Pillow.libs/libpng16-bedcb7ea.so.16.37.0 (ELF format: 'elf64-x86-64' abi: '0201003e00000000'; RPATH: '')
dpkg-shlibdeps: error: cannot continue due to the error above

This appears to be the same error as python-pillow/pillow-wheels#255, though it's unclear why this issue is only cropping up now.

Pillow did some PNG-related work in release 7.1.0, so that may be the cause. Pinning pillow to <v7.1.0 seems to work, so that's what we've done for now.

[anoadragon453]

Aaaaand unfortunately this workaround has prevented Synapse from starting after upgrading to v1.12.2 with Pillow 7.1.0 installed.

The plan for now is to have a v1.12.3 release with a fixed workaround. This will likely arrive tomorrow.

Note that if you can't downgrade from v1.12.2 for whatever reason, you can downgrade Pillow to v7.0.0 and then Synapse will start:

pip install --upgrade --force pillow==7.0.0

If you would like to downgrade instead, v1.12.1 is known working.

[richvdh]

So the problem is that Pillow 7.1.0 has changed where it installs its libraries. For 7.0.0, they are installed to site-packages/PIL/.libs:

$ ls env2/lib/python3.6/site-packages/PIL/.libs
libfreetype-69f25d5e.so.6.17.1  liblzma-6cd627ed.so.5.2.4     libtiff-bd1961ca.so.5.5.0       libwebpmux-40630b44.so.3.0.4
libjpeg-3b10b538.so.9.3.0       libopenjp2-b3d7668a.so.2.3.1  libwebp-3a2aeecf.so.7.0.5       libz-a147dcb0.so.1.2.3
liblcms2-a6801db4.so.2.0.8      libpng16-bedcb7ea.so.16.37.0  libwebpdemux-e9ec482e.so.2.0.6

But for 7.1.1:

$ ls env/lib/python3.6/site-packages/Pillow.libs/
libfreetype-69f25d5e.so.6.17.1  liblzma-99449165.so.5.2.5     libtiff-41910f6d.so.5.5.0       libwebpmux-ec1d5c76.so.3.0.5
libjpeg-ba7bf5af.so.9.4.0       libopenjp2-b3d7668a.so.2.3.1  libwebp-122bd20b.so.7.1.0       libz-a147dcb0.so.1.2.3
liblcms2-a6801db4.so.2.0.8      libpng16-bedcb7ea.so.16.37.0  libwebpdemux-2db559e5.so.2.0.6

Unfortunately we've hardcoded this path in the debian build scripts: https://github.com/matrix-org/synapse/blob/v1.12.1/debian/rules#L25-L28.

I'll make a PR to fix it.

[clokep]

This was fixed in #7212.