Fix Release Signatures on PyPI (404 Not Found)

[maltfield]

Describe the bug
The cryptographic signature of the python-gnupg module on PyPI has gone missing (404 Not Found)

To Reproduce

Steps to reproduce the behavior (following the instructions found here):

1. Fetch the URL of the python_gnupg-0.5.2 .whl file from the pypi.org simple API
2. Append .asc to the URL of the .whl file's URL and attempt to download it
3. Get 404 error

Example execution

ser@disp9050:~$ curl -s https://pypi.org/simple/python-gnupg/ | grep -i python_gnupg-0.5.2-py2.py3-none-any.whl
<a href="https://files.pythonhosted.org/packages/5f/86/18938c2378a8c8284910308b93e2bc521a965ced4ca028947e1ffa6e98dc/python_gnupg-0.5.2-py2.py3-none-any.whl#sha256=72ce142af6da7f07e433fef148b445fb3e07854acd2f88739008838745c0e9f5" data-dist-info-metadata="sha256=4a7fae421dc8ec32dc178aba722350ec58bb596a7197ae1f4345c9af9d7f0805" data-core-metadata="sha256=4a7fae421dc8ec32dc178aba722350ec58bb596a7197ae1f4345c9af9d7f0805">python_gnupg-0.5.2-py2.py3-none-any.whl</a><br />
user@disp9050:~$ 

user@disp9050:~$ wget https://files.pythonhosted.org/packages/5f/86/18938c2378a8c8284910308b93e2bc521a965ced4ca028947e1ffa6e98dc/python_gnupg-0.5.2-py2.py3-none-any.whl.asc
--2024-02-25 22:44:33--  https://files.pythonhosted.org/packages/5f/86/18938c2378a8c8284910308b93e2bc521a965ced4ca028947e1ffa6e98dc/python_gnupg-0.5.2-py2.py3-none-any.whl.asc
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.124.223, 2a04:4e42:1e::223
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.124.223|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-02-25 22:44:34 ERROR 404: Not Found.

user@disp9050:~$ 

Note that the file is available; just the signature is not.

user@disp9050:~$ wget https://files.pythonhosted.org/packages/5f/86/18938c2378a8c8284910308b93e2bc521a965ced4ca028947e1ffa6e98dc/python_gnupg-0.5.2-py2.py3-none-any.whl
--2024-02-25 22:45:05--  https://files.pythonhosted.org/packages/5f/86/18938c2378a8c8284910308b93e2bc521a965ced4ca028947e1ffa6e98dc/python_gnupg-0.5.2-py2.py3-none-any.whl
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.124.223, 2a04:4e42:1e::223
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.124.223|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 20964 (20K) [binary/octet-stream]
Saving to: ‘python_gnupg-0.5.2-py2.py3-none-any.whl’

python_gnupg-0.5.2- 100%[===================>]  20.47K  --.-KB/s    in 0.03s   

2024-02-25 22:45:06 (725 KB/s) - ‘python_gnupg-0.5.2-py2.py3-none-any.whl’ saved [20964/20964]

user@disp9050:~$ 

Expected behavior

I should be able to download both the python_gnupg payload and its cryptographic signature as before (and my builds shouldn't be breaking due to this bug).

[maltfield]

Note that this issue does appear to be specific to python_gnupg and not an upstream issue with PyPI.

I confirmed that the signature is still available for another project (borgbackup):

user@disp9050:~$ curl -s https://pypi.org/simple/borgbackup/ | grep -i borgbackup-1.1.13.tar.gz
<a href="https://files.pythonhosted.org/packages/97/68/27d96a12f54894223ad6676ce4d215ad61771e3e723580f3ee6e609e17b7/borgbackup-1.1.13.tar.gz#sha256=164a8666a61071ce2fa6c60627c7646f12e3a8e74cd38f046be72f5ea91b3821" >borgbackup-1.1.13.tar.gz</a><br />
user@disp9050:~$ 

user@disp9050:~$ wget https://files.pythonhosted.org/packages/97/68/27d96a12f54894223ad6676ce4d215ad61771e3e723580f3ee6e609e17b7/borgbackup-1.1.13.tar.gz
--2024-02-25 22:49:21--  https://files.pythonhosted.org/packages/97/68/27d96a12f54894223ad6676ce4d215ad61771e3e723580f3ee6e609e17b7/borgbackup-1.1.13.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.124.223, 2a04:4e42:1e::223
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.124.223|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3754457 (3.6M) [application/x-tar]
Saving to: ‘borgbackup-1.1.13.tar.gz’

borgbackup-1.1.13.t 100%[===================>]   3.58M  42.6KB/s    in 99s     

2024-02-25 22:51:13 (37.0 KB/s) - ‘borgbackup-1.1.13.tar.gz’ saved [3754457/3754457]

user@disp9050:~$ 

user@disp9050:~$ wget https://files.pythonhosted.org/packages/97/68/27d96a12f54894223ad6676ce4d215ad61771e3e723580f3ee6e609e17b7/borgbackup-1.1.13.tar.gz.asc
--2024-02-25 22:51:18--  https://files.pythonhosted.org/packages/97/68/27d96a12f54894223ad6676ce4d215ad61771e3e723580f3ee6e609e17b7/borgbackup-1.1.13.tar.gz.asc
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.124.223, 2a04:4e42:1e::223
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.124.223|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 862 [application/octet-stream]
Saving to: ‘borgbackup-1.1.13.tar.gz.asc’

borgbackup-1.1.13.t 100%[===================>]     862  --.-KB/s    in 0.007s  

2024-02-25 22:51:19 (115 KB/s) - ‘borgbackup-1.1.13.tar.gz.asc’ saved [862/862]

user@disp9050:~$ 

user@disp9050:~$ gpg --verify borgbackup-1.1.13.tar.gz.asc
gpg: assuming signed data in 'borgbackup-1.1.13.tar.gz'
gpg: Signature made Sat 06 Jun 2020 05:37:32 PM -05
gpg:                using RSA key 2F81AFFBAB04E11FE8EE65D4243ACFA951F78E01
gpg:                issuer "tw@waldmann-edv.de"
gpg: Good signature from "Thomas Waldmann <tw@waldmann-edv.de>" [unknown]
gpg:                 aka "Thomas Waldmann <thomas.j.waldmann@gmail.com>" [unknown]
gpg:                 aka "Thomas Waldmann <tw-public@gmx.de>" [unknown]
gpg:                 aka "Thomas Waldmann <twaldmann@thinkmo.de>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 6D5B EF9A DD20 7580 5747  B70F 9F88 FB52 FAF7 B393
     Subkey fingerprint: 2F81 AFFB AB04 E11F E8EE  65D4 243A CFA9 51F7 8E01
user@disp9050:~$ 

[vsajip]

PyPI does not support signatures any more, and does not support upload for signatures with new releases, though old releases might still have signatures available. See this issue. If you want signatures for 0.5.2 and other recent releases of this project, see the GitHub release and the corresponding assets for other releases. I'm planning to close this issue as not planned, as I don't see what more I can do to provide signatures for release artifacts.

[maltfield]

Sorry, but after reading that issue, I don't see anything about PGP signatures being deleted? It looks like they just removed it from the WUI, which has been the case for a very long time.

Are you saying that if you attempt to upload the signature with twine you get an error that you used to not get before?

[vsajip]

No error, but I don't believe it (twine) uploads the signatures. I use a command line of the form
twine upload -s -i <my-key> ... dist/*0.5.2* and while it invokes gpg which prompts me for the passphrase and creates the signatures, they're not uploaded. I think it's been that way for a while now - are you able to get signatures for 0.5.1? I haven't done anything different to the above for a long time - I generally pass -s to twine, get the passphrase prompt for signatures, and the signatures are created. The issue I linked is old; they removed UI support at first, but I think that now upload support is gone as well.

[vsajip]

$ ls dist/*0.5.2*asc
dist/python_gnupg-0.5.2-py2.py3-none-any.whl.asc  dist/python-gnupg-0.5.2.tar.gz.asc
$ twine upload -r python-gnupg dist/*0.5.2*asc
Uploading distributions to https://upload.pypi.org/legacy/

However, there is no progress feedback as you get when uploading archives, so it's not telling the whole truth.

Anyway, I've given you a source for signatures on GitHub. Why would I go to all the trouble of creating GitHub releases with signatures available if PyPI supported them? Have you looked for signatures with recent releases of another project? Your borgbackup example is from June 2020. What happens with borgbackup 1.2.7, from Dec 2023?

[maltfield]

I've been building the BusKill app with your python_gnupg-0.5.2 for a long time (years?), successfully downloading both the wheel and the signature from PyPI. It's only recently that it's become 404.

So you did successfully upload the signatures to PyPI in the past. It's unclear why your signature file is suddenly now 404 but others' are not. The only difference I can see is that your is a whl and the borgbackup one is a tarball.

[vsajip]

Then it's worth taking up with the PyPI project, though they obviously have views about the utility of signatures which you may not share ... there's certainly nothing I can do about those missing signatures, so I'll close this now ... but note that 0.5.2 was only released on 12 December 2023, so it's less than three months old!

[maltfield]

I have been inactive for a while, but my last successful build was on 2023-06-22:

• https://github.com/BusKill/buskill-app/actions/runs/5352933169

Looks like my build scrips are setup to just download the latest version of python-gnupg from PyPI, so--you're right--it probably wasn't v0.5.2 that worked last

• https://github.com/BusKill/buskill-app/blob/df68b8e111e9a01001200fef41ab8fb6f1c7c7ac/build/linux/buildAppImage.sh#L225-L241

Unfortunately, GitHub doesn't retain build logs for so long, so I can't check the build logs to see what version it downloaded.

it's worth taking up with the PyPI project,

That's exactly what I want to do, but I was hoping to get a command & error message from you to submit a proper bug report. Currently all I have is "I get 404 on this .asc file that doesn't exist on the WUI" -- they could easily deflect and say it's a dev problem, not their problem. We need some "steps to reproduce" that they can run to confirm it's an issue on their end.

they obviously have views about the utility of signatures which you may not share

If the PyPI devs are intentionally sabotaging the supply chain security of their packages and downstream consumers by removing the only possible method for authenticating packages, it would be extremely alarming.

[vsajip]

They're not sabotaging anything, it's just a commonly held view about the utility of signatures because of things like this. In any case, you can just ask them for clarity as to what the current status is of support for uploading/downloading signatures for release artifacts on PyPI. In any case, I've given you an alternative source of signatures - and I generate the signatures which were previously uploaded to PyPI, but now I (have to) use GitHub to make them available.