-
Notifications
You must be signed in to change notification settings - Fork 950
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use packaging.metadata to parse and validate upload metadata #14718
Conversation
2dfe7f9
to
cd87c92
Compare
Initial review note: It's not simple to identify the changes made, since there's two operations done in the main commit - moving functions to new modules, and then changes to those functions to adopt the new pattern. |
Currently blocked on a 24.0 release of pypa/packaging: pypa/packaging#755 |
packaging 24.0 has been released: https://pypi.org/project/packaging/24.0/ |
640d8d4
to
7e73d98
Compare
I think this is ready to merge @dstufft, can you review my edits? |
The edits look 💯 to me! Thanks for taking this across the finish line. |
this appears to regress some metadata validation:
|
…ypi#14718)" This reverts commit 155f61c.
…etadata (pypi#14718)" (pypi#15630)" This reverts commit 7b00f6b.
…try) (pypi#15631) * Revert "Revert "Use packaging.metadata to parse and validate upload metadata (pypi#14718)" (pypi#15630)" This reverts commit 7b00f6b. * Cast version to string when enqueueing task Fixes WAREHOUSE-PRODUCTION-1R3. * Ignore empty string values when parsing metadata * Add test coverage
This is the first piece of #14716 that has been split out.
It is currently blocked on pypa/packaging#733 and pypa/packaging#735 pypa/packaging#755.This removes (most) of the metadata handling from the
MetadataForm
, and instead uses thepackaging.metadata
module to interpret and validate the metadata (other than our custom validations we add on top of the standard validations).The
MetadataForm
is now theUploadForm
, and it has been drastically stripped down to include only the name, version, and the non metadata pieces of information (filetype, filename, etc).This PR should make our metadata validation more strict than it currently is, with no regression due to:
packaging.metadata
library tracks what version of the metadata a field was added, and makes it an error to use that field on older metadata versions.packaging.metadata
didn't already handle, have been layered on top of the built in validations.The main place this isn't true, is we no longer attempt to validate "legacy" specifiers like
requires
,provides
,obsoletes
, andrequires-external
, and we just allow them to be free form strings.The
UploadForm
continues to support fetching the name and version of the upload. This PR itself doesn't actually require that functionality, as we could just use theMetadata
object for everything, but future PRs will want to access the name/version before theMetadata
object has been created, so we leave them there to make that transition easier.We do add sanity checks to make sure that the name and version from the1UploadForm
are equivalent to the name and version from theMetadata
, which again is kind of silly currently, but will be more important in a later PR.Looking at the updated tests in the
test_legacy.py
file should give a good indication about the user visible changes, for the most part the tests largely just worked, and the changes needed to make them work were primarily adjusting the error messages or fixing test data that was actually invalid but the old validation routines allowed it.Footnotes
Removed because it was impossible to test until the future work that makes this actually possible to happen. ↩