Twine should require pkginfo > 1.10

[pfmoore]

Is there an existing issue for this?

• I have searched the existing issues (open and closed), and could not find an existing issue

What keywords did you use to search existing issues?

None, I scanned the closed issue list manually and found it.

Please describe why your using this option

This is linked to issue #1059

Without depending on pkginfo > 1.10, existing users of twine won't get the new pkginfo version by upgrading twine (pip does not upgrade dependencies by default unless necessary). As a result, users upgrading twine because they are getting an error when using metadata 2.3, will still get the issue after the upgrade, and will be confused as to how to address the problem. See pypa/packaging-problems#735 for such a case.

Anything else you'd like to mention?

No response

[henryiii]

FYI, maturin and now hatchling are producing metadata 2.3.

[sigmavirus24]

I don't believe we typically update the floor for dependencies like pkginfo because that typically gets us into hot water with distributors. We can, but I'm not sure it's worth the effort. I'm not certain the adoption of hatchling or maturin is enough that folks will encounter this.

[henryiii]

Small comment: Hatchling is the third most popular build backend on PyPI (after setuptools and Poetry) - it's the most popular backend to only support PEP 621 and maturin is the most popular compiled build backend (not counting setuptools). I would also assume other backends will update at some point now that PyPI supports 2.3 - if pyproject-metadata updates for example, that would affect pdm-backend, scikit-build-core, and meson-python.

Not saying it should be updated, but just commenting on this aspect of the reply. I agree that a tight floor would also cause issues (maybe good ones though?)

[umarbutler]

On 15 March, I updated a package using py -m twine upload --repository pypi dist/* and experienced no issues. Today, I built a new package, seemingly having not upgraded any of my build tools (although perhaps it is possible I did?) and experienced this issue: InvalidDistribution: Metadata is missing required fields: Name, Version.. This persisted even after upgrading my twine and trying Python 11 instead of 12. Only after upgrading pkginfo was the issue fixed. A higher version of pkginfo should be a requirement to ensure that this doesn't happen to others. At the very least, twine's error message should mention that upgrading pkginfo will probably solve the issue.

Related issues I have located are as follows:

• Check command doesn't recognize metadata 2.3 #1059
• Can't run a twine upload due to version problem pypi/warehouse#15611
• Twine should require pkginfo > 1.10 #1070
• Successfully build *.whl and *.tar.gz, but cannot upload them through twine packaging-problems#735

[pfmoore]

+1 for at least updating the error message to suggest manually upgrading pkginfo.

[umarbutler]

I don't believe we typically update the floor for dependencies like pkginfo because that typically gets us into hot water with distributors. We can, but I'm not sure it's worth the effort. I'm not certain the adoption of hatchling or maturin is enough that folks will encounter this.

@sigmavirus24 I’d suggest having a look at all the issues I referenced, there are a number of us that have already begun encountering problems. Hatchling is pretty popular.

[jaimergp]

This should be pkginfo>=1.10, right? 1.10 included, I mean.

[jaraco]

It seems that pkginfo 1.11 has added forward-compatibility for future metadata versions (along with a warning). Assuming #1123 is adopted, it's been suggested that twine set pkginfo >= 1.11 as the floor, after which the need to keep bumping the floor with metadata revisions goes away. As it currently stands, users get a more informative error message about "supported metadata versions", which provides a somewhat better signal when the metadata version is missing or unrecognized. Perhaps it makes sense to bump the minimum to >= 1.11 soonish to get the warning behavior for everyone.

[woodruffw]

Given that #1123 has landed with the immediate fix (and will be in a release shortly), I think we now need to decide how to handle this more generally. The options (as they currently appear to me, apologies if I'm missing something!):

1. Constrain pkginfo >= 1.11, and remove the tweaks needed for <= 1.10. This should be a small/less disruptive change, but keeps us tied to pkginfo.
2. Switch to packaging for metadata handling. This is a slightly bigger change, but has significant "pros" in my book: packaging is a PyPA project, and is the reference implementation for Python package metadata.

For (2), I think #1180 already does the majority of the work, and just needs to be deconflicted against the latest changes.

[dnicolodi]

Given that #1123 has landed with the immediate fix (and will be in a release shortly)

I think a release with only very partial support for metadata 2.4, see #1180 (comment), is not a good thing and will be seen just as a slightly less bad regression by the users. At least with the current situation where twine errors out on metadata 2.4, users can be naturally steered toward upload tools that send all the metadata fileds to PyPI, such at uv upload once they release their support for metadata 2.4. The partial support in #1123 would just have the effect of triggering a series of bugs reports toward PyPI and toward build backends by package authors that will not see license information shown on PyPI.

[woodruffw]

I think a release with only very partial support for metadata 2.4, see #1180 (comment), is not a good thing and will be seen just as a slightly less bad regression by the users.

I agree, although we've already landed a patch that puts us in that partial state.

I don't want to act unilaterally here, so I'm curious what @jaraco thinks: one option here would be to tag 6.0.0 with the current changes but not release it to PyPI. That would give integrators (distros) a migration path via a stable tag while allowing us to release a subsequent 6.1 (or whatever) that supports metadata 2.4 more thoroughly.

[dnicolodi]

I agree, although we've already landed a patch that puts us in that partial state.

The PR with proper support was ready when the PR with partial support was merged. The PR with partial support was open since June. I don't understand the need to merge partial support now. I understand even less the need to rush get a release out quickly with partial support. I don't see any drawback in waiting a few more days to have a release with proper support.

one option here would be to tag 6.0.0 with the current changes but not release it to PyPI. That would give integrators (distros) a migration path via a stable tag

A migration path toward what? Distributions that need or want to support twine with pkginfo 1.11 are already carrying downstream patches. Releasing 6.0 with incomplete support for metadata 2.4 only when paired with pkginfo 1.11 would just create busy work for distributors to package pkginfo 1.11 and twine 6.0 while we know that a new release with different requirements is imminent to properly support metadata 2.4.

Just waiting a bit for the release till the PR that adds proper support for metadata 2.4 (while dropping a dependency that downstream distributors need to package and keep updated) can be reviewed and merged is the solution that creates less work for everyone involved.

[woodruffw]

A migration path toward what? Distributions that need or want to support twine with pkginfo 1.11 are already carrying downstream patches. Releasing 6.0 with incomplete support for metadata 2.4 only when paired with pkginfo 1.11 would just create busy work for distributors to package pkginfo 1.11 and twine 6.0 while we know that a new release with different requirements is imminent to properly support metadata 2.4.

There's an explanation up-thread, and one in #1123 (comment) -- I'm not as familiar with the distributor-demand side, but people who know it better than I do seem to think that raising the minimum version on twine's dependencies without a clear migration path causes lots of pain for distributors.

(My understanding is that this is not just for the metadata 2.4 case, but also for the 2.3 case.)

Just waiting a bit for the release till the PR that adds proper support for metadata 2.4 (while dropping a dependency that downstream distributors need to package and keep updated) can be reviewed and merged is the solution that creates less work for everyone involved.

This is totally fine by me, and is what I meant by doing a 6.0 tag on GitHub only -- distributors and downstreams that don't need to deal with any of this can wait until 6.1 (or whatever), which will handle metadata 2.4 correctly.

[jaraco]

We'd already approved and merged the release commit, so the tag was just an administrative step in completing the release procedure. So 6.0 is released. Next, we can explore addressing this issue.

[dnicolodi]

It is very sad that we will have to recommend users to not use twine 6.0 and switch to uv publish.

[woodruffw]

This is all open source, so that's perfectly fine -- you should make whatever recommendations make sense for your users. But I don't understand how the release series here affects your decision process: if you were OK with waiting for a fully fixed release, then the intermediate release of a partial fix shouldn't affect you.

(Regardless, I want to thank you for your work on #1180 -- I appreciate that you've deconflicted it as well, and I'll do a full review of it tonight.)

[dnicolodi]

Before users got an explicit error that metadata 2.4 is not supported (although the error was a bit misleading). Now the upload will succeed but the metadata uploaded is incomplete. From the point of view of the maintainer of a build backend this is very undesirable. Users switching to PEP 639 license declarations will naturally come to the conclusion that the build backend is generating incomplete metadata.

The new release does not solve any issue for anyone. Looking at the recent issues opened for this project, the feeling is that the new release is mostly a way to crystallize the change in maintainers.

[jaraco]

Looking at the recent issues opened for this project, the feeling is that the new release is mostly a way to crystallize the change in maintainers.

Quite the contrary. This release was about releasing the fix to make pkginfo 1.11 viable.

The fact that the maintainership was at issue was because the pull requests by me were blocked by inactivity. That's been resolved.

Before users got an explicit error that metadata 2.4 is not supported (although the error was a bit misleading).

I see now how this appears as something of a regression, but given that it previously failed and now simply fails to honor a new expectation, I'd still consider this an improvement. I recommend we focus on getting resolution for each issue separately. If users are adversely impacted by the 6.0 release (beyond the benefits it provides), we can yank it and await the release that fixes the metadata 2.4 issue.

[dnicolodi]

pkginfo is not a dependency anymore. This can be closed.