Feedback about the release process

[sbidoul]

The current release processes bumps the version, generates things (NEWS.rst, AUTHORS.rst), then creates a git tag, then bumps the version again for development.

When, say, NEWS.rst needs to be tweaked, the process is a little bit cumbersome. One needs to update the file, commit, reorder the commits so the bump for development commits is last, then delete the tag and re-create it.

I open this issue to discuss how we could make things a little bit easier (or find out if I missed something).

One possibility would be to have two nox actions: one to prepare the release branch, and a second one to tag and bump for development.

[webknjaz]

Hey 👋 I'm obsessed with the release automation among other things, so I have something to share on this topic.

I like GHA-based workflows (I know that some people here don't like having such automation, but please, bear with me!). And in that context, I prefer using publishing on workflow_dispatch, not pushing tags or publishing GitHub Releases. This allows me to have a process where the first thing happening is building the dists (and pre-making a commit with changelog changes that's bundled into them too), then running tests against these dists and finally, publishing the exact same dists that were tested. After that, the workflow creates a Git tag, makes GitHub Releases, publishes sigstore signatures and other stuff.

I think what's important in this example is the order of the steps. And that order can be reproduced with plain nox commands without the CI/CD bit.

The idea is that I always rely on the event of publishing to the PyPI as a point of no return. So everything before it is ephemeral, so to speak. There's dists that are stored as artifacts and passed between the jobs, there's the changelog commit that's also passed around. But these are not yet set in stone. When the time comes to actually publish (a step where a CD workflow is usually paused until approved), the final decision on proceeding is made. So then, the publishing happens and after that, we just have to persist the changes making them permanent. And that includes applying the pre-cooked changelog commit to Git, making a tag, and for projects that don't use SCM-based versioning, also bumping a constant in a Python module.

I observed time and time again that in a lot of projects, making a Git tag (or even a GitHub Release that implies a tag!) is seen as a pre-requisite for publishing, something that's happening early in the release process. So they do this and related changes to Git first, and then publish. If publishing fails for some reason, they have to remove the tag, make other changes and so on. This means that for some time, there's a public tag pushed to the Git remote that many external observers may treat as "a release happened". And then, the actual release may happen and succeed but in hours. Or it may fail and that promise of immutable tags is not true anymore. Then, the embarrassing need to clean up emerges.

My point is that by making updates like tagging a Git commit a consequence/output of release rather than a trigger/input allows reducing the amount of things to clean up if something goes wrong. And I use the fact of succeeding publishing to the PyPI as a marker/flag with the semantics of "a release actually happened".

All this is to say that I think the the proposed separation makes sense, with a caveat that the twine publish step would probably also need to be separate.

[pradyunsg]

When, say, NEWS.rst needs to be tweaked, the process is a little bit cumbersome. One needs to update the file, commit, reorder the commits so the bump for development commits is last, then delete the tag and re-create it.

FWIW, I've filed #12764 to allow tweaking the NEWS file prior to the set of commits being made.

[webknjaz]

Oh, I thought it's already being reviewed/post-processed pre-release..

[sbidoul]

I'm happy with #12764, so closing this. Thanks @pradyunsg !