GHA: Publish release to test.pypi.org on demand

[jorisroovers]

Allow for on-demand publishing of the main branch to test.pypi.org.

[webknjaz]

I don't recommend using manual CLI invocations for publishing because you'll be missing out on the seamless GHA integrations.

For example, PyPI has recently implemented OIDC, and it's about to be enabled for private beta testing. It will allow to completely remove the GHA secrets and rely on secure short-lived tokens. Getting those tokens requires a few API requests to two different services that are specific to GHA, meaning the CLI tools will likely not implement this.

But the curated action https://github.com/marketplace/actions/pypi-publish will support it the moment OIDC is fully enabled on PyPI. With that, (Test)PyPI uploads will become zero-configuration for free, almost automatically.

Another point I'd like to make is that it's much more secure to perform publishing from a dedicated job — the secrets shouldn't be available to the build scripts or anything else that ends up being executed from the pre-publish jobs.

I also recommend using the environment: feature of GHA — it allows one to isolate the secrets, so they are not global and available to any job, but are only accessible from the job that has the proper environment set. Environments can also have branch protection — I usually make them require human approval for the job that publishes to PyPI. A real person needs to click a button on GitHub to allow such a job to start.

[jorisroovers]

Ok, multiple suggestions in here :) I'll definitely consider and probably adopt them.

But the curated action https://github.com/marketplace/actions/pypi-publish will support it the moment OIDC is fully enabled on PyPI. With that, (Test)PyPI uploads will become zero-configuration for free, almost automatically.

Thanks, will check it out!

Another point I'd like to make is that it's much more secure to perform publishing from a dedicated job — the secrets shouldn't be available to the build scripts or anything else that ends up being executed from the pre-publish jobs.

Hmm, I hadn't considered this. Is this because people could be opening a PR that modifies the CI pipeline that runs on every PR and modify it to extract the secrets?

I also recommend using the environment: feature of GHA — it allows one to isolate the secrets, so they are not global and available to any job, but are only accessible from the job that has the proper environment set.

Yup, makes total sense. Facepalm for not considering this, I've used environments before elsewhere.

I usually make them require human approval for the job that publishes to PyPI. A real person needs to click a button on GitHub to allow such a job to start

Yes this makes sense and I've done this before. Would need to check how we can marry that with our auto publishing of dev builds to PyPI on every commit which is already going on.

[webknjaz]

@jorisroovers

Hmm, I hadn't considered this. Is this because people could be opening a PR that modifies the CI pipeline that runs on every PR and modify it to extract the secrets?

TL;DR it is possible to smuggle some something that might access the secrets post-merge.

The secrets aren't available for runs triggered by the pull_request event. But they will be available for other events, post merge. If you use build tooling that has plugins, I imagine it might be possible to silently add a compromised plugin that would be loaded by said build tooling. It may be hard to track down if such third-party software does anything suspicious. And it's almost impossible to catch during reviews (unless you know to do audit of the entire transitive dependency tree being updated by such a change) — because it's expensive, and it's not the first thing most non-security-experienced reviewers would think of.

I've been seeing https://www.stepsecurity.io/products/harden-runner being integrated in many projects lately in an attempt to tighten the possibility of sneaking out sensitive data out of the CI. Maybe you should consider adding it too.

Yup, makes total sense. Facepalm for not considering this, I've used environments before elsewhere.

Yeah, they are not advertised as a match for the publishing use-case which is why it took me a while to come up with this approach. But I've started doing this in all of my projects a year (or maybe more) ago. I usually call the environments pypi and testpypi. Use the same secret name PYPI_API_TOKEN, meaning that I have two separate jobs publishing to different indexes. One environment needs approvals and the other is unconditional. Hopefully, with the OIDC support, the secrets bit will become unnecessary but the use for pre-approval use-case will still stand. I also like that I can render version URLs as environment params in CI, and they will be clickable in the workflow run summary graph. But this requires pre-calculating the expected version vars at the beginning of the workflow which is why I tend to have a whole separate job that runs first dedicated solely to calculating a number of variables that are shared across the jobs and are used for assertions sometimes.

Yes this makes sense and I've done this before. Would need to check how we can marry that with our auto publishing of dev builds to PyPI on every commit which is already going on.

Just a separate environment without protections, and a separate job for that too.