Policy around releases

[thibautjombart]

I start this thread to discuss how we can standardize our release process, following some conversations with @Bisaloo. It would be great to have everyone's input on this.

Release pace

I can see two extremes:

1. Plan releases long ahead, set up deadlines, try to meet them; typical waterfall
2. Release early, release often; less emphasis on long-term planning; typical Agile

I would advocate for 2, better suited for interacting with end-users, and often more realistic, as I often found sticking to plans made a long time ago is difficult, and/or the original plan becomes less relevant as a project develops.

Versioning

I would advocate using semantic for package versions, so we can reflect the kind of changes made to a package with a new release. I personally only ever use the first three digits, but no pre-release as I find it just complicates things and never felt a need for it.

Status

The status of the package is not entirely independent from versioning, but provides a high-level description or how 'reliable' the package will be to the user. @TimTaylor and I have described a lifecycle adapted from Rstudio for RECON summarized in this graph:

It is currently the one we use in some projects and on the general airtable database of the tool ecosystem. But we can use another one if we prefer to.

Project management

Issues

While issues can be broken down into tasks using tick boxes, I find issues are easier to tackle when they don't contain a long list of things to do. Ideally an issue could be closed by a single, not too extensive PR, so it is easier to review, and faster to merge.

Projects

I like using github projects to manage releases. The pros I can see are:

• it works particularly well in conjunction with small, self-contained issues
• it helps triage and prioritise issues, with the option of postponing issues which may turn out to be less of a priority, or more complicated, than initially thought
• it gives immediate visibility of progress with kanban boards

Github versus CRAN

General policy

I like the idea that the main branch on github remains functional at all time whilst holding the latest features and bug fixes, and CRAN hosts the 'stable' version. @Bisaloo was mentioning there is a limit to the number and frequency of releases we can make on CRAN. I am not sure if we want to set hard rules about when to submit a release on CRAN, as it may not be entirely related to semantic change. For instance, a patch fixing an important bug may need to go to CRAN urgently.

First release

First releases onto CRAN are a lot harder than further updates. I would advise not only for the package to be bullet proof when initially submitted, but also to have relatively stable API, so we avoid breaking backward compatibility with new updates. In terms of development stage, that would mean the package is probably towards the end of its experimental phase and getting towards 'maturing'.

I think that covers it for me. Thoughts?

[Bisaloo]

Related thread with some thoughts from the rest of the community: https://discuss.ropensci.org/t/release-cadence-how-do-developers-estimate-when-a-new-release-is-due/2861

Point by point comments & additional thoughts:

• Release pace: agreed with Agile approach. As an additional point, it is probably going crucial for us to not let too much time between releases to avoid dependency chain headaches, as in this RStudio/CRAN interaction:

  The dev version of reprex depends on the dev version of callr. So callr needs to submit first. But the dev version of callr depends on the dev version of processx. So processx needs to update first. And it has been stuck at CRAN for ?months? and the situation is very ... opaque. I explained this in an email to CRAN on Jan 23, which apparently was not read. I re-explained in another email yesterday, which has now been read and received a response. But now it is difficult to make it as if the orphaning had never happened, which would have been the right outcome.

• Versioning: I'm not using the 4th version component very assiduously but I provide an argument in favour of it in the README of our R-universe:

  Please note that as opposed to remotes (or devtools), installs via R-universe will not automatically detect every new commit as a new version. To make a change available via update.packages("yourproject"), you need to increment the version number of your package.

• General policy: CRAN will flag a manual review (while we want to go through automated review as much as possible to speed up the release process) if we submit more than 6 times in 6 months:

https://github.com/wch/r-source/blob/b12ffba7584825d6b11bba8b7dbad084a74c1c20/src/library/tools/R/QC.R#L7853-L7861

[thibautjombart]

Thanks, super useful comments / reads. Good point on the 4rth component of semantic versioning and package updates using R-universe, but my thoughts would be:

• if there are useful features/bug fixes, some of the first 3 components should be changed
• to update a specific package, one could merely run install.packages() (assuming the behaviour is OS consistent and does reinstall the package, whether it's there or not)
• for dev purposes, people might just use devtools::load_all() or devtools::install()?

[TimTaylor]

Thanks, super useful comments / reads. Good point on the 4rth component of semantic versioning and package updates using R-universe, but my thoughts would be:

• if there are useful features/bug fixes, some of the first 3 components should be changed

The problem is that ideally you want all 3 component releases on CRAN so without making use of the 4th component for development, and regularly bumping, there is no way for r-universe to pick up on the changes in dev. This kind of forces use of the 4th component during dev (else little benefit of r-universe for packages on CRAN).

[thibautjombart]

It makes sense. Any recommendations / standards on the 4rth component? Semantic is a bit vague on that point, but it would be again something that would be useful to standardise.

[TimTaylor]

IMO (don't put too much stock in it ;-) ) as long as the last digit goes up when you want to bump a dev release on r-universe I'm not sure I would worry about it too much. As long as everyone is clear that anything with a non-zero 4th component is dev, that is probably all you need. Were it not for the r-universe thing I'd do everything under x.x.x.9000 similar to the tidy people. However you're probably better starting at x.x.x.1 for reasons above.

The key point is that this only really matters for the r-universe integration so may be worth clarifying how beneficial/important that is at the same time.

[Bisaloo]

I mostly agree with @TimTaylor but I want to point out that there is a good case for starting at 9000 (actually, anything above 1234) because CRAN will then ensure you don't submit a package with a dev version number:

https://github.com/wch/r-source/blob/b12ffba7584825d6b11bba8b7dbad084a74c1c20/src/library/tools/R/QC.R#L6967-L6971