-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Time-based release process #12210
Comments
A more drastic change would be to switch to very frequent (say weekly) releases, and rely on fixing/reverting major regressions after the release in a patch release. Here the assumption is that we rarely have major regressions that affect a large fraction of users, since our regular testing process (mypy primer, test suite) already works quite well. For this to be practical, we'd need to streamline our releases a bit so that making a release only takes very little effort (e.g. auto-generate blog post draft from commit history). Having a release manager role would still be important, as somebody would have to push the button and make sure regressions get addressed in a timely manner, and make sure that blog posts go out (or maintain changelog in some other format). The role would look quite a bit different from the first proposal, though. |
Thanks for bringing this up! I think the slow release cadence has been a problem; it's slowed us down at typeshed and we regularly get comments from users asking when some merged PR will be released. I like the idea of having weekly, lightweight releases. We could perhaps stop having a blog post for each release and rely on GitHub releases to autogenerate a changelog for us. And if releases are weekly, we may not even need separate bugfix releases. We'll have to rethink the versioning scheme though, lest we end up with html5lib-style versions. Perhaps it's time to just jump to 1.0. |
Perhaps there is room for improved automation whereby we can detect regressions at the pull request stage. An increased test footprint and policy to require additional testing with pull requests may help. I am available to help in any way. |
I'm happy to help with releases. In terms of schedule, I'd be happy with: "monthly by default" + "whenever anyone feels like it" Thoughts on versioning scheme:
|
Vote for GitHub releases and prerelease scheme. |
Thanks for the ideas! Here is a more concrete proposal that incorporates some of the above ideas. SummaryThese are the biggest changes from the existing release process:
Release manager rotationWe'd have at least three release managers (RMs) in a rotation. Each shift could be three months and would typically include 2-3 feature releases. Beta releasesThe acting release manager makes sure we have a beta release at least monthly. These can be used in typeshed, and they make it easier to start using fixes or new features not yet included in a stable release. We'd encourage users to use them so that we can find regressions sooner. It's fine to publish beta releases more frequently. Each member of the release manager rotation (not just the currently acting RM) can publish a beta release at any point, assuming the CI build and wheel builds are green, and there are no known blocking issues (more about this below). Stable releasesOur goal would be to have a stable feature release around every 4 weeks (and at least every 8 weeks). A stable release would be based on a beta release that has been out for at least a week, to allow time for finding regressions. When making a beta release that will be used as the basis for a stable release, the RM will announce this intention by creating a GitHub release planning issue. The RM can cherry-pick low-risk fixes on top of the beta release during the stabilization period. If there are high-risk but important fixes, it's okay to make an additional beta release from the stable release branch. Feature development can continue in the master branch, and it's okay to make new beta releases from the master branch while waiting for a stable feature release to go out. Bugfix releasesAfter a stable feature release, we may publish one or more bugfix releases, mainly to address regressions (but any bug fixes are fine). These would often be released around 1-2 weeks after the stable release. The RM can decide when to publish additional bugfix releases and how many. Other members of the RM rotation can also make bugfix releases. Release notesWe could probably switch to using GitHub for release notes, instead of using the current blog. For beta releases and bugfix releases, we'd only provide auto-generated changelogs, to make it simple to make a release. For feature releases we'd continue to write more detailed release notes. They'd include manually edited "release highlights" followed by a detailed, auto-generated changelog, to make the process light-weight. Release numberingWe'd start with mypy 1.0. We'd follow a scheme that is influenced by semver (but we won't follow it strictly), where the major release number is increased when there is a major backward incompatible change, such as one of these:
In most feature releases we'll increase the minor version number. Bugfix releases increment the third version number. Beta versions would be numbered x.(y+1)bN, where x.y is the latest stable release or the stable release currently being released. So if 2.3 is the latest stable release, and 2.4 is being released on top of 2.3b1 (but not out yet), the next beta release would be 2.5b0 (if created from master). Release blockersIf there's a major regression in master, we can declare an issue as a release blocker, and no beta or feature release can go out until the issue is resolved. Dealing with regressionsThe default response to a regression or release blocker would be to revert the change that introduced it. This would be a normal and expected part of our development process, and contributors shouldn't feel bad if this happens, as regressions are impossible to prevent. The motivation is to unblock the release process as quickly as possible. Otherwise we'd risk accumulating multiple regressions that could take weeks or months to fix. Testing with internal reposWe'd continue to test against Dropbox internal code asynchronously. If there's a significant regression, we'd either revert the offending commit, implement a forward fix (at least if it's not release blocking), or declare a release blocker. The feature releases don't need to wait for a thumbs up from Dropbox employees, but the RM is allowed to wait for this (at their discretion). Other owners of private repos are also encouraged to test using beta releases or development releases, and they can also request reverts and declare release blockers. We'll need a detailed bug report for this. RM responsibilitiesThese would be the main tasks of the acting RM:
DiscussionI'm proposing 3-month RM shifts instead of, say, one release per shift, to reduce friction. There will be some overhead each time we change the RM, as the new RM will have to refresh their memory about the specifics of the role. We could also consider 6-month shifts, but that could be too big of a commitment. |
I'm very interested in volunteering as a release manager. |
I hadn't had a chance to read over Jukka's proposal until now, but I really like like it! I think the only part that concerns me is the versioning. I know we are approaching mypy 0.990 which will maybe force us to move to a different versioning scheme, but I am a bit leery to declare mypy 1.0, especially with big UX changes such as #10600, and the large number of crashes currently open. I also do not think we should declare the plugin API stable, even if we make a mypy 1.0, as I expect changes due to potential work such as #7724. I think if we do make a mypy 1.0, one thing we could do is version the plugin API separately, and stabilize it later. I also am not sure how effective having beta releases will be. In practice even CPython has issues of not enough testing of beta releases. It is of course easier to test beta mypy but I expect most people will just wait for the final release. |
Update on the release process: I'm currently trying out a monthly release process (+ optional bugfix releases), at least initially staffed by Dropbox employees. If we can keep making all scheduled releases for, say, 4-5 feature releases, we can look at onboarding additional release managers, also from outside Dropbox. If mypy 0.960 successfully goes out in May, it would be the third successful monthly feature release.
I wonder if having version numbers like 0.1000, 0.1010, etc. is supported. These would look a bit weird, but maybe it would be okay? If that doesn't work out, we can try to fix some highest-impact bugs and implement some planned disruptive changes before releasing 1.0, but with the assumption that some things would still be in flux. We can always release 2.0, 3.0, and so on if there are further major changes. We won't run out of major version numbers :-)
Agreed. It's too early to stabilize the plugin API.
I think that more frequent and predictable stable releases are more important than beta releases. We can consider supporting beta releases in the future, but I don't feel that it's the most important thing to focus on now.
Thanks for the offer! As I mentioned earlier, we may start onboarding additional release managers later this year. |
I think it would be useful to have automated (or mostly automated) beta releases that just release what's on master. As mentioned earlier, the current process of spending time putting together a release isn't really ideal for anyone (users get inconsistent delays between releases, maintainers spend a lot of time creating a release). I think we could reduce most of those problems by automating that process. In this system, we'd have beta releases that periodically (probably somewhere between weekly and monthly) just take the current master, make sure CI and any other tests are passing, and then publish a new version. Stable releases could then be created by taking a beta release, testing that on Dropbox code or doing any other verification necessary, fixing any relevant bugs, and then releasing a stable version of that same release. Some other related notes:
|
I think version numbers like 0.1000 are supported, but they're confusing, and we should just bump to mypy 1.0 — at our current schedule, this would be September or October :-) This is my issue for changes I'd see in an ideal 1.0 #11201 |
This commit will update to mypy 1. In the past, releases were managed by Dropbox (python/mypy#12210), and were done at infrequent intervals. They plan to release more regularly in the future. Mypy does not keep a changelog, but did provide a 1.0 release blog post: https://mypy-lang.blogspot.com/2023/02/mypy-10-released.html A changelog has been requested (python/mypy#2859, python/mypy#13279), but has been dismissed because "you can view the git history easily." The new release versioning scheme is in the format `major.minor.patch`, but is not SemVer-compliant, as explained in their blog post and wiki. https://github.com/python/mypy/wiki/Release-Process The update to mypy 1 was postponed until bug fixes were available in the 1.0.1 patch release (python/mypy#13685).
This commit will update to mypy 1. In the past, releases were managed by Dropbox (python/mypy#12210), and were done at infrequent intervals. They plan to release more regularly in the future. Mypy does not keep a changelog, but did provide a 1.0 release blog post: https://mypy-lang.blogspot.com/2023/02/mypy-10-released.html A changelog has been requested (python/mypy#2859, python/mypy#13279), but has been dismissed because "you can view the git history easily." The new release versioning scheme is in the format `major.minor.patch`, but is not SemVer-compliant, as explained in their blog post and wiki. https://github.com/python/mypy/wiki/Release-Process The update to mypy 1 was postponed until bug fixes were available in the 1.0.1 patch release (python/mypy#13685).
We've had a more regular release schedule recently, we've onboarded more people to work on releases, we have an action that sometimes automatically syncs typeshed, and we have a 1.x version scheme. |
For some time, we've had a pretty unpredictable release process, with long delays between releases. I see two main root causes for this. First, only Dropbox employees have been able to act as release managers -- mainly since we rely heavily on type checking our internal code base to avoid regressions and to validate new releases. This limits the available effort release managers have for releases (we also have our regular work responsibilities that often take precedence, unfortunately). Second, an infrequent release process tends to result in many regressions that take a lot of time to investigate and fix. This tends to make making a release a big undertaking, which also tends to reduce the frequency of releases, since fixing regressions is not a particularly fun activity.
I have some thoughts about how we might be able to make monthly releases, for example, hopefully fairly reliably. Here's a summary:
Thoughts? Are there any volunteers for the release manager role?
The text was updated successfully, but these errors were encountered: