Replies: 6 comments 3 replies
-
|
Thanks Bozhidar for opening this conversation, I think it's an extremely important conversation to have.
I would like to push back on the idea that SemVer is a good idea. All that SemVer says is to bump a number when you make breaking changes, a number which in the context of Emacs package management means virtually nothing. It is really just an excuse for making breaking changes as often as you like. For me the only correct baseline is to, as a rule, not make breaking changes. If there are cases where there are really compelling reasons to still consider a breaking change, because the benefits far outweigh the downsides, then those can be considered and discussed on a case-by-case basis. Keep in mind that for typical breaking changes we can know the benefits because they benefit us (the developers), whereas the cost of dealing with breaking changes is on the users.
But is is, the change to clj-refactor shows that. CIDER is really a collection of libraries. I also don't think we should rename or change the behavior of existing commands. People have these bounds to specific keys and have them fit into specific workflows. These should continue to work. I can see a case for keybindings and defaults settings to evolve over time, perhaps, but only if we then also provide a backwards compatibilty mode. Requiring a certain Emacs version is fine, there we just need a clear policy of what our current compatibility target is. Requiring a minimum version of libraries is also reasonable, but ideally those libraries would also have strong backward compatiblity commitments, mainly growing through accretion, so people aren't forced into dependency hell situations where we require a newer version of something, but another library only works with an older version. Maybe this will seem like an extreme standpoint to some, but if you have been following the discourse in the Clojure community the past few years then it shouldn't. This is the general trajectory that culturally we are moving towards, and it isn't as hard as it sounds. When a bit of extra care is taken then you can still introduce substantial changes and additions, without breaking existing consumers. I would like to make an appeal to authority here, in Rich Hickey's Spec-ulation talk, at 22:26 (and transcript) he explains clearly how you can achieve all the above while still growing your software. For more background on where I'm coming from you can also read my blog post from last year where I talk about becoming allergic to the churn. I have become allergic to the churn. We need to keep in mind always that many, many people rely on CIDER to do their work. When we end up breaking people's setup in the name of improvements, then that is a net negative. |
Beta Was this translation helpful? Give feedback.
-
I'm aware that SemVer is not a silver bullet (and that many people in the Clojure community dislike it), but it's still much better than what CIDER was doing for the past 8.5 years. I don't plan to make lots of breaking changes and quickly push the version to 20.x. I like the SemVer for two reasons - the versions start to carry some meaning, but more importantly for me, SemVer actually forces the maintainers to think harder about breaking changes. In my experience with it for other projects it certainly resulted in more stability. Clearly nothing beats accretive only changes, but if the cost of accretion becomes too high (e.g. you need to keep around some horrible design choices forever) I think it's not a bad idea to have some back up plan.
Fair point. In general I consider most of those libraries in a reasonable shape (one of my biggest achievements was to replace the legacy
Just as an FYI - most command renames were actually related to splitting As for changing the behavior of an existing command - I guess it depends on the nature of the change, as I think some changes that are a clear improvement are unlikely to bother someone. One can argue that bugfixes often change the behavior of commands as well. That's one aspect of accerition-only that never sat very well with me (providing new APIs instead of fixing the old ones).
I've documented some policies here https://docs.cider.mx/cider/1.0/about/compatibility.html There's obviously more ground to cover.
I totally agree. My only problem is and has always been lack of enough time. Obviously when you've got a limitted time for something you have to decide what's the best way to utilize it and in the past I've often optimized for me doing more work over the users experiencing less pain. Originally I felt that most people would be OK with some pain for a bigger gain. Over the years I saw that many people would rather not get any improvements than get some breakages and I've been more careful with potentially breaking changes for that reason. I have to admin, however, that this also impedes a bit my desire to work on CIDER as often when I calculate the work I have to do to change something in a non-disruptive manner I simply opt not to do it. I guess that's fine.
That's one of the few talks by Rich that I kind of disagree with it. 😉 I'm all for stability, but I also want us to be pragmatic about it. If I know that the impact of a change is going to be slim to none I'd rather just do it than provide a new version of the API or whatever (E.g. when it comes to bug fixes). Funny enough that's also why I like Clojure more than languages like Haskell - pure FP is great in theory, but painful in practice. And Clojure is pragmatic.
Will do!
No argument here. |
Beta Was this translation helpful? Give feedback.
-
|
Great discussion to start, Bozhidar. When evaluating this I think it's important to consider not just my current preferences, but what would've been best for me during the first few years I was using emacs/Clojure/CIDER. I learned them concurrently, so I wasn't necessarily proficient at all aspects of my config for a year or two. That inexperience meant that changes in CIDER were difficult to deal with. I think a lot of CIDER users are like me-during-those-first-years, and their concerns are what I think of in these discussions. This leads me to prefer a quite conservative approach. CIDER has a lot of users. It's reasonable to assume that any breaking change, however small, will break one of them. (Are there still widely-used configs that auto-update CIDER?) Many of those users depend on the tool for their livelihood. So I would hope that major semver version bumps be necessary, rare, and clearly explain what will break.
As we've just seen, we don't really know who uses what. At the very least I would expect a check across clojure-emacs tooling for anything that gets deprecated, removed, or renamed. Even then, semver-wise I would expect a major version bump from this kind of change, because it is.
How much more change do we expect to keybindings and defaults? Are these necessary changes, or nice-to-haves? Changes here are quite likely to break user workflows. I think one concern here is that CIDER has a lot of users, and few follow this repository. Discussions in tickets, or line items in the CHANGELOG, don't necessarily get noticed. (They should read the CHANGELOG before updating, but do we want to break their workflow if they don't? Are all the breaking changes clearly and prominently marked?) Both of these feel like an "incompatible API change" to me.
I think to a large extent this is already out of our control. The private API is already used by related tooling. Is it even possible to know how many people rely on the private API in their local configs? If it wasn't clearly marked |
Beta Was this translation helpful? Give feedback.
-
|
Those are all fair points. Let me address some of them.
That's what I hope as well. I definitely don't plan release a lot of major versions going forward.
In an ideal world - sure. In the real world I don't have the time to do everything by the book. While I'd like to run the project in a professional and non-disruptive manner, it's not really a professional project and from time to time I have to make some difficult choices. It's really just a matter of time and deciding where to invest ones limitted time to work on something. If people would rather be stuck with some legacy design decisions and idiosyncracies than deal with the fallout from them - vox populi, vox dei. I don't mind doing less work. 😆
Generally the only reason to change keybindings so far has been the simple fact we ran out of keybindings in the main keymap. Who could have envisioned how many commands CIDER would end up having 8 years ago? I expect such changes are unlikely to happen in the future, but we're still out of keybindings and some top-level key might be swapped for a nested keymap out of necessity. As for the settings - people do like to complain about them a lot, but there's rarely a good reason to change some. I think those have been fairly stable in the past years.
Yeah, that's true. Still, there are some bits of the API that are way more important than others - e.g. I really doubt someone's using low-level connection functions in their config/extensions. That's why it seems to me there might be some leeway for "breaking" changes, that are quite unlikely to break something. Calls this the pragmatic vs the purist view. :-) |
Beta Was this translation helpful? Give feedback.
-
|
Great discussion! Happy to see that an accretive approach is largely uncontroversial. As a humble contribution, I'd like to point out that there's no necessary confict between 'accretive' and semver. Ideally nothing would break ever, but as pointed out that can be occasionally difficult, and can mean that some bad designs are kept forever. So semver helps in those. But for it to be effective communication, it has to be as accurate and pessimmistic as possible. This is what has worked for me in a team setting where we did semver (and accretion):
|
Beta Was this translation helpful? Give feedback.
-
I wouldn't be too sure about this. I have frequently found myself implementing slight variations of CIDER functionality in my own configs based on the lower level functions. Sometimes I disagree with how CIDER expects me to interact with it, and so I reach for the underlying mechanisms. This seems to me like a very natural thing to do in Emacs land. The main point though is that we can't know, we can only assume. I've learned to become very pessimistic about changes in functionality/behavior not being noticed. Someone somewhere is always impacted.
Can you elaborate on this a bit Bozhidar? I understand CIDER is not your full time job, and I can imagine what you mean by this, but I think some people may be a little shocked to read this. CIDER gets a significant amount of funding (Clojurists Together, Cognitect/Nubank, Patreon) because people rely on it professionally. I think those supporters expect a certain degree of professionalism.
I can understand this may be disappointing, but I am actually happy to hear this. CIDER has seen less development recently, and it has been great. A good long stretch of nothing breaking! I do empathize with the motivational aspect, motivation is really the currency of open source, and once lost it can be hard to regain. But different people are (de)motivated by different things. For me the lack of dilligence is something that reduces my motivation to contribute to CIDER or be involved in the project.
There is an alternative here, which is rolling the major version into the artifact name, i.e. The benefits of this have been discussed elsewhere, it allows libraries to co-exist within the same runtime, and for dependent projects to upgrade on their own timeline. I know there are some particular considerations here because CIDER manages state, but I still think it's worth considering. It's also the only way to allow people who get CIDER from MELPA to be able to pick their version. MELPA simply deletes previous versions of a package off their servers. Which is a peculariaty of the Emacs ecosystem which makes things all the more poignant. Unless people build and install from source, or use something like straight.el, they simply are not able to pin to a previous version. This also means they are forced to deal with breaking changes at a time forced upon them, instead of being able to postpone the upgrade until a better time. |
Beta Was this translation helpful? Give feedback.
-
FWIW I run 15 emacs libs as git submodules, including various ...otherwise I agree with the general sentiment of the things you're saying, I'd simply thought I'd state a fact. |
Beta Was this translation helpful? Give feedback.
-
|
Hey, I hope I didn't come across as downplaying your input. Everything you say about semver is correct! And it's good to (re)iterate some of that. |
Beta Was this translation helpful? Give feedback.
-
Yeah, it's not professinal in the sense it's not my job and doesn't quite meet my standards of "professional", mostly due to time constraints and compromises derived from them (e.g. if I had more time the test suite would be much more extensive, the level of internal consistency would be higher, the documentation would be better, etc). Like many things in life the definition of "professional" is a subjective thing. Still, as far as Emacs projects go I believe the projects I maintain are probably as professional as they get. :-) I value the community around my projects immensely, but there's a limit to what can be done with the time and funding at my disposal. You'll also notice that I've put more emphasis on backwards compatibility to projects that are shared infrastructure (e.g. there hasn't been any breaking changes in nREPL sans the mandatory ns renaming, no breaking changes in a while on
Makes sense. And for me it's important to have such conversations so I can understand what different people are frustrated about.
No argument from me. I guess because everyone in Emacs-land breaks APIs like crazy and I know a bit more than average about all the libraries I use, this never bothered me much, but I understand that most people would likely be frustrated by having to track and address upstream changes.
Frankly, I've never been fond of this approach, mainly because I've noticed it confuses some users a lot (e.g. in cases like iTerm2 version 5). Often it confuses me a bit as well. :-) I get the benefits, but I think it's extremely unlikely that someone will want to use two versions of the same app/library at the same time and if we remove the limitations of Emacs's package manager - you can easily have two development branches of the same library/tool that can be maintained in parallel (e.g. version 1.x is bug-fix only mode, version 2.x is in active development). But there's also the real problem with maintaining multiple versions of something together - it requires more maintenance work. Still, I'm open to doing something like this down the road. |
Beta Was this translation helpful? Give feedback.
-
Now that CIDER made it to 1.0 and we have to be more careful with SemVer, it's important to agree on what exactly constitutes a breaking change. Clearly public API changes are breaking changes, but as CIDER is not a library, but a tool, we have to decide on things like:
And, of course, we should probably worker harder to clearly separate the public API from the private API. This topic is inspired by a conversation we've had with @plexus earlier to day related to some obsolete aliases I removed that caused a breakage in clj-refactor.el (see 4b6c0e9). I've got some ideas on how to proceed, but I'm also curious to hear what others think on the subject.
Backwards compatibility and API stability should be a high priority for us at any rate, but we have to decide how far we want to push in that direction.
Beta Was this translation helpful? Give feedback.
All reactions