Allow specifying plugin version requirement as semver range

[flash1293]

Picking up the discussion from #9019 (comment)

Our plugin system matured and at least the core APIs seems pretty stable by now. Based on the current state of affairs, forcing the user to pick a patch version compatibility seems too heavy handed for the case. Requiring any 3rd party plugin user to deal with bumping the version for each plugin they use on each version upgrade adds severe friction to Kibana plugin usage we can avoid. This is definitely adding the potential for unintended behavior, but as there are many cases in which plugins will work fine after upgrades it's worth allowing the plugin author to take over this responsibility.

Based on this reasoning, I propose allowing the user to specify compatibility as a semver range instead.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[pgayvallet]

I won't argue with the fact that maintaining a 3rd party plugin with the current tooling and support is a very tedious process, because let's be honest, it is (and being forced to release for each patch version is not the only part of this terrible DX).

Now, we're not currently following any non-breaking change or graceful evolution principles for our programmatic APIs, meaning that the signature and/or behavior of any of them can effectively change even during a patch version. This is the reason why we decided to force an exact version match between Kibana and any 3rd party plugin. Not 100% sure if that was the case before KP, but it likely was.

Ihmo, unless we achieve some kind of non-breaking change policy for these programmatic APIs, it's not realistic to allow to specify a range of version, as the developer will at least have no idea of any futur compatibility.

E.g during version 8.4, the developer will only be able to release a version of its plugin matching, say, 8.0-8.4, as it will have no idea if this version of its plugin will be compatible with kibana 8.5. Meaning that during 8.5, it will have to release a new version, and so on for 8.6, 8.7, or even any patch version of these minors.

So basically, without a proper NBC policy, it will only helps 3rd party developers to specify ranges of previous versions, so only helps during the initial release of the plugin.

Unless the developer takes the risk of stating that the plugin is compatible with future versions, but, well, do I need to elaborate on the risks here?

Our plugin system matured and at least the core APIs seems pretty stable by now

Arguable, but most importantly, 3rd party plugins can also depend on internal/official plugins, which don't necessarily have the same stability.

it's worth allowing the plugin author to take over this responsibility.

Thing is, when a licensed customer has an issue with a 3rd party plugins, we're the first to be contacted, and the number of 3rd party plugin related SDHs I encountered since I joined is non-negligible (and could drastically increase with this proposal).

Also, a misbehaving 3rd party plugin can potentially cause harm to the cluster or our system, which would redirect the responsibility to us.

If such proposal was free of risk and/or more work for support, I probably wouldn't mind, but in practice, it does.

[flash1293]

@pgayvallet Thanks for this insightful comment - I can agree with everything here. If there's room for that I would like to push forward the motion to get to a sensible backwards compatibility model for plugin APIs, I think there's quite some potential if we make it easier to develop plugins.

Maybe another take on the underlying issue for the near term:
I think the more important part isn't on the maintainer side (to keep it simple they can follow our releases like we do it with internal plugins and release one plugin version per stack version) but on the consumer side - every version bump requires a user to update every plugin and point it to the new version. Maybe there is an approach how we can make this kind of upgrade automatic (by having a data structure which encapsulates plugins for multiple versions or something like this)?

[lukeelmers]

If there's room for that I would like to push forward the motion to get to a sensible backwards compatibility model for plugin APIs, I think there's quite some potential if we make it easier to develop plugins.

Completely agree there's a lot of potential here, and so much more we could do to make authoring & consuming plugins easier. And backwards compatibility is an obvious place to start. But... it's tricky :)

We are working on a plan for addressing BWC in our REST APIs, but addressing it in our plugin APIs is a whole other (much larger) level of complexity, for reasons we don't have to get into here. Ultimately what we are implying is a move toward more stable/GA plugin APIs. And at the moment I'm not convinced we want to focus on that, especially as we still haven't determined what pluggability should look like in Cloud and whether the existing plugin model is the right path forward there (cc @alexfrancoeur @alexh97 @pauldotpower)

Once we have identified a strategy for Cloud, I expect it'll be easier to make decisions around how to best prioritize things like this.

[rudolf]

@pjhampton As a follow-up to your issue, I wonder if an alternative would be to provide plugin developers with tooling to automatically test, build and release new plugin versions when the stack gets bumped? E.g. using free CircleCI for open-source

The challenge is our internal APIs aren't following semver so this requirement is actually there to force plugin authors to re-test their plugin. We could remove this feature altogether and push the problem to users that just have to try out which versions work. But it feels like an easy test/build process would be more optimal and could potentially benefit users and plugin developers.

[pjhampton]

Maybe, @rudolf - it's something that I have started exploring - pjhampton/kibana-prometheus-exporter#279 in this file: https://github.com/pjhampton/kibana-prometheus-exporter/blob/main/.github/workflows/release-plugin.yml

For this to be worthwhile Elastic would need to be able to publish a new release version to plugin authors as a notification. I think for me this is about reducing plugin author and plugin user friction as much as possible - at the end of the day Elastics' internal releasing shouldn't matter much to the plugin author. I also think the likelihood of a patch release breaking a plugin is very low (IMO)

Edit - to double down on the above, I've been maintaining this plugin for a little while, and I think Kibana upstream has broken me once (in 7.10), so the releasing is a little laborious and needless for me.