experimental features in the platform libraries

[kabaros]

Overview

This RFC proposes an approach for adding "experimental" features to our platform libraries main release. This, as opposed to releases to a pre-release branch like alpha or beta, allows consumers to experiment with a potentially unstable feature in an otherwise stable release. The approach is common in open-source project - rust, gradle, ruby, node, react, vue, transtrack are among popular projects adopting such approach.

please respond to the RFC with: 👍🏿 (read it and sounds good to me) / 👀 (read it and don't have strong feelings) / or add a comment if you read it and have some questions or concerns

Motivation

The main motivation - already explained - is to enable users to test potentially unstable features in production in an otherwise stable release. As opposed to pre-release channels (alpha and beta) where the whole release channel could be unstable, and will likely contain multiple streams of unfinished work.

An important side effect of the approach is to simplify the release management process. While pre-release channels can be thought of as "experimental", they get more complicated to manage once there are more than one more experiment being implemented: these subprojects often overlap, and often reach maturity in an order different from when they were started, making merges to the main branch cumbersome and hard to track.

What makes a feature "experimental"?

The defintion below is from the VueJS project

Some features are shipped and documented in a stable version, but marked as experimental. Experimental features are typically features that have an associated RFC discussion with most of the design problems resolved on paper, but still lacking feedback from real-world usage.

The goal of experimental features is to allow users to provide feedback for them by testing them in a production setting, without having to use an unstable version of the main library. Experimental features themselves are considered unstable, and should only be used in a controlled manner, with the expectation that the feature may change between any release types.

This definition fits our DHIS2 context. Experiments would be initatives larger than a typical feature that are relatively long-running (i.e. the Plugin implementation) that we have them nailed in theory, but we want to gather further feedback for them in production before declaring the feature finalised.

Experiments are also always opt-in. The mechanism for opting-in might differ; consumers might opt-in by importing from /experimental or, in the future, by setting a config to opt-in to certain experiments.

Some initiatives might blur the line with experimental for example, adding support to TypeScript is a long-running project, a large-ish initiative but the path to it is well-defined and we have more certainty about the end goal. Removing dependency on create-react-app in the platform is unlikely to be done on an opt-in basis, and while it's a big chunk of work, the end result should is clear.

Experiments should not be added without design or deep thinking about a feature. But with experiments, as the VueJS definition mentions, there is a still some niggles we want to validate with a larger community without hindering our ability to move fast, change APIs and potentially break things.

Implementation

Consumers will be able to use these experimental features by importing from a /experimental subpath, for example:

import { Plugin } from '@dhis2/app-runtime/experimental'

Adding an experimental feature

If this is the first experiment in a project, then you will likely have to update the package.json with:

"exports": {
    "./": {
        "import": "./build/es/index.js",
        "require": "./build/cjs/index.js"
     },
     "/experimental": {
        "import": "./build/es/experimental.js",
        "require": "./build/cjs/experimental.js"
     }
},

The experimental.js (or experimental.ts) should be the output point for all experimental features.

For the feature itself, we should be tag it as experimental with JsDoc. For example:

@experimental This API is experimental and may change in future releases. Please consult the documentation https://developers.dhis2.org/docs/app-runtime/experimental/ for changes.

The feature should also be documented under the /experimental in the appropriate documentation site, and should have a clear section at the top explaining its experimental status similar to this example and this example.

Check list

• The experimental feature is only exported through /experimental subpath
• The experimental feature is marked with JsDoc with @experimental
• The feature should have its own section in the documentation under experimental features
• Add a console.warn to the body of the experiment mentioning: "you are using {Plugins}, an experimental feature of @dhis2/app-runtime."

Abandoning or "Graduating" an experimental feature

An experimental feature life-cycle ends with either it being removed and the experiment abandoned, or the experiment "graduating" into the main library and become ready to use without the experimental suffix.

In both cases, we should have a grace period for features to allow consumers to gracefully update their code. A deprecated feature will only be removed in the next major release.

During the grace period, a removed feature should be marked as @deprecated and a graduated feature should drop the @experimental tag. During the period, we should also add a warning in the entry point of the feature to instruct developers to update their code, i.e. in the body of the experimental Plugin feature, we can add:

export const Plugin = () => {
	console.warn('The Plugin feature has moved out of Experimental phase and can now be used without /experimental suffix. Consult DHIS2 docs at: https://developers.dhis2.org/docs/app-runtime/experimental/ for more info.
	// rest of plugin code..
}

Versioning

Breaking changes in an experiment are allowed and don't require a major release. We will try to minimise the effect of these changes, and whenever possible provide a graceful change where the consumers of an old API are warned about the change. But in principle, by opting-in to an experiment, the consumer takes more responsibility in keeping their code up to date than us - the library maintainers.

This is in conflict with semver principles, but it's a pragmatic approach that many popular open-source libraries use to simplify their release process, and allow experimentation faster without a lot of friction.

We should always communicate breaking changes even during an experiment to allow consumers to upgrade easily. That communication happens through two main channels:

• Documentation: any changes should be documented in the experiment page in the docs
• Code warnings: if possible, through developer logs, add messages to communicate changes to consumers using an old API, for example, highlighting their use of a different param structure when calling an API.

Finishing an experimental feature whether by removing it or graduating it does not have to trigger a major release. We can decide that a feature is important enough to highlight in a major release, but that's not necessary.

Alternative implementations

• An experimental (canary) channel: Some projects, like React, have a dedicated canary (or experimental) channel. The channel tracks the main release channel, and it's normally used along marking experimental APIs with a prefix. This approach would add to the maintenance overhead and complicates the release management process, and would be an overkill for a project our size.

• Config opt-in: In addition to the API, we could also let people opt-in through d2.config to experiments (individual or all in one). This is the approach used by cypress and other libraries. This adds a further layer of responsibility on the consumer, but we thought it's unnecessary at this stage for the experiments we're starting with. It can be a good way to go when we need a feature-toggle like approach for experiments. For example, maybe we can have an experimental server-side rendering that consumers can opt-in to try.

• EXPERIMENTAL prefix: some popular libraries like React choose a more explicit approach to naming their experimental API, by prefixing with the word experimental or unstable_ to the API name. This is more explicit and obvious than just having a /experimental suffix added to the import. We considered this approach, but we opted for a path that would make consumers' life easier once the feature is graduated successfully (they only have to remove the /experimental suffix). This sounds reasonable for the experiment we're starting with (plugins) as it has already been through many iterations and we don't anticipate major changes, but we could decide in the future to be more explicit about an API and prefix with a tag.

[amcgee]

Thanks for the great and thorough RFC @kabaros - this approach sounds really good to me and makes a lot of sense for platform libraries, and particularly the plugin functionality. 🚀

I hope we can in the future extend this to a broader maturity model covering most DHIS2 features and backend APIs, probably with a bit more strict controls on breaking changes in that case. In addition to being a great tool for our frontend libraries now this seems like a good stepping stone to something bigger as well.