Framework for defining the maturity of OpenTelemetry and its SIGs

[jpkrohling]

On 08 Mar 2023, the OpenTelemetry GC and TC held an OpenTelemetry Leadership summit, discussing various topics. One of the themes we discussed was establishing standard rules for describing the maturity of the OpenTelemetry project. This OTEP summarizes what was discussed there and is intended to have the wider community provide feedback.

This OTEP builds on what was previously communicated by the project, especially on the following page: https://opentelemetry.io/docs/reference/specification/versioning-and-stability.

The Collector’s stability levels inspired the maturity levels.

Signed-off-by: Juraci Paixão Kröhling juraci@kroehling.de

[jsuereth]

I like the goal of this proposal but not the details.

A few suggestions:

• Only components should be marked "tier 1" or "critical".
• Delegating decision on core components to SiGs makes sense.
• Regarding language support, we should have a notion of which languages are required for OTEL successful adoption and ensure these are "tier 1".
• Rather than defining "Signal" tier-ness, I think it should be based on the specification itself. I think everything we've defined as "stable" should actually be considered "tier 1". Basically, I think we already have a mechanism to determine what "tier 1" means there.
• OTLP should absolutely be called out as a component here.

[tigrannajaryan]

Do we beed definition of tiers and maturity/stability to be in one OTEP? Could we decouple these two discussions?

[jpkrohling]

Folks, thank you for the discussion and insights so far! Based on yesterday's GC/TC call, I reworked this OTEP to remove the parts where we determine which SIGs would be part of a future graduation, leaving only the definition of the maturity levels.

I ask you to reread the OTEP and provide your input on the open questions. Thank you again!

I'll address the open comments individually later, some/most of them might not be relevant anymore given the latest changes.

[tigrannajaryan]

I think we are missing the equivalent of "Feature Freeze/RC/Frozen" level that we have in some SIGs: open-telemetry/opentelemetry-specification#3459

[jpkrohling]

@marcalff, I agree in principle with you, and I had wording like that in the beginning: the idea was that a component would be stable only when all its dependencies were stable. As it turns out, we have good examples of cases where this isn't necessarily true, like for the Collector Contrib (with its 100s of components) and the Java Agent, with its pluggable pieces. So, that specific wording has been removed.

One way we solved the situation you mentioned was to have a notion of "tiered" features, stating that all "tier-1" features have to be stable before a component can be called stable. New features could start life as a tier-2 feature, eventually being promoted to tier-1 upon reaching a stable level, perhaps once all dependent parts supported it. We dropped that, as it seemed too complicated and restrictive for the moment. We might revive that part at a later point.

Given that we care the most about the user-facing side of this story, the idea now is that components and SIGs can define what's their external messaging, taking the state of their dependencies only as a source of information for them to make the decision.

[austinlparker]

One thing I didn't notice in this, and really nowhere in our maturity model, is documentation. Should that be split into its own OTEP?

[jpkrohling]

I like having documentation as part of the definition of done, but I'd leave it for individual SIGs to determine. A separate OTEP with recommendations would be nice, though.

[tigrannajaryan]

@jpkrohling can you please address/resolve all comments?

Everyone who reviewed so far and haven't approved: if you have objections please speak up / request changes to block the OTEP (or approve the OTEP if you agree with it).

[jpkrohling]

Folks, I marked a few conversations as resolved, as I believe they have been sufficiently addressed or discussed. As of right now, there are only two open items, one related to components "under development" being removed without notice, and one about stability guarantees for "stable" deliverables.

If you think there's still something to address in this OTEP, please reopen a discussion or open a new one.

[jsuereth]

Approving this with a few caveats for us to think about.

• I'm glad we got rid of identifying by community. We can expand over time.
• I suspect the difference between Alpha and Beta is material. I think we'll have trouble getting adoption of components without some strict compatibility guarantees. Specifically, I think Beta may not be strong enough for some components. I'm looking at, e.g. the Java API where they've created an experimental API, but actually support it through many versions. In practice, I think we'll need some better guarantees of stability of Beta to get the usage we desire prior to marking something Release. However, for now, consistency is better than perfection.
• I would like to see more strict rules around unmaintained to avoid squabbling. E.g. failure to patch bug-fixes vs. failure to insert features that may be quetionable to the maintainer. I think some flexibility here is good, but perhaps some guidance would also be good.

[jpkrohling]

Folks, what's missing to merge this one?

[tigrannajaryan]

Folks, what's missing to merge this one?

All comments must be resolved before the PR can be merged.

[flands]

Hey folks - just ran across this, so added some thoughts.

[jpkrohling]

@tigrannajaryan, I marked the last comments as resolved, as they are either being tracked by follow-up issues, or have been answered without further questions after at least a few days.

I think we are at a stage now where this should be merged, given we have more than 10 approvals. Further tweaks can be made via follow-up PRs.

[tigrannajaryan]

All comments resolved, enough approvals, 2 days since last comment, all merging conditions met. Merging.

[tigrannajaryan]

Thank you @jpkrohling and everyone who helped improve this OTEP!

[tigrannajaryan]

@jpkrohling I think it can be useful to announce this OTEP to all maintainers, perhaps even file issues in SDK repos so that they start labeling components according to the OTEP.

[jpkrohling]

Absolutely, I'm adding this to my to-do list, scheduled for executing in a week or so, to give time for everyone to come back from their PTOs and catch-up with the notifications.