clarify tested/supported versions of transitive or third-party dependencies

[liggitt]

Follow up from #11060, tracked in #12329

what is our stated support or testing for transitive or third-party dependencies?

I see three main categories of dependencies and sets of questions that are unanswered in current documentation:

• etcd
  • what versions are tested? what is the minimum and maximum supported version? should this info be in a machine-consumable format?
  • is there a tested or recommended upgrade order/process?
• dependencies (things the kubelet/apiserver/controllers interact with via CRI, CNI, CSI, KMS APIs)
  • what API versions are tested/supported
  • what specific implementations/versions are tested?
  • is there a tested or recommended upgrade order/process?
  • when we say we recommend container engine or CSI driver A at version B for k8s release 1.n, does that imply we expect k8s release 1.n+2 to support the same to cover maximum skew?
• addons that are bundled with kubernetes releases
  • what versions of these dependencies are tested and bundled in kubernetes release 1.X? should this info be in a machine-consumable format?
  • if there a tested or recommended upgrade order/process (e.g. if the add-on is defined in a daemonset, should the new daemonset be applied before/after upgrading the control plane, before/after upgrading the kubelets)?

Finally, where should this information be maintained (a unified page with a version matrix, per-release, or something else)?

@kubernetes/sig-testing @kubernetes/sig-release @kubernetes/sig-cluster-lifecycle

(#12233 is open specifically for etcd)

Page to Update:
https://kubernetes.io/docs/setup/version-skew-policy/

[fabriziopandini]

Xref kubernetes/kubeadm#1282

[liggitt]

/sig testing
/sig cluster-lifecycle
/sig release

[neolit123]

/assign @timothysc @luxas @roberthbailey
/assign @stevekuznetsov @spiffxp @fejta
sig-cluster-lifecycle, sig-testing: hello, we need feedback and approval for the proposals in this ticket from chairs with respect to LTS.

[roberthbailey]

Some initial thoughts:

etcd

I think of etcd as largely an implementation detail of the apiserver and owned by sig-apimachinery. I know that other folks in sig cluster lifecycle (e.g. @justinsb) have a different opinion, but I would love to see folks from apimachinery who work more closely with etcd (such as @jpbetz) help define the supported versions per release and the upgrade process.

My understanding is that right now we basically test a single version of etcd3 for each release (at least for the vast majority of tests) although we still claim support for etcd2 as well.

Dependencies

These are generally integrated / tested / qualified by the SIG that owns the interface, e.g. sig node would qualify various CRIs for a release or sig storage would qualify the CSI driver for a release. From what I've seen historically, as with etcd it's often been a single implementation that gets the vast majority of the test cycles and then an alternate version may be run through some tests to ensure compatibility.

Addons

Right now "addons" are implicitly bundled with the release in the form of container version references in yaml files in the cluster directory of the release tarball. I'm not sure who, other than kube-up, actually relies on this, but as we would like to get rid of the cluster directory hopefully the list of consumers of those yaml files is small.

@justinsb and @johnsonj have proposed a new way for managing addons, and along with that we will need to come up with a way to track addons per k8s release. One suggestion has been to use the bundle (https://github.com/GoogleCloudPlatform/k8s-cluster-bundle) as a schematic way to represent a set of components that have been tested and qualified together.

[tpepper]

/wg lts

[liggitt]

started a doc to organize work efforts related to kubernetes dependencies at https://docs.google.com/document/d/1WA8N7C48nkJmme9a96DU0o9jBpeycPhht8WF-Eam9QQ/edit?usp=sharing

[sftim]

/language en

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[BenTheElder]

/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[liggitt]

/reopen
/remove-lifecycle rotten

[k8s-ci-robot]

@liggitt: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@liggitt: This issue is currently awaiting triage.

SIG Docs takes a lead on issue triage for this website, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[wojtek-t]

/remove-lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.