Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Roadmap and Vision #1529

Merged
merged 6 commits into from
Aug 5, 2021
Merged

Add Roadmap and Vision #1529

Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
692bbc9
Add North Star Vision
saschagrunert Apr 20, 2021
bd4ce74
Move out of scope topics to in-scope for now
saschagrunert Apr 21, 2021
18a077a
Add deb/rpm packaging issue
saschagrunert Apr 21, 2021
85f7c5f
Add note about hermetic build process
saschagrunert Apr 22, 2021
d7c66a5
Rename 'North Star' to 'Roadmap and Vision'
saschagrunert Apr 23, 2021
0087f4a
Rephrase "Consumable" section
saschagrunert Apr 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 141 additions & 0 deletions north-star-vision.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# North Star Vision
saschagrunert marked this conversation as resolved.
Show resolved Hide resolved

## SIG Release Roadmap for 2021 and beyond

This document contains the SIG Release Roadmap for 2021. The status tracking can
be found at the bottom. More detailed information can be found on the [SIG
Release][0] and [Release Engineering][1] project boards.

[0]: https://github.com/orgs/kubernetes/projects/23
[1]: https://github.com/orgs/kubernetes/projects/30

### Primary Focus

Establish a **consumable**, **introspectable**, and **secure** supply chain for
Kubernetes. As a supply chain we understand the defining, building and
publishing of Kubernetes related artifacts.

1. **Consumable**: Improving the usability of artifacts by making their
consumption easier. This includes being process independent from external
companies like Google.
1. **Introspectable**: It is clear for users at which point and how Kubernetes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the "north star" just "documentation", or should it be a hermetic build process that is impervious to human interference, for all artifacts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's both: For example if we speak about package builds, then we want to completely automate that topic away from human interaction. On the other hand if we looks at the CVE process, then this will most of the time require human interaction.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear to me that, even for a CVE, a human needs to touch the built aartifacts, but I guess that path is open to discussion.

What I'd like to see as a strong "north star" statement is something like what I wrote - a hermetic build process that is impervious (or at least detectable) to human interference, for all official release artifacts including the official builds but also all of the container images.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I added the following statement:

All official release artifacts will be built by a hermetic process that is impervious to human interference.

artifacts are being built. This includes the documentation of all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are reproducible build artifacts included as part of this? I don't see them mentioned below. That has been a user-facing issue in the past (e.g. not being able to restart/recover from an issue during the release process and having to cut additional tags)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now it's not part of the roadmap. I see that kubernetes/kubernetes#70131 has been closed, but there is an open question at the end of the conversation. Maybe we want to add this topic as well. Thoughts on that @kubernetes/release-team-leads ?

deliverables as well as clarifying what we do not support. All official
release artifacts will be built by a hermetic process that is impervious to
human interference.
1. **Secure**: The artifacts we produce are verified for their integrity. This
applies to their functionality (we know what we deliver) as well as their
software security (we know when CVEs occur).

### Deliverables

The following deliverables are necessary to achieve the overall goal. Within
the following listing, all deliverables are sorted by their priority.

1. **Formalize supported release platforms (Introspectable)**

https://github.com/kubernetes/sig-release/issues/1337

Outcome: Definition of the life cycle for currently supported Kubernetes
artifacts and a guideline for the community about how to add new platforms.

1. **Implement a Bill of Materials (BOM) for release artifacts (Introspectable /
Secure)**

https://github.com/kubernetes/release/issues/1837

Outcome: An automated formal verification of produced release artifacts for
every future release.

1. **Enhance Kubernetes binary artifact management (Consumable)**

https://github.com/kubernetes/sig-release/issues/1372

Outcome: Being able to promote files as artifacts and using this mechanism
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a linked issue or more details for what this means?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one is the epic: #1372

cc @justaugustus @LappleApple if you have others in mind.

for Kubernetes releases.

1. **Define and collect metrics about Kubernetes releases (Introspectable)**

https://github.com/kubernetes/sig-release/issues/1527

Outcome: Being able to measure and interpret a set of defined metrics about
Kubernetes releases to associate actions with those.

1. **Define and implement the release cadence survey (Introspectable)**

https://github.com/kubernetes/sig-release/issues/1526

Outcome: A regular survey evaluating the user experience of the current
release cadence.

1. **Simplify CVE process for release management (Secure)**

https://github.com/kubernetes/sig-release/issues/896

https://github.com/kubernetes/release/issues/1354

Outcome: A documented and simple process for handling CVE information within
Kubernetes releases.

1. **Establish Cluster API as first-class signal for upstream releases
Copy link
Member

@aojea aojea Apr 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked this job https://testgrid.k8s.io/sig-cluster-lifecycle-cluster-api-provider-gcp#capg-conformance-v1alpha4-k8s-master&width=5 and there are several things to take into consideration:

  1. it runs a script https://github.com/kubernetes-sigs/cluster-api-provider-gcp/blob/master/scripts/ci-conformance.sh
    that install KIND
  2. it calls another script that creates some gcp infra with bash command "gcloud ..." and install packer https://github.com/kubernetes-sigs/cluster-api-provider-gcp/blob/master/scripts/ci-e2e.sh
  3. packer runs some ansible based on https://github.com/kubernetes-sigs/image-builder/tree/master/images/capi
  4. the it runs kind and does its stuff

It installs calico from master?

Deploy calico
kubectl --kubeconfig="/tmp/kubeconfig" apply -f https://docs.projectcalico.org/manifests/calico.yaml
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created

I see many moving parts, from my experience it will take a lot to stabilise this job ... I only ask to not replace current jobs until this has demonstrated stability for at least one release cycle

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No objections to improving support for this mechanism. I agree with @aojea that the current mechanisms should remain until the new approach demonstrates stability for at least a release cycle. As an aside, I would have guessed this would have fallen under "Consumable", not sure how it relates to "Secure".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed recently with SIG Scale, the GCP provider for CAPI (CAPG) is not in a good state due to no active maintainers, that can dedicate a certain % of workday to the project.
the existing CI scripts and setup for CAPG are a best effort currently.

(Consumable)**

Outcome: Cluster API provides a CI signal for blocking release test jobs.

1. **Enhance and simplify Kubernetes version markers (Consumable)**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have already have some issue for this item?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May @LappleApple knows more, I don't think that we have an umbrella issue for this one yet.

Some refs: kubernetes/release#1693, kubernetes/release#1711


Outcome: Clear documentation about available version markers as well as their
simplified automation.

1. **Moving deb/rpm package builds to community infrastructure (Consumable)**

https://github.com/kubernetes/release/issues/281

Outcome: Automated builds of `deb` and `rpm` Kubernetes packages within
community infrastructure.

1. **Create releases landing page (Consumable)**

https://github.com/kubernetes/website/issues/20293

Outcome: A releases page that is up to date and acts as canonical place for
release related information, for example links to release notes and support
timelines.

1. **Signing of release artifacts (Secure)**

https://github.com/kubernetes/release/issues/914

Outcome: Being able to GPG sign release artifacts, which also includes
container images.

saschagrunert marked this conversation as resolved.
Show resolved Hide resolved
### Known Risks

1. **We rely on different SIGs for our work**

We have a need to discuss our enhancements with different SIGs to get all
required information and drive the change. This can lead into helpful, but
maybe not expected input and delay the deliverable.

1. **Some topics require initial research**

We're not completely aware of all technical aspects for the changes. This
means that there is a risk of delaying because of investing more time in
pre-research.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be the decision taking in some of these topics a problem? Do we expect strong veto or not identified stakeholder going against some work?

### Requests to Other Teams

1. **SIG Architecture**

For the formalization of the released platforms and input about the overall
supply chain.

1. **SIG Cluster Lifecycle**

To get input for making Cluster API a first-class signal for upstream releases.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in addition to or replacing current jobs?

Copy link
Member Author

@saschagrunert saschagrunert Apr 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition. Maybe @kubernetes/sig-cluster-lifecycle can provide additional feedback on top of that point.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was a SIG CL proposal:
kubernetes/kubernetes#82532
"replace the existing PR and release blocking kube-up based jobs with CAPI jobs"

IIRC, SIG Testing and Release had some agreement on this, but i don't think it was discussed with other SIGs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that it would be useful to clarify what would this improve and what we gain from it.


### Current Status

| Topic | Lead | Status | Target Date | Links |
| ----- | ---- | ------ | ----------- | ----- |
| | | | | |