Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KEP-3784: Names DB (first draft) #3787

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

thockin
Copy link
Member

@thockin thockin commented Jan 26, 2023

First draft of a database for kube-owned names.

Issue link: #3784

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. labels Jan 26, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: thockin
Once this PR has been reviewed and has the lgtm label, please assign johnbelamaric for approval by writing /assign @johnbelamaric in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 26, 2023
@thockin thockin changed the title KEP-3784: Names DB first draft KEP-3784: Names DB (first draft) Jan 27, 2023
Copy link
Contributor

@sftim sftim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some thoughts

keps/sig-architecture/3784-database-of-names/README.md Outdated Show resolved Hide resolved
keps/sig-architecture/3784-database-of-names/README.md Outdated Show resolved Hide resolved
keps/sig-architecture/3784-database-of-names/README.md Outdated Show resolved Hide resolved
keps/sig-architecture/3784-database-of-names/README.md Outdated Show resolved Hide resolved
keps/sig-architecture/3784-database-of-names/README.md Outdated Show resolved Hide resolved
valueType: "String",
resources: [ "core/nodes", core/persistentvolumes ],
description: "The availability zone of the resource.",
docsLink: https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could store just the fragment identifier, even.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we think all docs go to the same page, sure. This is a little self-referential right now - do we generate this docs page from metadata, or do we generate this metadata from the docs, or do we demand contributors update 2 things (we know how well that works)?

I really want to hear your thougts on this specifically. The docs that exist are much more verbose than the examples I am providing here and include cross-refs and other things which might be awkward in the database text. But having 2 sources of truth is always suspect.

key: "topology.kubernetes.io/region",
sig: "Architecture",
context: "Label",
valueType: "String",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All labels and annotations have string values, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is an optional key to define the format of that string? (eg: valid JSON only please, or even specify a schema)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The transport type is string, but the actual value can be something else. E.g. annotation values are sometimes JSON-encoded structs. I meant this as a place to describe the "inner" type, but I see it's not clear.

Will update some text

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Watch out for whether you're defining the Kubernetes API (ie, as seen for conformance), or a particular implementation of that API.

If we change the internal implementation, could this drift?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know of any places where that is true, off the top of my head. In general annotations are either entirely impl-specific or they are generic - impls should not use a generic name that has impl-specific payload.

Do you know of any?

Comment on lines +382 to +387
Should we have a YAML per key - easier lookups and collision avoidance, but
lots of files. Would maybe need Context to be a list.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another potential home for one-file-per-key is k/website. Code that wants to could pull from the website repo and turn front matter YAML into something that other tools want to consume.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like ONE source of truth and the rest derived from that, if possible. Secondary concern is ease of adding/updating (which is why I suggest k/k/staging). But I am open...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Website is the current source of truth - but not machine readable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another problem with the website is that it lags releases. In other words, main documents the last minor release. That feels like the bigger challenge.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we put enough human-friendly text into this DB, would you want to generate the doc from it?

keps/sig-architecture/3784-database-of-names/README.md Outdated Show resolved Hide resolved
## Proposal

The "database" should live in a new git repo (proposed name:
"kubernetes/namesdb"). In this repo we will establish a files-and-directories
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have reserved things besides names too. eg https://github.com/kubernetes/kubernetes/blob/master/pkg/cluster/ports/ports.go. Would it be useful to step back one level and just have a generic "kubernetes reserved things" repository ("Kubernetes Assigned Numbers Authority"?), with names being one subdir and others potentially added later?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know that ports need all the same tooling and metadata, but I agree we don't do a great job of documenting them. Would it be enough to just convert that .go file into markdown for k/website?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be enough to just convert that .go file into markdown for k/website?

We (SIG Docs) can also work with JSON or YAML. JSON is handy because it has a defined media type and we can do HTTP fetches.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not concerned specifically with ports being documented/parseable, I was just wondering if we're going to need a bigger registry later, and thinking it would be nicer to have it all in one repo rather than having both a "namesdb" and a "portsdb" and who knows what else (Kubernetes-specific IPFIX elements?)

(Though the fact that ports was the first thing that came to mind is because OpenShift does have a centralized port registry because random operators kept trying to claim the same metrics ports. But this is a bigger management problem in OpenShift because it's spread across so many repos.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK - I don't object to leaving room at the root for an alternate hierarchy of of data (e.g. .../names/... vs .../ports/...) with different schema.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That changes "namesdb" to ... "iddb" or "nndb" (names and numbers) or "metadb" or something?


## Drawbacks

It's YET ANOTHER thing to update.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this make it harder to localize the label / annotation definitions?

(I think: probably it does, but it's still do-able)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great insight - yes. Perhaps we need to split the machine-data from the human-data? E.g. one of the fields in the record is a pointer the human-friendly strings which are localized and can feed docs? Ideally it would be one PR to update those - does that mean all the localized strings live in the same repo? Or do we put EN in this DB and non-EN translations are a secondary DB which is used by docs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the website, a common pattern is to generate [the relevant part of] the English docs using automation, and then have each localized version be generated and maintained by hand.

So the localized text would live only in k/website. I think that would work for this.

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 1, 2023
@thockin thockin removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 1, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 19, 2024
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Feb 18, 2024
@thockin thockin removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Feb 18, 2024
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 18, 2024
@thockin thockin removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 20, 2024
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 18, 2024
@thockin thockin removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants