Add resource version section to api-concepts documentation #16910

jpbetz · 2019-10-15T00:27:08Z

Clarify how resource version works, particularly the parameter on get, list and watch.
Clarify HTTP 410 Gone status codes work in conjunction with resource versions.
Add section about reflectors with pseudo-code for how to correctly "list and watch" from a client without "going back in time".

https://5da654db8fa033000817e6eb--kubernetes-io-vnext-staging.netlify.com/docs/reference/using-api/api-concepts/

k8sio-netlify-preview-bot · 2019-10-15T00:34:13Z

Deploy preview for kubernetes-io-vnext-staging processing.

Building with commit 10fabcd

https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5de5790d4c5394000856b040

netlify · 2019-10-15T00:35:40Z

Deploy preview for kubernetes-io-master-staging ready!

Built with commit 9d02e07

https://deploy-preview-16910--kubernetes-io-master-staging.netlify.com

jpbetz · 2019-10-24T17:50:42Z

@wojtek-t @smarterclayton @liggitt Looking for reviewers for this.

content/en/docs/reference/using-api/api-concepts.md

jpbetz · 2019-11-17T17:34:47Z

@wojtek-t @liggitt This is ready for another review pass.

daminisatya · 2019-11-27T17:57:14Z

@liggitt Is this PR good to go?

wojtek-t

LGTM (just two nits)

content/en/docs/reference/using-api/api-concepts.md

daminisatya · 2019-11-29T12:25:01Z

@jpbetz Can you look into @wojtek-t comment?

jpbetz · 2019-12-02T20:50:29Z

@jpbetz Can you look into @wojtek-t comment?

Feedback applied.

wojtek-t · 2019-12-02T21:15:40Z

LGTM - but I think @liggitt should be the one approving it.

daminisatya · 2019-12-03T15:37:19Z

/lgtm
/approve

k8s-ci-robot · 2019-12-03T15:37:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: daminisatya

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [daminisatya]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

liggitt · 2019-12-03T15:46:33Z

content/en/docs/reference/using-api/api-concepts.md

+
+| resourceVersion unset | resourceVersion="0" | resourceVersion="{non-zero version}" |
+|-----------------------|---------------------|--------------------------------------|
+| Most Recent           | Any                 | Not older than                       |


I think the "not older than" semantic for individual items needs clarification. If the server version is at rv=10, and an individual item is at rv=5, what will a get of the individual item with ?resourceVersion=8 do? Return the item at rv=5, because the server is at rv=10? Wait/return a "too large resource version" because the item is at rv=5? It's hard to tell from this description which to expect (it's especially confusing to request an individual item with resourceVersion=10 and get back a successful response of an item with resourceVersion=5)

I'll try adding this to the "not older than" semantic:" ... Note that this ensures only that the objects returned are no older than they were at the time of the provided resource version. The resource version in the ObjectMeta of individual object may be older than the provide resource version so long it is for the latest modification to the object at the time of the provided resource version."

liggitt · 2019-12-03T15:48:11Z

content/en/docs/reference/using-api/api-concepts.md

+
+**Get:**
+
+| resourceVersion unset | resourceVersion="0" | resourceVersion="{non-zero version}" |


nit: s/non-zero version/value other than 0/ to reinforce these are not necessarily numbers

liggitt · 2019-12-03T15:51:22Z

content/en/docs/reference/using-api/api-concepts.md

+| paging    | resourceVersion unset | resourceVersion="0" | resourceVersion="{non-zero version}" |
+|-----------|-----------------------|---------------------|--------------------------------------|
+| no limit  | Most Recent           | Any                 | Not older than                       |
+| limit="n" | Most Recent           | Any                 | Exact                                |


is it worth including a row for the continue token behavior, since you can't set resourceVersion separately, and the version you get back is based on the resourceVersion embedded in the continue token?

limit unset, continue unset limit="n", continue unset limit="n", continue="<token>"

liggitt · 2019-12-03T15:52:24Z

content/en/docs/reference/using-api/api-concepts.md

+
+- **Most Recent:** Return data at the most recent resource version. The returned data must be consistent (i.e. served from etcd via a quorum read).
+- **Any:** Return data at any resource version. The newest available resource version is preferred, but strong consistency is not required; data at any resource version may be served. It is possible for the request to return data at a much older resource version that the client has previously observed, particularly in high availabiliy configurations, due to partitions or stale caches. Clients that cannot tolerate this should not use this semantic.
+- **Not older than:** Return data at least as new as the provided resource version. The newest available resource version is preferred, but any data not older than this resource version may be served.


data at least as new as the provided resource version ... any data not older than this resource version may be served

this specifically needs clarification for the individual get case

liggitt · 2019-12-03T15:54:11Z

content/en/docs/reference/using-api/api-concepts.md

+- **Get State and Start at Any:** Warning: Watches initialize this way may return arbitrarily stale data! Please review this semantic before using it, and favor the other semantics where possible. Start a watch at any resource version, the most recent resource version available is preferred, but not required; any starting resource version is allowed. It is possible for the watch to start at a much older resource version that the client has previously observed, particularly in high availabiliy configurations, due to partitions or stale caches. Clients that cannot tolerate this should not start a watch with this semantic. To establish initial state, the watch begins with synthetic “Added” events for all resources instances that exist at the starting resource version. All following watch events are for all changes that occured after the resource version the watch started at.
+- **Start at Exact:** Start a watch at an exact resource version. The watch events are for all changes after the provided resource version. Unlike "Get State and Start at Most Recent" and "Get State and Start at Any", the watch is not started with synthetic "Added" events for the provided resource version. The client is assumed to already have the initial state at the starting resource version since the client provided the resource version.
+
+### "410 Gone" responses


what does the server return if you get/list a version newer than the server has available? I guess the only way you would have done that is if you constructed a numeric resourceVersion yourself or used one across resources served from different storage backends, which you're not supposed to do

Current behavior is:

List - waits briefly for the RV and then times out (https://gist.github.com/jpbetz/88a5413a5abbd8a1c6c2b69ac6b2e69f)

Get - same as list (https://gist.github.com/jpbetz/c622c02c8646c01ee3ff153db66a659c)

Watch - waits for events with sufficiently high RVs

I'll document.

liggitt · 2019-12-03T15:54:48Z

content/en/docs/reference/using-api/api-concepts.md

+
+Servers are not required to serve all older resource versions and may return a HTTP `410 (Gone)` status code if a client requests a resourceVersion older than the server has retained. Clients must be able to tolerate `410 (Gone)` responses. See [Efficient detection of changes](#efficient-detection-of-changes) for details on how to handle `410 (Gone)` responses when watching resources.
+
+For example, the kube-apiserver periodically compacts old resource versions from etcd based on its `--etcd-compaction-interval` setting. Also, the kube-apiserver's watch cache keeps `--watch-cache-sizes` resource versions in each resource cache. It depends on if a request is served from cache on which one of these limits applies, but if a resource version is unavailable in the one that applies, a `410 (Gone)` will be returned by the kube-apiserver.


It depends on if a request is served from cache on which one of these limits applies, but if a resource version is unavailable in the one that applies

nit: consider rephrasing to something like "If a resourceVersion is requested outside the applicable limit (depending on whether a request is served from cache or not), a 410 (Gone) will be returned by the kube-apiserver."

* Add Resource Version semantics section to api concepts * Clarify risks of going back in time, add details about compaction and watch cache sizes * Apply suggestions from liggitt Co-Authored-By: Jordan Liggitt <jordan@liggitt.net> * remove pesudocode, apply feedback * Fix typo * Clarify equality rules

* feat: graduate TaintNodesByCondition to GA (#17073) * Promote StartupProbe to beta (enabled by default). (#17164) * Watch bookmarks to GA (#17026) * feat: graduate ScheduleDaemonSetPods to GA (#17350) * Update Docker installation instructions (#17405) * Use exact version numbers for installing Docker in Ubuntu (#17428) * Move CSIMigration and CSIMigrationGCE to Beta in Kubernetes v1.17 (#17478) * Promote NodeLease feature to GA (#17189) * Update docs for csi topology ga (#17408) * Update RunAsUsername to beta (#17460) * doc:Update RunAsUsername to beta * doc: update samples - kubernetes.io/os is no longer beta * Updating based on review feedback * Promote Node-specific volume limits to GA (#17432) * Promote PodShareProcessNamespace to stable (#17192) * Promote PodShareProcessNamespace to stable * Add for_k8s_version to feature-state label Co-Authored-By: Tim Bannister <tim@scalefactory.com> * Readd version-check to shareProcessNamespace task * Update service load balancer finalizer doc for GA (#17438) * Update Topology Manager docs (#17451) * Added information on how device plugins can take advantage of Topology Manager * Updated the Topology Manager documentation to include additionalinformation and update some out of date sections * Fix broken Topology Manager link (#17746) Part of What's Next Device Plugin section * Update CRD defaulting docs for GA (#17450) * Add documentation for VolumeSnapshot Beta (#17233) * Updating EndpointSlice documentation for beta release in 1.17 (#17411) * (docs/dualstack): v1.17 updates (#17457) * Add placehold doc updates for dualstack in 1.17 Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Add Downward API and /etc/hosts Pod IP validation Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * remove addressed known issue via k/k pr 85246 Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Remove known issue and add flag as part of k/k 79993 Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * remove follow up placeholders Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Update verbiage Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Make IP addressing consistent throughout the task Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Update to status.podIPs Signed-off-by: Lachlan Evenson <lachlan.evenson@microsoft.com> * Update content/en/docs/tasks/network/validate-dual-stack.md Use set instead of env Co-Authored-By: Khaled Henidak (Kal) <khnidk@outlook.com> * add topology.kubernetes.io/zone, topology.kubernetes.io/region and node.kubernetes.io/instance-type labels to docs (#17498) Signed-off-by: Andrew Sy Kim <kiman@vmware.com> * Service topology alpha documentation (#17459) * Update list of feature flags for in-tree plugins migrated to CSI (#17533) Signed-off-by: Deep Debroy <ddebroy@docker.com> * Update Node concept for TaintNodesByCondition going GA (#17577) * feat: graduate ResourceQuotaScopeSelectors to GA in 1.17 (#17554) * kubeadm: update the upgrade documentation for 1.17 (#17587) * doc: Simplify Windows deployments with RuntimeClass (#16697) * doc: Simplify Windows deployments with RuntimeClass * Updating on review feedback * doc: Adding windows-build label from enhancement 1301 * update doc for kubelet option --reserved-cpus (#17648) * feat: update TaintNodesByCondition in feature gates table (#17377) * Update docs for v1 resource quota configuration (#17547) * AdmissionConfiguration v1 (#17548) * Update WebhookAdmissionConfiguration examples (#17549) * Update AWS EBS Migration Feature state (#16126) * Add resource version section to api-concepts documentation (#16910) * Add Resource Version semantics section to api concepts * Clarify risks of going back in time, add details about compaction and watch cache sizes * Apply suggestions from liggitt Co-Authored-By: Jordan Liggitt <jordan@liggitt.net> * remove pesudocode, apply feedback * Fix typo * Clarify equality rules * Cleanup kubectl generators docs (#17609) * Write ReplicationController without a space * Drop mentioning unsupported cluster versions * Fix capitalization for “API group” * Tweak wording * Avoid using deprecated generator in example * add Antrea description in dev-1.17 (#17919) * Promote VolumeSubpathEnvExpansion to GA * Reference Documentation for the Kubernetes API for 1.17 (#18019) * Update feature-gates.md (#18033) * Reference Documentation for kubectl Commands for 1.17 (#18017) * Update for v1.17 (#18034) * Update config.toml(release-1.17) for 1.17 (#18031)

jpbetz · 2019-12-10T23:24:04Z

Thanks for the review @liggitt. I've opened #18069 to address the post-merge comments

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. language/en Issues or PRs related to English language sig/docs Categorizes an issue or PR as relevant to SIG Docs. labels Oct 15, 2019

k8s-ci-robot requested review from lavalamp and smarterclayton October 15, 2019 00:27

jpbetz changed the base branch from master to dev-1.17 October 15, 2019 00:27

jpbetz force-pushed the resource-version-concepts branch from 9d02e07 to 987a15d Compare October 15, 2019 00:34

jpbetz force-pushed the resource-version-concepts branch 2 times, most recently from 99c9c61 to 8d04075 Compare October 15, 2019 01:12

jpbetz mentioned this pull request Oct 15, 2019

Document semantic of resourceVersion=0. #16944

Closed

jpbetz force-pushed the resource-version-concepts branch from 03bb838 to 494f4da Compare October 15, 2019 23:23

jpbetz commented Oct 24, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

liggitt reviewed Oct 25, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

liggitt reviewed Oct 25, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

liggitt reviewed Oct 25, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

liggitt reviewed Oct 25, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

liggitt reviewed Oct 25, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

liggitt reviewed Oct 25, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

liggitt reviewed Oct 25, 2019

View reviewed changes

jpbetz force-pushed the resource-version-concepts branch from 5376216 to adf3b21 Compare October 27, 2019 06:20

k8s-ci-robot removed language/id Issues or PRs related to Indonesian language language/ja Issues or PRs related to Japanese language language/ko Issues or PRs related to Korean language language/zh Issues or PRs related to Chinese language labels Nov 10, 2019

Fix typo

d04cb63

tengqm added this to the 1.17 milestone Nov 26, 2019

wojtek-t reviewed Nov 28, 2019

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

Clarify equality rules

10fabcd

jpbetz force-pushed the resource-version-concepts branch from 5a4a771 to 10fabcd Compare December 2, 2019 20:50

k8s-ci-robot assigned daminisatya Dec 3, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 3, 2019

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 3, 2019

k8s-ci-robot merged commit eda62da into kubernetes:dev-1.17 Dec 3, 2019

liggitt reviewed Dec 3, 2019

View reviewed changes

jpbetz mentioned this pull request Dec 10, 2019

Followup fixes for: Add resource version section to api-concepts #18069

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add resource version section to api-concepts documentation #16910

Add resource version section to api-concepts documentation #16910

jpbetz commented Oct 15, 2019 •

edited

Loading

k8sio-netlify-preview-bot commented Oct 15, 2019 •

edited

Loading

netlify bot commented Oct 15, 2019

jpbetz commented Oct 24, 2019

jpbetz commented Nov 17, 2019

daminisatya commented Nov 27, 2019

wojtek-t left a comment

daminisatya commented Nov 29, 2019

jpbetz commented Dec 2, 2019

wojtek-t commented Dec 2, 2019

daminisatya commented Dec 3, 2019

k8s-ci-robot commented Dec 3, 2019

liggitt Dec 3, 2019 •

edited

Loading

jpbetz Dec 10, 2019

liggitt Dec 3, 2019

liggitt Dec 3, 2019

liggitt Dec 3, 2019 •

edited

Loading

liggitt Dec 3, 2019

jpbetz Dec 10, 2019

liggitt Dec 3, 2019

jpbetz commented Dec 10, 2019 •

edited

Loading


		Get:

		\| resourceVersion unset \| resourceVersion="0" \| resourceVersion="{non-zero version}" \|


		Servers are not required to serve all older resource versions and may return a HTTP `410 (Gone)` status code if a client requests a resourceVersion older than the server has retained. Clients must be able to tolerate `410 (Gone)` responses. See [Efficient detection of changes](#efficient-detection-of-changes) for details on how to handle `410 (Gone)` responses when watching resources.

		For example, the kube-apiserver periodically compacts old resource versions from etcd based on its `--etcd-compaction-interval` setting. Also, the kube-apiserver's watch cache keeps `--watch-cache-sizes` resource versions in each resource cache. It depends on if a request is served from cache on which one of these limits applies, but if a resource version is unavailable in the one that applies, a `410 (Gone)` will be returned by the kube-apiserver.

Add resource version section to api-concepts documentation #16910

Add resource version section to api-concepts documentation #16910

Conversation

jpbetz commented Oct 15, 2019 • edited Loading

k8sio-netlify-preview-bot commented Oct 15, 2019 • edited Loading

netlify bot commented Oct 15, 2019

jpbetz commented Oct 24, 2019

jpbetz commented Nov 17, 2019

daminisatya commented Nov 27, 2019

wojtek-t left a comment

Choose a reason for hiding this comment

daminisatya commented Nov 29, 2019

jpbetz commented Dec 2, 2019

wojtek-t commented Dec 2, 2019

daminisatya commented Dec 3, 2019

k8s-ci-robot commented Dec 3, 2019

liggitt Dec 3, 2019 • edited Loading

Choose a reason for hiding this comment

jpbetz Dec 10, 2019

Choose a reason for hiding this comment

liggitt Dec 3, 2019

Choose a reason for hiding this comment

liggitt Dec 3, 2019

Choose a reason for hiding this comment

liggitt Dec 3, 2019 • edited Loading

Choose a reason for hiding this comment

liggitt Dec 3, 2019

Choose a reason for hiding this comment

jpbetz Dec 10, 2019

Choose a reason for hiding this comment

liggitt Dec 3, 2019

Choose a reason for hiding this comment

jpbetz commented Dec 10, 2019 • edited Loading

jpbetz commented Oct 15, 2019 •

edited

Loading

k8sio-netlify-preview-bot commented Oct 15, 2019 •

edited

Loading

liggitt Dec 3, 2019 •

edited

Loading

liggitt Dec 3, 2019 •

edited

Loading

jpbetz commented Dec 10, 2019 •

edited

Loading