-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates for Windows Server version 1709 with K8s v1.8 #6180
Updates for Windows Server version 1709 with K8s v1.8 #6180
Conversation
Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA. It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Deploy preview ready! Built with commit 6a6922c https://deploy-preview-6180--kubernetes-io-master-staging.netlify.com |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Love this as an introduction to the nitty-gritty users'll face when they read the guide. :) 👍
Review is mostly just minor organizational and formatting suggestions!
|
||
### Linux | ||
The above networking approach is already supported on Linux using a bridge interface, which essentially creates a private network local to the node. Similar to the Windows side, routes to all other pod CIDRs must be created in order to send packets via the "public" NIC. | ||
|
||
### Windows | ||
Windows supports the CNI network model and uses plugins to interface with the Windows Host Networking Service (HNS) to configure host networking and policy. An administrator creates a local host network using HNS PowerShell commands on each node as documented in the **_Windows Host Setup_** section below. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can just link to it directly via [Windows Host Setup](#windows-host-setup)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added direct link in JMesser81#1
Windows supports the CNI network model and uses plugins to interface with the Windows Host Networking Service (HNS) to configure host networking and policy. An administrator creates a local host network using HNS PowerShell commands on each node as documented in the **_Windows Host Setup_** section below. | ||
|
||
#### Upstream L3 Routing Topology | ||
In this topology, networking is achieved using L3 routing with static IP routes configured in an upstream Top of Rack (ToR) switch/router. Each cluster node is connected to the management network with a host IP. Additionally, each node uses a local 'l2bridge' network with a Pod CIDR assigned. All pods on a given worker node will be connected to the POD CIDR subnet ('l2bridge' network). In order to enable network communication between pods running on different nodes. The upstream router has static routes configured with POD CIDR prefix => Host IP. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kind of a nit-pick, but there's 3 different capitalizations of "pod" here; just say "pod."
Similarly nit-picking, there should be a comma instead of a period here:
...running on different nodes [here]. The upstream router has static...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This entire section is always true, not just for ToR routing:
Each cluster node is connected to the management network with a host IP. Additionally, each node uses a local 'l2bridge' network with a Pod CIDR assigned. All pods on a given worker node will be connected to the POD CIDR subnet ('l2bridge' network). In order to enable network communication between pods running on different nodes.
I recommend it be moved to somewhere more generic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed pod/POD/Pod nit in JMesser81#1
2. Kubenet network plugin setup on the Linux nodes. | ||
3. Windows Server 2016 (RTM version 10.0.14393 or later). | ||
4. Docker Version 1.12.2-cs2-ws-beta or later for Windows Server nodes (Linux nodes and Kubernetes control plane can run any Kubernetes supported Docker Version). | ||
3. Windows Server version 1709 (RTM version 10.0.16299.15 or later). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should temporarily mention the fact that these can't be Core builds until kubernetes/kubernetes#55031 is fixed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
v1.9.0-beta.1 has been released containing the fix for this issue.
## CNI Plugins | ||
Microsoft plans to publish code for two CNI plugins - win-l2bridge (host-gateway) and win-overlay (vxlan)) - per this [issue](https://github.com/containernetworking/plugins/issues/80). These two CNI plugins can either be used directly by WinCNI.exe or with Flannel [PR 832](https://github.com/coreos/flannel/pull/832). We have an [outstanding informational PR](https://github.com/containernetworking/plugins/pull/85) needed to complete this work. Windows Server platform work is complete. | ||
|
||
The selection of which network configuration and topology to deploy depends on the physical network topolgy and a user's ability to configure routes, performance concerns with encapsulation, and requirement to integrate with third-party network plugins. | ||
|
||
### Linux | ||
The above networking approach is already supported on Linux using a bridge interface, which essentially creates a private network local to the node. Similar to the Windows side, routes to all other pod CIDRs must be created in order to send packets via the "public" NIC. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This section doesn't seem relevant anymore here ("above networking approach" refers to?)? It seems like it belongs in either the host-gateway or upstream sections. Maybe it just needs clarification.
3. RRAS (Routing) Windows feature enabled - Allows routing between NICs on the box, and also "captures" packets that have the destination IP of a POD running on the node. To enable, open "Server Manager". Click on "Roles", "Add Roles". Click "Next". Select "Network Policy and Access Services". Click on "Routing and Remote Access Service" and the underlying checkboxes. | ||
4. Routes defined pointing to the other pod CIDRs via the "public" NIC - These routes are added to the built-in routing table as shown in **_Route Setup_** section below. | ||
The following diagram illustrates the Windows Server networking setup for Kubernetes using Upstream L3 Routing Setup: | ||
![K8s Cluster using L3 Routing with ToR](UpstreamRouting.png) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
freaking awesome diagram.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That diagram really helps to convey the setup
|
||
Note: this file assumes that a user previous created 'l2bridge' host networks on each Windows node using <Verb>-HNSNetwork cmdlets as shown in the start-kublet.ps1 and start-kubeproxy.ps1 scripts linked above |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: minor grammar issues, also <Verb>
is getting swallowed by the viewer (as HTML, I think?) so it just says "using -HNSNetwork." Escape it or wrap it in back-ticks. I'd wrap the script names in back-ticks too, for aesthetics :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ensure start-kublet.ps1
is changed to start-kubelet.ps1
, since the file was originally start-kublet.ps1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed these in JMesser81#1
} | ||
} | ||
] | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could run the JSON through this linter so that the spacing is prettier.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see the issue. There's a newline missing right before the code block.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
2. Run *kube-proxy* executable using the below command | ||
`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>` | ||
## Starting the Linux-based Control Plane | ||
Use your preferred method to setup and start Kubernetes cluster on Linux or follow the directions given in this [link](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md). Please note that Cluster CIDR might need to be updated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"...since it's hard-coded to 11.0.0.0/16
in the scripts." Should we make that parameterized?
Also, though it's in the linked guide, reiterate the necessity of copying the certificate configuration file from the master.
2. Run *kube-proxy* executable using the below command | ||
`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>` | ||
## Starting the Linux-based Control Plane | ||
Use your preferred method to setup and start Kubernetes cluster on Linux or follow the directions given in this [link](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md). Please note that Cluster CIDR might need to be updated. | ||
|
||
## Scheduling Pods on Windows | ||
Because your cluster has both Linux and Windows nodes, you must explicitly set the nodeSelector constraint to be able to schedule Pods to Windows nodes. You must set nodeSelector with the label beta.kubernetes.io/os to the value windows; see the following example: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: backticks on nodeSelector
and the label, pods caps
Also, you can write
```yaml
[the YAML file]
```
to get syntax highlighting! Likewise, use json
for the CNI config file. 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update: looks like they actually auto-detected the YAML it in the deployment, but the CNI config is broken for some reason. Either way, it'll make the GitHub preview prettier.
|
||
## Networking | ||
Network is achieved using L3 routing. Because third-party networking plugins (e.g. flannel, calico, etc) don't natively work on Windows Server, existing technology that is built into the Windows and Linux operating systems is relied on. In this L3 networking approach, a /16 subnet is chosen for the cluster nodes, and a /24 subnet is assigned to each worker node. All pods on a given worker node will be connected to the /24 subnet. This allows pods on the same node to communicate with each other. In order to enable networking between pods running on different nodes, routing features that are built into Windows Server 2016 and Linux are used. | ||
There are several supported network configurations with Windows Server version 1709 and K8s v1.8 including both Layer-3 routed and overlay topologies using third-party network plugins. | ||
1. Upstream L3 Routing - IP routes configured in upstream ToR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Direct links to each section? :D
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added direct links in JMesser81#1
I didn't see an easy way to find the preview of this page, so here it is: https://deploy-preview-6180--kubernetes-io-master-staging.netlify.com/docs/getting-started-guides/windows/ Maybe this is obvious for those that have updated the docs previously |
|
||
**Note:** Windows Server Containers on Kubernetes is an Alpha feature in Kubernetes 1.8. | ||
|
||
**Note:** There is one outstanding PR ([51063 Fixes to enable Windows CNI](https://github.com/kubernetes/kubernetes/pull/51063))which has not been merged into v1.8 and is required for Windows CNI to work with kubelet. Users will need to build a private kubelet binary to consume this change. Please refer to these [instructions](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md) for build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this it's necessary anymore, the PR got merged in v1.9, we can make v1.9 as the minimum requirement.
Anyway, if you really want, I would rather like to see the binaries pre-built if we want to provide them for kubernetes v1.8. Also in the instructions you are building from K8SREPO="github.com/madhanrm/kubernetes" which seem to have quite a lot of differences between what's upstream on kubernetes v1.8.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The note about 51063 Fixes to enable Windows CNI has been removed in PR to this branch JMesser81#1
It also makes mention and links to the releases page where pre-built binaries can be found. It does not address the K8SREPO referencing non-mainline K8S.
2. Kubenet network plugin setup on the Linux nodes. | ||
3. Windows Server 2016 (RTM version 10.0.14393 or later). | ||
4. Docker Version 1.12.2-cs2-ws-beta or later for Windows Server nodes (Linux nodes and Kubernetes control plane can run any Kubernetes supported Docker Version). | ||
3. Windows Server version 1709 (RTM version 10.0.16299.15 or later). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It shouldn't be a hard requirement to have Windows Server version 1709. The OVN/OVS CNI plugin will also support the previous versions of Windows with the limitation of one container per pod. You will be able to have mixed nodes in the same deployment with Windows Server 2016 and Windows server version 1709.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
|
||
## Networking | ||
Network is achieved using L3 routing. Because third-party networking plugins (e.g. flannel, calico, etc) don't natively work on Windows Server, existing technology that is built into the Windows and Linux operating systems is relied on. In this L3 networking approach, a /16 subnet is chosen for the cluster nodes, and a /24 subnet is assigned to each worker node. All pods on a given worker node will be connected to the /24 subnet. This allows pods on the same node to communicate with each other. In order to enable networking between pods running on different nodes, routing features that are built into Windows Server 2016 and Linux are used. | ||
There are several supported network configurations with Windows Server version 1709 and K8s v1.8 including both Layer-3 routed and overlay topologies using third-party network plugins. | ||
1. Upstream L3 Routing - IP routes configured in upstream ToR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think a new line is needed before 1, otherwise it won't list them properly. Preview link: https://deploy-preview-6180--kubernetes-io-master-staging.netlify.com/docs/getting-started-guides/windows/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added in JMesser81#1
There are several supported network configurations with Windows Server version 1709 and K8s v1.8 including both Layer-3 routed and overlay topologies using third-party network plugins. | ||
1. Upstream L3 Routing - IP routes configured in upstream ToR | ||
2. Host-Gateway - IP routes configured on each host | ||
3. OVN & OVS with Overlay - OVS switch extension and OVN controller creates VXLAN overlay network |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Direct links for this: https://github.com/apprenda/kubernetes-ovn-heterogeneous-cluster and https://github.com/openvswitch/ovn-kubernetes
Supported tunnel types for connecting nodes with overlay networks with OVN are Geneve and STT (doesn't need special NIC to offload encapsulation). Gateways may use geneve, vxlan, or stt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
4. [Future] Overlay - VXLAN or IP-in-IP encapsulation using Flannel | ||
5. [Future] Layer-3 Routing with BGP (Calico) | ||
|
||
## CNI Plugins |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other CNI plugins should be listed here as well, such as https://github.com/openvswitch/ovn-kubernetes and https://github.com/apprenda/kubernetes-ovn-heterogeneous-cluster
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
2. Build *kube-proxy* | ||
1. Linux/MacOS: `KUBE_BUILD_PLATFORMS=windows/amd64 make WHAT=cmd/kube-proxy` | ||
2. Windows: `go build cmd/kube-proxy/proxy.go` | ||
2. Configure Linux Master node using steps [here](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are different steps on how to setup the Linux master node depending on the chosen network topology. For the OVN/OVS case, the following guides can be used: https://github.com/apprenda/kubernetes-ovn-heterogeneous-cluster/tree/master/master#master-node and https://github.com/openvswitch/ovn-kubernetes#k8s-master-node-initialization
We should have different sections for Host Setup based on the network topology chosen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
|
||
## Setting up Windows Server Containers on Kubernetes | ||
To run Windows Server Containers on Kubernetes, you'll need to set up both your host machines and the Kubernetes node components for Windows and setup Routes for Pod communication on different nodes. | ||
To run Windows Server Containers on Kubernetes, you'll need to set up both your host machines and the Kubernetes node components for Windows and depending on your network topology, setup Routes for Pod communication on different nodes. | ||
|
||
### Host Setup |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest having different sections for Host Setup
based on the network topology chosen. The steps on how to setup the hosts could be different for each network topology.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
} | ||
} | ||
] | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Different sections are definitely needed. This does not apply to all CNI plugins.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
2. Run *kube-proxy* executable using the below command | ||
`.\proxy.exe --v=3 --proxy-mode=userspace --hostname-override=<ip address/hostname of the windows node> --master=<api server location> --bind-address=<ip address of the windows node>` | ||
## Starting the Linux-based Control Plane | ||
Use your preferred method to setup and start Kubernetes cluster on Linux or follow the directions given in this [link](https://github.com/Microsoft/SDN/blob/master/Kubernetes/HOWTO-on-prem.md). Please note that Cluster CIDR might need to be updated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Different sections needed here as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in JMesser81#1
} | ||
} | ||
] | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file should contain in the filename the network topology for which it applies. The filename suggests that this is valid for all CNI plugins.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
renamed to sample-l2bridge-wincni-config.json
Workaround for Jekyllr frontmatter
the user should use [::1] for the localhost
@JMesser81 i think the formatting is off now. something happened after |
@michmike - i'm not seeing any weird formatting when I render here: https://deploy-preview-6180--kubernetes-io-vnext-staging.netlify.com/docs/getting-started-guides/windows/ That section looks fine to me, can you be more specific? |
@JMesser81 the deploy preview does look perfect. i was looking at this view which in the past (up to yesterday) was accurate |
Reviewed 1 of 4 files at r1, 6 of 3034 files at r2, 2 of 3 files at r3, 1 of 1 files at r9. Comments from Reviewable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Huge update! The diagrams are amazing. Only a handful of minor edits for syntax--well done! 🌟
--- | ||
**Note:** These instructions were recently updated based on Windows Server platform enhancements and the Kubernetes v1.9 release |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Note:** These instructions were recently updated based on Windows Server platform enhancements and the Kubernetes v1.9 release.
{: .note}
So:
Note: These instructions were recently updated based on Windows Server platform enhancements and the Kubernetes v1.9 release.
{: .note}
**Note:** Windows Server Containers on Kubernetes is an Alpha feature in Kubernetes 1.5. | ||
The Kubernetes control plane (API Server, Scheduler, Controller Manager, etc) continue to run on Linux, while the kubelet and kube-proxy can be run on Windows Server 2016 or later | ||
|
||
**Note:** Windows Server Containers on Kubernetes is a Beta feature in Kubernetes v1.9 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Note:** Windows Server Containers on Kubernetes is a Beta feature in Kubernetes v1.9.
{: .note}
So:
Note: Windows Server Containers on Kubernetes is a Beta feature in Kubernetes v1.9.
{: .note}
# You will find the output binaries under the folder _output/local/bin/windows/ | ||
``` | ||
|
||
More detailed build instructions will be maintained and kept up to date [here](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/virtualization/windowscontainers/kubernetes/compiling-kubernetes-binaries.md) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a period at the end.
More detailed build instructions will be maintained and kept up to date [here](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/virtualization/windowscontainers/kubernetes/compiling-kubernetes-binaries.md).
|
||
**kubelet** | ||
More detailed instructions can be found [here](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/virtualization/windowscontainers/kubernetes/getting-started-kubernetes-windows.md) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a period at the end.
More detailed instructions can be found [here](https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/live/virtualization/windowscontainers/kubernetes/getting-started-kubernetes-windows.md).
thanks for the review @zacharysarah ! |
@JMesser81 👋 If you'll un-check and re-check the box to allow edits from maintainers (it's a workaround for a broken GitHub UI component), I'm happy to make edits for you. Otherwise, please go ahead and make the changes yourself. 👍 |
Sorry for the delay. Addressed comments. Thanks for the review! |
* Trivial change to open release branch * Undo trivial change * add service ipvs overview * Add instructions on how to setup kubectl * Document conntrack dependency for kube-proxy * Add an a This is kind of jarring / missing an article. I'm guessing it should either be ' to a rack of bare metal servers.' or '...to racks of bare metal servers.'. * adding example responses for common issues - support request - code bug report * Trivial change to open release branch * Undo trivial change * Signed-off-by: Ziqi Zhao <zhaoziqi@qiniu.com> (#5366) Fix the not-working test case yaml for /doc/concepts/storage/volumes.md * kubectl-overview * temp fix for broken pod and deployment links * Update Table of Solutions for Juju * Revise certificates documentation (#5965) * Update review-issues.md Some edits for clarity and condensed language. * Update init-containers.md Fix leading spaces in commands. * Update kubectl-overview.md Fix format. * Update clc.md Fix format. * Update openstack-heat.md The url no need. just highlight. * Typo I believe this should be "users" not "uses" * making explicit hostname uniq requirement * Update scheduling-hugepages.md * Update update-daemon-set.md * fix redirection of PersistentVolume * Update hpa.md * update kubectl instruction * Use the format of kubeadm init * fix spelling error guarnatees to guarantees * add matchLabels description (#6020) * search and replace for k8s.github.io to website (#6019) * fix scale command of object-management (#6011) * Update replicaset.md (#6009) * Update secret.md (#6008) * specify password for mysql image (#5990) * specify password for mysql image * specify password for mysql image * link error for run-stateless-application-deployment.md (#5985) * link error for run-stateless-application-deployment.md * link error for run-stateless-application-deployment.md * Add performance implications of inter-pod affinity/anti-affinity (#5979) * 404 monthly maintenance - October 2017 (#5977) * Updated redirects * More redirects * Add conjure-up to Turnkey Cloud Solutions list (#5973) * Add conjure-up to Turnkey Cloud Solutions list * Changed wording slightly * change the StatefulSet to ReplicaSet in reference (#5968) * Clarification of failureThreshold of probes (#5963) * Mention usage of block storage version param (#5925) Mention usage of block storage version (bs-version) parameter to workaround attachment issues using older K8S versions on an OpenStack cloud with path-based endpoints. Resolves: #5924 * Update sysctl-cluster.md (#5894) Include guide on enabling unsafe sysctls in minikube * Avoid Latin phrases & format note (#5889) * Avoid Latin phrases & format note according the Documentation Style Guide * Update scratch.md * Update scratch.md * resolves jekyll rendering error (#5976) - chinese isn't understood for keys in YAML frontmatter in jekyll, so replaced it with the english equivalent that doesn't throw the following error on rendering: Error reading file src/kubernetes.github.io/cn/docs/concepts/cluster-administration/device-plugins.md: (<unknown>): could not find expected ':' while scanning a simple key at line 4 column 1 * Change VM to pod. (#6022) * Add link to custom metrics. (#6023) * Rephrase core group. (#6024) * Added explanation on context to when joining (#6018) * Update create-cluster-kubeadm.md (#5761) Update Canal version in pod network apply commands * Fixes issue #5620 (#5869) * Fixes issue #5620 Signed-off-by: Brad Topol <btopol@us.ibm.com> * Restructured so that review process is for both current and upcoming releases. Added content describing the use of tech reviewers. * Removed incorrect Kubernetes reviewer link. * Fixed tech reviewer URL to now use website * Update pod-priority-preemption.md fix-wrong-link-to-pod-preemption * pod-security-policy.md: add links to the page about admission plugins. * Adding all files for BlaBlaCar case study (#5857) * Adding all files for BlaBlaCar case study * Update blablacar.html * Fix changed URL for google containers * Add /docs/reference/auto-generated directory * correct the downwardapi redirect * Remove links using "here" * Rename to /docs/reference/generated directory * add Concept template * Change title to just Ingress * Link mistake (#6038) * link mistake * link mistake * skip title check for skip_title_check.txt * skip title check for skip_title_check.txt * remove doesn't exist link. * Fix podpreset task (#5705) * Add a simple pod manifest to pod overview (#5986) * Split PodPreset concept out from task doc (#5984) * Add selector spec description (#5789) * Add selector spec description * Fix selector field explanation * Put orphaned topics in TOC. (#6051) * static-pod example bad format in the final page (#6050) * static-pod example bad format in the final page * static-pod example bad format in the final page * static-pod example bad format in the final page * static-pod example bad format in the final page * static-pod example bad format in the final page * Fix `backoffLimit` field misplacement (#6042) It should be placed in JobSpec according to: https://github.com/kubernetes/kubernetes/blob/master/api/swagger-spec/batch_v1.json#L1488-L1514 * Update addons.md (#6061) * add info about VMware NSX-T CNI plugin (#5987) * add info about VMware NSX-T CNI plugin Hello, I'm VMware Networking and Security Architect and would like to include short information about our CNI plugin implementation similar to what other vendors did Best regards Emil Gagala * Update networking.md * Update networking.md * Update networking.md * Update: Using universal zsh configuration (#5669) * Update install-kubectl.md Zsh is not only oh-my-zsh, so I added universal configuration for zsh that also can be used in prezto. * fix merge error after rebase * Operating etcd cluster for Kubernetes bad format in the final page (#6056) * Operating etcd cluster for Kubernetes bad format in the final page * Update configure-upgrade-etcd.md * Update configure-upgrade-etcd.md * Usage note and warning tags. (#6053) * Usage note and warning tags. * Update configure-upgrade-etcd.md * Update configure-upgrade-etcd.md * Document jekyll includes snippets * Add jekyll includes to docs home toc - Remove extra kubernetes home in toc * document docker cgroupdriver req (#5937) * Update test blacklists (#6063) * Update toc check blacklist * Update title check blacklist * wip * wip * Fix typo * Document unconfined apparmor profile * Revert "Document the unconfined profile for AppArmor" (#6268) * CRD Validation: remove alpha warning, change enable instructions to (#6066) disable * Documented service annotation for AWS ELB SSL policy * kubeadm: add a note about the new `--print-join-command` flag. This is a new flag for the `kubeadm token create` command. * Add a note to PDB page * Improve Kubeadm reference doc (#6103) * automatically-generated kubeadm reference doc * user-mantained kubeadm reference doc * Documentation for CSIPersistentVolume * change replicaset documentation to use apps/v1 APIs * Update service.md ipvs alpha version -> beta version * Updated Deployment concept docs (#6494) * Updated Deployment concept docs * Addressed comments * Documentation for volume scheduling alpha feature * Update admission control docs for webhooks * Improve DNS documentation (#6479) * update ds for 1.9 * Update service.md * Update service.md * Revert "begin updating webhook documentation" (#6575) * Update version numbers to include 1.9 (#6518) * Update site versions for 1.9 * Removed 1.4 docs * Update _config.yml * Update _config.yml * updates for raw block devices * rbac: docs for aggregated cluster roles (#6474) * Added IPv6 information for Kubelet arguments (#6498) * Added IPv6 info to kube-proxy arguments * Added IPv6 information for argument for kubelet * Update PVC resizing documentation (#6487) * Updates for Windows Server version 1709 with K8s v1.8 (#6180) * Updated for WSv1709 and K8s v1.8 * Updated picture and CNI config * Fixed formatting on CNI Config * Updated docs to reference Microsoft/SDN GitHub docs * fix typo * Workaround for Jekyllr frontmatter * Added section on features and limitations, with example yaml files. * Update index.md * Added kubeadm section, few other small fixes * Few minor grammar fixes * Update access-cluster.md with a comment that for IPv6 the user should use [::1] for the localhost * Addressed a number of issues brought up against the base PR * Fixed windows-host-setup link * Rewrite PodSecurityPolicy guide * Update index.md Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com> Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org> * Spelling correction and sentence capitalization. - Corrected the spelling error for storing, was put in as 'stoing'. - Capitalized list items. - Added '.' at end of sentences in the list items. * Update index.md * Update index.md * Addressed comments and rebased * Fixed formatting * Fixed formatting * Updated header link * Updated hyperlinks * Updated warning * formatting * formatting * formatting * Revert "Update access-cluster.md with a comment that for IPv6" This reverts commit 31e4dbd. * Revert "fix typo" This reverts commit c056787. * Revert "Workaround for Jekyllr frontmatter" This reverts commit b84ac59. * Fixed grammatical issues and reverted non-related commits * Revert "Rewrite PodSecurityPolicy guide" This reverts commit 5d39cfe. * Revert "Spelling correction and sentence capitalization." This reverts commit 47eed43. * Fixed auto-numbering * Minor formatting updates * CoreDNS feature documentation (#6463) * Initial placeholder PR for CoreDNS feature documentation * Remove from admin, add content * Fix missing endcapture * Add to tasks.yml * Review feedback * Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod (#6415) * Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod A new feature PVC Protection was added into K8s 1.9 that's why this documentation change is needed. * Added tag at the top of each new area. * Fix typo * Fix: switched on in (all kubelets) -> (all K8s components). * Added link to admission controller * Moved PVC Protection configuration into Before you begin section. * Added steps how to verify PVC Protection feature. * Fixes for admission controller plugin description and for PVC Protection description in PVC lifecycle. * Testing official rendering of enumerations (1., 2., 3., etc.) * Re-write to address comments from review. * Fixed definition when a PVC is in active use by a pod. * Change auditing docs page for 1.9 release (#6427) * Change auditing docs page for 1.9 release Signed-off-by: Mik Vyatskov <vmik@google.com> * Address review comments Signed-off-by: Mik Vyatskov <vmik@google.com> * Address review comments Signed-off-by: Mik Vyatskov <vmik@google.com> * Address review comments Signed-off-by: Mik Vyatskov <vmik@google.com> * Fix broken link Signed-off-by: Mik Vyatskov <vmik@google.com> * short circuit deny docs (#6536) * line wrap * short circuit deny * address comments * Add kubeadm 1.9 upgrade docs (#6485) * kubeadm: Improve kubeadm documentation for v1.9 (#6645) * Update admission control docs for webhooks (re-send #6368) (#6650) * Update admission control docs for webhooks * update in response to comments * Revamp rkt and add CRI-O as alternative runtime (#6371) Signed-off-by: Lorenzo Fontana <lo@linux.com> * Documented NLB for Kubernetes 1.9 (#6260) * Added IPV6 information to setup cluster using kubeadm (#6465) * Added IPV6 information to setup cluster using kubeadm * Updated kubeadm.md & create-cluster-kubeadm.md with IPv6 related information * Added IPv6 options for kubeadm --init & automated address binding for kube-proxy based on version of IP configured for API server) * Changes to kubeadm.md as per comments * Modified kubeadm.md and create-cluster-kubeadm.md * Implemented changes requested by zacharysarah * Removed autogenerated kubeadm.md changes * StatefulSet 1.9 updates. (#6550) * updates sts concept and tutorials to use 1.9 apps/v1 * Update statefulset.md * clarify pod name label * Garbage collection updates for 1.9 (#6555) * 1.9 gc policy update * carify deletion * Couple nits for dnsConfig doc (#6652) * Add doc for AllowedFlexVolume (#6563) * Update OpenStack Cloud Provider API support for v1.9 (#6638) * Flex volume is GA. Remove alpha notation. (#6666) * Update generated ref docs for Kubernetes and Federation components. (#6658) * Update generated ref docs for Kubernetes and Federation components. * Rename kubectl-options to kubectl. * Add title to kubectl. * Fix double synopsis. * Update Federation API ref docs for 1.9. (#6636) * Update federation API ref docs. * Move and redirect. * Move generated Federation docs to the generated directory. * Fix titles. * Type * Fix titles * Update auto-generated Kubernetes APi ref docs. (#6646) * Update kubectl commands for 1.9 (#6635) * add ExtendedResourceToleration admission controller (#6618) * Update API reference paths for v1.9 (#6681)
* Trivial change to open release branch * Undo trivial change * add service ipvs overview * Add instructions on how to setup kubectl * Document conntrack dependency for kube-proxy * Add an a This is kind of jarring / missing an article. I'm guessing it should either be ' to a rack of bare metal servers.' or '...to racks of bare metal servers.'. * adding example responses for common issues - support request - code bug report * Trivial change to open release branch * Undo trivial change * Signed-off-by: Ziqi Zhao <zhaoziqi@qiniu.com> (#5366) Fix the not-working test case yaml for /doc/concepts/storage/volumes.md * kubectl-overview * temp fix for broken pod and deployment links * Update Table of Solutions for Juju * Revise certificates documentation (#5965) * Update review-issues.md Some edits for clarity and condensed language. * Update init-containers.md Fix leading spaces in commands. * Update kubectl-overview.md Fix format. * Update clc.md Fix format. * Update openstack-heat.md The url no need. just highlight. * Typo I believe this should be "users" not "uses" * making explicit hostname uniq requirement * Update scheduling-hugepages.md * Update update-daemon-set.md * fix redirection of PersistentVolume * Update hpa.md * update kubectl instruction * Use the format of kubeadm init * fix spelling error guarnatees to guarantees * add matchLabels description (#6020) * search and replace for k8s.github.io to website (#6019) * fix scale command of object-management (#6011) * Update replicaset.md (#6009) * Update secret.md (#6008) * specify password for mysql image (#5990) * specify password for mysql image * specify password for mysql image * link error for run-stateless-application-deployment.md (#5985) * link error for run-stateless-application-deployment.md * link error for run-stateless-application-deployment.md * Add performance implications of inter-pod affinity/anti-affinity (#5979) * 404 monthly maintenance - October 2017 (#5977) * Updated redirects * More redirects * Add conjure-up to Turnkey Cloud Solutions list (#5973) * Add conjure-up to Turnkey Cloud Solutions list * Changed wording slightly * change the StatefulSet to ReplicaSet in reference (#5968) * Clarification of failureThreshold of probes (#5963) * Mention usage of block storage version param (#5925) Mention usage of block storage version (bs-version) parameter to workaround attachment issues using older K8S versions on an OpenStack cloud with path-based endpoints. Resolves: #5924 * Update sysctl-cluster.md (#5894) Include guide on enabling unsafe sysctls in minikube * Avoid Latin phrases & format note (#5889) * Avoid Latin phrases & format note according the Documentation Style Guide * Update scratch.md * Update scratch.md * resolves jekyll rendering error (#5976) - chinese isn't understood for keys in YAML frontmatter in jekyll, so replaced it with the english equivalent that doesn't throw the following error on rendering: Error reading file src/kubernetes.github.io/cn/docs/concepts/cluster-administration/device-plugins.md: (<unknown>): could not find expected ':' while scanning a simple key at line 4 column 1 * Change VM to pod. (#6022) * Add link to custom metrics. (#6023) * Rephrase core group. (#6024) * Added explanation on context to when joining (#6018) * Update create-cluster-kubeadm.md (#5761) Update Canal version in pod network apply commands * Fixes issue #5620 (#5869) * Fixes issue #5620 Signed-off-by: Brad Topol <btopol@us.ibm.com> * Restructured so that review process is for both current and upcoming releases. Added content describing the use of tech reviewers. * Removed incorrect Kubernetes reviewer link. * Fixed tech reviewer URL to now use website * Update pod-priority-preemption.md fix-wrong-link-to-pod-preemption * pod-security-policy.md: add links to the page about admission plugins. * Adding all files for BlaBlaCar case study (#5857) * Adding all files for BlaBlaCar case study * Update blablacar.html * Fix changed URL for google containers * Add /docs/reference/auto-generated directory * correct the downwardapi redirect * Remove links using "here" * Rename to /docs/reference/generated directory * add Concept template * Change title to just Ingress * Link mistake (#6038) * link mistake * link mistake * skip title check for skip_title_check.txt * skip title check for skip_title_check.txt * remove doesn't exist link. * Fix podpreset task (#5705) * Add a simple pod manifest to pod overview (#5986) * Split PodPreset concept out from task doc (#5984) * Add selector spec description (#5789) * Add selector spec description * Fix selector field explanation * Put orphaned topics in TOC. (#6051) * static-pod example bad format in the final page (#6050) * static-pod example bad format in the final page * static-pod example bad format in the final page * static-pod example bad format in the final page * static-pod example bad format in the final page * static-pod example bad format in the final page * Fix `backoffLimit` field misplacement (#6042) It should be placed in JobSpec according to: https://github.com/kubernetes/kubernetes/blob/master/api/swagger-spec/batch_v1.json#L1488-L1514 * Update addons.md (#6061) * add info about VMware NSX-T CNI plugin (#5987) * add info about VMware NSX-T CNI plugin Hello, I'm VMware Networking and Security Architect and would like to include short information about our CNI plugin implementation similar to what other vendors did Best regards Emil Gagala * Update networking.md * Update networking.md * Update networking.md * Update: Using universal zsh configuration (#5669) * Update install-kubectl.md Zsh is not only oh-my-zsh, so I added universal configuration for zsh that also can be used in prezto. * fix merge error after rebase * Operating etcd cluster for Kubernetes bad format in the final page (#6056) * Operating etcd cluster for Kubernetes bad format in the final page * Update configure-upgrade-etcd.md * Update configure-upgrade-etcd.md * Usage note and warning tags. (#6053) * Usage note and warning tags. * Update configure-upgrade-etcd.md * Update configure-upgrade-etcd.md * Document jekyll includes snippets * Add jekyll includes to docs home toc - Remove extra kubernetes home in toc * document docker cgroupdriver req (#5937) * Update test blacklists (#6063) * Update toc check blacklist * Update title check blacklist * wip * wip * Fix typo * Document unconfined apparmor profile * Revert "Document the unconfined profile for AppArmor" (#6268) * CRD Validation: remove alpha warning, change enable instructions to (#6066) disable * Documented service annotation for AWS ELB SSL policy * kubeadm: add a note about the new `--print-join-command` flag. This is a new flag for the `kubeadm token create` command. * Add a note to PDB page * Improve Kubeadm reference doc (#6103) * automatically-generated kubeadm reference doc * user-mantained kubeadm reference doc * Documentation for CSIPersistentVolume * change replicaset documentation to use apps/v1 APIs * Update service.md ipvs alpha version -> beta version * Updated Deployment concept docs (#6494) * Updated Deployment concept docs * Addressed comments * Documentation for volume scheduling alpha feature * Update admission control docs for webhooks * Improve DNS documentation (#6479) * update ds for 1.9 * Update service.md * Update service.md * Revert "begin updating webhook documentation" (#6575) * Update version numbers to include 1.9 (#6518) * Update site versions for 1.9 * Removed 1.4 docs * Update _config.yml * Update _config.yml * updates for raw block devices * rbac: docs for aggregated cluster roles (#6474) * Added IPv6 information for Kubelet arguments (#6498) * Added IPv6 info to kube-proxy arguments * Added IPv6 information for argument for kubelet * Update PVC resizing documentation (#6487) * Updates for Windows Server version 1709 with K8s v1.8 (#6180) * Updated for WSv1709 and K8s v1.8 * Updated picture and CNI config * Fixed formatting on CNI Config * Updated docs to reference Microsoft/SDN GitHub docs * fix typo * Workaround for Jekyllr frontmatter * Added section on features and limitations, with example yaml files. * Update index.md * Added kubeadm section, few other small fixes * Few minor grammar fixes * Update access-cluster.md with a comment that for IPv6 the user should use [::1] for the localhost * Addressed a number of issues brought up against the base PR * Fixed windows-host-setup link * Rewrite PodSecurityPolicy guide * Update index.md Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com> Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org> * Spelling correction and sentence capitalization. - Corrected the spelling error for storing, was put in as 'stoing'. - Capitalized list items. - Added '.' at end of sentences in the list items. * Update index.md * Update index.md * Addressed comments and rebased * Fixed formatting * Fixed formatting * Updated header link * Updated hyperlinks * Updated warning * formatting * formatting * formatting * Revert "Update access-cluster.md with a comment that for IPv6" This reverts commit 31e4dbd. * Revert "fix typo" This reverts commit c056787. * Revert "Workaround for Jekyllr frontmatter" This reverts commit b84ac59. * Fixed grammatical issues and reverted non-related commits * Revert "Rewrite PodSecurityPolicy guide" This reverts commit 5d39cfe. * Revert "Spelling correction and sentence capitalization." This reverts commit 47eed43. * Fixed auto-numbering * Minor formatting updates * CoreDNS feature documentation (#6463) * Initial placeholder PR for CoreDNS feature documentation * Remove from admin, add content * Fix missing endcapture * Add to tasks.yml * Review feedback * Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod (#6415) * Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod A new feature PVC Protection was added into K8s 1.9 that's why this documentation change is needed. * Added tag at the top of each new area. * Fix typo * Fix: switched on in (all kubelets) -> (all K8s components). * Added link to admission controller * Moved PVC Protection configuration into Before you begin section. * Added steps how to verify PVC Protection feature. * Fixes for admission controller plugin description and for PVC Protection description in PVC lifecycle. * Testing official rendering of enumerations (1., 2., 3., etc.) * Re-write to address comments from review. * Fixed definition when a PVC is in active use by a pod. * Change auditing docs page for 1.9 release (#6427) * Change auditing docs page for 1.9 release Signed-off-by: Mik Vyatskov <vmik@google.com> * Address review comments Signed-off-by: Mik Vyatskov <vmik@google.com> * Address review comments Signed-off-by: Mik Vyatskov <vmik@google.com> * Address review comments Signed-off-by: Mik Vyatskov <vmik@google.com> * Fix broken link Signed-off-by: Mik Vyatskov <vmik@google.com> * short circuit deny docs (#6536) * line wrap * short circuit deny * address comments * Add kubeadm 1.9 upgrade docs (#6485) * kubeadm: Improve kubeadm documentation for v1.9 (#6645) * Update admission control docs for webhooks (re-send #6368) (#6650) * Update admission control docs for webhooks * update in response to comments * Revamp rkt and add CRI-O as alternative runtime (#6371) Signed-off-by: Lorenzo Fontana <lo@linux.com> * Documented NLB for Kubernetes 1.9 (#6260) * Added IPV6 information to setup cluster using kubeadm (#6465) * Added IPV6 information to setup cluster using kubeadm * Updated kubeadm.md & create-cluster-kubeadm.md with IPv6 related information * Added IPv6 options for kubeadm --init & automated address binding for kube-proxy based on version of IP configured for API server) * Changes to kubeadm.md as per comments * Modified kubeadm.md and create-cluster-kubeadm.md * Implemented changes requested by zacharysarah * Removed autogenerated kubeadm.md changes * StatefulSet 1.9 updates. (#6550) * updates sts concept and tutorials to use 1.9 apps/v1 * Update statefulset.md * clarify pod name label * Garbage collection updates for 1.9 (#6555) * 1.9 gc policy update * carify deletion * Couple nits for dnsConfig doc (#6652) * Add doc for AllowedFlexVolume (#6563) * Update OpenStack Cloud Provider API support for v1.9 (#6638) * Flex volume is GA. Remove alpha notation. (#6666) * Update generated ref docs for Kubernetes and Federation components. (#6658) * Update generated ref docs for Kubernetes and Federation components. * Rename kubectl-options to kubectl. * Add title to kubectl. * Fix double synopsis. * Update Federation API ref docs for 1.9. (#6636) * Update federation API ref docs. * Move and redirect. * Move generated Federation docs to the generated directory. * Fix titles. * Type * Fix titles * Update auto-generated Kubernetes APi ref docs. (#6646) * Update kubectl commands for 1.9 (#6635) * add ExtendedResourceToleration admission controller (#6618) * Update API reference paths for v1.9 (#6681)
This updated getting-started guide targets Windows Server version 1709 (GA October 2017) using Kubernetes v1.8.2 with several additional PRs required for building kubelet and kube-proxy to work on Windows. The guide references different supported network topologies and available CNI plugins.
Links to Microsoft/SDN repo reference instructions needed for building windows K8s binaries locally as well as instructions for running scripts to join Windows node to an existing K8s cluster.
This change is