Skip to content

Commit

Permalink
feat:add support for provisioning windows node pools
Browse files Browse the repository at this point in the history
  • Loading branch information
g-awmalik authored Sep 27, 2022
1 parent 6472909 commit 92d7c67
Show file tree
Hide file tree
Showing 65 changed files with 2,415 additions and 34 deletions.
13 changes: 13 additions & 0 deletions .kitchen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,19 @@ suites:
backend: gcp
controls:
- gcp
- name: "simple_windows_node_pool"
driver:
root_module_directory: test/fixtures/simple_windows_node_pool
verifier:
systems:
- name: gcloud
backend: local
controls:
- gcloud
- name: gcp
backend: gcp
controls:
- gcp
- name: "deploy_service"
driver:
root_module_directory: test/fixtures/deploy_service
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ Then perform the following commands on the root folder:
| subnetwork | The subnetwork to host the cluster in (required) | `string` | n/a | yes |
| timeouts | Timeout for cluster operations. | `map(string)` | `{}` | no |
| upstream\_nameservers | If specified, the values replace the nameservers taken by default from the node’s /etc/resolv.conf | `list(string)` | `[]` | no |
| windows\_node\_pools | List of maps containing Windows node pools | `list(map(string))` | `[]` | no |
| zones | The zones to host the cluster in (optional if regional cluster / required if zonal) | `list(string)` | `[]` | no |

## Outputs
Expand Down Expand Up @@ -232,6 +233,9 @@ Then perform the following commands on the root folder:
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

## node_pools variable

> Use this variable for provisioning linux based node pools. For Windows based node pools use [windows_node_pools](#windows\_node\_pools-variable)
The node_pools variable takes the following parameters:

| Name | Description | Default | Requirement |
Expand Down Expand Up @@ -270,6 +274,11 @@ The node_pools variable takes the following parameters:
| tags | The list of instance tags applied to all nodes | | Required |
| value | The value for the taint | | Required |
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |

## windows_node_pools variable
The windows_node_pools variable takes the same parameters as [node_pools](#node\_pools-variable) but is reserved for provisioning Windows based node pools only. This variable is introduced to satisfy a [specific requirement](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows#create_a_cluster_and_node_pools) for the presence of at least one linux based node pool in the cluster before a windows based node pool can be created.


## Requirements

Before this module can be used on a project, you must ensure that the following pre-requisites are fulfilled:
Expand Down
8 changes: 8 additions & 0 deletions autogen/main/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,9 @@ Then perform the following commands on the root folder:

{% if autopilot_cluster != true %}
## node_pools variable

> Use this variable for provisioning linux based node pools. For Windows based node pools use [windows_node_pools](#windows\_node\_pools-variable)
The node_pools variable takes the following parameters:

| Name | Description | Default | Requirement |
Expand Down Expand Up @@ -228,7 +231,12 @@ The node_pools variable takes the following parameters:
| tags | The list of instance tags applied to all nodes | | Required |
| value | The value for the taint | | Required |
| version | The Kubernetes version for the nodes in this pool. Should only be set if auto_upgrade is false | " " | Optional |

## windows_node_pools variable
The windows_node_pools variable takes the same parameters as [node_pools](#node\_pools-variable) but is reserved for provisioning Windows based node pools only. This variable is introduced to satisfy a [specific requirement](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows#create_a_cluster_and_node_pools) for the presence of at least one linux based node pool in the cluster before a windows based node pool can be created.

{% endif %}

## Requirements

Before this module can be used on a project, you must ensure that the following pre-requisites are fulfilled:
Expand Down
18 changes: 17 additions & 1 deletion autogen/main/cluster.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -500,7 +500,7 @@ locals {
# resources where "ForceNew" is "true". schemaNodeConfig can be found in node_config.go at
# https://github.com/terraform-providers/terraform-provider-google/blob/master/google/node_config.go#L22
resource "random_id" "name" {
for_each = local.node_pools
for_each = merge(local.node_pools, local.windows_node_pools)
byte_length = 2
prefix = format("%s-", lookup(each.value, "name"))
keepers = merge(
Expand Down Expand Up @@ -569,13 +569,22 @@ resource "random_id" "name" {

{% endif %}
{% if autopilot_cluster != true %}
{% for i in range(2) %}
{% if i == 0 %}
resource "google_container_node_pool" "pools" {
{% else %}
resource "google_container_node_pool" "windows_pools" {
{% endif %}
{% if beta_cluster %}
provider = google-beta
{% else %}
provider = google
{% endif %}
{% if i == 0 %}
for_each = local.node_pools
{% else %}
for_each = local.windows_node_pools
{% endif %}
{% if update_variant %}
name = { for k, v in random_id.name : k => v.hex }[each.key]
{% else %}
Expand Down Expand Up @@ -756,6 +765,7 @@ resource "google_container_node_pool" "pools" {
}
}

{% if i == 0 %}
dynamic "linux_node_config" {
for_each = length(merge(
local.node_pools_linux_node_configs_sysctls["all"],
Expand All @@ -770,6 +780,7 @@ resource "google_container_node_pool" "pools" {
}
}
{% endif %}
{% endif %}

shielded_instance_config {
enable_secure_boot = lookup(each.value, "enable_secure_boot", false)
Expand All @@ -790,5 +801,10 @@ resource "google_container_node_pool" "pools" {
update = lookup(var.timeouts, "update", "45m")
delete = lookup(var.timeouts, "delete", "45m")
}

{% if i == 1 %}
depends_on = [google_container_node_pool.pools[0]]
{% endif %}
}
{% endfor %}
{% endif %}
13 changes: 11 additions & 2 deletions autogen/main/main.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ locals {
// Build a map of maps of node pools from a list of objects
node_pool_names = [for np in toset(var.node_pools) : np.name]
node_pools = zipmap(local.node_pool_names, tolist(toset(var.node_pools)))
windows_node_pool_names = [for np in toset(var.windows_node_pools) : np.name]
windows_node_pools = zipmap(local.windows_node_pool_names, tolist(toset(var.windows_node_pools)))
{% endif %}

release_channel = var.release_channel != null ? [{ channel : var.release_channel }] : []
Expand Down Expand Up @@ -164,8 +166,15 @@ locals {
}]

{% if autopilot_cluster != true %}
cluster_output_node_pools_names = concat([for np in google_container_node_pool.pools : np.name], [""])
cluster_output_node_pools_versions = { for np in google_container_node_pool.pools : np.name => np.version }
cluster_output_node_pools_names = concat(
[for np in google_container_node_pool.pools : np.name], [""],
[for np in google_container_node_pool.windows_pools : np.name], [""]
)

cluster_output_node_pools_versions = merge(
{ for np in google_container_node_pool.pools : np.name => np.version },
{ for np in google_container_node_pool.windows_pools : np.name => np.version },
)
{% endif %}

cluster_master_auth_list_layer1 = local.cluster_output_master_auth
Expand Down
6 changes: 6 additions & 0 deletions autogen/main/variables.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,12 @@ variable "node_pools" {
]
}

variable "windows_node_pools" {
type = list(map(string))
description = "List of maps containing Windows node pools"
default = []
}

variable "node_pools_labels" {
type = map(map(string))
description = "Map of maps containing node labels by node-pool name"
Expand Down
20 changes: 20 additions & 0 deletions autogen/main/variables_defaults.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : {}]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : {}]
),
var.node_pools_labels
)

Expand All @@ -38,6 +42,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : {}]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : {}]
),
var.node_pools_metadata
)

Expand All @@ -48,6 +56,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : []]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : []]
),
var.node_pools_taints
)

Expand All @@ -58,6 +70,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : []]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : []]
),
var.node_pools_tags
)

Expand All @@ -68,6 +84,10 @@ locals {
[for node_pool in var.node_pools : node_pool["name"]],
[for node_pool in var.node_pools : []]
),
zipmap(
[for node_pool in var.windows_node_pools : node_pool["name"]],
[for node_pool in var.windows_node_pools : []]
),
var.node_pools_oauth_scopes
)
{% if beta_cluster %}
Expand Down
1 change: 1 addition & 0 deletions autogen/safer-cluster/main.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ module "gke" {
initial_node_count = (var.initial_node_count == 0) ? 1 : var.initial_node_count

node_pools = var.node_pools
windows_node_pools = var.windows_node_pools
node_pools_labels = var.node_pools_labels
node_pools_metadata = var.node_pools_metadata
node_pools_taints = var.node_pools_taints
Expand Down
6 changes: 6 additions & 0 deletions autogen/safer-cluster/variables.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,12 @@ variable "node_pools" {
]
}

variable "windows_node_pools" {
type = list(map(string))
description = "List of maps containing node pools"
default = []
}

variable "node_pools_labels" {
type = map(map(string))
description = "Map of maps containing node labels by node-pool name"
Expand Down
15 changes: 15 additions & 0 deletions build/int.cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,21 @@ steps:
- verify beta-cluster-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do destroy beta-cluster-local']
- id: converge simple-windows-node-pool-local
waitFor:
- create all
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do converge simple-windows-node-pool-local']
- id: verify simple-windows-node-pool-local
waitFor:
- converge simple-windows-node-pool-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do verify simple-windows-node-pool-local']
- id: destroy simple-windows-node-pool-local
waitFor:
- verify simple-windows-node-pool-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do destroy simple-windows-node-pool-local']
- id: converge deploy-service-local
waitFor:
- create all
Expand Down
Loading

0 comments on commit 92d7c67

Please sign in to comment.