Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to add Shielded VMs support in google_dataproc_workflow_templ… #6481

Merged
merged 6 commits into from
Sep 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion mmv1/third_party/terraform/go.mod.erb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.18

require (
cloud.google.com/go/bigtable v1.16.0
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.18.0
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.19.0
github.com/apparentlymart/go-cidr v1.1.0
github.com/client9/misspell v0.3.4
github.com/davecgh/go-spew v1.1.1
Expand Down
4 changes: 2 additions & 2 deletions mmv1/third_party/terraform/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24 h1:sHglBQTwgx+rWPdisA5ynNEsoARbiCBOyGcJM4/OzsM=
github.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24/go.mod h1:4UJr5HIiMZrwgkSPdsjy2uOQExX/WEILpIrO9UPGuXs=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.18.0 h1:0NWhlgF7eg/jpecAhG/QUwp0MH7D4jYekpljiLUaE74=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.18.0/go.mod h1:i6Pmzp7aolLmJY86RaJ9wjqm/HFleMeN7Vl5uIWLwE8=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.19.0 h1:4YAtk4xuOCxUSkGdwlDhkX7DTP4VwLZCoebGGEsU+U4=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.19.0/go.mod h1:i6Pmzp7aolLmJY86RaJ9wjqm/HFleMeN7Vl5uIWLwE8=
github.com/Masterminds/goutils v1.1.0/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
github.com/Masterminds/semver v1.4.2/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y=
github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,35 @@ func TestAccDataprocWorkflowTemplate_basic(t *testing.T) {
})
}

func TestAccDataprocWorkflowTemplate_withShieldedVMs(t *testing.T) {
t.Parallel()

context := map[string]interface{}{
"random_suffix": randString(t, 10),
"project": getTestProjectFromEnv(),
"version": "2.0.35-debian10",
}

vcrTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Providers: testAccProviders,
CheckDestroy: funcAccTestDataprocWorkflowTemplateCheckDestroy(t),
ExternalProviders: map[string]resource.ExternalProvider{
"random": {},
},
Steps: []resource.TestStep{
{
Config: testAccDataprocWorkflowTemplate_withShieldedVMs(context),
},
{
ImportState: true,
ImportStateVerify: true,
ResourceName: "google_dataproc_workflow_template.shielded_vms_template",
},
},
})
}

func testAccDataprocWorkflowTemplate_basic(context map[string]interface{}) string {
return Nprintf(`
resource "google_dataproc_workflow_template" "template" {
Expand Down Expand Up @@ -95,6 +124,67 @@ resource "google_dataproc_workflow_template" "template" {
`, context)
}

func testAccDataprocWorkflowTemplate_withShieldedVMs(context map[string]interface{}) string {
return Nprintf(`
resource "google_dataproc_workflow_template" "shielded_vms_template" {
name = "template%{random_suffix}"
location = "us-central1"
placement {
managed_cluster {
cluster_name = "my-shielded-cluster"
config {
gce_cluster_config {
zone = "us-central1-a"
tags = ["foo", "bar"]
shielded_instance_config {
enable_secure_boot = true
enable_vtpm = true
enable_integrity_monitoring = true
}
}
master_config {
num_instances = 1
machine_type = "n1-standard-1"
disk_config {
boot_disk_type = "pd-ssd"
boot_disk_size_gb = 15
}
}
worker_config {
num_instances = 3
machine_type = "n1-standard-2"
disk_config {
boot_disk_size_gb = 10
num_local_ssds = 2
}
}

secondary_worker_config {
num_instances = 2
}
software_config {
image_version = "%{version}"
}
}
}
}
jobs {
step_id = "someJob"
spark_job {
main_class = "SomeClass"
}
}
jobs {
step_id = "otherJob"
prerequisite_step_ids = ["someJob"]
presto_job {
query_file_uri = "someuri"
}
}
}
`, context)
}

func funcAccTestDataprocWorkflowTemplateCheckDestroy(t *testing.T) func(s *terraform.State) error {
return func(s *terraform.State) error {
for name, rs := range s.RootModule().Resources {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -730,6 +730,10 @@ The `gce_cluster_config` block supports:
* `service_account_scopes` -
(Optional)
Optional. The URIs of service account scopes to be included in Compute Engine instances. The following base set of scopes is always included: * https://www.googleapis.com/auth/cloud.useraccounts.readonly * https://www.googleapis.com/auth/devstorage.read_write * https://www.googleapis.com/auth/logging.write If no scopes are specified, the following defaults are also provided: * https://www.googleapis.com/auth/bigquery * https://www.googleapis.com/auth/bigtable.admin.table * https://www.googleapis.com/auth/bigtable.data * https://www.googleapis.com/auth/devstorage.full_control

* `shielded_instance_config` -
(Optional)
Optional. Shielded Instance Config for clusters using [Compute Engine Shielded VMs](https://cloud.google.com/security/shielded-cloud/shielded-vm). Structure [defined below](#nested_shielded_instance_config).

* `subnetwork` -
(Optional)
Expand Down Expand Up @@ -762,6 +766,32 @@ The `reservation_affinity` block supports:
* `values` -
(Optional)
Optional. Corresponds to the label values of reservation resource.

<a name="nested_shielded_instance_config"></a>The `shielded_instance_config` block supports:

```hcl
cluster_config {
gce_cluster_config {
shielded_instance_config {
enable_secure_boot = true
enable_vtpm = true
enable_integrity_monitoring = true
}
}
}
```

* `enable_secure_boot` -
(Optional)
Optional. Defines whether instances have [Secure Boot](https://cloud.google.com/compute/shielded-vm/docs/shielded-vm#secure-boot) enabled.

* `enable_vtpm` -
(Optional)
Optional. Defines whether instances have the [vTPM](https://cloud.google.com/compute/shielded-vm/docs/shielded-vm#vtpm) enabled.

* `enable_integrity_monitoring` -
(Optional)
Optional. Defines whether instances have [Integrity Monitoring](https://cloud.google.com/compute/shielded-vm/docs/shielded-vm#integrity-monitoring) enabled.

The `gke_cluster_config` block supports:

Expand Down
3 changes: 2 additions & 1 deletion tpgtools/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.18

require (
bitbucket.org/creachadair/stringset v0.0.9
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.18.0
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.19.0
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/hashicorp/errwrap v1.0.0
github.com/hashicorp/hcl v1.0.0
Expand All @@ -23,6 +23,7 @@ require (
github.com/golang/protobuf v1.4.2 // indirect
github.com/google/go-cmp v0.5.8 // indirect
github.com/google/go-cpy v0.0.0-20211218193943-a9c933c06932 // indirect

github.com/hashicorp/go-cty v1.4.1-0.20200414143053-d3edf31b6320 // indirect
github.com/hashicorp/go-multierror v1.0.0 // indirect
github.com/hashicorp/go-uuid v1.0.1 // indirect
Expand Down
4 changes: 2 additions & 2 deletions tpgtools/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.18.0 h1:0NWhlgF7eg/jpecAhG/QUwp0MH7D4jYekpljiLUaE74=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.18.0/go.mod h1:i6Pmzp7aolLmJY86RaJ9wjqm/HFleMeN7Vl5uIWLwE8=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.19.0 h1:4YAtk4xuOCxUSkGdwlDhkX7DTP4VwLZCoebGGEsU+U4=
github.com/GoogleCloudPlatform/declarative-resource-client-library v1.19.0/go.mod h1:i6Pmzp7aolLmJY86RaJ9wjqm/HFleMeN7Vl5uIWLwE8=
github.com/agext/levenshtein v1.2.1/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558=
github.com/agext/levenshtein v1.2.2 h1:0S/Yg6LYmFJ5stwQeRp6EeOcCbj7xiqQSdNelsXvaqE=
github.com/agext/levenshtein v1.2.2/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558=
Expand Down