Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AlloyDB user support #8431

Closed
103 changes: 103 additions & 0 deletions mmv1/products/alloydb/User.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Copyright 2023 Google Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

--- !ruby/object:Api::Resource
name: 'User'
self_link: '{{cluster}}/users/{{user_id}}'
base_url: '{{cluster}}/users'
create_url: '{{cluster}}/users?userId={{user_id}}'
update_verb: :PATCH
DanielRieske marked this conversation as resolved.
Show resolved Hide resolved
update_mask: true
description: 'An AlloyDB User.'
references: !ruby/object:Api::Resource::ReferenceLinks
guides:
'AlloyDB': 'https://cloud.google.com/alloydb/docs/'
api: 'https://cloud.google.com/alloydb/docs/reference/rest/v1/projects.locations.clusters.users/create'
async: !ruby/object:Provider::Terraform::PollAsync
check_response_func_existence: transport_tpg.PollCheckForExistence
target_occurrences: 10
actions: ['create', 'update', 'delete']
import_format: ['{{cluster}}/users/{{user_id}}']
custom_code: !ruby/object:Provider::Terraform::CustomCode
custom_import: templates/terraform/custom_import/alloydb_user.go.erb
autogen_async: true
skip_sweeper: true
examples:
- !ruby/object:Provider::Terraform::Examples
name: 'alloydb_user'
primary_resource_id: 'default'
# Fine-grained resource need different autogenerated tests, as
# we need to check destroy during a test step where the parent resource
# still exists, rather than during CheckDestroy (when read returns
# nothing because the parent resource has then also been destroyed)
skip_test: true
vars:
alloydb_user_id: 'me'
alloydb_user_password: 'changeme'
alloydb_cluster_id: 'alloydb-cluster'
alloydb_instance_id: 'alloydb-instance'
- !ruby/object:Provider::Terraform::Examples
name: 'alloydb_user_iam_user'
primary_resource_id: 'default'
# Fine-grained resource need different autogenerated tests, as
# we need to check destroy during a test step where the parent resource
# still exists, rather than during CheckDestroy (when read returns
# nothing because the parent resource has then also been destroyed)
skip_test: true
vars:
alloydb_user_id: 'me@example.com'
alloydb_cluster_id: 'alloydb-cluster-iam'
alloydb_instance_id: 'alloydb-instance-iam'
parameters:
- !ruby/object:Api::Type::String
name: 'userId'
required: true
immutable: true
url_param_only: true
description: |
The ID of the alloydb user.
- !ruby/object:Api::Type::ResourceRef
name: 'cluster'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it preferable to have this set up as a single parent resource rather than a series of project/location/cluster?

Generally we have each broken out into an individual field.

cc @GauravJain21

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would mean that the user has to set it's location and I'd argue this resource isn't a regional object on it's own but rather part of a regional object.

As the cluster dictates what region the user resides in I wouldn't give the user the option to specify location.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. On the other hand existing resources like google_sql_user work this way, where the project and instance are both specified independently: https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_user#project

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The biggest difference here is that google_sql_user doesn't rely on a region in his end-point nor is it a field that can be set.
It is implied that it lives in the region where the instance resides in.

POST https://sqladmin.googleapis.com/v1/projects/{project}/instances/{instance}/users

I am curious what others believe the right approach is here.

description: |
Identifies the alloydb cluster. Must be in the format
'projects/{project}/locations/{location}/clusters/{cluster_id}'
required: true
immutable: true
resource: 'Cluster'
imports: 'name'
url_param_only: true
properties:
- !ruby/object:Api::Type::String
name: 'name'
output: true
description: |
Output only. Name of the resource in the form of projects/{project}/locations/{location}/cluster/{cluster}/users/{user}.
- !ruby/object:Api::Type::String
name: 'password'
description: |
Input only. Password for the user. This field is required but shouldn't be set if user_type is set to `ALLOYDB_IAM_USER`
sensitive: true
ignore_read: true
- !ruby/object:Api::Type::Array
name: 'databaseRoles'
item_type: Api::Type::String
description:
Optional. List of database roles this user has. The database role strings are subject to the PostgreSQL naming conventions.
- !ruby/object:Api::Type::Enum
name: 'userType'
description: 'Optional. Type of this user.'
default_value: :ALLOYDB_BUILT_IN
values:
- :USER_TYPE_UNSPECIFIED
DanielRieske marked this conversation as resolved.
Show resolved Hide resolved
- :ALLOYDB_BUILT_IN
- :ALLOYDB_IAM_USER
17 changes: 17 additions & 0 deletions mmv1/templates/terraform/custom_import/alloydb_user.go.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
config := meta.(*transport_tpg.Config)

// current import_formats can't import fields with forward slashes in their value
if err := tpgresource.ParseImportId([]string{
"(?P<cluster>.+)/users/(?P<user_id>[^/]+)",
}, d, config); err != nil {
return nil, err
}

// Replace import id for the resource id
id, err := tpgresource.ReplaceVars(d, config, "{{cluster}}/users/{{user_id}}")
if err != nil {
return nil, fmt.Errorf("Error constructing id: %s", err)
}
d.SetId(id)

return []*schema.ResourceData{d}, nil
44 changes: 44 additions & 0 deletions mmv1/templates/terraform/examples/alloydb_user.tf.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
resource "google_alloydb_user" "<%= ctx[:primary_resource_id] %>" {
user_id = "<%= ctx[:vars]['alloydb_user_id'] %>"
password = "<%= ctx[:vars]['alloydb_user_password'] %>"

database_roles = [
"postgres"
]

cluster = google_alloydb_cluster.<%= ctx[:primary_resource_id] %>.id

depends_on = [google_alloydb_cluster.<%= ctx[:primary_resource_id] %>]
}

resource "google_compute_network" "<%= ctx[:primary_resource_id] %>" {
name = "alloydb-cluster"
}

resource "google_alloydb_cluster" "<%= ctx[:primary_resource_id] %>" {
cluster_id = "<%= ctx[:vars]['alloydb_cluster_id'] %>"
location = "us-central1"
network = google_compute_network.<%= ctx[:primary_resource_id] %>.id
}

resource "google_alloydb_instance" "<%= ctx[:primary_resource_id] %>" {
cluster = google_alloydb_cluster.<%= ctx[:primary_resource_id] %>.name
instance_id = "<%= ctx[:vars]['alloydb_instance_id'] %>"
instance_type = "PRIMARY"

depends_on = [google_service_networking_connection.vpc_connection]
}

resource "google_compute_global_address" "private_ip_alloc" {
name = "<%= ctx[:vars]['alloydb_cluster_id'] %>"
address_type = "INTERNAL"
purpose = "VPC_PEERING"
prefix_length = 16
network = google_compute_network.<%= ctx[:primary_resource_id] %>.id
}

resource "google_service_networking_connection" "vpc_connection" {
network = google_compute_network.<%= ctx[:primary_resource_id] %>.id
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.private_ip_alloc.name]
}
44 changes: 44 additions & 0 deletions mmv1/templates/terraform/examples/alloydb_user_iam_user.tf.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
resource "google_alloydb_user" "<%= ctx[:primary_resource_id] %>" {
user_id = "<%= ctx[:vars]['alloydb_user_id'] %>"
user_type = "ALLOYDB_IAM_USER"

database_roles = [
"alloydbiamuser"
]

cluster = google_alloydb_cluster.<%= ctx[:primary_resource_id] %>.id

depends_on = [google_alloydb_instance.<%= ctx[:primary_resource_id] %>]
}

resource "google_compute_network" "<%= ctx[:primary_resource_id] %>" {
name = "alloydb-cluster"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should change to a parameterized value so it gets a random suffix attached. Right now this is causing the failure because it collides with another network named the same

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slevenick I am skipping the generation of tests because this particular resource does not work well with the autogenerated tests.

If we destroy the cluster or instance during the CheckDestroy phase that will result in Internal 500 errors when trying to verify if an user still exists, therefore I need to check it while the parent resource still exists.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's.... frightening. What do we expect the user experience to be when they created a user within Terraform and then they destroyed it via another tool? They would hit the 500 and be confused about what is happening

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you share either the error message in logs or a use case that triggers it? I can reach out to the internal team because returning a 500 on an attempted GET is pretty bad.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slevenick I did not realise the impact in certain scenario's on users and didn't report it, for future contributions I will keep this in mind.

There are a few scenario's where this can happens but usually boils down to not having an instance in it's cluster.

I tested the following 2 scenario's:

You create a cluster and try to add an user without creating an instance.


2023-07-26T20:57:40.102+0200 [DEBUG] provider.terraform-provider-google: ---[ REQUEST ]---------------------------------------
2023-07-26T20:57:40.102+0200 [DEBUG] provider.terraform-provider-google: POST /v1/projects/playground-x/locations/europe-west4/clusters/tf-test-alloydb-cluster/users?alt=json&userId=test HTTP/1.1
2023-07-26T20:57:40.102+0200 [DEBUG] provider.terraform-provider-google: Host: alloydb.googleapis.com
2023-07-26T20:57:40.102+0200 [DEBUG] provider.terraform-provider-google: User-Agent: Terraform/1.5.0 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/dev
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: Content-Length: 67
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: Content-Type: application/json
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: Accept-Encoding: gzip
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: 
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: {
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google:  "databaseRoles": [
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google:   "alloydbiamuser"
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google:  ],
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google:  "userType": "ALLOYDB_IAM_USER"
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: }
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: 
2023-07-26T20:57:40.103+0200 [DEBUG] provider.terraform-provider-google: -----------------------------------------------------
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: 2023/07/26 20:57:40 [DEBUG] Google API Response Details:
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: ---[ RESPONSE ]--------------------------------------
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: HTTP/2.0 500 Internal Server Error
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Cache-Control: private
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Content-Type: application/json; charset=UTF-8
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Date: Wed, 26 Jul 2023 18:57:40 GMT
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Server: ESF
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Vary: Origin
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Vary: X-Origin
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: Vary: Referer
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: X-Content-Type-Options: nosniff
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: X-Frame-Options: SAMEORIGIN
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: X-Xss-Protection: 0
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: 
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google: {
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google:   "error": {
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google:     "code": 500,
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google:     "message": "An internal error has occurred (4e2d52b2-e97b-466a-9d22-a1a2c9b499bc)",
2023-07-26T20:57:40.263+0200 [DEBUG] provider.terraform-provider-google:     "status": "INTERNAL"
2023-07-26T20:57:40.264+0200 [DEBUG] provider.terraform-provider-google:   }
2023-07-26T20:57:40.264+0200 [DEBUG] provider.terraform-provider-google: }

│ Error: Error creating User: timeout while waiting for state to become 'success' (timeout: 1m0s)
│ 
│   with google_alloydb_user.default,
│   on main.tf line 61, in resource "google_alloydb_user" "user":
│   34: resource "google_alloydb_user" "user" {

You create a cluster, instance & user through Terraform and we delete the instance and try to pull the state of the user resource


2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: ---[ REQUEST ]---------------------------------------
2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: GET /v1/projects/playground-x/locations/europe-west4/clusters/tf-test-alloydb-cluster/users/test?alt=json HTTP/1.1
2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: Host: alloydb.googleapis.com
2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: User-Agent: Terraform/1.5.0 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/dev
2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: Content-Type: application/json
2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: Accept-Encoding: gzip
2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: 
2023-07-26T20:23:52.799+0200 [DEBUG] provider.terraform-provider-google: 
2023-07-26T20:23:52.800+0200 [DEBUG] provider.terraform-provider-google: -----------------------------------------------------
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: 2023/07/26 20:23:52 [DEBUG] Google API Response Details:
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: ---[ RESPONSE ]--------------------------------------
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: HTTP/2.0 500 Internal Server Error
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Cache-Control: private
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Content-Type: application/json; charset=UTF-8
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Date: Wed, 26 Jul 2023 18:23:52 GMT
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Server: ESF
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Vary: Origin
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Vary: X-Origin
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: Vary: Referer
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: X-Content-Type-Options: nosniff
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: X-Frame-Options: SAMEORIGIN
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: X-Xss-Protection: 0
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: 
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: {
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google:   "error": {
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google:     "code": 500,
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google:     "message": "An internal error has occurred (044b08c2-93bd-4981-a397-05c5c1b32f2b)",
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google:     "status": "INTERNAL"
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google:   }
2023-07-26T20:23:52.837+0200 [DEBUG] provider.terraform-provider-google: }

│ Error: Error when reading or editing AlloydbUser "projects/playground-x/locations/europe-west4/clusters/tf-test-alloydb-cluster/users/test": googleapi: Error 500: An internal error has occurred (570d762a-8316-4854-b15c-a40116fb4e97)
│ 
│   with google_alloydb_user.user,
│   on main.tf line 61, in resource "google_alloydb_user" "user":
│   61: resource "google_alloydb_user" "user" {

While I was at it I tested what happens with an invalid cluster name and it got rejected as a 404.

Given these results I am unsure if this should be merged at this time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GauravJain21 please take a look at this

}

resource "google_alloydb_cluster" "<%= ctx[:primary_resource_id] %>" {
cluster_id = "<%= ctx[:vars]['alloydb_cluster_id'] %>"
location = "us-central1"
network = google_compute_network.<%= ctx[:primary_resource_id] %>.id
}

resource "google_alloydb_instance" "<%= ctx[:primary_resource_id] %>" {
cluster = google_alloydb_cluster.<%= ctx[:primary_resource_id] %>.name
instance_id = "<%= ctx[:vars]['alloydb_instance_id'] %>"
instance_type = "PRIMARY"

depends_on = [google_service_networking_connection.vpc_connection]
}

resource "google_compute_global_address" "private_ip_alloc" {
name = "<%= ctx[:vars]['alloydb_cluster_id'] %>"
address_type = "INTERNAL"
purpose = "VPC_PEERING"
prefix_length = 16
network = google_compute_network.<%= ctx[:primary_resource_id] %>.id
}

resource "google_service_networking_connection" "vpc_connection" {
network = google_compute_network.<%= ctx[:primary_resource_id] %>.id
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.private_ip_alloc.name]
}
Loading