Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle aggregatedList case for generated sweepers #2941

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build/terraform
2 changes: 1 addition & 1 deletion build/terraform-beta
10 changes: 0 additions & 10 deletions products/compute/terraform.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,6 @@ overrides: !ruby/object:Overrides::ResourceOverrides
custom_code: !ruby/object:Provider::Terraform::CustomCode
post_create: templates/terraform/post_create/labels.erb
Autoscaler: !ruby/object:Overrides::Terraform::ResourceOverride
# TODO: Remove when aggregatedList is handled in sweeper generation
skip_sweeper: true
examples:
- !ruby/object:Provider::Terraform::Examples
name: "autoscaler_single_instance"
Expand Down Expand Up @@ -1010,8 +1008,6 @@ overrides: !ruby/object:Overrides::ResourceOverrides
decoder: templates/terraform/decoders/unwrap_resource.go.erb
encoder: templates/terraform/encoders/compute_network_endpoint.go.erb
NetworkEndpointGroup: !ruby/object:Overrides::Terraform::ResourceOverride
# TODO: Remove when aggregatedList is handled in sweeper generation
skip_sweeper: true
examples:
- !ruby/object:Provider::Terraform::Examples
name: "network_endpoint_group"
Expand All @@ -1030,8 +1026,6 @@ overrides: !ruby/object:Overrides::ResourceOverrides
required: false
default_from_api: true
NodeGroup: !ruby/object:Overrides::Terraform::ResourceOverride
# TODO: Remove when aggregatedList is handled in sweeper generation
skip_sweeper: true
docs: !ruby/object:Provider::Terraform::Docs
warning: |
Due to limitations of the API, Terraform cannot update the
Expand Down Expand Up @@ -1343,8 +1337,6 @@ overrides: !ruby/object:Overrides::ResourceOverrides
snapshotSchedulePolicy.snapshotProperties.storageLocations: !ruby/object:Overrides::Terraform::PropertyOverride
is_set: true
Reservation: !ruby/object:Overrides::Terraform::ResourceOverride
# TODO: Remove when aggregatedList is handled in sweeper generation
skip_sweeper: true
examples:
- !ruby/object:Provider::Terraform::Examples
name: "reservation_basic"
Expand Down Expand Up @@ -1904,8 +1896,6 @@ overrides: !ruby/object:Overrides::ResourceOverrides
id: !ruby/object:Overrides::Terraform::PropertyOverride
name: proxyId
TargetInstance: !ruby/object:Overrides::Terraform::ResourceOverride
# TODO: Remove when aggregatedList is handled in sweeper generation
skip_sweeper: true
examples:
- !ruby/object:Provider::Terraform::Examples
name: "target_instance_basic"
Expand Down
37 changes: 35 additions & 2 deletions templates/terraform/sweeper_file.go.erb
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ import (
<%
sweeper_name = product_ns + object.name
wrap_path = object&.nested_query&.keys&.first || object.collection_url_key
listUrlTemplate = object.__product.base_url + object.base_url

listUrlTemplate.sub! "zones/{{zone}}", "aggregated"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming there's basically no non-GCE zonal resources, so we'll only replace GCE resources with this?

Copy link
Member Author

@c2thorn c2thorn Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, other resources use location/ in the base url and they don't support aggregatedList. I am assuming some consistency in the future, otherwise we will have to use some manual flag.

aggregatedList = listUrlTemplate.include? "/aggregated/"

deleteUrlTemplate = object.__product.base_url + object.delete_uri
-%>

func init() {
Expand All @@ -39,7 +45,7 @@ func testSweep<%= sweeper_name -%>(region string) error {
return err
}

listTemplate := strings.Split("<%= object.__product.base_url + object.base_url -%>","?")[0]
listTemplate := strings.Split("<%= listUrlTemplate -%>","?")[0]

d := &ResourceDataMock{
FieldsInSchema: map[string]interface{}{
Expand Down Expand Up @@ -72,8 +78,24 @@ func testSweep<%= sweeper_name -%>(region string) error {
log.Printf("[INFO] Nothing found in response.")
return nil
}
<% if aggregatedList -%>
var rl []interface{}
zones := resourceList.(map[string]interface{})
// Loop through every zone in the list response
for _, zonesValue := range zones {
zone := zonesValue.(map[string]interface{})
for k, v := range zone {
// Zone map either has resources or a warning stating there were no resources found in the zone
if k != "warning" {
resourcesInZone := v.([]interface{})
rl = append(rl, resourcesInZone...)
}
}
}
<% else -%>

rl := resourceList.([]interface{})
<% end -%>

log.Printf("[INFO] Found %d items in %s list response.", len(rl), resourceName)
// items who don't match the tf-test prefix
Expand All @@ -93,7 +115,18 @@ func testSweep<%= sweeper_name -%>(region string) error {
nonPrefixCount++
continue
}
deleteTemplate := "<%= object.__product.base_url + object.delete_uri -%>"
deleteTemplate := "<%= deleteUrlTemplate -%>"
<% if aggregatedList -%>

if obj["zone"] == nil {
log.Printf("[INFO] %s resource zone was nil", resourceName)
return nil
}
zoneSegs := strings.Split(obj["zone"].(string),"/")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't see that helper! I actually do the same thing here to parse the name. I will replace both instances at the same time in a separate PR since it will cause diffs for a ton of existing sweeper files

zone := zoneSegs[len(zoneSegs)-1]
deleteTemplate = strings.Replace(deleteTemplate, "{{zone}}", zone, -1)

<% end -%>
deleteUrl, err := replaceVars(d, config, deleteTemplate)
if err != nil {
log.Printf("[INFO] error preparing delete url: %s", err)
Expand Down