resource/aws_ecs_service: Change type of placement_strategy to TypeList

[atsushi-ishibashi]

#1466

[atsushi-ishibashi]

This PR is compatible with the current master👍

[comebackoneyear]

Can't wait to get this merged! 👍

[comebackoneyear]

Tested it on our stack and it works.

[catsby]

I believe your idea is correct here, this should be a list, however the change presents at least two issues I see:

1. Because placement_strategy is a ForceNew attribute, users may see a need to re-create their ECS Services because the ordering of your configuration may not match the ordering AWS has. I believe this would only occur if you created the service, then happened to rearrange the strategies in your configuration. It’s not very likely but it’s still there, and we’d need to mark this as a breaking change.
2. The current implementation normalizes host to instanceId for the user:

• https://github.com/terraform-providers/terraform-provider-aws/pull/1476/files#diff-51a2b869d60207ffb83c18f0daef501dL136 .
  This code does not, so if the user is using host, they’ll see a diff. We need to either normalize, or update the documentation to note that you should use instanceId instead.

Let me know what you think about these issues

[atsushi-ishibashi]

@catsby Thank you for review.

1

I agree with you. And from the first, does placement_strategy need not be a ForceNew attribute?
If there is no problem with changing ForceNew, I will change it.

2

I just didn’t consider normalization. Set field is valid for a TypeSet so I removed the field. We must implement normalization for host. 
Do you have any code to refer to about implementing normalization for TypeList?

[atsushi-ishibashi]

And should we update the documentation to reflect that the order of placement_strategy in resource_aws_ecs_service is associated to the order of placement strategy in ecs service?

[atsushi-ishibashi]

About normalization, I misunderstood.
Normalization which replaces "host" with "instanceId" is used only to generate hashcode to sort set, right?
You're worrying that the current state of sorting by hashcode is not equal to the state of this PR.

uhhh.. it's impossible to hold the order depending on the current AWS state.
It seems to be appropriate to mark this as a breaking change.
Or changing ForceNew attribute solves this?

[atsushi-ishibashi]

Normalization which replaces "host" with "instanceId" is used only to generate hashcode to sort set

It maybe incorrect... So far I couldn't understand the behaviour of *schema.Set completely.
Sorry for confusing.

[atsushi-ishibashi]

@catsby
https://github.com/hashicorp/terraform/blob/master/helper/schema/set.go#L189
The key of Set.m is the hash generated by SchemaSetFunc so normalization is used to sort.
But the value of Set.m doesn't seem to be normalized and item interface{} is directly set to the value so I'm wondering whether the current implementation normalizes host to instanceId for the user in state file.

It'd be great if you could correct me when I misunderstand or miss any workflows🙇

[atsushi-ishibashi]

Sorry:bow:
I can confirm that no change appears after changing "instanceId" -> "host".

[comebackoneyear]

Is it possible to get eyes on this. We can't deploy our stack with this bug present as the priority is basicly randomized.

[atsushi-ishibashi]

@catsby It'd be great if you could review again 🙇

[atsushi-ishibashi]

make testacc TEST=./aws TESTARGS='-run=TestAccAWSEcsServiceWithPlacementStrategy'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSEcsServiceWithPlacementStrategy -timeout 120m
=== RUN   TestAccAWSEcsServiceWithPlacementStrategy
--- PASS: TestAccAWSEcsServiceWithPlacementStrategy (343.48s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	343.513s

[atsushi-ishibashi]

@catsby @radeksimko What is the status of this PR? Will be merged?

[catsby]

This is indeed a breaking change, unfortunately. I'll discuss internally where that leaves the the PR as far as what release it goes into, 1.3.0 or delay to 2.0.0

[atsushi-ishibashi]

@catsby I got it. Thank you👍

[catsby]

Hey @radeksimko since this is potentially a breaking change, should we delay this to a 2.0.0 release?

[radeksimko]

I think we could make it non-breaking change with state migration, but at this point it is BC.

[atsushi-ishibashi]

@radeksimko Could you give me advice about how to implement without breaking change? state migration?

[catsby]

Hey there! We have some way of migrating the state from one version to the next. All states start at 0 unless specified. Here are some example migrations (they have matching test files, too)

• https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_instance_migrate.go
• https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_kinesis_firehose_delivery_stream_migrate.go
• https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_route53_record_migrate.go

You trigger them by adding this info into the schema for the resource:

https://github.com/terraform-providers/terraform-provider-aws/blob/88deb6ef63a91e419341870fd92228a0cd459d1b/aws/resource_aws_instance.go#L22-L34

In this example, we tell the schema to invoke resourceAwsInstanceMigrateState to handle the migration from whatever state it's in, to the declared version 1. That method is defined in the first link I provide above.

Unfortunately I'm not immediately aware of examples of migrating from TypeSet to TypeList, I believe it's something like this:

• create slice to contain the items in a new list format (`[]interface{})
• grab the count of strategies in the is.Attributes["placement_strategy.#"] value, for reference and later validation
• loop through each value in is.Attributes, it's a map[string]interface{}, and do a string prefix check on the index, looking for placement_strategy that is not placement_strategy.# (that's the count)
• for each of those found, add them to the list from step 1, and delete it from the attributes: delete(is.Attributes, "placement_strategy.<hash>")
• when the new slice is built up, the is.Attributes should be free of any placement_strategy.<hash> values. Compare the length of your temporary slice to the count from step 2 and verify they are the same
• loop through the new slice
  • for each item, do essentially is.Attributes["placement_strategy.<index of loop>"] = item.

I believe those are the basics here. I understand this is asking a lot of you. If you're not up to it I completely understand, you've done so much already. We can likely take it from here and attempt the above, but I can't promise it will be any time week or next.

Please let me know if you have any questions!

[s-maj]

AWS ECS Service is partially immutable (see here) so every change related to placement strategy/constraints should result in forcenew.

[catsby]

@s-maj correct! With this pull request as is, there will likely be users who are confronted with a ForceNew in subsequent terraform plan invocations. Hopefully a state migration can lower the number of people who need them.

The changes made in this pull request are changes made to how we store the configuration of placement strategies. It could be that with this change, no actual changes to the ECS Service are needed. For example, and I could be wrong, but I suspect that anyone with a single placement strategy will sees no change, Terraform simply changes how it's stored internally.

For people with more than 1, the migration could possibly convert them into their proper order, and there's no difference. It's likely though with increasing number of strategies defined that this likelihood would go down though. If the ordering is different, then we'll see a ForceNew plan. Long term that's a good thing, it means that in their current state, the users placement strategies are not actually in the order that's defined in the configuration, and that should be corrected. Unfortunate that it's a ForceNew and an interruption, but probably better than the current situation where the strategies are not be in the order the user expects them to be.

I would need to tinker to verify 🤔

[s-maj]

Ok, I see your point now :) I assumed that all folks are using default "spread(attribute:ecs.availability-zone), spread(instanceId)" or something evenly complex but some might have simpler strategies.

…

On Wed, Nov 8, 2017 at 4:24 PM, Clint ***@***.***> wrote: @s-maj <https://github.com/s-maj> correct! With this pull request *as is*, there will likely be users who are confronted with a ForceNew in subsequent terraform plan invocations. Hopefully a state migration can lower the number of people who need them. The changes made in this pull request are changes made to how we store the configuration of placement strategies. It could be that with this change, no actual changes to the ECS Service are needed. For example, and I could be wrong, but I suspect that anyone with a single placement strategy will sees no change, Terraform simply changes how it's stored internally. I would need to tinker to verify 🤔 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1476 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOhaWvw3D1G8JlWYQQm9D5kOb5v81Yt4ks5s0dXYgaJpZM4O_cie> .

[atsushi-ishibashi]

@catsby Thank you for your guidance. It seems so fun that I'm eager to try it. But when I have a trouble, I'll ask your help:bow:

[atsushi-ishibashi]

@catsby @radeksimko Have you already decided to merge this PR for v2.0.0?

[radeksimko]

We will come back to this PR as well as to other PRs which are labelled as breaking-change when the time comes to ship 2.0.0. We have no date set for this next major release yet.

[atsushi-ishibashi]

@radeksimko @catsby Cloud you review migration? I think it avoid breaking change so this PR can be merged before v2.0.0.
Imposibility to configure placement_strategy correctly would have a bad effect on our projects 🙇

[chrisdewar]

@radeksimko, @catsby given that @atsushi-ishibashi has resolved the issue that would make this a breaking change, is there a possibility to get this in before v2.0.0? If we can't rely on the order of the placement strategies, our resiliency is quite severely impacted...

[chrisdewar]

@radeksimko, @catsby - can we please look at re-prioritising this one? this bug breaks high availability in a lot of scenarios, and it's seemingly just waiting on a review... we're willing to sponsor this fix to get it out the door.

[idubinskiy]

@radeksimko @catsby Adding another voice to the chorus asking if there's any update on this. I believe this should be considered a major bug in the provider, as the incorrect (and unexpected) ordering can cause real issues when using ECS.

@atsushi-ishibashi Looks like there's a merge conflict now. Do you think you can clear that up so this can be merged once the maintainers response? Thanks so much for providing this fix.

[bflad]

Hi Everyone! 👋 Sorry this is causing a lot of pain and frustration due to the unexpected behavior of Terraform. Thank you very much for your efforts here.

Indeed the implementation details of this PR are very problematic due to how Terraform internally stores state versus the actual configuration. I'm personally not aware of a good way to seamlessly migrate an existing attribute from TypeSet to TypeList in the cases where there are more than one element. As mentioned previously (#1476 (comment)), the likelihood of an in-place migration not working increases. Since this change risks recreating the resource, I would personally be very cautious in accepting this solution even for major upgrades of the provider.

All is not lost though! We recently needed to do something similar for CloudFront distributions to migrate from an existing unordered attribute to an ordered one: #4117

You'll notice there that we opted to create a brand new attribute (prepended with ordered_) that is correctly implemented with TypeList, ConflictsWith the old TypeSet one, and deprecates the old one while still supporting it until we release the next major version of the AWS provider. This allows for an opt-in and iterative configuration update rather than a much harder or unexpected upgrade path.

To that end, I'm going to close this PR in preference for an implementation similar to #4117 for this situation, rather than keep this old and unlikely to be accepted PR open. Again, thank you @atsushi-ishibashi for your work here. For those not familiar, @atsushi-ishibashi does a bunch of great work in this provider. 👍 Please reach out with any questions!

[idubinskiy]

@bflad Thanks for the great explanation. Do you know if anyone is working on a #4117-style fix for this issue?

[bflad]

I'm not aware of anyone at HashiCorp working or planning to work on this at this time, but if I see a pull request come through I'll happily look at it.

[atsushi-ishibashi]

I will 💪

[chrisdewar]

Thank you @atsushi-ishibashi - the hero the ECS community needs.

[bflad]

For those following this issue you'll probably be interested in knowing we have implemented a new argument ordered_placement_strategy and deprecated the existing placement_strategy argument so you can opt into the new behavior. This will be released with v1.17.0 of the AWS provider, likely in two days. We will remove placement_strategy in the next major version of the AWS provider.

[pascal-de-ladurantaye]

Hi there! We are currently working on fixing our deprecation warnings within our terraform scripts and this issue is one of them. I'm simply wondering if there is a way for us to do a state migration for all of our ecs services as they are ForcedNew by this change. If it helps, all of our services have a single placement strategy. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!