Skip to content

Commit

Permalink
Issue hashicorp#459 - EBS information required during cluster config …
Browse files Browse the repository at this point in the history 
…change (hashicorp#1131)

* Issue hashicorp#459 - EBS information required during cluster configuration change

AWS demands to pass information about EBS volume during cluster configuration change like e.g. number of data nodes.

Issue: hashicorp#459
Debug data from issue:
```
  cluster_config.0.instance_count: "2" => "1"

2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: 2017/07/12 12:47:01 [DEBUG] [aws-sdk-go] DEBUG: Request es/UpdateElasticsearchDomainConfig Details:
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: ---[ REQUEST POST-SIGN ]-----------------------------
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: POST /2015-01-01/es/domain/daniel-test/config HTTP/1.1
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: Host: es.us-east-1.amazonaws.com
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: User-Agent: aws-sdk-go/1.10.8 (go1.8.1; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.9.8
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: Content-Length: 150
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: Authorization: AWS4-HMAC-SHA256 Credential=/20170712/us-east-1/es/aws4_request, SignedHeaders=content-length;host;x-amz-date, Signature=
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: X-Amz-Date: 20170712T114701Z
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: Accept-Encoding: gzip
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws:
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: {"ElasticsearchClusterConfig":{"DedicatedMasterEnabled":false,"InstanceCount":2,"InstanceType":"t2.small.elasticsearch","ZoneAwarenessEnabled":false}}
2017/07/12 12:47:01 [DEBUG] plugin: terraform-provider-aws: -----------------------------------------------------
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: 2017/07/12 12:47:02 [DEBUG] [aws-sdk-go] DEBUG: Response es/UpdateElasticsearchDomainConfig Details:
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: ---[ RESPONSE ]--------------------------------------
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: HTTP/1.1 400 Bad Request
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: Content-Length: 188
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: Content-Type: application/json
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: Date: Wed, 12 Jul 2017 11:47:01 GMT
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: X-Amzn-Errortype: ValidationException
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: X-Amzn-Requestid:
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws:
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws:
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: -----------------------------------------------------
2017/07/12 12:47:02 [DEBUG] plugin: terraform-provider-aws: 2017/07/12 12:47:02 [DEBUG] [aws-sdk-go] {"message":"New cluster configuration has insufficient storage capacity for your current data. The new configuration only supports up to 0.0GB. Your current Elasticsearch usage is 0.00GB"}
```

Debug data after fix:
```
cluster_config.0.instance_count: "2" => "1"
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: 2017/07/12 13:42:35 [DEBUG] [aws-sdk-go] DEBUG: Request es/UpdateElasticsearchDomainConfig Details:
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: ---[ REQUEST POST-SIGN ]-----------------------------
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: POST /2015-01-01/es/domain/daniel-test/config HTTP/1.1
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: Host: es.us-east-1.amazonaws.com
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: User-Agent: aws-sdk-go/1.10.8 (go1.8.1; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.9.8
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: Content-Length: 218
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: Authorization: AWS4-HMAC-SHA256 Credential=/20170712/us-east-1/es/aws4_request, SignedHeaders=content-length;host;x-amz-date, Signature=
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: X-Amz-Date: 20170712T124235Z
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: Accept-Encoding: gzip
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws:
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: {"EBSOptions":{"EBSEnabled":true,"VolumeSize":10,"VolumeType":"gp2"},"ElasticsearchClusterConfig":{"DedicatedMasterEnabled":false,"InstanceCount":1,"InstanceType":"t2.small.elasticsearch","ZoneAwarenessEnabled":false}}
2017/07/12 13:42:35 [DEBUG] plugin: terraform-provider-aws: -----------------------------------------------------
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: 2017/07/12 13:42:36 [DEBUG] [aws-sdk-go] DEBUG: Response es/UpdateElasticsearchDomainConfig Details:
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: ---[ RESPONSE ]--------------------------------------
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: HTTP/1.1 200 OK
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: Content-Length: 1355
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: Content-Type: application/json
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: Date: Wed, 12 Jul 2017 12:42:36 GMT
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: X-Amzn-Requestid:
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws:
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws:
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: -----------------------------------------------------
2017/07/12 13:42:36 [DEBUG] plugin: terraform-provider-aws: 2017/07/12 13:42:36 [DEBUG] [aws-sdk-go] {"DomainConfig":{"AccessPo...<CUT>
```

Please verify.
Thanks

* fix formatting

* add test for hashicorp#459

Simple test, just to check if number of instances is correct.

* fix and adjust testing method

* fix function name for as an entrypoint for Config

* test: Randomize name of the cluster

* test: Use even number of data nodes

* test: Specify valid snapshot hours

* test: Fix test functions

* Fix tests
  • Loading branch information
@jocgir
DanKans authored and jocgir committed Aug 28, 2017
1 parent 42af1df commit 2cdf86a
Show file tree
Hide file tree
Showing 2 changed files with 88 additions and 11 deletions.
23 changes: 12 additions & 11 deletions aws/resource_aws_elasticsearch_domain.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -358,10 +358,10 @@ func resourceAwsElasticSearchDomainUpdate(d *schema.ResourceData, meta interface

if err := setTagsElasticsearchService(conn, d, d.Id()); err != nil {
return err
} else {
d.SetPartial("tags")
}

d.SetPartial("tags")

input := elasticsearch.UpdateElasticsearchDomainConfigInput{
DomainName: aws.String(d.Get("domain_name").(string)),
}
Expand All @@ -374,7 +374,7 @@ func resourceAwsElasticSearchDomainUpdate(d *schema.ResourceData, meta interface
input.AdvancedOptions = stringMapToPointers(d.Get("advanced_options").(map[string]interface{}))
}

if d.HasChange("ebs_options") {
if d.HasChange("ebs_options") || d.HasChange("cluster_config") {
options := d.Get("ebs_options").([]interface{})

if len(options) > 1 {
Expand All @@ -383,17 +383,18 @@ func resourceAwsElasticSearchDomainUpdate(d *schema.ResourceData, meta interface
s := options[0].(map[string]interface{})
input.EBSOptions = expandESEBSOptions(s)
}
}

if d.HasChange("cluster_config") {
config := d.Get("cluster_config").([]interface{})
if d.HasChange("cluster_config") {
config := d.Get("cluster_config").([]interface{})

if len(config) > 1 {
return fmt.Errorf("Only a single cluster_config block is expected")
} else if len(config) == 1 {
m := config[0].(map[string]interface{})
input.ElasticsearchClusterConfig = expandESClusterConfig(m)
if len(config) > 1 {
return fmt.Errorf("Only a single cluster_config block is expected")
} else if len(config) == 1 {
m := config[0].(map[string]interface{})
input.ElasticsearchClusterConfig = expandESClusterConfig(m)
}
}

}

if d.HasChange("snapshot_options") {
Expand Down
76 changes: 76 additions & 0 deletions aws/resource_aws_elasticsearch_domain_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,54 @@ func TestAccAWSElasticSearchDomain_tags(t *testing.T) {
})
}

func TestAccAWSElasticSearchDomain_update(t *testing.T) {
var input elasticsearch.ElasticsearchDomainStatus
ri := acctest.RandInt()

resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Providers: testAccProviders,
CheckDestroy: testAccCheckESDomainDestroy,
Steps: []resource.TestStep{
{
Config: testAccESDomainConfig_ClusterUpdate(ri, 2, 22),
Check: resource.ComposeTestCheckFunc(
testAccCheckESDomainExists("aws_elasticsearch_domain.example", &input),
testAccCheckESNumberOfInstances(2, &input),
testAccCheckESSnapshotHour(22, &input),
),
},
{
Config: testAccESDomainConfig_ClusterUpdate(ri, 4, 23),
Check: resource.ComposeTestCheckFunc(
testAccCheckESDomainExists("aws_elasticsearch_domain.example", &input),
testAccCheckESNumberOfInstances(4, &input),
testAccCheckESSnapshotHour(23, &input),
),
},
}})
}

func testAccCheckESSnapshotHour(snapshotHour int, status *elasticsearch.ElasticsearchDomainStatus) resource.TestCheckFunc {
return func(s *terraform.State) error {
conf := status.SnapshotOptions
if *conf.AutomatedSnapshotStartHour != int64(snapshotHour) {
return fmt.Errorf("Snapshots start hour differ. Given: %d, Expected: %d", *conf.AutomatedSnapshotStartHour, snapshotHour)
}
return nil
}
}

func testAccCheckESNumberOfInstances(numberOfInstances int, status *elasticsearch.ElasticsearchDomainStatus) resource.TestCheckFunc {
return func(s *terraform.State) error {
conf := status.ElasticsearchClusterConfig
if *conf.InstanceCount != int64(numberOfInstances) {
return fmt.Errorf("Number of instances differ. Given: %d, Expected: %d", *conf.InstanceCount, numberOfInstances)
}
return nil
}
}

func testAccLoadESTags(conf *elasticsearch.ElasticsearchDomainStatus, td *elasticsearch.ListTagsOutput) resource.TestCheckFunc {
return func(s *terraform.State) error {
conn := testAccProvider.Meta().(*AWSClient).esconn
Expand Down Expand Up @@ -224,6 +272,34 @@ resource "aws_elasticsearch_domain" "example" {
`, randInt)
}

func testAccESDomainConfig_ClusterUpdate(randInt, instanceInt, snapshotInt int) string {
return fmt.Sprintf(`
resource "aws_elasticsearch_domain" "example" {
domain_name = "tf-test-%d"
advanced_options {
"indices.fielddata.cache.size" = 80
}
ebs_options {
ebs_enabled = true
volume_size = 10
}
cluster_config {
instance_count = %d
zone_awareness_enabled = true
instance_type = "t2.micro.elasticsearch"
}
snapshot_options {
automated_snapshot_start_hour = %d
}
}
`, randInt, instanceInt, snapshotInt)
}

func testAccESDomainConfig_TagUpdate(randInt int) string {
return fmt.Sprintf(`
resource "aws_elasticsearch_domain" "example" {
Expand Down

0 comments on commit 2cdf86a

Please sign in to comment.