Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Resource aws_rds_cluster_activity_stream #22097

Merged
merged 51 commits into from
Apr 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
3836feb
Add resource: aws_rds_cluster_activity_stream
Fedomn Jul 18, 2020
c36565b
fix awsproviderlint and terrafmt checked issues
Fedomn Jul 18, 2020
591bcb3
rename arn to resource_arn to align with API naming
Fedomn Jul 19, 2020
7b99331
add ValidateFunc for resource_arn
Fedomn Jul 19, 2020
1277b03
use enums ActivityStreamMode
Fedomn Jul 19, 2020
e7e5347
use testAccCheckResourceDisappears instead of custom logic
Fedomn Jul 19, 2020
0b456f7
move all waiter functions and logic to internal rds package
Fedomn Jul 19, 2020
20891dd
use RandomWithPrefix instead of building a string
Fedomn Jul 20, 2020
c612054
use TestCheckResourceAttrPair to compare resources used
Fedomn Jul 20, 2020
8eb244e
fix dangling resources error when resource disappear
Fedomn Jul 20, 2020
21ce864
remove apply_immediately in arguments
Fedomn Jul 21, 2020
2163bdb
upgrade terraform-plugin-sdk to v2
Fedomn Nov 7, 2020
56be708
use tfawserr to replace awserr
Fedomn Nov 7, 2020
823e3f2
fix terrafmt and markdown-lint errors
Fedomn Nov 7, 2020
abcadf4
fix validate-terraform errors
Fedomn Nov 10, 2020
4be2a82
add ErrorCheck and fix doc code block language
Fedomn Mar 23, 2021
776a061
add rds cluster activity stream resource
jdstuart Dec 3, 2021
fa03d06
add rds cluster activity stream resource
jdstuart Dec 5, 2021
3b78253
update documentation for and register as a resource
jdstuart Dec 7, 2021
0569c3a
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Dec 7, 2021
b8f7c5b
fix bug in wait code
jdstuart Dec 7, 2021
34d0891
remove Timing from resource
jdstuart Dec 7, 2021
65e70fd
add lookup by DB Cluster ARN
jdstuart Dec 7, 2021
b4a60e9
remove debug print statements
jdstuart Dec 7, 2021
d8be772
update test to use acctest.ConfigCompose
jdstuart Dec 7, 2021
06a73e0
update test to use acctest.ConfigCompose
jdstuart Dec 7, 2021
a50be24
update acceptance test
jdstuart Dec 7, 2021
fbfff0a
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Dec 7, 2021
3188405
update docs
jdstuart Dec 7, 2021
b974f42
add .changelog entry
jdstuart Dec 7, 2021
fa70992
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Dec 8, 2021
cba58dc
fix website linting issue
jdstuart Dec 8, 2021
9a27cef
fix import and non used parameters
jdstuart Dec 8, 2021
a58cc9a
fix terrafmt errors
jdstuart Dec 8, 2021
58997bf
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Dec 8, 2021
968a99a
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Feb 5, 2022
13551b9
remove test and use instead of
jdstuart Feb 5, 2022
cea2b8c
remove tests and
jdstuart Feb 5, 2022
139e5e9
remove unused functions
jdstuart Feb 5, 2022
91f8a47
update website documentation. add input variable
jdstuart Feb 5, 2022
cd36d20
made engine_native_audit_fields_included optional
jdstuart Feb 6, 2022
fa5c677
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Feb 11, 2022
3a36e79
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Feb 15, 2022
14d11b5
fixed acceptance tests
jdstuart Feb 15, 2022
9e3d885
fixed acceptance tests
jdstuart Feb 16, 2022
b819118
Merge remote-tracking branch 'hashicorp/main' into f-aws_rds_cluster_…
jdstuart Feb 19, 2022
ad54701
fix semgrep errors
jdstuart Feb 19, 2022
0aa0d33
resolve merge conflicts
johnsonaj Apr 5, 2022
864d85b
r/aws_rds_cluster_activity_stream: add to provider
johnsonaj Apr 5, 2022
af4c2fd
r/aws_rds_cluster_activity_stream: minor changes to finder
johnsonaj Apr 6, 2022
ca4093e
docs: update to include avaiable regions
johnsonaj Apr 6, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .changelog/22097.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:new-resource
aws_rds_cluster_activity_stream
```
1 change: 1 addition & 0 deletions internal/provider/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -1703,6 +1703,7 @@ func Provider() *schema.Provider {
"aws_db_snapshot": rds.ResourceSnapshot(),
"aws_db_subnet_group": rds.ResourceSubnetGroup(),
"aws_rds_cluster": rds.ResourceCluster(),
"aws_rds_cluster_activity_stream": rds.ResourceClusterActivityStream(),
"aws_rds_cluster_endpoint": rds.ResourceClusterEndpoint(),
"aws_rds_cluster_instance": rds.ResourceClusterInstance(),
"aws_rds_cluster_parameter_group": rds.ResourceClusterParameterGroup(),
Expand Down
138 changes: 138 additions & 0 deletions internal/service/rds/cluster_activity_stream.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
package rds

import (
"context"
"fmt"
"log"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/rds"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation"
"github.com/hashicorp/terraform-provider-aws/internal/conns"
"github.com/hashicorp/terraform-provider-aws/internal/tfresource"
"github.com/hashicorp/terraform-provider-aws/internal/verify"
)

func ResourceClusterActivityStream() *schema.Resource {
return &schema.Resource{
CreateContext: resourceAwsRDSClusterActivityStreamCreate,
ReadContext: resourceAwsRDSClusterActivityStreamRead,
DeleteContext: resourceAwsRDSClusterActivityStreamDelete,
Importer: &schema.ResourceImporter{
StateContext: schema.ImportStatePassthroughContext,
},

Schema: map[string]*schema.Schema{
"resource_arn": {
Type: schema.TypeString,
Required: true,
ForceNew: true,
ValidateFunc: verify.ValidARN,
},
"kms_key_id": {
Type: schema.TypeString,
Required: true,
ForceNew: true,
},
"mode": {
Type: schema.TypeString,
Required: true,
ForceNew: true,
ValidateFunc: validation.StringInSlice(rds.ActivityStreamMode_Values(), false),
},
"kinesis_stream_name": {
Type: schema.TypeString,
Computed: true,
},
"engine_native_audit_fields_included": {
Type: schema.TypeBool,
Optional: true,
Default: false,
ForceNew: true,
},
},
}
}

func resourceAwsRDSClusterActivityStreamCreate(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
conn := meta.(*conns.AWSClient).RDSConn

resourceArn := d.Get("resource_arn").(string)

startActivityStreamInput := &rds.StartActivityStreamInput{
jdstuart marked this conversation as resolved.
Show resolved Hide resolved
ResourceArn: aws.String(resourceArn),
ApplyImmediately: aws.Bool(true),
KmsKeyId: aws.String(d.Get("kms_key_id").(string)),
Mode: aws.String(d.Get("mode").(string)),
EngineNativeAuditFieldsIncluded: aws.Bool(d.Get("engine_native_audit_fields_included").(bool)),
}

log.Printf("[DEBUG] RDS Cluster start activity stream input: %s", startActivityStreamInput)

resp, err := conn.StartActivityStream(startActivityStreamInput)
if err != nil {
return diag.FromErr(fmt.Errorf("error creating RDS Cluster Activity Stream: %s", err))
}

log.Printf("[DEBUG]: RDS Cluster start activity stream response: %s", resp)

d.SetId(resourceArn)

err = waitActivityStreamStarted(ctx, conn, d.Id())
if err != nil {
return diag.FromErr(err)
}

return resourceAwsRDSClusterActivityStreamRead(ctx, d, meta)
}

func resourceAwsRDSClusterActivityStreamRead(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
conn := meta.(*conns.AWSClient).RDSConn

log.Printf("[DEBUG] Finding DB Cluster (%s)", d.Id())
resp, err := FindDBClusterWithActivityStream(conn, d.Id())

if tfresource.NotFound(err) {
log.Printf("[WARN] RDS Cluster (%s) not found, removing from state", d.Id())
d.SetId("")
return nil
}

if err != nil {
return diag.FromErr(fmt.Errorf("error describing RDS Cluster (%s): %s", d.Id(), err))
}

d.Set("resource_arn", resp.DBClusterArn)
d.Set("kms_key_id", resp.ActivityStreamKmsKeyId)
d.Set("kinesis_stream_name", resp.ActivityStreamKinesisStreamName)
d.Set("mode", resp.ActivityStreamMode)

return nil
}

func resourceAwsRDSClusterActivityStreamDelete(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
conn := meta.(*conns.AWSClient).RDSConn

stopActivityStreamInput := &rds.StopActivityStreamInput{
ApplyImmediately: aws.Bool(true),
ResourceArn: aws.String(d.Id()),
}

log.Printf("[DEBUG] RDS Cluster stop activity stream input: %s", stopActivityStreamInput)

resp, err := conn.StopActivityStream(stopActivityStreamInput)
if err != nil {
return diag.FromErr(fmt.Errorf("error stopping RDS Cluster Activity Stream: %w", err))
}

log.Printf("[DEBUG] RDS Cluster stop activity stream response: %s", resp)

err = waitActivityStreamStopped(ctx, conn, d.Id())
if err != nil {
return diag.FromErr(err)
}

return nil
}
207 changes: 207 additions & 0 deletions internal/service/rds/cluster_activity_stream_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
package rds_test

import (
"fmt"
"regexp"
"testing"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/rds"
sdkacctest "github.com/hashicorp/terraform-plugin-sdk/v2/helper/acctest"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-plugin-sdk/v2/terraform"
"github.com/hashicorp/terraform-provider-aws/internal/acctest"
"github.com/hashicorp/terraform-provider-aws/internal/conns"
tfrds "github.com/hashicorp/terraform-provider-aws/internal/service/rds"
"github.com/hashicorp/terraform-provider-aws/internal/tfresource"
)

func TestAccAWSRDSClusterActivityStream_basic(t *testing.T) {
var dbCluster rds.DBCluster
clusterName := sdkacctest.RandomWithPrefix("tf-testacc-aurora-cluster")
instanceName := sdkacctest.RandomWithPrefix("tf-testacc-aurora-instance")
resourceName := "aws_rds_cluster_activity_stream.test"
rdsClusterResourceName := "aws_rds_cluster.test"
kmsKeyResourceName := "aws_kms_key.test"

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, rds.EndpointsID),
Providers: acctest.Providers,
CheckDestroy: testAccCheckAWSClusterActivityStreamDestroy,
Steps: []resource.TestStep{
{
Config: testAccAWSClusterActivityStreamConfig(clusterName, instanceName),
Check: resource.ComposeTestCheckFunc(
testAccCheckAWSRDSClusterActivityStreamExists(resourceName, &dbCluster),
testAccCheckAWSRDSClusterActivityStreamAttributes(&dbCluster),
acctest.MatchResourceAttrRegionalARN(resourceName, "resource_arn", "rds", regexp.MustCompile("cluster:"+clusterName)),
resource.TestCheckResourceAttrPair(resourceName, "resource_arn", rdsClusterResourceName, "arn"),
resource.TestCheckResourceAttrPair(resourceName, "kms_key_id", kmsKeyResourceName, "key_id"),
resource.TestCheckResourceAttrSet(resourceName, "kinesis_stream_name"),
resource.TestCheckResourceAttr(resourceName, "mode", rds.ActivityStreamModeAsync),
),
},
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"engine_native_audit_fields_included"},
},
},
})
}

func TestAccAWSRDSClusterActivityStream_disappears(t *testing.T) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be worth to add a disappears test if the db is delete whether the code knows how to handle this case

var dbCluster rds.DBCluster
clusterName := sdkacctest.RandomWithPrefix("tf-testacc-aurora-cluster")
instanceName := sdkacctest.RandomWithPrefix("tf-testacc-aurora-instance")
resourceName := "aws_rds_cluster_activity_stream.test"

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acctest.PreCheck(t) },
ErrorCheck: acctest.ErrorCheck(t, rds.EndpointsID),
Providers: acctest.Providers,
CheckDestroy: testAccCheckAWSClusterActivityStreamDestroy,
Steps: []resource.TestStep{
{
Config: testAccAWSClusterActivityStreamConfig(clusterName, instanceName),
Check: resource.ComposeTestCheckFunc(
testAccCheckAWSRDSClusterActivityStreamExists(resourceName, &dbCluster),
acctest.CheckResourceDisappears(acctest.Provider, tfrds.ResourceClusterActivityStream(), resourceName),
),
ExpectNonEmptyPlan: true,
},
},
})
}

func testAccCheckAWSRDSClusterActivityStreamExists(resourceName string, dbCluster *rds.DBCluster) resource.TestCheckFunc {
return testAccCheckAWSRDSClusterActivityStreamExistsWithProvider(resourceName, dbCluster, acctest.Provider)
}

func testAccCheckAWSRDSClusterActivityStreamExistsWithProvider(resourceName string, dbCluster *rds.DBCluster, provider *schema.Provider) resource.TestCheckFunc {
return func(s *terraform.State) error {
rs, ok := s.RootModule().Resources[resourceName]
if !ok {
return fmt.Errorf("not found: %s", resourceName)
}

if rs.Primary.ID == "" {
return fmt.Errorf("DBCluster ID is not set")
}

conn := provider.Meta().(*conns.AWSClient).RDSConn

response, err := tfrds.FindDBClusterWithActivityStream(conn, rs.Primary.ID)

if err != nil {
return err
}

*dbCluster = *response
return nil
}
}

func testAccCheckAWSRDSClusterActivityStreamAttributes(v *rds.DBCluster) resource.TestCheckFunc {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is these need if attributes are already checked?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It checks properties returned from AWS and not only attributes stored with the resource. i.e. the ActivityStreamStatus which is expected to be in state Started

return func(s *terraform.State) error {

if aws.StringValue(v.DBClusterArn) == "" {
return fmt.Errorf("empty RDS Cluster arn")
}

if aws.StringValue(v.ActivityStreamKmsKeyId) == "" {
return fmt.Errorf("empty RDS Cluster activity stream kms key id")
}

if aws.StringValue(v.ActivityStreamKinesisStreamName) == "" {
return fmt.Errorf("empty RDS Cluster activity stream kinesis stream name")
}

if aws.StringValue(v.ActivityStreamStatus) != rds.ActivityStreamStatusStarted {
return fmt.Errorf("incorrect activity stream status: expected: %s, got: %s", rds.ActivityStreamStatusStarted, aws.StringValue(v.ActivityStreamStatus))
}

if aws.StringValue(v.ActivityStreamMode) != rds.ActivityStreamModeSync && aws.StringValue(v.ActivityStreamMode) != rds.ActivityStreamModeAsync {
return fmt.Errorf("incorrect activity stream mode: expected: sync or async, got: %s", aws.StringValue(v.ActivityStreamMode))
}

return nil
}
}

func testAccCheckAWSClusterActivityStreamDestroy(s *terraform.State) error {
return testAccCheckAWSClusterActivityStreamDestroyWithProvider(s, acctest.Provider)
}

func testAccCheckAWSClusterActivityStreamDestroyWithProvider(s *terraform.State, provider *schema.Provider) error {
conn := provider.Meta().(*conns.AWSClient).RDSConn

for _, rs := range s.RootModule().Resources {
if rs.Type != "aws_rds_cluster_activity_stream" {
continue
}

var err error

_, err = tfrds.FindDBClusterWithActivityStream(conn, rs.Primary.ID)
if err != nil {
// Return nil if the cluster is already destroyed
if tfresource.NotFound(err) {
return nil
}
return err
}

return err
}

return nil
}

func testAccAWSClusterActivityStreamConfigBase(clusterName, instanceName string) string {
return fmt.Sprintf(`
data "aws_availability_zones" "available" {
state = "available"
}

resource "aws_kms_key" "test" {
description = "Testing for AWS RDS Cluster Activity Stream"
deletion_window_in_days = 7
}

resource "aws_rds_cluster" "test" {
cluster_identifier = "%[1]s"
availability_zones = ["${data.aws_availability_zones.available.names[0]}", "${data.aws_availability_zones.available.names[1]}", "${data.aws_availability_zones.available.names[2]}"]
master_username = "foo"
master_password = "mustbeeightcharaters"
skip_final_snapshot = true
deletion_protection = false
engine = "aurora-postgresql"
engine_version = "11.9"
}

resource "aws_rds_cluster_instance" "test" {
identifier = "%[2]s"
cluster_identifier = aws_rds_cluster.test.id
engine = aws_rds_cluster.test.engine
instance_class = "db.r6g.large"
}
`, clusterName, instanceName)
}

func testAccAWSClusterActivityStreamConfig(clusterName, instanceName string) string {
return acctest.ConfigCompose(
testAccAWSClusterActivityStreamConfigBase(clusterName, instanceName),
`
resource "aws_rds_cluster_activity_stream" "test" {
resource_arn = aws_rds_cluster.test.arn
kms_key_id = aws_kms_key.test.key_id
mode = "async"

depends_on = [aws_rds_cluster_instance.test]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should avoid this and add some retry logic in create if it needs to wait for the instance to be created.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's no depends_on it can also cause a race condition when deleting the entire stack. i.e. If AWS is busy destroying either aws_rds_cluster or aws_rds_cluster_activity_stream then the destroy operation of the other will fail stating the cluster is in invalid state to perform the operation.

We can wait at initialization, but to wait at destroy would require changes to the aws_rds_cluster resources too.

}
`)
}
Loading