Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Support Amazon VPC Flow Logs to Kinesis Data Firehose #27340

Merged
merged 4 commits into from
Oct 25, 2022

Conversation

BhanuCh
Copy link

@BhanuCh BhanuCh commented Oct 19, 2022

Description

Adds validation for VPC Flow logs destination type as Kinesis Data Firehose

Relations

Closes #26730

Output from Acceptance Testing

$make testacc TESTARGS='-run=TestAccVPCFlowLog' PKG=ec2
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/ec2/... -v -count 1 -parallel 2  -run=TestAccVPCFlowLog -timeout 180m
=== RUN   TestAccVPCFlowLog_vpcID
=== PAUSE TestAccVPCFlowLog_vpcID
=== RUN   TestAccVPCFlowLog_logFormat
=== PAUSE TestAccVPCFlowLog_logFormat
=== RUN   TestAccVPCFlowLog_subnetID
=== PAUSE TestAccVPCFlowLog_subnetID
=== RUN   TestAccVPCFlowLog_transitGatewayID
=== PAUSE TestAccVPCFlowLog_transitGatewayID
=== RUN   TestAccVPCFlowLog_transitGatewayAttachmentID
=== PAUSE TestAccVPCFlowLog_transitGatewayAttachmentID
=== RUN   TestAccVPCFlowLog_LogDestinationType_cloudWatchLogs
=== PAUSE TestAccVPCFlowLog_LogDestinationType_cloudWatchLogs
=== RUN   TestAccVPCFlowLog_LogDestinationType_kinesisFirehose
=== PAUSE TestAccVPCFlowLog_LogDestinationType_kinesisFirehose
=== RUN   TestAccVPCFlowLog_LogDestinationType_s3
=== PAUSE TestAccVPCFlowLog_LogDestinationType_s3
=== RUN   TestAccVPCFlowLog_LogDestinationTypeS3_invalid
=== PAUSE TestAccVPCFlowLog_LogDestinationTypeS3_invalid
=== RUN   TestAccVPCFlowLog_LogDestinationTypeS3DO_plainText
=== PAUSE TestAccVPCFlowLog_LogDestinationTypeS3DO_plainText
=== RUN   TestAccVPCFlowLog_LogDestinationTypeS3DOPlainText_hiveCompatible
=== PAUSE TestAccVPCFlowLog_LogDestinationTypeS3DOPlainText_hiveCompatible
=== RUN   TestAccVPCFlowLog_LogDestinationTypeS3DO_parquet
=== PAUSE TestAccVPCFlowLog_LogDestinationTypeS3DO_parquet
=== RUN   TestAccVPCFlowLog_LogDestinationTypeS3DOParquet_hiveCompatible
=== PAUSE TestAccVPCFlowLog_LogDestinationTypeS3DOParquet_hiveCompatible
=== RUN   TestAccVPCFlowLog_LogDestinationTypeS3DOParquetHiveCompatible_perHour
=== PAUSE TestAccVPCFlowLog_LogDestinationTypeS3DOParquetHiveCompatible_perHour
=== RUN   TestAccVPCFlowLog_LogDestinationType_maxAggregationInterval
=== PAUSE TestAccVPCFlowLog_LogDestinationType_maxAggregationInterval
=== RUN   TestAccVPCFlowLog_tags
=== PAUSE TestAccVPCFlowLog_tags
=== RUN   TestAccVPCFlowLog_disappears
=== PAUSE TestAccVPCFlowLog_disappears
=== CONT  TestAccVPCFlowLog_vpcID
=== CONT  TestAccVPCFlowLog_LogDestinationTypeS3DO_plainText
--- PASS: TestAccVPCFlowLog_LogDestinationTypeS3DO_plainText (21.89s)
=== CONT  TestAccVPCFlowLog_LogDestinationType_cloudWatchLogs
--- PASS: TestAccVPCFlowLog_vpcID (34.36s)
=== CONT  TestAccVPCFlowLog_LogDestinationTypeS3_invalid
--- PASS: TestAccVPCFlowLog_LogDestinationType_cloudWatchLogs (20.21s)
=== CONT  TestAccVPCFlowLog_transitGatewayID
--- PASS: TestAccVPCFlowLog_LogDestinationTypeS3_invalid (11.94s)
=== CONT  TestAccVPCFlowLog_LogDestinationType_s3
--- PASS: TestAccVPCFlowLog_LogDestinationType_s3 (21.65s)
=== CONT  TestAccVPCFlowLog_transitGatewayAttachmentID
--- PASS: TestAccVPCFlowLog_transitGatewayID (175.61s)
=== CONT  TestAccVPCFlowLog_LogDestinationType_kinesisFirehose
--- PASS: TestAccVPCFlowLog_transitGatewayAttachmentID (264.53s)
=== CONT  TestAccVPCFlowLog_subnetID
--- PASS: TestAccVPCFlowLog_subnetID (19.00s)
=== CONT  TestAccVPCFlowLog_LogDestinationTypeS3DOParquetHiveCompatible_perHour
--- PASS: TestAccVPCFlowLog_LogDestinationType_kinesisFirehose (146.47s)
=== CONT  TestAccVPCFlowLog_LogDestinationTypeS3DO_parquet
--- PASS: TestAccVPCFlowLog_LogDestinationTypeS3DOParquetHiveCompatible_perHour (18.98s)
=== CONT  TestAccVPCFlowLog_disappears
--- PASS: TestAccVPCFlowLog_LogDestinationTypeS3DO_parquet (18.68s)
=== CONT  TestAccVPCFlowLog_LogDestinationTypeS3DOParquet_hiveCompatible
--- PASS: TestAccVPCFlowLog_disappears (15.63s)
=== CONT  TestAccVPCFlowLog_tags
--- PASS: TestAccVPCFlowLog_LogDestinationTypeS3DOParquet_hiveCompatible (18.87s)
=== CONT  TestAccVPCFlowLog_LogDestinationTypeS3DOPlainText_hiveCompatible
--- PASS: TestAccVPCFlowLog_LogDestinationTypeS3DOPlainText_hiveCompatible (17.75s)
=== CONT  TestAccVPCFlowLog_LogDestinationType_maxAggregationInterval
--- PASS: TestAccVPCFlowLog_tags (40.92s)
=== CONT  TestAccVPCFlowLog_logFormat
--- PASS: TestAccVPCFlowLog_LogDestinationType_maxAggregationInterval (17.21s)
--- PASS: TestAccVPCFlowLog_logFormat (32.05s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/ec2	460.028s

@github-actions
Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/L Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/vpc Issues and PRs that pertain to the vpc service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. needs-triage Waiting for first response or review from a maintainer. labels Oct 19, 2022
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @BhanuCh 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTOR guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@ewbankkit ewbankkit added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Oct 20, 2022
Copy link
Contributor

@schuylr schuylr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I also verified that the vpc_flow_log.go file pulls the validation directly from the AWS SDK:

ValidateFunc: validation.StringInSlice(ec2.LogDestinationType_Values(), false),

Therefore, no modification was necessary for the resource itself. Only acceptance tests and documentation updates were needed.

internal/service/ec2/vpc_flow_log_test.go Outdated Show resolved Hide resolved
internal/service/ec2/vpc_flow_log_test.go Outdated Show resolved Hide resolved
website/docs/r/flow_log.html.markdown Outdated Show resolved Hide resolved
@BhanuCh BhanuCh requested a review from schuylr October 21, 2022 22:33
Copy link
Contributor

@schuylr schuylr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for implementing those changes and submitting this PR! 🚀

Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

% make testacc TESTS=TestAccVPCFlowLog_LogDestinationType_kinesisFirehose PKG=ec2
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/ec2/... -v -count 1 -parallel 20 -run='TestAccVPCFlowLog_LogDestinationType_kinesisFirehose'  -timeout 180m
=== RUN   TestAccVPCFlowLog_LogDestinationType_kinesisFirehose
=== PAUSE TestAccVPCFlowLog_LogDestinationType_kinesisFirehose
=== CONT  TestAccVPCFlowLog_LogDestinationType_kinesisFirehose
--- PASS: TestAccVPCFlowLog_LogDestinationType_kinesisFirehose (136.62s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/ec2	140.774s

@ewbankkit
Copy link
Contributor

@BhanuCh Thanks for the contribution 🎉 👏.

@ewbankkit ewbankkit merged commit 8432b28 into hashicorp:main Oct 25, 2022
@github-actions github-actions bot added this to the v4.37.0 milestone Oct 25, 2022
@github-actions
Copy link

This functionality has been released in v4.37.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/vpc Issues and PRs that pertain to the vpc service. size/L Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

VPC Flow Logs can now be delivered to Amazon Kinesis Firehose
3 participants