When building infrastructure in AWS, there are always things to consider for any project. For example, security, IAM, cost, log storage and notification, etc... It is quite difficult to build a separate Terraform for each project for all the things that must be considered.
In this repository, we use Terraform to build the baseline configuration.
- Base
- Functions
- Settings
- Logs
- Terraform
https://www.terraform.io/ - Slack
For notifications, you will need Slack, OAuthToken, and ChannelID.
https://slack.com/
https://slack.dev/node-slack-sdk/getting-started
AWS CloudTrail is a service for governance, compliance, operational and risk auditing of AWS accounts.CloudTrail enables you to log, continuously monitor and retain account activity across your AWS infrastructure.
After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, Slack notifications will be sent.
You will be notified with a message similar to the following.
AWS Config is a service that allows you to evaluate, audit, and review the configuration of AWS resources. Config continuously monitors and records the configuration of AWS resources and automatically evaluates the recorded configuration against the desired settings. Config allows you to review configuration and association changes between AWS resources, examine detailed resource configuration history, and verify overall compliance with settings specified in company guidelines. This simplifies compliance audits, security analysis, change management, and operational troubleshooting.
After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, Slack notifications will be sent.
You will be notified with a message similar to the following.
Amazon GuardDuty is a threat detection service that continuously monitors for malicious or unauthorized activity in order to protect AWS accounts, workloads, and data stored in Amazon S3.
After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, Slack notifications will be sent.
You will be notified with a message similar to the following.
The AWS Security Hub provides a comprehensive view of security alerts and security status across all your AWS accounts. A wide range of sophisticated security tools are at your disposal, from firewall and endpoint protection to vulnerability and compliance scanners.
The three security standards provided in the Security Hub are addressed to the best of our ability.
- AWS Foundational Security Best Practices
- CIS AWS Foundations Benchmark
- PCI DSS v3.2.1
The following is the security score when only this Terraform is applied.
You should be aware that the score will not be accurate until you re-evaluate it after building.
AWS Budgets provides the ability to set up custom budgets and be alerted when costs or usage exceed (or are expected to exceed) the budgeted amount or amounts.
After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, you will receive Slack notifications at the specified time (default is 18:00 JST daily). An email will also be sent if the specified cost limit is exceeded.
AWS Compute Optimizer recommends optimal AWS resources for your workloads to reduce costs and improve performance by using machine learning to analyze historical utilization metrics. Over-provisioning resources can lead to unnecessary infrastructure cost, and under-provisioning resources can lead to poor application performance. Compute Optimizer helps you choose optimal configurations for three types of AWS resources: Amazon EC2 instances, Amazon EBS volumes, and AWS Lambda functions, based on your utilization data.
You can set the policy to assign to IAM groups. You can also make the virtual MFA setting mandatory as a base policy.
You can also configure the IAM Switch Role.
You can create an IAM User and Group.
Overall, all resources created in Terraform will have the same TAG, and Resource Groups will be filtered by that TAG.
AWS Trusted Advisor is a fully managed service that provides guidance on how to follow AWS best practices. Improve security and performance, reduce costs, and monitor service limitations.
After configuring the Slack channel, adding the Slack app, and setting the OAuthToken, you will be able to receive Slack notifications at the specified time (default is 9:00 JST daily).
However, Trusted Advisor requires the support plan to be signed up for the Business or Enterprise plan. The default setting is false.
-
Remove the access key from the root account
Since this is a security issue, let's remove the access key from the root account from the management console. -
Manual creation of IAM user and IAM group to run Terraform
Create an IAM user and an IAM group from the management console in order to run Terraform. Create an IAM group (pseudonym: deploy). Attach AdministratorAccess as the policy. Create an IAM user (pseudonym: terraform), giving it only Programmatic access for Access Type, and add it to the IAM group (pseudonym: deploy). -
Create an S3 to store the Terraform State
Create an S3 from the management console to manage the Terraform State. However, if you have an environment where you can run the aws command and profile already configured, you can create an S3 by running the following command.
https://github.com/y-miyazaki/cloud-commands/blob/master/cmd/awstfinitstate
# awstfinitstate -h
This command creates a S3 Bucket for Terraform State.
You can also add random hash to bucket name suffix.
Usage:
awstfinitstate -r {region} -b {bucket name} -p {profile}[<options>]
awstfinitstate -r ap-northeast-1 -b terraform-state
awstfinitstate -r ap-northeast-1 -b terraform-state -p default -s
Options:
-b {bucket name} S3 bucket name
-p {aws profile name} Name of AWS profile
-r {region} S3 region
-s If set, a random hash will suffix bucket name.
-h Usage awstfinitstate
# awstfinitstate -r ap-northeast-1 -b terraform-state -p default -s
~
~
~
~
~
~
~
~
~
~
~
~
~
--------------------------------------------------------------
bucket_name: terraform-state-xxxxxxxxxx
region: ap-northeast-1
---------------------------------------------------------------
terraform.{environment}.tfvars file to configure for each environment
You need to rename the linked file terraform.example.tfvars and change each variable for your environment. The variables that need to be changed are marked with TODO comments; search for them in TODO. -
main_provider.tf file to set for each environment
Rename the linked file main_provider.tf.example to main_provider.tf. After that, you need to change each parameter. The variables that need to be changed are marked with TODO comments, search for them in TODO.
#--------------------------------------------------------------
# Terraform Provider
#--------------------------------------------------------------
terraform {
required_version = ">=0.13"
required_providers {
aws = {
source = "hashicorp/aws"
version = ">=3.29.1"
}
}
backend "s3" {
# TODO: need to change bucket for terraform state.
bucket = "xxxxxxxxxxxxxxxx"
# TODO: need to change bucket key for terraform state.
key = "xxxxxxxxxx"
# TODO: need to change profile for terraform state.
profile = "default"
# TODO: need to change region for terraform state.
region = "ap-northeast-1"
}
}
#--------------------------------------------------------------
# AWS Provider
# access key and secret key should not use.
#--------------------------------------------------------------
provider "aws" {
# TODO: need to change profile.
profile = "default"
# TODO: need to change region.
region = "ap-northeast-1"
}- Running Terraform
Run the terraform command: terraform init followed by terraform apply. You may find that terraform apply fails due to conflicts or other problems, so run it again and it will succeed.
bash-5.1# terraform init
There are some problems with the CLI configuration:
Error: The specified plugin cache dir /root/.terraform.d/plugin-cache cannot be opened: stat /root/.terraform.d/plugin-cache: no such file or directory
As a result of the above problems, Terraform may not behave as intended.
Initializing modules...
Initializing the backend...
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/template from the dependency lock file
- Installing hashicorp/aws v3.29.1...
- Installed hashicorp/aws v3.29.1 (signed by HashiCorp)
- Installing hashicorp/random v3.1.0...
- Installed hashicorp/random v3.1.0 (signed by HashiCorp)
- Installing hashicorp/template v2.2.0...
- Installed hashicorp/template v2.2.0 (signed by HashiCorp)
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.bash-5.1# terraform apply --auto-approve -var-file=terraform.example.tfvars
module.aws_recipes_s3_bucket_log_log.random_id.this: Creating...
random_id.this: Creating...
module.aws_recipes_s3_bucket_log_logdom_id.this: Creation complete after 0s [id=wiatHg]
random_id.this: Creation complete after 0s [id=uqe0bU7J]
module.aws_recipes_security_default_vpc.aws_default_subnet.this[1]: Creating...
...
...
...
Apply complete! resources: x added, x changed, 0 destroyed.This is a description of the S3 bucket that will be created and the data in the bucket.
| Category | bucket | Directory | Description | Note |
|---|---|---|---|---|
| AWS Config | aws-log-common | /AWSLogs/{accountID}/Config/{region}/yyyy/m/d/ConfigHistory/ | AWS Config Compliance History Timeline for Resources. | https://docs.aws.amazon.com/config/latest/developerguide/view-compliance-history.html |
| AWS Config | aws-log-common | /AWSLogs/{accountID}/Config/{region}/yyyy/m/d/ConfigSnapshot/ | AWS Config snapshot. | https://docs.aws.amazon.com/config/latest/developerguide/deliver-snapshot-cli.html |
| AWS Config | aws-log-common | /AWSLogs/{accountID}/Config/ConfigWritabilityCheckFile/yyyy/m/d | This is a test file to confirm that Config can be written to the S3 bucket normally. | |
| AWS Config | aws-log-common | /AWSLogs/{accountID}/CloudTrail/AccessLog | This is the access log of the CloudTrail bucket. | |
| AWS CloudTrail | aws-log-cloudtrail | /AWSLogs/{accountID}/CloudTrail-Digest/{region}/yyyy/mm/dd | Each digest file contains the names of the log files that were delivered to your Amazon S3 bucket during the last hour, the hash values for those log files, and the digital signature of the previous digest file. The signature for the current digest file is stored in the metadata properties of the digest file object. The digital signatures and hashes are used for validating the integrity of the log files and of the digest file itself. | https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-validation-digest-file-structure.html |
| AWS CloudTrail | aws-log-cloudtrail | /AWSLogs/{accountID}/CloudTrail-Insight/{region}/yyyy/mm/dd | CloudTrail Insights can help you detect unusual API activity in your AWS account by raising Insights events. CloudTrail Insights measures your normal patterns of API call volume, also called the baseline, and generates Insights events when the volume is outside normal patterns. Insights events are generated for write management APIs. | https://docs.aws.amazon.com/awscloudtrail/latest/userguide/log-insights-events-with-cloudtrail.html |
| AWS CloudTrail | aws-log-cloudtrail | /AWSLogs/{accountID}/CloudTrail/{region}/yyyy/mm/dd | It is recorded as an event in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface. | https://docs.aws.amazon.com/awscloudtrail/latest/userguide/get-and-view-cloudtrail-log-files.html |
| AWS Log | aws-log-application | /Logs | Application log from CloudWatch Logs. |
Author: Yoshiaki Miyazaki
Contact: https://github.com/y-miyazaki