Skip to content

Commit

Permalink
Improve cloudwatch and awsfargate documentation (#23862)
Browse files Browse the repository at this point in the history
  • Loading branch information
kaiyan-sheng authored Feb 5, 2021
1 parent 4cb42cd commit 35ca7e3
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 3 deletions.
32 changes: 32 additions & 0 deletions x-pack/metricbeat/module/aws/cloudwatch/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,38 @@ will get lost. Metrics from namespace AWS/Billing are sent to Cloudwatch every
several hours. By querying from AWS/Billing namespace every 300 seconds,
additional costs will occur.

[float]
==== Example 3
Depends on the configuration and number of services in the AWS account, the number
of API calls may get too big to cause high API cost. In order to reduce the number
of API calls, we recommend users to use this configuration below as an example.

* *metrics.name*: Only collect a sub list of metrics that are useful to your use case.
* *metrics.statistic*: By default, cloudwatch metricset will make API calls to
get all stats like average, max, min, sum and etc. If the user knows which
statistics method is most useful, specify it in the configuration.
* *metrics.dimensions*: Different AWS services report different dimensions in their
CloudWatch metrics. For example, https://docs.aws.amazon.com/emr/latest/ManagementGuide/UsingEMR_ViewingMetrics.html[EMR metrics]
can have either `JobFlowId` dimension or `JobId` dimension. If user knows which
specific dimension is useful, it can be specified in this configuration option.

[source,yaml]
----
- module: aws
period: 5m
metricsets:
- cloudwatch
regions: us-east-1
metrics:
- namespace: AWS/ElasticMapReduce
name: ["S3BytesWritten", "S3BytesRead", "HDFSUtilization", "TotalLoad"]
resource_type: elasticmapreduce
statistic: ["Average"]
dimensions:
- name: JobId
value: "*"
----

[float]
=== More examples
With the configuration below, users will be able to collect cloudwatch metrics
Expand Down
2 changes: 1 addition & 1 deletion x-pack/metricbeat/module/awsfargate/cloudformation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Resources:
ExecutionRoleArn: !Ref ExecutionRole
ContainerDefinitions:
- Name: metricbeat-container
Image: docker.elastic.co/beats/metricbeat:7.11.0-SNAPSHOT
Image: docker.elastic.co/beats/metricbeat:8.0.0-SNAPSHOT
Secrets:
- Name: ELASTIC_CLOUD_ID
ValueFrom: !Ref CloudIDArn
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Resources:
ExecutionRoleArn: !Ref ExecutionRole
ContainerDefinitions:
- Name: metricbeat-container
Image: docker.elastic.co/beats/metricbeat:7.11.0-SNAPSHOT
Image: docker.elastic.co/beats/metricbeat:8.0.0-SNAPSHOT
Secrets:
- Name: ELASTIC_CLOUD_ID
ValueFrom: !Ref CloudIDArn
Expand Down Expand Up @@ -158,7 +158,9 @@ Resources:

[float]
==== Create CloudFormation Stack
Once the CloudFormation template is saved locally into `clouformation.yml`, AWS
When copying the CloudFormation template, please make sure the Metricbeat
container image is the correct version.
Once the template is saved locally into `clouformation.yml`, AWS
CLI can be used to create a stack using one command:
----
aws --region us-east-1 cloudformation create-stack --stack-name <your-stack-name> --template-body file://./cloudformation.yml --capabilities CAPABILITY_NAMED_IAM --parameters ParameterKey=SubnetID,ParameterValue=<subnet-id> ParameterKey=CloudAuthArn,ParameterValue=<cloud-auth-arn> ParameterKey=CloudIDArn,ParameterValue=<cloud-id-arn> ParameterKey=ClusterName,ParameterValue=<cluster-name> ParameterKey=RoleName,ParameterValue=<role-name> ParameterKey=TaskName,ParameterValue=<task-name> ParameterKey=ServiceName,ParameterValue=<service-name> ParameterKey=LogGroupName,ParameterValue=<log-group-name>
Expand Down

0 comments on commit 35ca7e3

Please sign in to comment.