Skip to content

Commit

Permalink
Merge branch 'v2' into feat/parallel-batch
Browse files Browse the repository at this point in the history
  • Loading branch information
@jeromevdl
jeromevdl authored Jun 25, 2024
2 parents af2fda9 + 1317da4 commit b51a6de
Show file tree
Hide file tree
Showing 11 changed files with 127 additions and 93 deletions.
112 changes: 37 additions & 75 deletions docs/core/metrics.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,7 @@ If you're new to Amazon CloudWatch, there are two terminologies you must be awar

## Install

Depending on your version of Java (either Java 1.8 or 11+), the configuration slightly changes.

=== "Maven Java 11+"
=== "Maven"

```xml hl_lines="3-7 16 18 24-27"
<dependencies>
Expand Down Expand Up @@ -75,52 +73,7 @@ If you're new to Amazon CloudWatch, there are two terminologies you must be awar
</build>
```

=== "Maven Java 1.8"

```xml hl_lines="3-7 16 18 24-27"
<dependencies>
...
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-metrics</artifactId>
<version>{{ powertools.version }}</version>
</dependency>
...
</dependencies>
...
<!-- configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lambda-powertools-java aspects into your project -->
<build>
<plugins>
...
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>aspectj-maven-plugin</artifactId>
<version>1.14.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<complianceLevel>1.8</complianceLevel>
<aspectLibraries>
<aspectLibrary>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-metrics</artifactId>
</aspectLibrary>
</aspectLibraries>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
...
</plugins>
</build>
```

=== "Gradle Java 11+"
=== "Gradle"

```groovy hl_lines="3 11"
plugins {
Expand All @@ -140,34 +93,14 @@ If you're new to Amazon CloudWatch, there are two terminologies you must be awar
targetCompatibility = 11
```

=== "Gradle Java 1.8"

```groovy hl_lines="3 11"
plugins {
id 'java'
id 'io.freefair.aspectj.post-compile-weaving' version '6.6.3'
}
repositories {
mavenCentral()
}
dependencies {
aspect 'software.amazon.lambda:powertools-metrics:{{ powertools.version }}'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
```

## Getting started

Metric has two global settings that will be used across all metrics emitted:

Setting | Description | Environment variable | Constructor parameter
------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------- | -------------------------------------------------
**Metric namespace** | Logical container where all metrics will be placed e.g. `ServerlessAirline` | `POWERTOOLS_METRICS_NAMESPACE` | `namespace`
**Service** | Optionally, sets **service** metric dimension across all metrics e.g. `payment` | `POWERTOOLS_SERVICE_NAME` | `service`
| Setting | Description | Environment variable | Constructor parameter |
|----------------------|---------------------------------------------------------------------------------|--------------------------------|-----------------------|
| **Metric namespace** | Logical container where all metrics will be placed e.g. `ServerlessAirline` | `POWERTOOLS_METRICS_NAMESPACE` | `namespace` |
| **Service** | Optionally, sets **service** metric dimension across all metrics e.g. `payment` | `POWERTOOLS_SERVICE_NAME` | `service` |

!!! tip "Use your application or main service as the metric namespace to easily group all metrics"

Expand Down Expand Up @@ -198,7 +131,7 @@ Setting | Description | Environment variable | Constructor parameter
@Override
@Metrics(namespace = "ExampleApplication", service = "booking")
public Object handleRequest(Object input, Context context) {
...
// ...
}
}
```
Expand All @@ -224,7 +157,7 @@ You can create metrics using `putMetric`, and manually create dimensions for all
public Object handleRequest(Object input, Context context) {
metricsLogger.putDimensions(DimensionSet.of("environment", "prod"));
metricsLogger.putMetric("SuccessfulBooking", 1, Unit.COUNT);
...
// ...
}
}
```
Expand All @@ -234,6 +167,35 @@ You can create metrics using `putMetric`, and manually create dimensions for all
!!! note "Metrics overflow"
CloudWatch EMF supports a max of 100 metrics. Metrics utility will flush all metrics when adding the 100th metric while subsequent metrics will be aggregated into a new EMF object, for your convenience.


### Adding high-resolution metrics

You can create [high-resolution metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#high-resolution-metrics)
passing a `storageResolution` to the `putMetric` method:

=== "HigResMetricsHandler.java"

```java hl_lines="3 13"
import software.amazon.lambda.powertools.metrics.Metrics;
import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
import software.amazon.cloudwatchlogs.emf.model.StorageResolution;

public class MetricsEnabledHandler implements RequestHandler<Object, Object> {

MetricsLogger metricsLogger = MetricsUtils.metricsLogger();

@Override
@Metrics(namespace = "ExampleApplication", service = "booking")
public Object handleRequest(Object input, Context context) {
// ...
metricsLogger.putMetric("SuccessfulBooking", 1, Unit.COUNT, StorageResolution.HIGH);
}
}
```

!!! info "When is it useful?"
High-resolution metrics are data with a granularity of one second and are very useful in several situations such as telemetry, time series, real-time incident management, and others.

### Flushing metrics

The `@Metrics` annotation **validates**, **serializes**, and **flushes** all your metrics. During metrics validation,
Expand Down
2 changes: 1 addition & 1 deletion examples/powertools-examples-batch/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<aspectj.version>1.9.20.1</aspectj.version>
<sdk.version>2.25.26</sdk.version>
<sdk.version>2.26.7</sdk.version>
</properties>

<dependencies>
Expand Down
3 changes: 3 additions & 0 deletions examples/powertools-examples-core-utilities/sam/src/main/java/helloworld/App.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
import software.amazon.cloudwatchlogs.emf.model.StorageResolution;
import software.amazon.cloudwatchlogs.emf.model.Unit;
import software.amazon.lambda.powertools.logging.Logging;
import software.amazon.lambda.powertools.metrics.Metrics;
Expand Down Expand Up @@ -64,6 +65,8 @@ public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEv
metric.setDimensions(DimensionSet.of("AnotherService1", "CustomService1"));
});

metricsLogger().putMetric("CustomMetric3", 1, Unit.COUNT, StorageResolution.HIGH);

MDC.put("test", "willBeLogged");

APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent()
Expand Down
8 changes: 4 additions & 4 deletions pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@
<log4j.version>2.20.0</log4j.version>
<log4j.version>2.23.1</log4j.version>
<slf4j.version>2.0.7</slf4j.version>
<jackson.version>2.17.0</jackson.version>
<jackson.version>2.17.1</jackson.version>
<aws.sdk.version>2.25.35</aws.sdk.version>
<aws.xray.recorder.version>2.15.1</aws.xray.recorder.version>
<payloadoffloading-common.version>2.1.3</payloadoffloading-common.version>
<aws.xray.recorder.version>2.16.0</aws.xray.recorder.version>
<payloadoffloading-common.version>2.2.0</payloadoffloading-common.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<lambda.core.version>1.2.3</lambda.core.version>
<lambda.events.version>3.11.2</lambda.events.version>
Expand All @@ -88,7 +88,7 @@
<maven-source-plugin.version>3.3.0</maven-source-plugin.version>
<maven-gpg-plugin.version>3.2.4</maven-gpg-plugin.version>
<junit.version>5.10.2</junit.version>
<aws-embedded-metrics.version>1.0.6</aws-embedded-metrics.version>
<aws-embedded-metrics.version>4.1.2</aws-embedded-metrics.version>
<jmespath.version>0.6.0</jmespath.version>
<elastic.version>1.6.0</elastic.version>

Expand Down
14 changes: 13 additions & 1 deletion ...-tests/handlers/metrics/src/main/java/software/amazon/lambda/powertools/e2e/Function.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,15 @@
import com.amazonaws.services.lambda.runtime.RequestHandler;
import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
import software.amazon.cloudwatchlogs.emf.model.StorageResolution;
import software.amazon.cloudwatchlogs.emf.model.Unit;
import software.amazon.lambda.powertools.metrics.Metrics;
import software.amazon.lambda.powertools.metrics.MetricsUtils;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.time.temporal.ChronoUnit;
import java.time.Instant;


public class Function implements RequestHandler<Input, String> {

Expand All @@ -29,11 +35,17 @@ public class Function implements RequestHandler<Input, String> {
@Metrics(captureColdStart = true)
public String handleRequest(Input input, Context context) {

Instant currentTimeTruncatedPlusThirty =
LocalDateTime.now().truncatedTo(ChronoUnit.MINUTES).toInstant(ZoneOffset.UTC).plusSeconds(30);
metricsLogger.setTimestamp(currentTimeTruncatedPlusThirty);

DimensionSet dimensionSet = new DimensionSet();
input.getDimensions().forEach((key, value) -> dimensionSet.addDimension(key, value));
metricsLogger.putDimensions(dimensionSet);

input.getMetrics().forEach((key, value) -> metricsLogger.putMetric(key, value, Unit.COUNT));
input.getMetrics().forEach((key, value) -> metricsLogger.putMetric(key, value, Unit.COUNT,
input.getHighResolution().equalsIgnoreCase("true") ? StorageResolution.HIGH :
StorageResolution.STANDARD));

return "OK";
}
Expand Down
10 changes: 10 additions & 0 deletions ...e2e-tests/handlers/metrics/src/main/java/software/amazon/lambda/powertools/e2e/Input.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ public class Input {

private Map<String, String> dimensions;

private String highResolution;

public Input() {
}

Expand All @@ -32,6 +34,14 @@ public void setMetrics(Map<String, Double> metrics) {
this.metrics = metrics;
}

public String getHighResolution() {
return highResolution;
}

public void setHighResolution(String highResolution) {
this.highResolution = highResolution;
}

public Map<String, String> getDimensions() {
return dimensions;
}
Expand Down
39 changes: 34 additions & 5 deletions powertools-e2e-tests/src/test/java/software/amazon/lambda/powertools/MetricsE2ET.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@
import static software.amazon.lambda.powertools.testutils.Infrastructure.FUNCTION_NAME_OUTPUT;
import static software.amazon.lambda.powertools.testutils.lambda.LambdaInvoker.invokeFunction;

import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.time.temporal.ChronoUnit;
import java.util.Collections;
import java.util.List;
import java.util.Map;
Expand Down Expand Up @@ -66,12 +70,20 @@ public static void tearDown() {
@Test
public void test_recordMetrics() {
// GIVEN

Instant currentTimeTruncatedToMinutes =
LocalDateTime.now().truncatedTo(ChronoUnit.MINUTES).toInstant(ZoneOffset.UTC);

String event1 =
"{ \"metrics\": {\"orders\": 1, \"products\": 4}, \"dimensions\": { \"Environment\": \"test\"} }";
"{ \"metrics\": {\"orders\": 1, \"products\": 4}, \"dimensions\": { \"Environment\": \"test\"}, \"highResolution\": \"false\"}";

String event2 =
"{ \"metrics\": {\"orders\": 1, \"products\": 8}, \"dimensions\": { \"Environment\": \"test\"}, \"highResolution\": \"true\"}";
// WHEN
InvocationResult invocationResult = invokeFunction(functionName, event1);

invokeFunction(functionName, event2);

// THEN
MetricsFetcher metricsFetcher = new MetricsFetcher();
List<Double> coldStart =
Expand All @@ -84,18 +96,35 @@ public void test_recordMetrics() {
List<Double> orderMetrics =
metricsFetcher.fetchMetrics(invocationResult.getStart(), invocationResult.getEnd(), 60, namespace,
"orders", Collections.singletonMap("Environment", "test"));
assertThat(orderMetrics.get(0)).isEqualTo(1);
assertThat(orderMetrics.get(0)).isEqualTo(2);
List<Double> productMetrics =
metricsFetcher.fetchMetrics(invocationResult.getStart(), invocationResult.getEnd(), 60, namespace,
"products", Collections.singletonMap("Environment", "test"));
assertThat(productMetrics.get(0)).isEqualTo(4);

// When searching across a 1 minute time period with a period of 60 we find both metrics and the sum is 12

assertThat(productMetrics.get(0)).isEqualTo(12);

orderMetrics =
metricsFetcher.fetchMetrics(invocationResult.getStart(), invocationResult.getEnd(), 60, namespace,
"orders", Collections.singletonMap("Service", service));
assertThat(orderMetrics.get(0)).isEqualTo(1);
assertThat(orderMetrics.get(0)).isEqualTo(2);
productMetrics =
metricsFetcher.fetchMetrics(invocationResult.getStart(), invocationResult.getEnd(), 60, namespace,
"products", Collections.singletonMap("Service", service));
assertThat(productMetrics.get(0)).isEqualTo(4);
assertThat(productMetrics.get(0)).isEqualTo(12);

Instant searchStartTime = currentTimeTruncatedToMinutes.plusSeconds(15);
Instant searchEndTime = currentTimeTruncatedToMinutes.plusSeconds(45);

List<Double> productMetricDataResult =
metricsFetcher.fetchMetrics(searchStartTime, searchEndTime, 1, namespace,
"products", Collections.singletonMap("Environment", "test"));

// We are searching across the time period the metric was created but with a period of 1 second. Only the high resolution metric will be available at this point

assertThat(productMetricDataResult.get(0)).isEqualTo(8);


}
}
11 changes: 11 additions & 0 deletions ...ls-metrics/src/test/java/software/amazon/lambda/powertools/metrics/MetricsLoggerTest.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.function.Consumer;
import org.junit.jupiter.api.AfterEach;
Expand All @@ -35,6 +37,7 @@
import software.amazon.cloudwatchlogs.emf.config.SystemWrapper;
import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
import software.amazon.cloudwatchlogs.emf.model.StorageResolution;
import software.amazon.cloudwatchlogs.emf.model.Unit;

class MetricsLoggerTest {
Expand Down Expand Up @@ -245,6 +248,7 @@ private void testLogger(Consumer<Consumer<MetricsLogger>> methodToTest) {
{
metricsLogger.setDimensions(DimensionSet.of("Dimension1", "Value1"));
metricsLogger.putMetric("Metric1", 1, Unit.COUNT);
metricsLogger.putMetric("Metric2", 1, Unit.COUNT, StorageResolution.HIGH);
});

assertThat(out.toString())
Expand All @@ -263,6 +267,13 @@ private void testLogger(Consumer<Consumer<MetricsLogger>> methodToTest) {
assertThat(aws.get("CloudWatchMetrics"))
.asString()
.contains("Namespace=GlobalName");

ArrayList cloudWatchMetrics = (ArrayList) aws.get("CloudWatchMetrics");
LinkedHashMap<String, Object> values =
(java.util.LinkedHashMap<String, Object>) cloudWatchMetrics.get(0);
ArrayList metricArray = (ArrayList) values.get("Metrics");
LinkedHashMap<String, Object> metricValues = (LinkedHashMap<String, Object>) metricArray.get(1);
assertThat(metricValues).containsEntry("StorageResolution", 1);
});
}
}
Expand Down
11 changes: 6 additions & 5 deletions .../amazon/lambda/powertools/metrics/handlers/PowertoolsMetricsTooManyDimensionsHandler.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,14 @@
public class PowertoolsMetricsTooManyDimensionsHandler implements RequestHandler<Object, Object> {

@Override
@Metrics
@Metrics(namespace = "ExampleApplication",service = "booking")
public Object handleRequest(Object input, Context context) {
MetricsLogger metricsLogger = metricsLogger();

metricsLogger.setDimensions(IntStream.range(1, 15)
.mapToObj(value -> DimensionSet.of("Dimension" + value, "DimensionValue" + value))
.toArray(DimensionSet[]::new));
DimensionSet dimensionSet = new DimensionSet();
for (int i = 0; i < 35; i++) {
dimensionSet.addDimension("Dimension" + i, "value" + i);
}
metricsLogger.setDimensions(dimensionSet);

return null;
}
Expand Down
Loading

0 comments on commit b51a6de

Please sign in to comment.