Amazon EC2 Metadata Mock (AEMM) is a tool to simulate Amazon EC2 instance metadata service for local testing.
- Project Summary
- Major Features
- Supported Metadata Categories
- Getting Started
- Configuration
- Usage
- Troubleshooting
- Integrations
- Building
- Communication
- Contributing
- License
AWS EC2 Instance metadata is data about your instance that you can use to configure or manage the running instance. Instance metadata is divided into categories like hostname, instance id, maintenance events, spot instance action, autoscaling target-lifecycle-state. See the complete list of metadata categories here.
The instance metadata can be accessed from within the instance. Some instance metadata is available only when an instance is affected by the event. E.g. A Spot instance's metadata item spot/instance-action is available only when AWS decides to interrupt the Spot instance.
These bring forth some challenges like not being able to test one's application in the event of Spot interruption or other such events and requiring an EC2 instance for testing.
This project attempts to bridge these gaps by providing mocks for most of these metadata categories. The mock responses are designed to replicate those from the actual instance metadata service for accurate, local testing.
- Simulate Spot Instance Interruption (ITN) & EC2 Rebalance Recommendation events for Spot instances
- Delay mock response from the mock serve start time
- Configure metadata in mock responses via CLI flags, config file, env variables
- IMDSv1 and v2 support (configurable for IMDSv2 support only)
- Save processed configuration to a local file
AEMM supports most metadata categories except for:
- ancestor-ami-ids
- events/maintenance/history
PRs for any of the above paths are always welcome! Please see our Contributing section for details.
Paths specific to Windows instances such as elastic-gpus/associations/elastic-gpu-id will not be supported at this time. Feel free to open
an issue to discuss if you would like AEMM to support these paths in the future.
AEMM is simple to get up and running.
brew tap aws/tap
brew install ec2-metadata-mock
curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.12.0/ec2-metadata-mock-`uname | tr '[:upper:]' '[:lower:]'`-amd64
chmod +x ec2-metadata-mock
curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.12.0/ec2-metadata-mock-linux-arm
curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.12.0/ec2-metadata-mock-linux-arm64
curl -Lo ec2-metadata-mock https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.12.0/ec2-metadata-mock-windows-amd64.exe
docker pull public.ecr.aws/aws-ec2/amazon-ec2-metadata-mock:v1.12.0
docker run -it --rm -p 1338:1338 public.ecr.aws/aws-ec2/amazon-ec2-metadata-mock:v1.12.0
- Kubernetes >= 1.14
We are hosting helm-charts for Amazon EC2 Metadata Mock in ecr-public. The chart for this project is hosted in helm/amazon-ec2-metadata-mock.
Detailed instructions on installing Amazon EC2 Metadata Mock using Helm can be found here Helm README
kubectl apply -f https://github.com/aws/amazon-ec2-metadata-mock/releases/download/v1.12.0/all-resources.yaml
Use ec2-metadata-mock --help to view examples and explanations of supported flags and commands:
$ ec2-metadata-mock --help
ec2-metadata-mock is a tool to mock Amazon EC2 instance metadata.
Usage:
ec2-metadata-mock <command> [arguments] [flags]
ec2-metadata-mock [command]
Examples:
ec2-metadata-mock --mock-delay-sec 10 mocks all metadata paths
ec2-metadata-mock spot --action terminate mocks spot ITN only
Available Commands:
events Mock EC2 maintenance events
help Help about any command
spot Mock EC2 Spot interruption notice
asglifecycle Mock ASG target-lifecycle-state changes from InService to Terminated
Flags:
--asg-termination-delay-sec int asg termination delay in seconds, relative to the application start time (default: 0 seconds)
--asg-termination-trigger-time int asg termination trigger time in RFC3339 format. This takes priority over asg-termination-delay-sec (default: none)
-c, --config-file string config file for cli input parameters in json format (default: $HOME/aemm-config.json)
-h, --help help for ec2-metadata-mock
-n, --hostname string the HTTP hostname for the mock url (default: 0.0.0.0)
-I, --imdsv2 whether to enable IMDSv2 only, requiring a session token when submitting requests (default: false, meaning both IMDS v1 and v2 are enabled)
-d, --mock-delay-sec int spot itn delay in seconds, relative to the application start time (default: 0 seconds)
-x, --mock-ip-count int number of IPs in a cluster that can receive a Spot Interrupt Notice and/or Scheduled Event (default 2)
--mock-trigger-time string spot itn trigger time in RFC3339 format. This takes priority over mock-delay-sec (default: none)
-p, --port string the HTTP port where the mock runs (default: 1338)
--rebalance-delay-sec int rebalance rec delay in seconds, relative to the application start time (default: 0 seconds)
--rebalance-trigger-time string rebalance rec trigger time in RFC3339 format. This takes priority over rebalance-delay-sec (default: none)
-s, --save-config-to-file whether to save processed config from all input sources in .ec2-metadata-mock/.aemm-config-used.json in $HOME or working dir, if homedir is not found (default: false)
-v, --version version for ec2-metadata-mock
Use "ec2-metadata-mock [command] --help" for more information about a command.
Starting AEMM with default configurations using ec2-metadata-mock will start the server on the default host and port:
$ ec2-metadata-mock
Initiating ec2-metadata-mock for all mocks on port 1338
Serving the following routes: /latest/meta-data/product-codes, /latest/meta-data/iam/info, /latest/meta-data/instance-type, ...(truncated for readability)
With the server running, send a request to one of the supported routes above using curl localhost:1338/<route>. Example request to display all supported routes:
$ curl localhost:1338/latest/meta-data
ami-id
ami-launch-index
ami-manifest-path
autoscaling/
block-device-mapping/
elastic-inference/
events/
hostname
iam/
instance-action
instance-id
instance-life-cycle
instance-type
kernel-id
local-hostname
local-ipv4
mac
network/
placement/
product-codes
public-hostname
public-ipv4
public-keys/
ramdisk-id
reservation-id
security-groups
services/
spot/
tags/
AEMM's wide-range of configurability ranges from overriding port numbers to enabling IMDSv2-only to updating specific metadata values and paths. These configurations can be loaded from various sources with a deterministic precedence.
Defaults for AEMM configuration are sourced throughout code. Examples below:
- CLI flags
- Metadata mock responses
- Commands
AEMM supports configuration from various sources including: cli flags, env variables, and config files. Details regarding configuration steps, behavior, and precedence are outlined here.
AEMM is primarily used as a developer tool to help test behavior related to Metadata Service. Popular use cases include: emulating spot instance interrupts after a designated delay, mocking scheduled maintenance events, IMDSv2 migrations, and requesting static metadata. This section outlines the common use cases of AEMM; advanced usage and behavior are documented here.
To view the available flags for the Spot Interruption command use spot --help:
$ ec2-metadata-mock spot --help
Mock EC2 Spot interruption notice
Usage:
ec2-metadata-mock spot [--action ACTION] [flags]
Aliases:
spot
Examples:
ec2-metadata-mock spot -h spot help
ec2-metadata-mock spot -d 5 --action terminate mocks spot interruption only
Flags:
-a, --action string action in the spot interruption notice (default: terminate)
action can be one of the following: terminate,hibernate,stop
-h, --help help for spot
-r, --rebalance-rec-time string rebalance rec time specifies the approximate time when the rebalance recommendation notification will be emitted in RFC3339 format
-t, --time string termination time specifies the approximate time when the spot instance will receive the shutdown signal in RFC3339 format to execute instance action E.g. 2020-01-07T01:03:47Z (default: request time + 2 minutes in UTC)
Global Flags:
--asg-termination-delay-sec int asg termination delay in seconds, relative to the application start time (default: 0 seconds)
--asg-termination-trigger-time int asg termination trigger time in RFC3339 format. This takes priority over asg-termination-delay-sec (default: none)
-c, --config-file string config file for cli input parameters in json format (default: $HOME/aemm-config.json)
-h, --help help for ec2-metadata-mock
-n, --hostname string the HTTP hostname for the mock url (default: 0.0.0.0)
-I, --imdsv2 whether to enable IMDSv2 only, requiring a session token when submitting requests (default: false, meaning both IMDS v1 and v2 are enabled)
-d, --mock-delay-sec int spot itn delay in seconds, relative to the application start time (default: 0 seconds)
-x, --mock-ip-count int number of IPs in a cluster that can receive a Spot Interrupt Notice and/or Scheduled Event (default 2)
--mock-trigger-time string spot itn trigger time in RFC3339 format. This takes priority over mock-delay-sec (default: none)
-p, --port string the HTTP port where the mock runs (default: 1338)
--rebalance-delay-sec int rebalance rec delay in seconds, relative to the application start time (default: 0 seconds)
--rebalance-trigger-time string rebalance rec trigger time in RFC3339 format. This takes priority over rebalance-delay-sec (default: none)
-s, --save-config-to-file whether to save processed config from all input sources in .ec2-metadata-mock/.aemm-config-used.json in $HOME or working dir, if homedir is not found (default: false)
-v, --version version for ec2-metadata-mock
1.) Starting AEMM with spot: spot routes available immediately:
$ ec2-metadata-mock spot
Initiating ec2-metadata-mock for EC2 Spot interruption notice on port 1338
Serving the following routes: ... (truncated for readability)
Send the request:
$ curl localhost:1338/latest/meta-data/spot/instance-action
{
"action": "terminate",
"time": "2020-04-24T17:11:44Z"
}
2.) Starting AEMM with spot after Delay: Users can apply a delay duration in seconds for when the spot metadata will become available:
$ ec2-metadata-mock spot -d 10
Initiating ec2-metadata-mock for EC2 Spot interruption notice on port 1338
Flags:
mock-delay-sec: 10
Serving the following routes: ... (truncated for readability)
Sending a request to spot paths before the delay has passed will return 404 - Not Found:
$ curl localhost:1338/latest/meta-data/spot/instance-action
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>404 - Not Found</title>
</head>
<body>
<h1>404 - Not Found</h1>
</body>
</html>
// Server log
Delaying the response by 10s as requested. The mock response will be available in 2s. Returning `notFoundResponse` for now
Once the delay is complete, querying spot paths return expected results:
$ curl localhost:1338/latest/meta-data/spot/instance-action
{
"action": "terminate",
"time": "2020-04-24T17:19:32Z"
}
Alternatively a trigger time can be configured using --mock-trigger-time which can be useful to synchronize spot interruption simulation over multiple instances.
The Rebalance Recommendation notification is also available under the spot command as it is sent when EC2 emits this signal when Spot Instances are at an elevated risk of interruption:
$ ec2-metadata-mock spot
Initiating ec2-metadata-mock for EC2 Spot interruption notice on port 1338
Serving the following routes: ... (truncated for readability)
Send the request:
$ curl localhost:1338/latest/meta-data/events/recommendations/rebalance
{
"noticeTime": "2020-10-16T19:18:24Z"
}
Note: although Rebalance Recommendation path contains events it will not be available when starting AEMM with the events command
Similar to spot, the events command, view the local flags using events --help:
$ ec2-metadata-mock events --help
Mock EC2 Scheduled Events
Usage:
ec2-metadata-mock events [--code CODE] [--state STATE] [--not-after] [--not-before-deadline] [flags]
Aliases:
events, se, scheduledevents
Examples:
ec2-metadata-mock events -h events help
ec2-metadata-mock events -o instance-stop --state active -d mocks an active and upcoming scheduled event for instance stop with a deadline for the event start time
Flags:
-o, --code string event code in the scheduled event (default: system-reboot)
event-code can be one of the following: instance-reboot,system-reboot,system-maintenance,instance-retirement,instance-stop
-h, --help help for events
-a, --not-after string the latest end time for the scheduled event in RFC3339 format E.g. 2020-01-07T01:03:47Z default: application start time + 7 days in UTC))
-b, --not-before string the earliest start time for the scheduled event in RFC3339 format E.g. 2020-01-07T01:03:47Z (default: application start time in UTC)
-l, --not-before-deadline string the deadline for starting the event in RFC3339 format E.g. 2020-01-07T01:03:47Z (default: application start time + 9 days in UTC)
-t, --state string state of the scheduled event (default: active)
state can be one of the following: active,completed,canceled
(Truncated Global Flags for readability)
1.) Starting AEMM with events: events route available immediately and spot routes will no longer be available due to the implementation of Commands detailed here:
$ ec2-metadata-mock events --code instance-reboot -a 2020-01-07T01:03:47Z -b 2020-01-01T01:03:47Z -l 2020-01-10T01:03:47Z --state completed
Initiating ec2-metadata-mock for EC2 Events on port 1338
Serving the following routes: ... (truncated for readability)
Send the request:
$ curl localhost:1338/latest/meta-data/events/maintenance/scheduled
{
"Code": "instance-reboot",
"Description": "The instance is scheduled for instance-reboot",
"State": "completed",
"EventId": "instance-event-1234567890abcdef0",
"NotBefore": "1 Jan 2020 01:03:47 GMT",
"NotAfter": "7 Jan 2020 01:03:47 GMT",
"NotBeforeDeadline": "10 Jan 2020 01:03:47 GMT"
}
The asglifecycle command will generate a asg lifecycle termination event after a user-specified delay or termination time.
$ ec2-metadata-mock asglifecycle --help
Mock EC2 Auto Scaling Group Lifecycle Termination State
Usage:
ec2-metadata-mock asglifecycle
Aliases:
asglifecycle, autoscaling, asg
Send the request:
$ curl localhost:1338/latest/meta-data/autoscaling/target-lifecycle-state
Terminated
AEMM supports both versions of Instance Metadata service. By default, AEMM starts with supporting v1 and v2; however, it is possible to enable IMDSv2 only via overrides.
1.) Starting AEMM with IMDSv2 only: session tokens are required for all requests; v1 requests will return 401 - Unauthorized:
$ ec2-metadata-mock --imdsv2
Send a v1 request:
$ curl localhost:1338/latest/meta-data/mac
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>401 - Unauthorized</title>
</head>
<body>
<h1>401 - Unauthorized</h1>
</body>
</html>
Send a v2 request:
TOKEN=`curl -X PUT "localhost:1338/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" localhost:1338/latest/meta-data/mac
0e:49:61:0f:c3:11
Requesting a token outside the TTL bounds (between 1-2600 seconds) will return 400 - Bad Request:
$ curl -X PUT "localhost:1338/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 0"
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>400 - Bad Request</title>
</head>
<body>
<h1>400 - Bad Request</h1>
</body>
</html>
Providing an expired token is synonymous to no token at all resulting in 401 - Unauthorized.
Static metadata is classified as instance-specific metadata that is always available regardless of which command is used to start the tool.
Examples of static metadata include all metadata categories from the non-dynamic category table (ami-id, instance-id, mac) except for events and spot categories (classified as commands in AEMM).
Note that 'static' naming is used within the context of this tool ONLY
1.) Requesting static metadata instance-id:
$ ec2-metadata-mock
Send the request:
$ curl localhost:1338/latest/meta-data/instance-id
i-1234567890abcdef0
Details on overriding static metadata values and behavior can be found here
| Warning Displayed | Cause | Effect |
|---|---|---|
| Warning: Config File file name Not Found in locations | input configuration file not found | other input sources are used (See above for details) |
| Warning: Failed to save the final configuration to local file - Failed to create directory for final configuration at path/to/dir: error string | Failure to create the hidden directory .amazon-ec2-metadata-mock to store the final configuration file |
configuration used by the tool is NOT saved to a local file. The tool continues with its primary job of mocking metadata paths |
| Warning: Failed to save the final configuration to local file - The destination 'path/to/dir' for saving the configuration already exists, but is not a directory | Failure to create the hidden directory .amazon-ec2-metadata-mock to store the final configuration file, because a resource by that name already exists |
configuration used by the tool is NOT saved to a local file. The tool continues with its primary job of mocking metadata paths |
| Warning: Failed to save the final configuration to local file path/to/local/file: error string | Failure to save final configuration to a file | configuration used by the tool is NOT saved to a local file. The tool continues with its primary job of mocking metadata paths |
| Warning: Failed to find home directory due to error: error string | Failure to get home directory | working directory is used instead |
aws-node-termination-handler uses AEMM in its e2e test suite to mock the metadata service and interrupt events.
In the NTH imds-v2-test, Helm is used to download the latest AEMM release, install it onto the cluster, and start it with imdsv2-only access. NTH then acquires the v2 token from AEMM, consumes the interrupt event, then cordons the worker node and evicts the test pod, thus validating NTH functionality. For more details on NTH e2e tests refer to the documentation here.
For build instructions, please consult BUILD.md
If you've run into a bug or have a new feature request, please open an issue.
Check out the open source Amazon EC2 Spot Instances Integrations Roadmap to see what we're working on and give us feedback!
Contributions are welcome! Please read our guidelines and our Code of Conduct
This project is licensed under the Apache-2.0 License.