This script scans your AWS resources across multiple regions and services, and generates a detailed inventory of them. It's written in Python, and leverages Boto3, the official Amazon Web Services (AWS) SDK for Python, to interact with AWS services.
As the adoption of cloud technologies continues to rise, businesses have started leveraging AWS to host an increasingly diverse range of services. As your cloud presence grows, it becomes increasingly important to maintain a clear overview of your AWS resources across all services and regions.
However, AWS's expansive list of services and geographically diverse infrastructure can make it a daunting task to maintain visibility of all resources. AWS provides over 200+ services, each of which can contain numerous different resource types. These services can be spread across 24 geographic regions worldwide.
To address these issues, we have developed the AWS Auto Inventory, a robust, efficient, and flexible Python script that scans your AWS environment to provide a comprehensive overview of your resources.
The script provides:
- Thorough scanning across specified (or all) AWS regions and services.
- Parallel scanning capabilities, leveraging Python's multithreading to process multiple regions and services concurrently.
- Error-handling mechanisms for AWS's API call rate limits, including retry logic with exponential backoff.
- Configurable retry parameters.
- Verbose logging, useful for troubleshooting and understanding the scanning process.
- Results stored in JSON files, making it easy to feed the output into further data processing or visualization tools.
As this tool is a Python script, you need Python 3.6+ installed on your machine. If you have Python installed, you can directly run the script. Make sure you have installed the required Python libraries. Install them using pip:
pip install -r requirements.txt
Before you start, make sure to set your AWS credentials either by using the AWS CLI:
aws configure
Or by setting the following environment variables:
export AWS_ACCESS_KEY_ID=<your_access_key>
export AWS_SECRET_ACCESS_KEY=<your_secret_key>
export AWS_SESSION_TOKEN=<your_session_token> # If using temporary credentials
You can run the AWS Auto Inventory using the command line. Here is the general syntax:
python scan.py -s <scanfile> [parameters]
Parameters:
--scan
: The path to the JSON file or URL containing the AWS services to scan.. This is a required parameter.--regions
: Optional parameter specifying a list of AWS regions to scan. If not provided, all accessible regions will be scanned.--output_dir
: Optional parameter specifying the directory to store the results. Default is "output".--log_level
: Optional parameter to set the logging level (e.g., DEBUG, INFO, WARNING, ERROR, CRITICAL). Default is "INFO".--max-retries
: Optional parameter to set the maximum number of retries for each service. Default is 3.--retry-delay
: Optional parameter to set the delay (in seconds) before each retry. Default is 2.--concurrent-regions
: Optional parameter to set the number of regions to process concurrently. Default is None, which means the script will use as many as possible.--concurrent-services
: Optional parameter to set the number of services to process concurrently for each region. Default is None, which means the script will use as many as possible.
Here are some examples of how to use the tool:
- Scan all resources specified in
scanfile.json
across all regions:
python scan.py -s scanfile.json
Note: scanfile.json
can be local path or an URL, example:
python scan.py -s https://raw.githubusercontent.com/aws-samples/aws-auto-inventory/main/scan/sample/services/iam.json
- Scan resources specified in
scanfile.json
across specific regions (us-east-1 and eu-west-1), with a maximum of 5 retries and a retry delay of 3 seconds:
python scan.py -s scanfile.json -r us-east-1 eu-west-1 --max-retries 5 --retry-delay 3
- Scan resources specified in
scanfile.json
across all regions, processing up to 3 regions and 5 services concurrently:
python scan.py -s scanfile.json --concurrent-regions 3 --concurrent-services 5
In the scan directory, you'll find numerous configurations available for use. However, please note that not all function
properties have been thoroughly tested and there's a possibility of failure. Thus, it's recommended to use this information as a reference only.
The AWS Auto Inventory requires a JSON file specifying which AWS services and resources to scan. The format of this file is as follows:
[
{
"service": "s3",
"function": "list_buckets"
},
{
"service": "ec2",
"function": "describe_instances",
"result_key": "Reservations"
}
]
Each JSON object represents a single AWS service to scan. The service
key specifies the service, and the function
key specifies the boto3 client function to call. The optional result_key
can be used to specify a key in the returned JSON object that contains the desired data. If not provided, the full JSON response will be used.
You can download the latest releases here.
- AWS Code Habits - A library with Make targets, Ansible playbooks, Jinja templates (and more) designed to boost common software development tasks and enhance governance.
This project is licensed under the Apache License 2.0 License. See the LICENSE file.
Copyright Amazon, Inc. or its affiliates. All Rights Reserved.