___ ______ _ _ _ _ _
/ \ \ / / ___| | | | | |_(_) |___
/ _ \ \ /\ / /\___ \ | | | | __| | / __|
/ ___ \ V V / ___) | | |_| | |_| | \__ \
/_/ \_\_/\_/ |____/ \___/ \__|_|_|___/
The project is a means for getting more familiar with the AWS Ruby SDKs in practice. It sets up rake tasks for querying AWS resources using the published APIs. The intention is to enable viewing AWS resources, aggregated, sliced and diced, without having to piece things together in the AWS console.
The rake tasks are non-destructive by design, i.e. informational only. Actually manipulation of resources should probably be done with some IaC tool like Terraform.
This project has been written, tested, and run exclusively with Ruby 3 on MacOS, and the Prerequisites below follow accordingly. AWS Utils should run on other *nix OSes, and on Windows too, with some adjustments.
- clone the repo into your desired directory
git clone git@github.com:avonderluft/aws_utils.git
cd aws_utils
- Running
./bin/setup
should be all you need to get going.
./bin/setup
assumes you have Homebrew installed, and will use rbenv as your Ruby Version Manager. If you want to do things differently, you can use the ./bin/setup
script as general guide.
- Homebrew, if you are on MacOs. See https://brew.sh/
- awscli installed and configured, e.g.
brew install awscli
- AWS configuration set up in ~/.aws/config with at least a 'default' profile
- AWS programatic access via key and secret, normally saved in ~/.aws/credentials, or in ENV variables
- IAM permissions for the specific services you wish to query
- ruby installed, with gems rake and bundler (these are pre-installed with current ruby versions)
- Optional: a ruby version manager, e.g. rbenv or chruby (RVM is not recommended)
- e.g. rbenv - see docs at https://github.com/rbenv/rbenv
- After updating it, (re-)source your shell.rc, e.g.
source ~/.bashrc
orsource ~/.zshrc
- Optional: install other rubies according to preference
- To improve performance results are cached for 30 minutes locally by default, except for regions which are cached for a day
- cache expiration time can be changed by editing
./config/config.yaml
(see Config below) - To force clear caches on any rake task, append 'cache=no', e.g.
rake ec2s:new cache=no
in ./config/config.yaml
Change the integers for minutes cached: regions_cache_expire_minutes
for regions, and cache_expire_minutes
for everything else.
Default is one year (365 days) - adjust according to your standards
You can speed up your queries by limiting the regions searched in ./config/config.yaml
, adding strings which will be matched with region names. For example, if you wanted to limit regions to Europe and USA, you could update the YAML config like so:
---
cache_expire_minutes: 30
regions_cache_expire_minutes: 1440 # 1 day
stale_key_days: 365
region_filters:
- 'eu-'
- 'us-'
To search in all regions accessible to you, just leave the regions_filter
element empty:
---
stale_key_days: 365
region_filters:
After you change region_filters config.yaml
, run rake regions cache=no
to apply the changes
Note: Follow the installation instuctions above, and you should not need to prefix bundle exec
to your rake
commands, since the gems will be installed in a common location for the ruby, not in an application-specific location, e.g vendor/bundle.
If you do need bundle exec
it is helpful to create an alias in your shell.rc e.g. alias be="bundle exec "
. This done, you can substitute 'be' for 'bundle exec' in your rake commands, if needed
- Display all available rake tasks:
rake -T
- To filter for specific AWS services, add argument, e.g.
rake -T ec2
If your AWS account requires MFA for CLI access, you will be prompted to set up an MFA session interactively, to set a new session token. You will need your AWS username, and the 6 digit rotating token from your MFA device.
- Show EC2 instances, grouped by region:
rake ec2s
- Show all volumes, grouped by region:
rake vols
- Show all unencrypted volumes, grouped by region:
rake vols:unencrypted
- Show all unencrypted snapshots:
rake snaps:unencrypted
- Show all S3 buckets without logging enabled:
rake s3s:no_logging
- Show all KMS keys which include 'EKS' in their description:
rake keys:desc[EKS]
Audit reports are created in ./audit_reports/ directory for each service with date stamp in the filename. Each subsequent run creates a diff file to show changes since the last run.
- Display all audit tasks:
rake -T audit
The 'lambda' directory contains a few example scripts
- Run
rake
to run the tests. You might need to prefixbundle exec
- You can view current at
./coverage/index.html
in a web browser - You can also run
bin/console
for an interactive prompt that will allow you to experiment with objects
- to Jason Davila for the original bash scripts which inspired this project
- to my employer Excella for time on the bench to work on it
- to all the contributors to Ruby which make it such a marvelous language
- code for more AWS services, e.g. ECS, and expanding EKS
- write tests for all the services covered
- write test for rake tasks
The basic structure is in place, to which you can add tasks to slice and dice according to taste. Follow the general structure, and create some new tasks, and/or improve what is already here. Then submit a pull request. Thanks!