Commits on Apr 23, 2023

1. [QT-525] enos: use spot instances for Vault targets (#20037) …

   The previous strategy for provisioning infrastructure targets was to use
   the cheapest instances that could reliably perform as Vault cluster
   nodes. With this change we introduce a new model for target node
   infrastructure. We've replaced on-demand instances for a spot
   fleet. While the spot price fluctuates based on dynamic pricing, 
   capacity, region, instance type, and platform, cost savings for our
   most common combinations range between 20-70%.
   
   This change only includes spot fleet targets for Vault clusters.
   We'll be updating our Consul backend bidding in another PR.
   
   * Create a new `vault_cluster` module that handles installation,
     configuration, initializing, and unsealing Vault clusters.
   * Create a `target_ec2_instances` module that can provision a group of
     instances on-demand.
   * Create a `target_ec2_spot_fleet` module that can bid on a fleet of
     spot instances.
   * Extend every Enos scenario to utilize the spot fleet target acquisition
     strategy and the `vault_cluster` module.
   * Update our Enos CI modules to handle both the `aws-nuke` permissions
     and also the privileges to provision spot fleets.
   * Only use us-east-1 and us-west-2 in our scenario matrices as costs are
     lower than us-west-1.
   
   Signed-off-by: Ryan Cragun <me@ryan.ec>

   

   ryancragun committed Apr 23, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 724ab7d
   • Browse repository at this point

   Copy the full SHA

   724ab7d View commit details

   Browse the repository at this point in the history

2. [QT-530] enos: allow-list all public IP addresses (#20304) …

   The security groups that allow access to remote machines in Enos
   scenarios have been configured to only allow port 22 (SSH) from the
   public IP address of machine executing the Enos scenario. To achieve
   this we previously utilized the `enos_environment.public_ip_address`
   attribute. Sometime in mid March we started seeing sporadic SSH i/o
   timeout errors when attempting to execute Enos resources against SSH
   transport targets. We've only ever seen this when communicating from
   Azure hosted runners to AWS hosted machines.
   
   While testing we were able to confirm that in some cases the public IP
   address resolved using DNS over UDP4 to Google and OpenDNS name servers
   did not match what was resolved when using the HTTPS/TCP IP address
   service hosted by AWS. The Enos data source was implemented in a way
   that we'd attempt resolution of a single name server and only attempt
   resolving from the next if previous name server could not get a result.
   We'd then allow-list that single IP address. That's a problem if we can
   resolve two different public IP addresses depending our endpoint address.
   
   This change utlizes the new `enos_environment.public_ip_addresses`
   attribute and subsequent behavior change. Now the data source will
   attempt to resolve our public IP address via name servers hosted by
   Google, OpenDNS, Cloudflare, and AWS. We then return a unique set of
   these IP addresses and allow-list all of them in our security group. It
   is our hope that this resolves these i/o timeout errors that seem like
   they're caused by the security group black-holing our attempted access
   because the IP we resolved does not match what we're actually exiting
   with.
   
   Signed-off-by: Ryan Cragun <me@ryan.ec>

   

   ryancragun committed Apr 23, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 3f2758c
   • Browse repository at this point

   Copy the full SHA

   3f2758c View commit details

   Browse the repository at this point in the history