Add GovCloud region name to validation set.

[triplepoint]

It looks like the GovCloud region was left off the enumerated set of region names.

It's a bit difficult to parse Amazon's documentation, but we believe 'us-gov-west-1' is the only GovCloud region:
http://docs.aws.amazon.com/govcloud-us/latest/UserGuide/using-govcloud-arns.html

We discovered this through mitchellh/vagrant-aws, which uses this library to validate Vagrant parameters when building ec2 instances.

[geemus]

Thanks!

[triplepoint]

I was curious why you were validating input on the client side. It seems to me that you're liable to get into trouble when AWS inevitably changes allowed values (like adding regions, for example).

Is it not workable to pass unvalidated calls to AWS and let the AWS API response error inform the user that they passed invalid values? What was the motivation for the engineyard pull request that added this client side validation?

[lanej]

• when you're looking for something, it's nice to have a list of regions to search.
• the list is not frozen, can be modified by users very easily.
• without validation, exceptions occur on first request instead of initialization even though the initialized object will never be able to make a request.
• the error returned is not from AWS. Since region is needed to point to a specific API endpoint, you get a DNS lookup error.

irb(main):001:0> Fog::Compute::AWS.new(region: "mike")
ArgumentError: Unknown region: "mike"
    from /Users/deploy/p/fog-aws/lib/fog/aws.rb:232:in `validate_region!'
    from /Users/deploy/p/fog-aws/lib/fog/aws/compute.rb:479:in `initialize'
    from /Users/deploy/.gem/ruby/2.2.2/gems/fog-core-1.32.1/lib/fog/core/service.rb:115:in `new'
    from /Users/deploy/.gem/ruby/2.2.2/gems/fog-core-1.32.1/lib/fog/core/service.rb:115:in `new'
    from (irb):1
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/cli/console.rb:14:in `run'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/cli.rb:308:in `console'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/cli.rb:10:in `start'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/bin/bundle:20:in `block in <top (required)>'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/friendly_errors.rb:7:in `with_friendly_errors'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/bin/bundle:18:in `<top (required)>'
    from /Users/deploy/.gem/ruby/2.2.2/bin/bundle:23:in `load'
irb(main):002:0> Fog::AWS.regions << "mike"
=> ["ap-northeast-1", "ap-southeast-1", "ap-southeast-2", "eu-central-1", "eu-west-1", "us-east-1", "us-west-1", "us-west-2", "sa-east-1", "cn-north-1", "mike"]
irb(main):003:0> Fog::Compute::AWS.new(region: "mike").servers.all
Excon::Errors::SocketError: getaddrinfo: nodename nor servname provided, or not known (SocketError)
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/socket.rb:108:in `getaddrinfo'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/socket.rb:108:in `connect'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/ssl_socket.rb:148:in `connect'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/socket.rb:28:in `initialize'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/ssl_socket.rb:8:in `initialize'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/connection.rb:387:in `new'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/connection.rb:387:in `socket'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/connection.rb:106:in `request_call'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/middlewares/mock.rb:47:in `request_call'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/middlewares/base.rb:15:in `request_call'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/middlewares/base.rb:15:in `request_call'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/middlewares/base.rb:15:in `request_call'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/middlewares/instrumentor.rb:22:in `request_call'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/connection.rb:233:in `request'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/middlewares/idempotent.rb:26:in `error_call'
    from /Users/deploy/.gem/ruby/2.2.2/gems/excon-0.45.5/lib/excon/middlewares/base.rb:10:in `error_call'
... 19 levels...
    from /Users/deploy/p/fog-aws/lib/fog/aws/compute.rb:520:in `request'
    from /Users/deploy/p/fog-aws/lib/fog/aws/requests/compute/describe_instances.rb:81:in `describe_instances'
    from /Users/deploy/p/fog-aws/lib/fog/aws/models/compute/servers.rb:62:in `all'
    from (irb):3
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/cli/console.rb:14:in `run'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/cli.rb:308:in `console'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/cli.rb:10:in `start'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/bin/bundle:20:in `block in <top (required)>'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/lib/bundler/friendly_errors.rb:7:in `with_friendly_errors'
    from /Users/deploy/.gem/ruby/2.2.2/gems/bundler-1.10.6/bin/bundle:18:in `<top (required)>'
    from /Users/deploy/.gem/ruby/2.2.2/bin/bundle:23:in `load'
    from /Users/deploy/.gem/ruby/2.2.2/bin/bundle:23:in `<main>'irb(main):004:0>

much better to validate up front.

[triplepoint]

I see. The immediate failure mode that comes to mind (and which is the one that affected us) is that not every user of this library is a direct developer with access to the fog AWS object's configuration. So modifying the list of known regions isn't necessarily easy or even possible, and so the default behavior becomes much more important.

For example, the vagrant-aws vagrant plugin takes configuration values out of a Vagrantfile and passes them on to fog to instantiate EC2 instances. If the vagrant user specifies a legitimate AWS region that's unknown to fog (like the govcloud region was), the tool fails. And because fog was a "hidden" dependency from the point of view of the end user, it's not clear to the user why the failure happened or what to do about it.

In general, it's brittle practice to validate a client library's allowed values in lockstep with a 3rd-party API. Inevitable changes in behavior on the API's end will manifest as sudden breakages for deployments which haven't incorporated the latest library code. And the client development team must maintain constant vigilance for even minor changes in the API. Even the swiftest response on the part of the development team will still leave users in a state of breakage for an unknown period of time.

If it's a hard requirement that invalid regions be detected on object creation, then it seems to me that the correct behavior would be a call out to AWS to fetch the set of allowed regions, with the input validated against that list. I'm unfamiliar with the details of the AWS API, but perhaps this would provide that information? http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeRegions.html

In any case, thank you again for your quick turn around on our pull request.

[lanej]

@triplepoint you make some excellent points.

100% understand the issues with hardcoding the regions, exacerbated by fog not being a direct dependency. I will pitch this to the other owners.

Might be worth adding an option to disable validation.

[geemus]

An easy compromise might be to change this from an error to a warning. It wouldn't "just work" for govcloud as a region, but I think it would allow you to explicitly pass that as the region name as well as the endpoint host/path and (probably) get it to work? Might be worth a try anyway. What do you think?