Merge pull request #624 from inspec/support-azure-nw-sgs

Support caching of azure nw sgs

docs/resources/azure_network_security_group.md

@@ -21,33 +21,39 @@ For more information, refer to the resource pack [README](../../README.md).
 
 ### Installation
 
-This resource is available in the [InSpec Azure resource pack](https://github.com/inspec/inspec-azure). 
+This resource is available in the [InSpec Azure resource pack](https://github.com/inspec/inspec-azure).
 For an example `inspec.yml` file and how to set up your Azure credentials, refer to resource pack [README](../../README.md#Service-Principal).
 
 ## Syntax
 
 An `azure_network_security_group` resource block identifies a Network Security Group by `name` and `resource_group` or the `resource_id`.
+
 ```ruby
 describe azure_network_security_group(resource_group: 'example', name: 'GroupName') do
   it { should exist }
 end
 ```
+
 ```ruby
 describe azure_network_security_group(resource_id: '/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Network/networkSecurityGroups/{nsgName}') do
   it { should exist }
 end
 ```
+
 ## Parameters
 
 | Name                           | Description                                                                      |
 |--------------------------------|----------------------------------------------------------------------------------|
 | resource_group                 | Azure resource group that the targeted resource resides in.`MyResourceGroup`     |
 | name                           | Name of the Azure resource to test. `MyNSG`                                      |
 | resource_id                    | The unique resource ID. `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Network/networkSecurityGroups/{nsgName}` |
+| resource_data                  | In-memory cached Azure Network security group data. Passing data to this parameter can increase performance since it avoids multiple network calls to the same Azure resource. When provided, it binds the values directly to the resource. Data passed to the `resource_data` parameter could be stale. It is the user's responsibility to refresh the data. |
+
+Provide one of the following parameter sets for a valid query:
 
-Either one of the parameter sets can be provided for a valid query:
 - `resource_id`
 - `resource_group` and `name`
+- `resource_data`
 
 ## Properties
 
@@ -72,13 +78,13 @@ Therefore, it is recommended to use `allow`, `allow_in` or `allow_out` propertie
 <superscript>**</superscript> These properties do not compare criteria defined by explicit ip ranges with the security rules defined by [Azure service tags](https://docs.microsoft.com/en-us/azure/virtual-network/service-tags-overview) and vice versa.
 For example, providing that a network security group has a single security rule allowing all traffics from internet by using `Internet` service tag in the source will fail the `allow_in(ip_range: '64.233.160.0')` test due to incompatible source definitions.
 This is because InSpec Azure resource pack has no control over which ip ranges are defined in Azure service tags.
-Therefore, tests using these methods should be written explicitly for service tags and ip ranges. 
+Therefore, tests using these methods should be written explicitly for service tags and ip ranges.
 For more information about network security groups and security rules refer to [here](https://docs.microsoft.com/en-us/azure/virtual-network/security-overview).
-`*ip_range` used in these methods support IPv4 and IPv6. The ip range criteriaom should be written in CIDR notation.   
+`*ip_range` used in these methods support IPv4 and IPv6. The ip range criteriaom should be written in CIDR notation.
 
 For properties applicable to all resources, such as `type`, `name`, `id`, `properties`, refer to [`azure_generic_resource`](azure_generic_resource.md#properties).
 
-Also, refer to [Azure documentation](https://docs.microsoft.com/en-us/rest/api/virtualnetwork/networksecuritygroups/get#networksecuritygroup) for other properties available. 
+Also, refer to [Azure documentation](https://docs.microsoft.com/en-us/rest/api/virtualnetwork/networksecuritygroups/get#networksecuritygroup) for other properties available.
 Any property in the response may be accessed with the key names separated by dots (`.`).
 
 ## Examples
@@ -94,37 +100,50 @@ end
 describe azure_network_security_group(resource_group: 'example', name: 'GroupName') do
   it { should allow_ssh_from_internet }
 end
-```    
+```
 ### Test that a Network Security Group Allows Inbound Traffics from a Certain Ip Range in Any Port and Any Protocol
 ```ruby
 describe azure_network_security_group(resource_group: 'example', name: 'GroupName') do
   it { should allow(source_ip_range: '10.0.0.0/24', direction: 'inbound') }
   it { should allow_in(ip_range: '10.0.0.0/24') }    # same test with the specific inbound rule check
 end
-```    
+```
 ### Test that a Network Security Group Allows Inbound Traffics from Internet Service Tag in Port `80` and `TCP` Protocol
 ```ruby
 describe azure_network_security_group(resource_group: 'example', name: 'GroupName') do
   it { should allow(source_service_tag: 'Internet', destination_port: '22', protocol: 'TCP', direction: 'inbound') }
   it { should allow_in(service_tag: 'Internet', port: '22', protocol: 'TCP') }    # same test with the specific inbound rule check
 end
-```        
+```
 ### Test that a Network Security Group Allows Inbound Traffics from Virtual Network Service Tag in a Range of Ports and Any Protocol
 ```ruby
 describe azure_network_security_group(resource_group: 'example', name: 'GroupName') do
   it { should allow(source_service_tag: 'VirtualNetwork', destination_port: %w{22 8080 56-78}, direction: 'inbound') }
   it { should allow_in(service_tag: 'VirtualNetwork', port: %w{22 8080 56-78}) }    # same test with the specific inbound rule check
 end
-```            
+```
 ### Test that a Network Security Group Allows Outbound Traffics to a Certain Ip Range in any Port and Any Protocol
 ```ruby
 describe azure_network_security_group(resource_group: 'example', name: 'GroupName') do
   it { should allow(destination_ip_range: '10.0.0.0/24', direction: 'outbound') }
   it { should allow_out(ip_range: '10.0.0.0/24') }   # same test with the specific outbound rule check
 end
-```    
-Please note that `allow` requires `direction` parameter is set to either `inbound` or `outbound` and prefix the `ip_range`, `service_tag` and `port` with either `source_` or `destination_` identifiers.     
-
+```
+
+### Loop through multiple network security groups and verify that each group does not allow inbound traffic from already cached data
+
+```ruby
+azure_network_security_groups.entries.each do |azure_network_security_group_data|
+  describe azure_network_security_group(resource_data: azure_network_security_group_data) do
+    it { should_not allow(destination_ip_range: '10.0.0.0/24', direction: 'inbound') }
+    it { should_not allow_in(ip_range: '10.0.0.0/24') }   # same test with the specific outbound rule check
+  end
+end
+```
+
+Please note that `allow` requires the `direction` parameter be set to either `inbound` or `outbound`
+and you must prefix the `ip_range`, `service_tag`, and `port` with either `source_` or `destination_` identifiers.
+
 ## Matchers
 
 This InSpec audit resource has the following special matchers. For a full list of available matchers, please visit our [Universal Matchers page](https://www.inspec.io/docs/reference/matchers/).
@@ -146,5 +165,5 @@ end
 
 ## Azure Permissions
 
-Your [Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal) must be setup with a `contributor` role on the subscription you wish to test.
+Your [Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal) must be setup with a minimum of `reader` role on the subscription you wish to test.
 

docs/resources/azure_network_security_groups.md

@@ -53,6 +53,7 @@ The name of the resource group.
 | names         | A list of all the network security group names.                                      | `name`          |
 | tags          | A list of `tag:value` pairs defined on the resources.                                | `tags`          |
 | etags         | A list of etags defined on the resources.                                            | `etag`          |
+| properties    | A list of all properties of all the resources.                                       | `properties`    | 
 
 <superscript>*</superscript> For information on how to use filter criteria on plural resources refer to [FilterTable usage](https://github.com/inspec/inspec/blob/master/dev-docs/filtertable-usage.md).
 
@@ -98,4 +99,4 @@ end
 ```
 ## Azure Permissions
 
-Your [Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal) must be setup with a `contributor` role on the subscription you wish to test.
+Your [Service Principal](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal) must be setup with a minimum of `reader` role on the subscription you wish to test.

libraries/azure_generic_resource.rb

@@ -33,6 +33,9 @@ def initialize(opts = {}, static_resource = false) # rubocop:disable Style/Optio
                     end
     if @opts[:is_uri_a_url]
       query_params = { resource_uri: @opts[:resource_uri] }
+    elsif @opts[:resource_data].present?
+      @opts[:resource_data] = @opts[:resource_data].to_h
+      query_params = { resource_uri: @resource_id, resource_data: @opts[:resource_data] }
     else
       resource_fail('There is not enough input to create an Azure resource ID.') if @resource_id.empty?
       # This is the last check on resource_id before talking to resource manager endpoint to get the detailed information.
@@ -140,9 +143,17 @@ def validate_static_resource
     required_parameters = %i(resource_group resource_provider name)
     required_parameters += @opts[:required_parameters] if @opts.key?(:required_parameters)
     allowed_parameters = %i(resource_path resource_identifiers resource_id resource_uri add_subscription_id
-                            query_parameters is_uri_a_url audience transform_keys)
+                            query_parameters is_uri_a_url audience transform_keys resource_data)
     allowed_parameters += @opts[:allowed_parameters] if @opts.key?(:allowed_parameters)
 
+
+    if @opts.key?(:resource_data)
+      @opts[:resource_data] = @opts[:resource_data].to_h
+      validate_parameters(allow: allowed_parameters + required_parameters)
+      @resource_id = @opts[:resource_data][:id]
+      return
+    end
+
     if @opts.key?(:resource_id)
       validate_parameters(required: %i(resource_provider resource_id), allow: allowed_parameters + required_parameters)
       @resource_id = @opts[:resource_id]

libraries/azure_network_security_group.rb

@@ -45,6 +45,7 @@ def initialize(opts = {})
     #       not to accept a different `resource_provider`.
     #
     opts[:resource_provider] = specific_resource_constraint('Microsoft.Network/networkSecurityGroups', opts)
+    opts[:allowed_parameters] = %i(resource_data)
 
     # static_resource parameter must be true for setting the resource_provider in the backend.
     super(opts, true)
@@ -211,6 +212,12 @@ def not_icmp?(properties)
     !properties.protocol.match?(/ICMP/)
   end
 
+  private
+
+  def get_resource(opts = {})
+    opts[:resource_data].presence || super
+  end
+
   # Code for backward compatibility ends here <<<<<<<<
 end
 

libraries/azure_network_security_groups.rb

@@ -52,19 +52,7 @@ def initialize(opts = {})
     # It is recommended to check that after every usage of inherited methods or making API calls.
     return if failed_resource?
 
-    # Define the column and field names for FilterTable.
-    # In most cases, the `column` should be the pluralized form of the `field`.
-    # @see https://github.com/inspec/inspec/blob/master/docs/dev/filtertable-usage.md
-    table_schema = [
-      { column: :names, field: :name },
-      { column: :etags, field: :etag },
-      { column: :tags, field: :tags },
-      { column: :ids, field: :id },
-      { column: :locations, field: :location },
-    ]
-
-    # FilterTable is populated at the very end due to being an expensive operation.
-    AzureGenericResources.populate_filter_table(:table, table_schema)
+    populate_filter_table_from_response
   end
 
   def to_s

libraries/backend/helpers.rb

@@ -243,7 +243,7 @@ def self.validate_params_require_any_of(resource_name = nil, require_any_of, opt
   # @return [Array] Allowed parameters
   # @param allow [Array]
   def self.validate_params_allow(allow, opts, skip_length = false) # rubocop:disable Style/OptionalBooleanParameter TODO: Fix this.
-    unless skip_length
+    if !opts[:resource_data] && !skip_length
       raise ArgumentError, 'Arguments or values can not be longer than 500 characters.' if opts.any? { |k, v| k.size > 100 || v.to_s.size > 500 } # rubocop:disable Style/SoleNestedConditional TODO: Fix this.
     end
     raise ArgumentError, 'Scalar arguments not supported.' unless defined?(opts.keys)