diff --git a/docs/resources/azure_generic_resource.md b/docs/resources/azure_generic_resource.md
index 1cab898bd..daa8e5672 100644
--- a/docs/resources/azure_generic_resource.md
+++ b/docs/resources/azure_generic_resource.md
@@ -31,16 +31,18 @@ where
The following parameters can be passed for targeting a specific Azure resource.
-| Name | Description |
-|-------------------|----------------------------------------------------------------------------------------------------------|
-| resource_group | Azure resource group that the targeted resource has been created in. `MyResourceGroup` |
-| name | Name of the Azure resource to test. `MyVM` |
-| resource_provider | Azure resource provider of the resource to be tested. `Microsoft.Compute/virtualMachines` |
-| resource_path | Relative path to the resource if it is defined on another resource. Resource path of a subnet in a virtual network would be: `{virtualNetworkName}/subnets`. |
-| resource_id | Unique id of Azure resource to be tested. `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Compute/virtualMachines/{vmName}` |
+| Name | Description |
+|--------------------------------------|----------------------------------------------------------------------------------------------------------|
+| resource_group | Azure resource group that the targeted resource has been created in. `MyResourceGroup` |
+| name | Name of the Azure resource to test. `MyResourceName` |
+| resource_provider | Azure resource provider of the resource to be tested. `Microsoft.Compute/virtualMachines` |
+| resource_path | Relative path to the resource if it is defined on another resource. Resource path of a subnet in a virtual network would be: `{virtualNetworkName}/subnets`. |
+| resource_id | Unique id of Azure resource to be tested. `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Compute/virtualMachines/{vmName}` |
+| resource_uri | Azure REST API URI of the resource to be tested. This parameter should be used when a resource does not reside in a resource group. It requires `add_subscription_id` and `name` parameters to be provided together. `/subscriptions/{subscriptionId}/providers/Microsoft.Authorization/policyDefinitions/` |
+| add_subscription_id | Indicates whether the `resource_uri` contains the subscription id. `true` or `false` |
| tag_name* | Tag name defined on the Azure resource. `name` |
-| tag_value | Tag value of the tag defined with the `tag_name`. `external_linux` |
-| api_version | API version to use when interrogating the resource. If not set or the provided api version is not supported by the resource provider then the latest version for the resource provider will be used. `2017-10-9`, `latest`, `default` |
+| tag_value | Tag value of the tag defined with the `tag_name`. `external_linux` |
+| api_version | API version to use when interrogating the resource. If not set or the provided api version is not supported by the resource provider then the latest version for the resource provider will be used. `2017-10-9`, `latest`, `default` |
* When resources are filtered by a tag name and value, the tags for each resource are not returned in the results.
@@ -50,9 +52,10 @@ Either one of the parameter sets can be provided for a valid query:
- `name`
- `resource_group`, `resource_provider` and `name`
- `resource_group`, `resource_provider`, `resource_path` and `name`
+- `add_subscription_id`, `resource_uri` and `name`
- `tag_name` and `tag_value`
-Different parameter combinations can be tried. If it is not supported either the InSpec resource or the Azure Rest API will raise an error.
+Different parameter combinations can be tried. If it is not supported, either the InSpec resource or the Azure Rest API will raise an error.
If the Azure Resource Manager endpoint returns multiple resources for a given query, this singular generic resource will fail. In that case, the [plural generic resource](azure_generic_resources.md) should be used.
@@ -72,7 +75,7 @@ The following properties are applicable to almost all resources.
| tags | The tag `key:value pairs` if defined on the resource. |
| properties | The resource properties. |
-For more properties, refer to [Azure documents](https://docs.microsoft.com/en-us/rest/api/resources/resources/list#genericresourceexpanded).
+For more properties, refer to specific Azure documents for the resource being tested.
## Examples
@@ -110,6 +113,22 @@ describe azure_generic_resource(resource_provider: 'Microsoft.DevTestLab/labs',
its('properties.allowClaim') { should cmp false }
end
```
+### Test a Resource Group
+```ruby
+describe azure_generic_resource(add_subscription_id: true, resource_uri: '/resourcegroups/', name: 'my_group') do
+ it { should exist }
+ its('tags') { should include(:owner) }
+ its('tags') { should include(owner: 'John Doe') }
+end
+```
+### Test a Policy Definition
+```ruby
+describe azure_generic_resource(add_subscription_id: true, resource_uri: 'providers/Microsoft.Authorization/policyDefinitions', name: 'my_policy') do
+ it { should exist }
+ its('properties.policyRule.then.effect') { should cmp 'deny' }
+ its('properties.policyType') { should cmp 'Custom' }
+end
+```
For more examples, please see the [integration tests](/test/integration/verify/controls/azure_generic_resource.rb).
## Matchers
diff --git a/docs/resources/azure_generic_resources.md b/docs/resources/azure_generic_resources.md
index 0d060a15c..efca67c49 100644
--- a/docs/resources/azure_generic_resources.md
+++ b/docs/resources/azure_generic_resources.md
@@ -38,6 +38,8 @@ All of them are optional.
| resource_provider | Azure resource provider of the resources to be tested. | `Microsoft.Compute/virtualMachines` |
| tag_name* | Tag name defined on the Azure resources. | `name` |
| tag_value | Tag value of the tag defined with the `tag_name`. | `external_linux` |
+| resource_uri | Azure REST API URI of the resources to be tested. This parameter should be used when resources do not reside in resource groups. It requires `add_subscription_id` parameter to be provided together. `/subscriptions/{subscriptionId}/providers/Microsoft.Authorization/policyDefinitions/` |
+| add_subscription_id | Indicates whether the `resource_uri` contains the subscription id. `true` or `false` |
* When resources are filtered by a tag name and value, the tags for each resource are not returned in the results.
@@ -52,6 +54,7 @@ Either one of the parameter sets can be provided for a valid query:
- `substring_of_resource_group` and `resource_provider`
- `tag_name`
- `tag_name` and `tag_value`
+- `add_subscription_id` and `resource_uri`
Different parameter combinations can be tried. If it is not supported either the InSpec resource or the Azure Rest API will raise an error.
@@ -66,12 +69,14 @@ It is advised to use these parameter sets to narrow down the targeted resources
| tags | A list of `tag:value` pairs defined on the resources. | `tags`|
| types | A list of resource types. | `type`|
| locations | A list of locations where resources are created in. | `location`|
-| created_times | A list of created times of the resources. | `created_time`|
-| changed_times | A list of changed times of the resources. | `changed_time`|
-| provisioning_states | A list of provisioning states of the resources. | `provisioning_state`|
+| created_times** | A list of created times of the resources. | `created_time`|
+| changed_times** | A list of changed times of the resources. | `changed_time`|
+| provisioning_states** | A list of provisioning states of the resources. | `provisioning_state`|
* For information on how to use filter criteria on plural resources refer to [FilterTable usage](https://github.com/inspec/inspec/blob/master/docs/dev/filtertable-usage.md#a-where-method-you-can-call-with-hash-params-with-loose-matching).
+** These properties are not available when `resource_uri` is used.
+
## Examples
### Test All Virtual Machines in Your Subscription
@@ -111,6 +116,12 @@ describe azure_generic_resources.where{ created_time > Time.now - 86400 } do
it { should exist }
end
```
+### Test Policy Definitions
+```ruby
+describe azure_generic_resources(add_subscription_id: true, resource_uri: 'providers/Microsoft.Authorization/policyDefinitions') do
+ it { should exist }
+end
+```
Please see [here](https://github.com/inspec/inspec/blob/master/docs/dev/filtertable-usage.md) for more information on how to leverage FilterTable capabilities on plural resources.
For more examples, please see the [integration tests](/test/integration/verify/controls/azure_generic_resources.rb).
diff --git a/libraries/azure_backend.rb b/libraries/azure_backend.rb
index 4855cb73b..3beb139b1 100644
--- a/libraries/azure_backend.rb
+++ b/libraries/azure_backend.rb
@@ -151,8 +151,10 @@ def rescue_wrong_api_call(url, params = {})
response = @azure.rest_get_call(url, params)
rescue UnsuccessfulAPIQuery::UnexpectedHTTPResponse::InvalidApiVersionParameter => e
api_version_suggested = e.suggested_api_version(params['api-version'])
- Inspec::Log.warn "Incompatible api version: #{params['api-version']}\n"\
+ unless params['api-version'] == 'failed_attempt'
+ Inspec::Log.warn "Incompatible api version: #{params['api-version']}\n"\
"Trying with the latest api version suggested by the Azure Rest API: #{api_version_suggested}."
+ end
if api_version_suggested.nil?
Inspec::Log.warn 'Failed to acquire suggested api version from the Azure Rest API.'
else
@@ -234,33 +236,22 @@ def construct_resource_id
#
def get_resource(opts = {})
Helpers.validate_parameters(resource_name: @__resource_name__,
- require_any_of: %i(resource_uri resource_provider resource_path resource_group),
+ required: %i(resource_uri),
allow: %i(api_version),
opts: opts)
api_version = opts[:api_version] || 'latest'
- argument_error_message = 'Parameters error. For singular resources `resource_uri`; '\
- 'for plural resources `resource_provider` and/or `resource_path` should be provided.'
- if opts.key?(:resource_uri) && opts.keys.any? { |k| %i(resource_provider resource_path).include?(k) }
- raise ArgumentError, argument_error_message
- end
- if opts.key?(:resource_uri)
- uri_subdomain = opts[:resource_uri]
- else
- uri_subdomain = ["/subscriptions/#{@azure.credentials[:subscription_id]}/providers",
- opts[:resource_provider],
- opts[:resource_path]].compact.join('/').gsub('//', '/')
- end
- _resource_group, provider, r_type = Helpers.res_group_provider_type_from_uri(uri_subdomain)
- # Add resource_group if provided.
- unless opts[:resource_group].nil?
- uri_subdomain = uri_subdomain.sub('/providers/', "/resourceGroups/#{opts[:resource_group]}/providers/")
+ if opts[:resource_uri].include?('providers')
+ # If the resource provider is unknown then this method can't find the api_version.
+ # The latest api_version will de acquired from the error message via #rescue_wrong_api_call method.
+ _resource_group, provider, r_type = Helpers.res_group_provider_type_from_uri(opts[:resource_uri])
end
# Some resource names can contain spaces. Decode them before parsing with URI.
- url = URI.join(@azure.resource_manager_endpoint_url, uri_subdomain.gsub(' ', '%20'))
+ url = URI.join(@azure.resource_manager_endpoint_url, opts[:resource_uri].gsub(' ', '%20'))
api_version = api_version.downcase
if %w{latest default}.include?(api_version)
+ api_version_info = {}
# api_version is not a specific version yet: latest or default.
- api_version_info = get_api_version(provider, r_type, api_version)
+ api_version_info = get_api_version(provider, r_type, api_version) if provider
# Something was wrong at get_api_version, and we will try to get a valid api_version via rescue_wrong_api_call
# by providing an invalid api_version intentionally.
api_version_info[:api_version] = 'failed_attempt' if api_version_info[:api_version].nil?
@@ -305,20 +296,23 @@ def get_api_version(provider, resource_type, api_version_status = 'latest')
end
return response unless response[:api_version].nil?
- # If the resource manager api version is updated earlier, use that.
- api_version_mgm = @resource_manager_endpoint_api || @azure.resource_manager_endpoint_api_version
- url = Helpers.construct_url([
- @azure.resource_manager_endpoint_url,
- 'subscriptions',
- @azure.credentials[:subscription_id], 'providers',
- provider
- ])
- provider_details, suggested_api_version = rescue_wrong_api_call(url, { 'api-version' => api_version_mgm })
- # If suggested_api_version is not nil, then the resource manager api version should be updated.
- unless suggested_api_version.nil?
- @resource_manager_endpoint_api = suggested_api_version
- Inspec::Log.warn "Resource manager endpoint api version should be updated with #{suggested_api_version} in `libraries/backend/helpers.rb`"
+ # Use the cached provider details if exist.
+ if @azure.provider_details[provider.to_sym].nil?
+ # If the resource manager api version is updated earlier, use that.
+ api_version_mgm = @resource_manager_endpoint_api || @azure.resource_manager_endpoint_api_version
+ url = Helpers.construct_url([@azure.resource_manager_endpoint_url, 'subscriptions',
+ @azure.credentials[:subscription_id], 'providers',
+ provider])
+ provider_details, suggested_api_version = rescue_wrong_api_call(url, { 'api-version' => api_version_mgm })
+ # If suggested_api_version is not nil, then the resource manager api version should be updated.
+ unless suggested_api_version.nil?
+ @resource_manager_endpoint_api = suggested_api_version
+ Inspec::Log.warn "Resource manager endpoint api version should be updated to #{suggested_api_version} in `libraries/backend/helpers.rb`"
+ end
+ else
+ provider_details = @azure.provider_details[provider.to_sym]
end
+
resource_type_details = provider_details[:resourceTypes].select { |rt| rt[:resourceType] == resource_type }&.first
# For some resource types the api version might be available with their parent resource.
if resource_type_details.nil? && resource_type.include?('/')
@@ -329,6 +323,8 @@ def get_api_version(provider, resource_type, api_version_status = 'latest')
Inspec::Log.warn "Couldn't get the #{api_version_status} API version for `#{provider}/#{resource_type}`. " \
'Please make sure that the provider/resourceType are in the correct format, e.g. `Microsoft.Compute/virtualMachines`.'
else
+ # Caching provider details.
+ @azure.provider_details[provider.to_sym] = provider_details if @azure.provider_details[provider.to_sym].nil?
api_versions = resource_type_details[:apiVersions]
api_versions_stable = api_versions.reject { |a| a.include?('preview') }
api_versions_preview = api_versions.select { |a| a.include?('preview') }
@@ -377,6 +373,25 @@ def validate_short_desc(resource_list, filter, singular = true)
end
end
+ def validate_resource_uri
+ Helpers.validate_params_required(%i(add_subscription_id), @opts)
+ if @opts[:add_subscription_id] == true
+ @opts[:resource_uri] = "/subscriptions/#{@azure.credentials[:subscription_id]}/#{@opts[:resource_uri]}"
+ .gsub('//', '/')
+ end
+ end
+
+ def validate_resource_provider
+ # Ensure that the provided resource id is for the correct resource provider.
+ if @opts.key?(:resource_id) && !@opts[:resource_id].downcase.include?(@opts[:resource_provider].downcase)
+ raise ArgumentError, "Resource provider must be #{@opts[:resource_provider]}."
+ end
+ if @opts.key?(:resource_uri) && !@opts[:resource_uri].downcase.include?(@opts[:resource_provider].downcase)
+ raise ArgumentError, "Resource provider must be #{@opts[:resource_provider]}."
+ end
+ true
+ end
+
# Get the paginated result.
# The next_link url won't be validated since it is provided by the Azure Rest API.
# @see https://docs.microsoft.com/en-us/rest/api/azure/#async-operations-throttling-and-paging
@@ -416,7 +431,7 @@ def specific_resource_constraint(resource_provider, opts)
# This should be used to ensure to fail the resources properly if they can not be created.
def catch_failed_resource_queries
yield
- # Inform user if it is an API incompatibility issue and recommend how to solve it.
+ # Inform user if it is an API incompatibility issue and recommend how to solve it.
rescue UnsuccessfulAPIQuery::UnexpectedHTTPResponse::InvalidApiVersionParameter => e
api_version_suggested_list = e.suggested_api_version
message = "Incompatible api version is provided.\n"\
@@ -652,7 +667,7 @@ def include?(opt)
end
if opt.is_a?(Hash)
raise ArgumentError, 'Only one item can be provided' if opt.keys.size > 1
- return @item[opt.keys.first] == opt.values.first
+ return @item[opt.keys.first&.to_sym] == opt.values.first
end
@item.key?(opt.to_sym)
end
diff --git a/libraries/azure_generic_resource.rb b/libraries/azure_generic_resource.rb
index 63af731a4..339ff6bbb 100644
--- a/libraries/azure_generic_resource.rb
+++ b/libraries/azure_generic_resource.rb
@@ -11,30 +11,11 @@ class AzureGenericResource < AzureResourceBase
def initialize(opts = {}, static_resource = false)
super(opts)
-
- if static_resource && !@opts.key?(:resource_id)
- if @opts[:resource_identifiers]
- raise ArgumentError, '`:resource_identifiers` have to be provided within a list.' \
- unless @opts[:resource_identifiers].is_a?(Array)
- # The `name` parameter should have been required in the static resource.
- # Since it is a mandatory field, it is better to make sure that it is in the required list before validations.
- @opts[:resource_identifiers] << :name unless @opts[:resource_identifiers].include?(:name)
- provided = Helpers.validate_params_only_one_of(@__resource_name__, @opts[:resource_identifiers], @opts)
- # Remove resource identifiers other than `:name`.
- unless provided == :name
- @opts[:name] = @opts[provided]
- @opts.delete(provided)
- end
- end
- required_params = %i(resource_group name)
- required_params += @opts[:required_parameters] if @opts.key?(:required_parameters)
- validate_parameters(required: required_params, allow: %i(resource_path resource_identifiers resource_provider))
- elsif static_resource && @opts.key?(:resource_id)
- # Ensure that the provided resource id is for the correct resource provider.
- raise ArgumentError, "Resource provider must be #{@opts[:resource_provider]}." \
- unless @opts[:resource_id].downcase.include?(@opts[:resource_provider].downcase)
- @opts.delete(:resource_provider)
- validate_parameters(required: %i(resource_id), allow: %i(resource_path resource_identifiers resource_provider))
+ if @opts.key?(:resource_provider)
+ validate_resource_provider
+ end
+ if static_resource
+ validate_static_resource
else
# Either one of the following sets can be provided for a valid short description query (to get the resource_id).
# resource_group + name
@@ -43,34 +24,37 @@ def initialize(opts = {}, static_resource = false)
# resource_group + resource_provider + name
# resource_id: no other parameters (within above mentioned) should exist
#
- # If there are static resource specific validations they can be passed here:
- # required parameters via `opts[:required_parameters]`
validate_parameters(require_any_of: %i(resource_group
resource_path
name
tag_name
tag_value
resource_id
- resource_provider))
+ resource_uri
+ resource_provider
+ add_subscription_id))
end
- @display_name = @opts.slice(:resource_group, :resource_provider, :name, :tag_name, :tag_value, :resource_id)
- .values.join(' ')
-
- # Use the latest api_version unless provided.
- api_version = @opts[:api_version] || 'latest'
+ @display_name = @opts.slice(:resource_group, :resource_provider, :name, :tag_name, :tag_value, :resource_id,
+ :resource_uri).values.join(' ')
# Get/create or acquire the resource_id.
# The resource_id is a MUST to get the detailed resource information.
#
# Use the provided resource_id
- if @opts[:resource_id]
+ if @opts.key?(:resource_uri)
+ if static_resource
+ validate_parameters(required: %i(resource_uri add_subscription_id name), allow: %i(resource_provider))
+ else
+ validate_parameters(required: %i(resource_uri add_subscription_id name))
+ end
+ validate_resource_uri
+ @resource_id = [@opts[:resource_uri], @opts[:name]].join('/').gsub('//', '/')
+ elsif @opts.key?(:resource_id)
@resource_id = @opts[:resource_id]
-
- # Construct the resource_id from parameters if they are sufficient
+ # Construct the resource_id from parameters if they are sufficient
elsif %i(resource_group resource_provider name).all? { |param| @opts.keys.include?(param) }
@resource_id = construct_resource_id
-
- # Query the resource management endpoint to get the resource_id with the provided parameters.
+ # Query the resource management endpoint to get the resource_id with the provided parameters.
else
filter = @opts.slice(:resource_group, :name, :resource_provider, :tag_name, :tag_value, :location)
catch_failed_resource_queries do
@@ -99,6 +83,8 @@ def initialize(opts = {}, static_resource = false)
# This is the last check on resource_id before talking to resource manager endpoint to get the detailed information.
Helpers.validate_resource_uri(@resource_id)
+ # Use the latest api_version unless provided.
+ api_version = @opts[:api_version] || 'latest'
catch_failed_resource_queries do
params = { resource_uri: @resource_id, api_version: api_version }
@resource_long_desc = get_resource(params)
@@ -163,4 +149,29 @@ def additional_resource_properties(opts = {})
create_resource_methods({ opts[:property_name].to_sym => properties })
public_send(opts[:property_name].to_sym) if respond_to?(opts[:property_name])
end
+
+ private
+
+ def validate_static_resource
+ if @opts.key?(:resource_id) || @opts.key?(:resource_uri)
+ return
+ end
+ if @opts[:resource_identifiers]
+ raise ArgumentError, '`:resource_identifiers` should be a list.' unless @opts[:resource_identifiers].is_a?(Array)
+ # The `name` parameter should have been required in the static resource.
+ # Since it is a mandatory field, it is better to make sure that it is in the required list before validations.
+ @opts[:resource_identifiers] << :name unless @opts[:resource_identifiers].include?(:name)
+ provided = Helpers.validate_params_only_one_of(@__resource_name__, @opts[:resource_identifiers], @opts)
+ # Remove resource identifiers other than `:name`.
+ unless provided == :name
+ @opts[:name] = @opts[provided]
+ @opts.delete(provided)
+ end
+ end
+ required_parameters = %i(resource_group resource_provider name)
+ allowed_parameters = %i(resource_path resource_identifiers)
+ required_parameters += @opts[:required_parameters] if @opts.key?(:required_parameters)
+ allowed_parameters += @opts[:allowed_parameters] if @opts.key?(:allowed_parameters)
+ validate_parameters(required: required_parameters, allow: allowed_parameters)
+ end
end
diff --git a/libraries/azure_generic_resources.rb b/libraries/azure_generic_resources.rb
index c3f6bb3bb..6bd21b652 100644
--- a/libraries/azure_generic_resources.rb
+++ b/libraries/azure_generic_resources.rb
@@ -14,41 +14,45 @@ class AzureGenericResources < AzureResourceBase
def initialize(opts = {}, static_resource = false)
# A HTTP client will be created in the backend.
super(opts)
-
- @display_name = @opts.slice(:resource_group, :resource_path, :name, :resource_provider, :tag_name, :tag_value).values.join(' ')
+ # Ensure that the provided resource id is for the correct resource provider.
+ if @opts.key?(:resource_provider)
+ validate_resource_provider
+ end
+ @display_name = @opts.slice(:resource_group, :resource_path, :name, :resource_provider,
+ :tag_name,
+ :tag_value,
+ :resource_uri)
+ .values.join(' ')
+ # @table = fetch_data
+ table_schema = [
+ { column: :ids, field: :id },
+ { column: :names, field: :name },
+ { column: :tags, field: :tags },
+ { column: :types, field: :type },
+ { column: :locations, field: :location },
+ { column: :created_times, field: :createdTime },
+ { column: :changed_times, field: :changedTime },
+ { column: :provisioning_states, field: :provisioningState },
+ ]
if static_resource
- raise ArgumentError, 'Warning for the resource author: `resource_provider` must be defined.' \
- unless opts.key?(:resource_provider)
- @table = []
- @resources = {}
- opts[:api_version] = 'latest' unless opts.key?(:api_version)
- # These are the parameters created in the static resource code, NOT provided by the user.
- allowed_params = %i(resource_path resource_group resource_provider)
- # User provided parameters will be passed here for validation with:
- # opts[:required_parameters]
- # opts[:allowed_parameters]
- allowed_params += opts[:allowed_parameters] unless opts[:allowed_parameters].nil?
- parameters_to_validate = {
- required: opts[:required_parameters],
- allow: allowed_params,
- }.each_with_object({}) { |(k, v), acc| acc[k] = v unless v.nil? }
- validate_parameters(**parameters_to_validate)
- @display_name = @opts[:display_name] unless @opts[:display_name].nil?
- get_resources(opts[:resource_path])
+ validate_static_resource
return
end
-
+ if @opts.key?(:resource_uri)
+ validate_parameters(required: %i(resource_uri add_subscription_id), allow: %i(api_version))
+ validate_resource_uri
+ collect_resources
+ AzureGenericResources.populate_filter_table(:table, table_schema)
+ return
+ end
+ raise ArgumentError, "#{@__resource_name__}: The `api_version` parameter is not allowed." if opts.key?(:api_version)
# Either one of the following sets can be provided for a valid short description query.
# resource_group
# name
# substring_of_name, substring_of_resource_group
# tag_name + tag_value
# resource_group + resource_provider
- raise ArgumentError, "#{@__resource_name__}: The `api_version` parameter is not allowed." if opts.key?(:api_version)
- validate_parameters(allow: %i(name
- substring_of_name
- resource_group
- substring_of_resource_group
+ validate_parameters(allow: %i(name substring_of_name resource_group substring_of_resource_group
resource_provider
tag_name
tag_value
@@ -70,18 +74,6 @@ def initialize(opts = {}, static_resource = false)
# However, an empty FilterTable should still be created to be able to response `should_not exist` test.
return unless validated || @resources.empty?
@table = @resources.empty? ? [] : @resources
-
- # @table = fetch_data
- table_schema = [
- { column: :ids, field: :id },
- { column: :names, field: :name },
- { column: :tags, field: :tags },
- { column: :types, field: :type },
- { column: :locations, field: :location },
- { column: :created_times, field: :createdTime },
- { column: :changed_times, field: :changedTime },
- { column: :provisioning_states, field: :provisioningState },
- ]
AzureGenericResources.populate_filter_table(:table, table_schema)
end
@@ -129,26 +121,37 @@ def self.populate_filter_table(raw_data, table_scheme)
# Call this in the static resources.
# Get plural resource details and populate @table to be used in FilterTable.
# Paginate API responses if necessary.
- # @param resource_path [String, nil] A part of the URL that will be used to query resources.
- # If the endpoint is
- # `https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/
- # virtualMachines?api-version=2019-12-01`
- # resource_path should be: nil
- #
+ # resource_path [String] A part of the URL that will be used to query resources.
# If the endpoint is
# `https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/
# Microsoft.DBforMySQL/servers/{serverName}/databases?api-version=2017-12-01`
# resource_path should be: `{serverName}/databases`
#
+ # resource_uri [String] URI of the resources.
+ # If the endpoint is
+ # `https://management.azure.com/providers/Microsoft.Authorization/policyDefinitions?api-version=2019-09-01`
+ # resource_uri should be: `providers/Microsoft.Authorization/policyDefinitions`
+ #
# `resource_group` will be added if provided at resource initialization.
#
- def get_resources(resource_path = nil)
+ def collect_resources
# Get details of resources and populate the FilterTable via @table.
# @see https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/listall
# GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/virtualMachines?
# api-version=2019-12-01
- query_params = @opts.slice(:api_version, :resource_group, :resource_provider)
- query_params[:resource_path] = resource_path unless resource_path.nil?
+ if @opts.key?(:resource_uri)
+ # query_params = { resource_uri: @opts[:resource_uri], api_version: @opts.dig(:api_version) }
+ query_params = { resource_uri: @opts[:resource_uri] }
+ else
+ resource_uri = ["/subscriptions/#{@azure.credentials[:subscription_id]}/providers",
+ @opts[:resource_provider],
+ @opts[:resource_path]].compact.join('/').gsub('//', '/')
+ unless @opts[:resource_group].nil?
+ resource_uri = resource_uri.sub('/providers/', "/resourceGroups/#{@opts[:resource_group]}/providers/")
+ end
+ query_params = { resource_uri: resource_uri }
+ end
+ query_params[:api_version] = @opts[:api_version] if @opts.key?(:api_version)
catch_failed_resource_queries do
@api_response = get_resource(query_params)
end
@@ -181,4 +184,29 @@ def get_resources(resource_path = nil)
end
nil
end
+
+ def validate_static_resource
+ raise ArgumentError, 'Warning for the resource author: `resource_provider` must be defined.' \
+ unless @opts.key?(:resource_provider)
+ @table = []
+ @resources = {}
+ @opts[:api_version] = 'latest' unless @opts.key?(:api_version)
+ # These are the parameters created in the static resource code, NOT provided by the user.
+ allowed_params = %i(resource_path resource_group resource_provider add_subscription_id resource_uri)
+ # User provided parameters will be passed here for validation with:
+ # opts[:required_parameters]
+ # opts[:allowed_parameters]
+ allowed_params += @opts[:allowed_parameters] unless @opts[:allowed_parameters].nil?
+ parameters_to_validate = {
+ required: @opts[:required_parameters],
+ allow: allowed_params,
+ }.each_with_object({}) { |(k, v), acc| acc[k] = v unless v.nil? }
+ validate_parameters(**parameters_to_validate)
+ @display_name = @opts[:display_name] unless @opts[:display_name].nil?
+
+ if @opts.key?(:resource_uri)
+ validate_resource_uri
+ end
+ collect_resources
+ end
end
diff --git a/libraries/azure_key_vaults.rb b/libraries/azure_key_vaults.rb
index 248fe72d3..f2e6fbe0e 100644
--- a/libraries/azure_key_vaults.rb
+++ b/libraries/azure_key_vaults.rb
@@ -65,14 +65,6 @@ def initialize(opts = {})
{ column: :properties, field: :properties },
]
- # Talk to Azure Rest API and gather resources data in @resources.
- # Paginate if necessary.
- # Use the `populate_table` method (if defined) for filling the @table with the desired resource attributes.
- get_resources
-
- # Check if the resource is failed.
- return if failed_resource?
-
# FilterTable is populated at the very end due to being an expensive operation.
AzureGenericResources.populate_filter_table(:table, table_schema)
end
diff --git a/libraries/azure_virtual_networks.rb b/libraries/azure_virtual_networks.rb
index fd0e1b6bb..cf9a66c0f 100644
--- a/libraries/azure_virtual_networks.rb
+++ b/libraries/azure_virtual_networks.rb
@@ -63,16 +63,6 @@ def initialize(opts = {})
{ column: :locations, field: :location },
]
- # Before calling the `get_resources` method, a private `populate_table` method has to be defined for this static resource.
- # Talk to Azure Rest API and gather resources data in @resources.
- # Paginate if necessary.
- # Use the `populate_table` method for filling the @table with the desired resource attributes according to the
- # table_schema layout.
- get_resources
-
- # Check if the resource is failed.
- return if failed_resource?
-
# FilterTable is populated at the very end due to being an expensive operation.
AzureGenericResources.populate_filter_table(:table, table_schema)
end
diff --git a/libraries/backend/azure_connection.rb b/libraries/backend/azure_connection.rb
index f0387e524..aa866cf3b 100644
--- a/libraries/backend/azure_connection.rb
+++ b/libraries/backend/azure_connection.rb
@@ -10,6 +10,8 @@
# - make the access token available to use.
class AzureConnection
@@token_data = HashRecursive.recursive
+ @@provider_details = {}
+
# This will be included in headers for statistical purposes.
INSPEC_USER_AGENT = 'pid-18d63047-6cdf-4f34-beed-62f01fc73fc2'
@@ -74,6 +76,10 @@ def initialize(client_args)
end
end
+ def provider_details
+ @@provider_details
+ end
+
# Make a HTTP GET request to Azure Rest API.
#
# Azure Rest API requires access token for every query.
@@ -152,21 +158,28 @@ def fail_api_query(resp, message = nil)
message += "HTTP #{resp.status}.\n"
body = resp.body
unless body.empty?
+ code = body[:code]
+ error_message = body[:message]
error = body[:error]
if error&.is_a?(Hash)
- code = error[:code]
- error_message = error[:message]
- message += "#{code} #{error_message}"
+ code ||= error[:code]
+ error_message ||= error[:message]
end
- message += resp.body.to_s if code.nil?
+ message += code.nil? ? "#{code} #{error_message}" : resp.body.to_s
end
resource_not_found_codes = %w{Request_ResourceNotFound ResourceGroupNotFound ResourceNotFound NotFound}
- wrong_api_keyword = ['The supported api-versions are', 'The supported versions']
- invalid_api_codes = %w{InvalidApiVersionParameter NoRegisteredProviderFound InvalidResourceType}
+ resource_not_found_keywords = ['could not be found']
+ wrong_api_keywords = ['The supported api-versions are', 'The supported versions are']
+ explicit_invalid_api_code = 'InvalidApiVersionParameter'
+ possible_invalid_api_codes = %w{InvalidApiVersionParameter NoRegisteredProviderFound InvalidResourceType}
+ code = code.to_s
if code
- if invalid_api_codes.include?(code) && wrong_api_keyword.any? { |kw| error_message&.include?(kw) }
+ if code == explicit_invalid_api_code \
+ || possible_invalid_api_codes.include?(code) && wrong_api_keywords.any? { |word| error_message.include?(word) }
raise UnsuccessfulAPIQuery::UnexpectedHTTPResponse::InvalidApiVersionParameter, error_message
- elsif resource_not_found_codes.include?(code)
+ elsif resource_not_found_codes.include?(code) \
+ || resource_not_found_codes.any? { |not_found_code| code.include?(not_found_code) } \
+ && resource_not_found_keywords.any? { |word| error_message.include?(word) }
raise UnsuccessfulAPIQuery::ResourceNotFound, error_message
end
end
diff --git a/libraries/backend/helpers.rb b/libraries/backend/helpers.rb
index 0089e72f0..cbc95e935 100644
--- a/libraries/backend/helpers.rb
+++ b/libraries/backend/helpers.rb
@@ -189,7 +189,7 @@ def self.validate_parameters(resource_name: nil, allow: [], required: nil, requi
# @return [String] Provided parameter within require only one of parameters.
# @param require_only_one_of [Array]
- def self.validate_params_only_one_of(resource_name, require_only_one_of, opts)
+ def self.validate_params_only_one_of(resource_name = nil, require_only_one_of, opts)
# At least one of them has to exist.
Helpers.validate_params_require_any_of(resource_name, require_only_one_of, opts)
provided = require_only_one_of.select { |i| opts.key?(i) }
@@ -202,14 +202,14 @@ def self.validate_params_only_one_of(resource_name, require_only_one_of, opts)
# @return [Array] Required parameters
# @param required [Array]
- def self.validate_params_required(resource_name, required, opts)
+ def self.validate_params_required(resource_name = nil, required, opts)
raise ArgumentError, "#{resource_name}: `#{required}` must be provided" unless opts.is_a?(Hash) && required.all? { |req| opts.key?(req) && !opts[req].nil? && opts[req] != '' }
required
end
# @return [Array] Require any of parameters
# @param require_any_of [Array]
- def self.validate_params_require_any_of(resource_name, require_any_of, opts)
+ def self.validate_params_require_any_of(resource_name = nil, require_any_of, opts)
raise ArgumentError, "#{resource_name}: One of `#{require_any_of}` must be provided." unless opts.is_a?(Hash) && require_any_of.any? { |req| opts.key?(req) && !opts[req].nil? && opts[req] != '' }
require_any_of
end
@@ -219,9 +219,10 @@ def self.validate_params_require_any_of(resource_name, require_any_of, opts)
def self.validate_params_allow(allow, opts)
raise ArgumentError, 'Arguments or values can not be longer than 256 characters.' if opts.any? { |k, v| k.size > 100 || v.to_s.size > 500 }
raise ArgumentError, 'Scalar arguments not supported' unless defined?(opts.keys)
- raise ArgumentError, 'Unexpected arguments found' unless opts.keys.all? { |a| allow.include?(a) }
+ raise ArgumentError, "Unexpected arguments found. Allowed: #{allow}, Found: #{opts}" unless opts.keys.all? { |a| allow.include?(a) }
raise ArgumentError, 'Provided parameter should not be empty' unless opts.values.all? do |a|
return true if a.class == Integer
+ return true if [TrueClass, FalseClass].include?(a.class)
!a.empty?
end
end
@@ -270,7 +271,7 @@ def self.validate_resource_uri(resource_uri)
resource_uri_format = '/subscriptions/{subscription_id}/resourceGroups/{resource_group}/providers/'\
'Microsoft.Compute/virtualMachines/{resource_name}'
raise ArgumentError, "Resource URI should be in the format of #{resource_uri_format}" \
- unless resource_uri.start_with?('/subscriptions/') && resource_uri.include?('/providers/')
+ unless resource_uri.start_with?('/subscriptions/') || resource_uri.include?('/providers/')
end
# Disassemble resource_id and extract the resource_group, provider and resource_provider.
diff --git a/test/integration/verify/controls/azure_generic_resource.rb b/test/integration/verify/controls/azure_generic_resource.rb
index 2f3147323..9552e0c4e 100644
--- a/test/integration/verify/controls/azure_generic_resource.rb
+++ b/test/integration/verify/controls/azure_generic_resource.rb
@@ -15,6 +15,11 @@
its('zones') { should be_nil }
end
+ describe azure_generic_resource(resource_uri: "/resourceGroups/#{resource_group}/providers/Microsoft.Compute/virtualMachines", name: win_name, add_subscription_id: true) do
+ it { should exist }
+ its('name') { should eq win_name }
+ end
+
# If api_version is not provided, latest version should be used.
describe azure_generic_resource(resource_group: resource_group, name: win_name) do
its('api_version_used_for_query_state') { should eq 'latest' }
diff --git a/test/integration/verify/controls/azure_generic_resources.rb b/test/integration/verify/controls/azure_generic_resources.rb
index 2c8edb3c0..c7ff083e2 100644
--- a/test/integration/verify/controls/azure_generic_resources.rb
+++ b/test/integration/verify/controls/azure_generic_resources.rb
@@ -23,6 +23,11 @@
its('provisioning_states') { should include('Succeeded') }
end
+ describe azure_generic_resources(resource_uri: 'resourcegroups', add_subscription_id: true) do
+ it { should exist }
+ its('types.uniq') { should cmp ['Microsoft.Resources/resourceGroups'] }
+ end
+
describe azure_generic_resources(resource_group: 'fake-group') do
it { should_not exist }
end
diff --git a/test/unit/resources/azure_generic_resource_test.rb b/test/unit/resources/azure_generic_resource_test.rb
index 2be093f1b..44ed11466 100644
--- a/test/unit/resources/azure_generic_resource_test.rb
+++ b/test/unit/resources/azure_generic_resource_test.rb
@@ -18,4 +18,9 @@ def test_only_resource_id_ok
def test_invalid_endpoint
assert_raises(ArgumentError) { AzureGenericResource.new(endpoint: 'fake_endpoint') }
end
+
+ def test_resource_uri
+ # add_subscription_id, name and resource_uri have to be provided together
+ assert_raises(ArgumentError) { AzureGenericResource.new(resource_uri: 'test') }
+ end
end
diff --git a/test/unit/resources/azure_generic_resources_test.rb b/test/unit/resources/azure_generic_resources_test.rb
index 62e6c5f12..59b178051 100644
--- a/test/unit/resources/azure_generic_resources_test.rb
+++ b/test/unit/resources/azure_generic_resources_test.rb
@@ -10,4 +10,9 @@ def test_resource_id_not_ok
def test_api_version_not_ok
assert_raises(ArgumentError) { AzureGenericResources.new(api_version: '2020-01-01') }
end
+
+ def test_resource_uri
+ # add_subscription_id and resource_uri have to be provided together
+ assert_raises(ArgumentError) { AzureGenericResources.new(resource_uri: 'test') }
+ end
end