add description field to service_account for Terraform

docs/resources/google_storage_bucket_object.md

@@ -25,6 +25,8 @@ end
 Properties that can be accessed from the `google_storage_bucket_object` resource:
 
 
+  * `bucket`: The name of the bucket.
+
   * `object`: The name of the object.
 
   * `content_type`: The Content-Type of the object data.  See https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types for more information on possible Content-Types

docs/resources/google_storage_bucket_objects.md

@@ -1,75 +1,45 @@
 ---
-title: About the google_storage_bucket_objects Resource
+title: About the google_storage_bucket_objects resource
 platform: gcp
 ---
 
-# google\_storage\_bucket\_objects
-
-Use the `google_storage_bucket_objects` InSpec audit resource to test properties of a GCP storage bucket objects.
-
-<br>
-
 ## Syntax
-
-A `google_storage_bucket_objects` resource block collects GCP bucket objects by project then tests that group.
-
-    describe google_storage_bucket_objects(bucket: 'bucket-name') do
-      it { should exist }
-    end
-
-Use this InSpec resource to enumerate IDs then test in-depth using `google_storage_bucket_object`.
-
-    google_storage_bucket_objects(bucket: 'bucket-name').object_names.each do |object_name|
-      describe google_storage_bucket_object(bucket: bucket_name, object: object_name) do
-        it { should exist }
-      end
-    end
-
-<br>
+A `google_storage_bucket_objects` is used to test a Google BucketObject resource
 
 ## Examples
-
-The following examples show how to use this InSpec audit resource.
-
-### Test that there are no more than a specified number of storage buckets for the project
-
-    describe google_storage_bucket_objects(bucket: 'bucket-name') do
-      its('count') { should be <= 100 }
-    end
-
-
-### Test that an expected named bucket is available
-
-    describe google_storage_bucket_objects(bucket: 'bucket-name') do
-      its('object_buckets'){ should include 'my_expected_bucket' }
-    end
-
-### Test that an expected named bucket is available
-
-    describe google_storage_bucket_objects(bucket: 'bucket-name') do
-      its('object_names'){ should include 'my_expected_object' }
-    end
-
-### Test a filtered group of bucket objects created within the last 24hrs
-
-    describe google_storage_bucket_objects(bucket: 'bucket-name').where(object_created_time > Time.now - 60*60*24) do
-      it { should exist }
-    end
-<br>
-
-## Filter Criteria
-
-This resource supports the following filter criteria:  `object_bucket`; `object_name` and `object_created_time`. Any of these may be used with `where`, as a block or as a method.
+```
+describe google_storage_bucket_objects(bucket: 'bucket-with-object') do
+	its('object_names') { should include 'image1' }
+	its('count') { should be <= 10 }
+end
+```
 
 ## Properties
+Properties that can be accessed from the `google_storage_bucket_objects` resource:
+
+See [google_storage_bucket_object.md](google_storage_bucket_object.md) for more detailed information
+  * `object_buckets`: an array of `google_storage_bucket_object` bucket
+  * `objects`: an array of `google_storage_bucket_object` object
+  * `content_types`: an array of `google_storage_bucket_object` content_type
+  * `crc32cs`: an array of `google_storage_bucket_object` crc32c
+  * `etags`: an array of `google_storage_bucket_object` etag
+  * `generations`: an array of `google_storage_bucket_object` generation
+  * `ids`: an array of `google_storage_bucket_object` id
+  * `md5_hashes`: an array of `google_storage_bucket_object` md5_hash
+  * `media_links`: an array of `google_storage_bucket_object` media_link
+  * `metagenerations`: an array of `google_storage_bucket_object` metageneration
+  * `object_names`: an array of `google_storage_bucket_object` name
+  * `sizes`: an array of `google_storage_bucket_object` size
+  * `storage_classes`: an array of `google_storage_bucket_object` storage_class
+  * `object_created_times`: an array of `google_storage_bucket_object` time_created
+  * `time_deleteds`: an array of `google_storage_bucket_object` time_deleted
+  * `time_storage_class_updateds`: an array of `google_storage_bucket_object` time_storage_class_updated
+  * `time_updateds`: an array of `google_storage_bucket_object` time_updated
 
-*  `object_buckets` - an array of google_storage_bucket identifier strings
-*  `object_names` - an array of google_storage_bucket_object name strings
-*  `object_created_times` - an array of time created Time objects
-
-<br
-
+## Filter Criteria
+This resource supports all of the above properties as filter criteria, which can be used
+with `where` as a block or a method.
 
 ## GCP Permissions
 
-Ensure the [Google Cloud Storage API](https://console.cloud.google.com/apis/api/storage-component.googleapis.com/) is enabled.
+Ensure the [Google Cloud Storage](https://console.cloud.google.com/apis/library/storage-component.googleapis.com/) is enabled for the current project.

libraries/google_storage_bucket_object.rb

@@ -22,6 +22,7 @@ class StorageBucketObject < GcpResourceBase
   supports platform: 'gcp'
 
   attr_reader :params
+  attr_reader :bucket
   attr_reader :object
   attr_reader :content_type
   attr_reader :crc32c
@@ -47,6 +48,7 @@ def initialize(params)
   end
 
   def parse
+    @bucket = @fetched['bucket']
     @object = @fetched['object']
     @content_type = @fetched['contentType']
     @crc32c = @fetched['crc32c']
@@ -78,10 +80,6 @@ def to_s
     "BucketObject #{@params[:object]}"
   end
 
-  def bucket
-    @params[:bucket]
-  end
-
   # rubocop:disable Lint/DuplicateMethods
   def size
     @size&.to_i

libraries/google_storage_bucket_objects.rb

@@ -1,48 +1,116 @@
-# frozen_string_literal: true
+# frozen_string_literal: false
 
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    AUTO GENERATED CODE     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
 require 'gcp_backend'
+class StorageBucketObjects < GcpResourceBase
+  name 'google_storage_bucket_objects'
+  desc 'BucketObject plural resource'
+  supports platform: 'gcp'
 
-module Inspec::Resources
-  class GoogleStorageBucketObjects < GcpResourceBase
-    name 'google_storage_bucket_objects'
-    desc 'Verifies settings for GCP storage bucket objects in bulk'
+  attr_reader :table
 
-    example "
-      describe google_storage_bucket_objects(bucket: 'bucket-name') do
-        it { should exist }
-      end
-    "
+  filter_table_config = FilterTable.create
 
-    def initialize(opts = {})
-      # Call the parent class constructor
-      super(opts)
-      @bucket = opts[:bucket]
-    end
+  filter_table_config.add(:object_buckets, field: :object_bucket)
+  filter_table_config.add(:objects, field: :object)
+  filter_table_config.add(:content_types, field: :content_type)
+  filter_table_config.add(:crc32cs, field: :crc32c)
+  filter_table_config.add(:etags, field: :etag)
+  filter_table_config.add(:generations, field: :generation)
+  filter_table_config.add(:ids, field: :id)
+  filter_table_config.add(:md5_hashes, field: :md5_hash)
+  filter_table_config.add(:media_links, field: :media_link)
+  filter_table_config.add(:metagenerations, field: :metageneration)
+  filter_table_config.add(:object_names, field: :object_name)
+  filter_table_config.add(:sizes, field: :size)
+  filter_table_config.add(:storage_classes, field: :storage_class)
+  filter_table_config.add(:object_created_times, field: :object_created_times)
+  filter_table_config.add(:time_deleteds, field: :time_deleted)
+  filter_table_config.add(:time_storage_class_updateds, field: :time_storage_class_updated)
+  filter_table_config.add(:time_updateds, field: :time_updated)
 
-    # FilterTable setup
-    filter_table_config = FilterTable.create
-    filter_table_config.add(:object_names, field: :object_name)
-    filter_table_config.add(:object_buckets, field: :object_bucket)
-    filter_table_config.add(:object_created_times, field: :object_created_time)
-    filter_table_config.connect(self, :fetch_data)
-
-    def fetch_data
-      bucket_object_rows = []
-      next_page = nil
-      loop do
-        catch_gcp_errors do
-          @bucket_objects = @gcp.gcp_storage_client.list_objects(@bucket, page_token: next_page)
-        end
-        return [] if !@bucket_objects || !@bucket_objects.items
-        @bucket_objects.items.map do |object|
-          bucket_object_rows+=[{ object_name: object.name,
-                                 object_bucket: object.bucket,
-                                 object_created_time: Time.parse(object.time_created.to_s) }]
+  filter_table_config.connect(self, :table)
+
+  def initialize(params = {})
+    super(params.merge({ use_http_transport: true }))
+    @params = params
+    @table = fetch_wrapped_resource('objects')
+  end
+
+  def fetch_wrapped_resource(wrap_path)
+    # fetch_resource returns an array of responses (to handle pagination)
+    result = @connection.fetch_all(product_url, resource_base_url, @params, 'Get')
+    return if result.nil?
+
+    # Conversion of string -> object hash to symbol -> object hash that InSpec needs
+    converted = []
+    result.each do |response|
+      next if response.nil? || !response.key?(wrap_path)
+      response[wrap_path].each do |hash|
+        hash_with_symbols = {}
+        hash.each_key do |key|
+          name, value = transform(key, hash)
+          hash_with_symbols[name] = value
         end
-        next_page = @bucket_objects.next_page_token
-        break unless next_page
+        converted.push(hash_with_symbols)
       end
-      @table = bucket_object_rows
     end
+
+    converted
+  end
+
+  def transform(key, value)
+    return transformers[key].call(value) if transformers.key?(key)
+
+    [key.to_sym, value]
+  end
+
+  def transformers
+    {
+      'bucket' => ->(obj) { return :object_bucket, obj['bucket'] },
+      'object' => ->(obj) { return :object, obj['object'] },
+      'contentType' => ->(obj) { return :content_type, obj['contentType'] },
+      'crc32c' => ->(obj) { return :crc32c, obj['crc32c'] },
+      'etag' => ->(obj) { return :etag, obj['etag'] },
+      'generation' => ->(obj) { return :generation, obj['generation'] },
+      'id' => ->(obj) { return :id, obj['id'] },
+      'md5Hash' => ->(obj) { return :md5_hash, obj['md5Hash'] },
+      'mediaLink' => ->(obj) { return :media_link, obj['mediaLink'] },
+      'metageneration' => ->(obj) { return :metageneration, obj['metageneration'] },
+      'name' => ->(obj) { return :object_name, obj['name'] },
+      'size' => ->(obj) { return :size, obj['size'] },
+      'storageClass' => ->(obj) { return :storage_class, obj['storageClass'] },
+      'timeCreated' => ->(obj) { return :object_created_times, parse_time_string(obj['timeCreated']) },
+      'timeDeleted' => ->(obj) { return :time_deleted, parse_time_string(obj['timeDeleted']) },
+      'timeStorageClassUpdated' => ->(obj) { return :time_storage_class_updated, parse_time_string(obj['timeStorageClassUpdated']) },
+      'updated' => ->(obj) { return :time_updated, parse_time_string(obj['updated']) },
+    }
+  end
+
+  # Handles parsing RFC3339 time string
+  def parse_time_string(time_string)
+    time_string ? Time.parse(time_string) : nil
+  end
+
+  private
+
+  def product_url
+    'https://www.googleapis.com/storage/v1/'
+  end
+
+  def resource_base_url
+    'b/{{bucket}}/o/{{object}}'
   end
 end

test/integration/build/gcp-mm.tf

@@ -661,6 +661,7 @@ resource "google_storage_bucket_object" "object" {
 }
 
 resource "google_app_engine_standard_app_version" "default" {
+  count = "${var.gcp_organization_id == "" ? 0 : var.gcp_enable_privileged_resources}"
   project         = var.gcp_project_id
   version_id      = var.standardappversion["version_id"]
   service         = var.standardappversion["service"]

test/integration/verify/controls/google_appengine_standard_app_version.rb

@@ -23,10 +23,12 @@
   "entrypoint": "node ./app.js",
   "port": "8080"
 }, description: 'Cloud App Engine definition')
+gcp_enable_privileged_resources = attribute(:gcp_enable_privileged_resources, default:0, description:'Flag to enable privileged resources requiring elevated privileges in GCP.')
 control 'google_appengine_standard_app_version-1.0' do
   impact 1.0
   title 'google_appengine_standard_app_version resource test'
 
+  only_if { gcp_enable_privileged_resources.to_i == 1 }
 
   describe google_appengine_standard_app_version(project: gcp_project_id, location: gcp_location, version_id: standardappversion['version_id'], service: standardappversion['service']) do
     it { should exist }

test/integration/verify/controls/google_appengine_standard_app_versions.rb

@@ -23,10 +23,12 @@
   "entrypoint": "node ./app.js",
   "port": "8080"
 }, description: 'Cloud App Engine definition')
+gcp_enable_privileged_resources = attribute(:gcp_enable_privileged_resources, default:0, description:'Flag to enable privileged resources requiring elevated privileges in GCP.')
 control 'google_appengine_standard_app_versions-1.0' do
   impact 1.0
   title 'google_appengine_standard_app_versions resource test'
 
+  only_if { gcp_enable_privileged_resources.to_i == 1 }
 
   describe google_appengine_standard_app_versions(project: gcp_project_id, location: gcp_location,service: standardappversion['service']) do
     its('runtimes') { should include standardappversion['runtime'] }

test/integration/verify/controls/google_storage_bucket_objects.rb

@@ -1,18 +1,31 @@
-title 'Storage Bucket Objects Properties'
+# ----------------------------------------------------------------------------
+#
+#     ***     AUTO GENERATED CODE    ***    AUTO GENERATED CODE     ***
+#
+# ----------------------------------------------------------------------------
+#
+#     This file is automatically generated by Magic Modules and manual
+#     changes will be clobbered when the file is regenerated.
+#
+#     Please read more about how to change this file in README.md and
+#     CONTRIBUTING.md located at the root of this package.
+#
+# ----------------------------------------------------------------------------
 
-gcp_storage_bucket_object = attribute(:gcp_storage_bucket_object, default: '', description: 'The GCP bucket with objects.')
-gcp_storage_bucket_object_name = attribute(:gcp_storage_bucket_object_name, default: '', description: 'The GCP bucket object name.')
-gcp_enable_privileged_resources = attribute(:gcp_enable_privileged_resources,default:0,description:'Flag to enable privileged resources requiring elevated privileges in GCP.')
+title 'Test GCP google_storage_bucket_objects resource.'
 
-control 'gcp-storage-bucket-objects-1.0' do
-
-  only_if { gcp_enable_privileged_resources.to_i == 1 }
+gcp_project_id = attribute(:gcp_project_id, default: 'gcp_project_id', description: 'The GCP project identifier.')
+gcp_storage_bucket_object = attribute(:gcp_storage_bucket_object, default: 'gcp_storage_bucket_object', description: 'The name of the storage bucket with an object')
+gcp_service_account_display_name = attribute(:gcp_service_account_display_name, default: 'gcp_service_account_display_name', description: 'The name of the service account assigned permissions')
+gcp_enable_privileged_resources = attribute(:gcp_enable_privileged_resources, default: 'gcp_enable_privileged_resources', description: 'If we are running tests with escalated permissions(required for this test)')
+gcp_storage_bucket_object_name = attribute(:gcp_storage_bucket_object_name, default: 'gcp_storage_bucket_object_name', description: 'The name of the object')
+control 'google_storage_bucket_objects-1.0' do
   impact 1.0
-  title 'Ensure that the Storage Bucket Objects have the correct properties in bulk'
+  title 'google_storage_bucket_objects resource test'
 
+  only_if { gcp_enable_privileged_resources.to_i == 1 }
   describe google_storage_bucket_objects(bucket: gcp_storage_bucket_object) do
-    it { should exist }
-    its('object_buckets') { should include gcp_storage_bucket_object }
-    its('object_names') { should include gcp_storage_bucket_object_name}
+  	its('object_names') { should include gcp_storage_bucket_object_name }
+  	its('count') { should be <= 10 }
   end
-end
+end