profile scan is reported every chef-client run even if compliance_profile resource wasn't executed

[jeremiahsnapp]

Cookbook version

0.14.4

Chef-client version

Chef: 12.14.89

Platform Details

Ubuntu 14.04

Scenario:

After working around issue #101 the audit cookbook properly fetches and executes compliance profiles according to the interval set in attributes. However, even when the compliance_profile resource does not get executed because of the interval the compliance_report resource still reports the results of the profile using the report results from previous chef-client runs that get cached in /var/chef/cache/compliance.

Here is the code that reads the cached report.

https://github.com/chef-cookbooks/audit/blob/v0.14.4/libraries/report.rb#L85

Seems like a possible solution would be to delete cached reports at the beginning of the audit recipe. Doing it at the beginning seems safer than at the end because if the chef-client run should fail somehow the cached files could accidentally remain in place for the next chef-client run

Steps to Reproduce:

Use the workaround in issue #101 to get interval settings to work properly.

Then use the following attributes in the audit wrapper cookbook.

default['audit']['profiles']['base/ssh'] = true
default['audit']['interval']['enabled'] = true
default['audit']['interval']['time'] = 2

Then run chef-client twice (one run right after the other) with this wrapper cookbook in the run list.

Expected Result:

Cached profile reports from previous chef-client runs should not get accidentally sent to the server.

Actual Result:

As you can see below the fetch and execute action for the ssh compliance profile was skipped but its cached report was still read, summarized in this output and posted to the server.

  * compliance_profile[ssh] action fetch[2016-09-29T11:13:08+00:00] INFO: Processing compliance_profile[ssh] action fetch (audit::default line 78)
 (skipped due to only_if)
  * compliance_profile[ssh] action execute[2016-09-29T11:13:08+00:00] INFO: Processing compliance_profile[ssh] action execute (audit::default line 78)
 (skipped due to only_if)
  * compliance_report[chef-server] action execute[2016-09-29T11:13:08+00:00] INFO: Processing compliance_report[chef-server] action execute (audit::default line 93)
[2016-09-29T11:13:08+00:00] INFO: Summary for ssh {"duration":0.162846326,"example_count":69,"failure_count":44,"skip_count":0}
[2016-09-29T11:13:08+00:00] INFO: Report to Chef Server: https://chef.lxc/compliance/organizations/demo/inspec

Workaround

Putting the following in my audit wrapper cookbook's default.rb recipe worked for me. It deletes the reports before including the audit cookbook.

compliance_cache_directory = ::File.join(Chef::Config[:file_cache_path], 'compliance')
Dir.glob(File.join(compliance_cache_directory, '*report.json')).each do |f|
  file f do
    action :delete
  end
end

include_recipe 'audit'

[jeremiahsnapp]

This issue is closed by #104