Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: store constraint status audit results in sorted order #3293

Merged
merged 7 commits into from
Mar 20, 2024

Conversation

prachirp
Copy link
Contributor

  1. Implement LimitQueue, a max Priority Queue on StatusViolations which only hold items up to a certain limit L. Because the size is bounded by a constant limit, LimitQueue operations provide O(1) time complexity guarantees for all operations. The data structure only uses constant space so will not affect the audit manager's RAM usage.
  2. Unit test LimitQueue and SVQueue classes to verify functionality.
  3. Use LimitQueue as the value in the updateLists map instead of []updateListEntry so we can store audit results in sorted order up to constraintViolationsLimit.
  4. Remove the intermediate type updateListEntry in favor of saving the StatusViolation directly. The only difference between the two is that updateListEntry used util.EnforcementAction while StatusViolation uses string. Update logViolation to log a string enforcement action. Keep the same logic to track totalViolationsPerEnforcementAction.
  5. Move some variable definitions closer to where they will be used, for example details, labels, uid, and rv.

What this PR does / why we need it:

As stated in issue #3261, the goal is to be able to compare violations from the status of the same constraint A at different times, for example before and after cluster restart.

So originally the constraint status could return a random subset of 20 out of 40 findings in an unsorted fashion (Z,Y,X and dropping A,B,C) at time 0 and another random subset of 20 out of 40 findings still in an unsorted fashion (Y,X,A and dropping B,C,Z) at time 1. The first PR updated to return a random subset of 20 out of 40 findings in a sorted fashion (X,Y,Z and dropping A,B,C) at time 0 and another random subset of 20 out of 40 findings in a sorted fashion (A,X,Y and dropping B,C,Z). This is definitely helpful to make the diffs less noisy.

In order to have a valid comparison, we want to get the first 20 out of 40 findings each time and return them in a sorted fashion. So ideally return the subset (A,B,C and dropping X,Y,Z) at time 0 and return the subset (A,B,C and dropping X,Y,Z) at time 1. This way we can make valid comparisons about the status of the same constraint over time and this will also make diffs much less noisy.

This PR meets this goal by using a LimitQueue data structure.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #3261

Special notes for your reviewer:

@prachirp prachirp requested a review from a team as a code owner February 29, 2024 00:56
@prachirp prachirp changed the title Store constraint status audit results in sorted order. fix: store constraint status audit results in sorted order Feb 29, 2024
@prachirp prachirp force-pushed the master branch 3 times, most recently from 8982b1f to 5938845 Compare February 29, 2024 01:35
@codecov-commenter
Copy link

codecov-commenter commented Feb 29, 2024

Codecov Report

Attention: Patch coverage is 43.47826% with 52 lines in your changes are missing coverage. Please review.

Project coverage is 54.50%. Comparing base (3350319) to head (51ee38a).
Report is 38 commits behind head on master.

Files Patch % Lines
pkg/audit/manager.go 43.47% 49 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##           master    #3293     +/-   ##
=========================================
  Coverage   54.49%   54.50%             
=========================================
  Files         134      134             
  Lines       12329    10294   -2035     
=========================================
- Hits         6719     5611   -1108     
+ Misses       5116     4174    -942     
- Partials      494      509     +15     
Flag Coverage Δ
unittests 54.50% <43.47%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

1. Implement LimitQueue, a max Priority Queue on StatusViolations which only hold items up to a certain limit L. Because the size is bounded by a constant limit, LimitQueue operations provide O(1) time complexity guarantees for all operations. The data structure only uses constant space so will not affect the audit manager's RAM usage.
2. Unit test LimitQueue and SVQueue classes to verify functionality.
3. Use LimitQueue as the value in the updateLists map instead of []updateListEntry so we can store audit results in sorted order up to constraintViolationsLimit.
4. Remove the intermediate type updateListEntry in favor of saving the StatusViolation directly. The only difference between the two is that updateListEntry used util.EnforcementAction while StatusViolation uses string. Update logViolation to log a string enforcement action. Keep the same logic to track totalViolationsPerEnforcementAction.
5. Move some variable definitions closer to where they will be used, for example details, labels, uid, and rv.

Signed-off-by: Prachi Pendse <prachirp@google.com>
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice solution!

pkg/audit/manager.go Outdated Show resolved Hide resolved
pkg/audit/manager.go Show resolved Hide resolved
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure exactly why audit results have stopped updating, but added some possibilities

pkg/audit/manager.go Outdated Show resolved Hide resolved
pkg/audit/manager.go Outdated Show resolved Hide resolved
pkg/audit/manager.go Outdated Show resolved Hide resolved
pkg/audit/manager.go Show resolved Hide resolved
@maxsmythe
Copy link
Contributor

running make lint locally may also help suss out the reason. Not sure why results aren't being written to GitHub

Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@prachirp
Copy link
Contributor Author

prachirp commented Mar 3, 2024

@JaydipGabani Please take a look, thanks!

Copy link
Contributor

@JaydipGabani JaydipGabani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, after resolving the pending comments from Rita

Copy link
Contributor Author

@prachirp prachirp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reviews! I realize I don't have time to benchmark the performance differences between the old and new behaviors so I hope we can submit this change as-is.

pkg/audit/manager.go Show resolved Hide resolved
pkg/audit/manager.go Show resolved Hide resolved
Signed-off-by: Prachi Pendse <prachirp@google.com>
Copy link
Contributor

@JaydipGabani JaydipGabani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@ritazh ritazh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ritazh ritazh merged commit 7584f4a into open-policy-agent:master Mar 20, 2024
16 checks passed
leewoobin789 pushed a commit to softlee-io/gatekeeper that referenced this pull request Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Constraint status content is non-deterministic
5 participants