-
Notifications
You must be signed in to change notification settings - Fork 737
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: store constraint status audit results in sorted order #3293
Conversation
8982b1f
to
5938845
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3293 +/- ##
=========================================
Coverage 54.49% 54.50%
=========================================
Files 134 134
Lines 12329 10294 -2035
=========================================
- Hits 6719 5611 -1108
+ Misses 5116 4174 -942
- Partials 494 509 +15
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
1. Implement LimitQueue, a max Priority Queue on StatusViolations which only hold items up to a certain limit L. Because the size is bounded by a constant limit, LimitQueue operations provide O(1) time complexity guarantees for all operations. The data structure only uses constant space so will not affect the audit manager's RAM usage. 2. Unit test LimitQueue and SVQueue classes to verify functionality. 3. Use LimitQueue as the value in the updateLists map instead of []updateListEntry so we can store audit results in sorted order up to constraintViolationsLimit. 4. Remove the intermediate type updateListEntry in favor of saving the StatusViolation directly. The only difference between the two is that updateListEntry used util.EnforcementAction while StatusViolation uses string. Update logViolation to log a string enforcement action. Keep the same logic to track totalViolationsPerEnforcementAction. 5. Move some variable definitions closer to where they will be used, for example details, labels, uid, and rv. Signed-off-by: Prachi Pendse <prachirp@google.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice solution!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure exactly why audit results have stopped updating, but added some possibilities
running |
…sults in sorted order Signed-off-by: Prachi Pendse <prachirp@google.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@JaydipGabani Please take a look, thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, after resolving the pending comments from Rita
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the reviews! I realize I don't have time to benchmark the performance differences between the old and new behaviors so I hope we can submit this change as-is.
Signed-off-by: Prachi Pendse <prachirp@google.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…cy-agent#3293) Signed-off-by: Prachi Pendse <prachirp@google.com>
What this PR does / why we need it:
As stated in issue #3261, the goal is to be able to compare violations from the status of the same constraint A at different times, for example before and after cluster restart.
So originally the constraint status could return a random subset of 20 out of 40 findings in an unsorted fashion (Z,Y,X and dropping A,B,C) at time 0 and another random subset of 20 out of 40 findings still in an unsorted fashion (Y,X,A and dropping B,C,Z) at time 1. The first PR updated to return a random subset of 20 out of 40 findings in a sorted fashion (X,Y,Z and dropping A,B,C) at time 0 and another random subset of 20 out of 40 findings in a sorted fashion (A,X,Y and dropping B,C,Z). This is definitely helpful to make the diffs less noisy.
In order to have a valid comparison, we want to get the first 20 out of 40 findings each time and return them in a sorted fashion. So ideally return the subset (A,B,C and dropping X,Y,Z) at time 0 and return the subset (A,B,C and dropping X,Y,Z) at time 1. This way we can make valid comparisons about the status of the same constraint over time and this will also make diffs much less noisy.
This PR meets this goal by using a LimitQueue data structure.
Which issue(s) this PR fixes (optional, using
fixes #<issue number>(, fixes #<issue_number>, ...)
format, will close the issue(s) when the PR gets merged):Fixes #3261
Special notes for your reviewer: