Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] Implement array of scalar values diff algorithm #180162

Closed
10 tasks done
Tracked by #174168
jpdjere opened this issue Apr 5, 2024 · 5 comments
Closed
10 tasks done
Tracked by #174168

[Security Solution] Implement array of scalar values diff algorithm #180162

jpdjere opened this issue Apr 5, 2024 · 5 comments
Assignees
@dplumlee
Labels
8.16 candidate enhancement New value added to drive a business result Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.16.0

Comments

@jpdjere
Copy link
Contributor

jpdjere commented Apr 5, 2024 •
edited by dplumlee
Loading

Epics: https://github.com/elastic/security-team/issues/1974 (internal), #174168

Summary

Implement an algorithm for diffing and merging changes in array of scalar values type of fields of detection rules.

Context from the Rule Customization RFC:

To do
@jpdjere jpdjere added triage_needed Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area labels Apr 5, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@jpdjere jpdjere mentioned this issue Apr 5, 2024
Open
@banderror banderror changed the title [Security Solution] Implement array of scalar value fields diff algorithm [Security Solution] Implement array of scalar value fields diff algorithm (DRAFT) Apr 17, 2024
@banderror banderror mentioned this issue May 24, 2024
Open
5 tasks
@banderror banderror changed the title [Security Solution] Implement array of scalar value fields diff algorithm (DRAFT) [Security Solution] Implement array of scalar values diff algorithm May 24, 2024
@dplumlee
Copy link
Contributor

dplumlee commented Jun 6, 2024 •
edited
Loading

Here are the proposed fields that would utilize this diff algorithm:

Common fields

  • tags
  • references
  • author

Threat match fields

  • threat_index

New terms fields

  • new_terms_fields

This was referenced Jun 17, 2024
Merged
Merged
banderror pushed a commit that referenced this issue Jul 2, 2024
@dplumlee @elasticmachine
[Security Solution] Adds diff algorithm and unit tests for scalar arr…
73b695d 
…ay values (#186323)

## Summary

Related ticket: #180162

Adds the diff algorithm for arrays of scalar values (right now we only
have fields of strings) and unit tests to cover all the base cases of
what the algorithm can return.


### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@banderror banderror added the enhancement New value added to drive a business result label Jul 5, 2024
@dplumlee dplumlee mentioned this issue Jul 8, 2024
Merged
3 tasks
dplumlee added a commit that referenced this issue Jul 9, 2024
@dplumlee
[Security Solution] Array of scalar values diff algorithm test plan (#…
0b405a0 
…186325)

## Summary

Related ticket: #180162

Adds test plan for diff algorithm for arrays of scalar values
implemented here: #186323

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jul 9, 2024
@dplumlee
[Security Solution] Array of scalar values diff algorithm test plan (e…
1d32085 
…lastic#186325)

## Summary

Related ticket: elastic#180162

Adds test plan for diff algorithm for arrays of scalar values
implemented here: elastic#186323

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 0b405a0)
kibanamachine referenced this issue Jul 10, 2024
@kibanamachine @dplumlee
[8.15] [Security Solution] Array of scalar values diff algorithm test…
a98ac32 
… plan (#186325) (#187873)

# Backport

This will backport the following commits from `main` to `8.15`:
- [[Security Solution] Array of scalar values diff algorithm test plan
(#186325)](#186325)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Davis
Plumlee","email":"56367316+dplumlee@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-07-09T14:33:04Z","message":"[Security
Solution] Array of scalar values diff algorithm test plan
(#186325)\n\n## Summary\r\n\r\nRelated ticket:
https://github.com/elastic/kibana/issues/180162\r\n\r\nAdds test plan
for diff algorithm for arrays of scalar values\r\nimplemented here:
https://github.com/elastic/kibana/pull/186323\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"0b405a0d656e496775a2c6efa1392ec3ed3fdd4f","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","v8.15.0","v8.16.0"],"title":"[Security Solution] Array of scalar
values diff algorithm test
plan","number":186325,"url":"https://github.com/elastic/kibana/pull/186325","mergeCommit":{"message":"[Security
Solution] Array of scalar values diff algorithm test plan
(#186325)\n\n## Summary\r\n\r\nRelated ticket:
https://github.com/elastic/kibana/issues/180162\r\n\r\nAdds test plan
for diff algorithm for arrays of scalar values\r\nimplemented here:
https://github.com/elastic/kibana/pull/186323\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"0b405a0d656e496775a2c6efa1392ec3ed3fdd4f"}},"sourceBranch":"main","suggestedTargetBranches":["8.15"],"targetPullRequestStates":[{"branch":"8.15","label":"v8.15.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/186325","number":186325,"mergeCommit":{"message":"[Security
Solution] Array of scalar values diff algorithm test plan
(#186325)\n\n## Summary\r\n\r\nRelated ticket:
https://github.com/elastic/kibana/issues/180162\r\n\r\nAdds test plan
for diff algorithm for arrays of scalar values\r\nimplemented here:
https://github.com/elastic/kibana/pull/186323\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"0b405a0d656e496775a2c6efa1392ec3ed3fdd4f"}}]}]
BACKPORT-->

Co-authored-by: Davis Plumlee <56367316+dplumlee@users.noreply.github.com>
dplumlee added a commit that referenced this issue Jul 11, 2024
@dplumlee
[Security Solution] Integration tests for scalar array diff algorithm (…
76cc844 
…#187778)

## Summary

Completes #180162

Switches fields to use the scalar array diff algorithms assigned to them
in the [overarching
ticket](#180162 (comment))

Adds integration tests in accordance to
#186325 for the upgrade/_review
API endpoint for the scalar array diff algorithm.

### Checklist

Delete any items that are not applicable to this PR.


- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed



### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
@dplumlee
Copy link
Contributor

Resolved by:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dplumlee dplumlee

Labels
8.16 candidate enhancement New value added to drive a business result Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.16.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jpdjere @banderror @elasticmachine @dplumlee