[Security Solution] Users can Customize Prebuilt Detection Rules: Milestone 2

[banderror]

Epic: https://github.com/elastic/security-team/issues/1974 (internal)
Milestones: << • >>

Summary

Milestone 2: Implement basic rule upgrade and installation workflows (flyouts, callout) without adding support for prebuilt rule customization just yet. The UI should be simplified: basic 2-way diffs, no conflict resolution.

This meta ticket is created to simplify tracking of various tickets related to the epic, and to make this public information so our users can track the progress.

Implementation tasks

UI/UX design

Beta Give feedback

1. Design basic rule installation UI
2. Design basic rule upgrade UI
3. Design prebuilt rule details flyout with full rule JSON diff
4. Design prebuilt rule details flyout with per-field text-based diffs
5. Design prebuilt rule details flyout with per-field custom diffs

API for prebuilt rule upgrade and installation workflows

Beta Give feedback

1. [Security Solution] Introduce an endpoint to use for the detection rules table filters/data #137428
   Feature:Rule Management Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp performance technical debt v8.7.0 +7
2. [Security Solution] Add an experimental feature flag for the new prebuilt rule upgrade and installation workflows #148181
   8.8 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp +5
   
3. [Security Solution] Implement prebuilt rule status API endpoint #148182
   8.8 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp +5
   
4. [Security Solution] Implement prebuilt rule upgrade/_review API endpoint #148183
   8.8 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp +5
   
5. [Security Solution] Implement prebuilt rule upgrade/_perform API endpoint #148184
   Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp v8.9.0 +5
6. [Security Solution] Implement prebuilt rule installation/_review API endpoint #148185
   8.8 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp +5
   
7. [Security Solution] Implement prebuilt rule installation/_perform API endpoint #148186
   8.9 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp +5
   
8. [Security Solution] Implement overall rule diff algorithm #148189
   Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp v8.9.0 +5
9. [Security Solution] Prebuilt rule status API endpoint returns incorrect total number of prebuilt rules #152836
   8.8 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp bug impact:high +7
   

UI for prebuilt rule upgrade and installation workflows

Beta Give feedback

1. [Security Solution] Write test plan and add test coverage for the new workflows of installing and upgrading prebuilt rules #148192
   8.11 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp v8.10.0 v8.11.0 v8.9.0 +8
   
2. [Security Solution] Refactor RulesTableContext into separate Installed, Upgrade, & New RulesContexts #154614
   8.9 candidate Feature:Prebuilt Detection Rules Feature:Rule Management Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp refactoring technical debt v8.9.0 +9
   
3. [Security Solution] Create new Add Rules page #154615
   8.9 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp v8.9.0 +6
   

Prebuilt rule details flyout MVP

Beta Give feedback

1. [Security Solution] Implement a rule details flyout for installing and upgrading prebuilt rules #162334
   8.10 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp enhancement v8.10.0 +7
   
2. [Security Solution] Write a test plan for the Overview and Investigation Guide tabs of prebuilt rule flyout #166160
   8.12 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp test-plan v8.11.0 v8.12.0 +8
   
3. [Security Solution] Add test coverage for the Overview and Investigation Guide tabs of prebuilt rule flyout #166161
   8.12 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp test-coverage v8.11.0 v8.12.0 +8
   

Prebuilt rule details flyout with full rule JSON diff

Beta Give feedback

1. [Security Solution] Implement a JSON diff for the prebuilt rule upgrade flyout #169160
   8.12 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp enhancement v8.12.0 +7
   
2. [Security Solution] Tweak JSON diffs highlighting styles to improve readability #173264
   8.12 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp enhancement release_note:skip +7
   
3. [Security Solution] Prebuilt rule flyout: Allow customisation by passing additional components as props #166164
   8.12 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp refactoring +6
   
4. Prebuilt Rule Customization — JSON diff security-docs#4371
   2 of 2
   Docset: ESS Docset: Serverless Effort: Medium Feature: Prebuilt rules Feature: Rules Priority: Medium Team: Detections/Response new-feature v8.12.0 +9
   
5. [Security Solution] Write a test plan for the JSON diff tab of prebuilt rule flyout #166162
   8.13 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp test-plan +6
   
6. [Security Solution] Add test coverage for the JSON diff tab of prebuilt rule flyout #166163
   8.13 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp test-coverage +6
   

Prebuilt rule details flyout with per-field text-based diffs

Beta Give feedback

1. [Security Solution] Create a mock for the Diff tab in rule preview flyout #166159
   8.12 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp +5
   
2. [Security Solution] Implement per-field text-based diffs for the prebuilt rule upgrade flyout #166489
   8.13 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp v8.13.0 v8.14.0 +7
   
3. [Security Solution] Write a test plan for the per-field diff tab of prebuilt rule flyout #176473
   8.13 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp test-plan +6
   
4. Prebuilt rule customization - per-field rule diffs security-docs#4783
   2 of 2
   Docset: ESS Docset: Serverless Feature: Prebuilt rules Feature: Rules Team: Detections/Response v8.13.0 +6
   

Documentation

Beta Give feedback

1. [DOCS] Document the new UI for installing and upgrading prebuilt detection rules security-docs#3496
   Effort: Large Feature: Prebuilt rules Feature: Rules Priority: High Team: Detections/Response Team: Docs v8.9.0 +7
   
2. [DOCS] Document the new revision rule field security-docs#3116
   0 of 4
   Feature: Prebuilt rules Feature: Rules Team: Detections/Response Team: Docs v8.8.0 +5
   
3. Prebuilt Rule Customization — JSON diff security-docs#4371
   2 of 2
   Docset: ESS Docset: Serverless Effort: Medium Feature: Prebuilt rules Feature: Rules Priority: Medium Team: Detections/Response new-feature v8.12.0 +9
   
4. Prebuilt rule customization - per-field rule diffs security-docs#4783
   2 of 2
   Docset: ESS Docset: Serverless Feature: Prebuilt rules Feature: Rules Team: Detections/Response v8.13.0 +6
   

Bugs

Beta Give feedback

1. PUT /api/detection_engine/rules/prepackaged API endpoint error #133983
   Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp bug impact:medium +6
2. [Security Solution] Side effects of no-content version bumps in prebuilt rules #130576
   8.9 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp bug impact:medium +7
   
3. [Security Solution] Add Elastic Rules page is empty after rule assets are installed in background #160396
   8.9 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp bug impact:high v8.9.0 +8
   
4. [Security Solution] Hide Rule Updates tab when no updates are available #161195
   8.9 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp bug impact:high v8.9.0 +8
   
5. [Security Solution] Rule Updates table has broken and users can't upgrade prebuilt rules #161094
   8.9 candidate Feature:Prebuilt Detection Rules QA:Validated Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp bug fixed impact:critical v8.9.0 +10
   

   
6. [Security Solution] Prebuilt rule is duplicated during upgrade when its new version has a different rule type #161305
   8.9 candidate Feature:Prebuilt Detection Rules QA:Validated Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp blocker bug fixed impact:critical v8.9.0 +11
   
7. [Security Solution] User with Security: Read privileges can install and upgrade prebuilt rules via UI #161443
   8.9 candidate Feature:Prebuilt Detection Rules QA:Validated Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp blocker bug fixed impact:critical v8.9.0 +11
   
8. [Security Solution] JSON diffs for prebuilt rules: Technical / runtime properties are displayed #174844
   8.13 candidate Feature:Prebuilt Detection Rules Team: SecuritySolution Team:Detection Rule Management Team:Detections and Resp bug impact:medium v8.12.1 +8
   

TRADE team dependencies

Beta Give feedback

1. [FR] Update Security Docs Release Process to Add Index Rules Page Per Stack detection-rules#2867
   7 of 7
   documentation enhancement v8.9.0 +3
   

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[banderror]

Milestone 2 has been reached:

• We have reworked prebuilt rule installation and upgrade workflows. We have a new UI for them that allows users to selectively install and upgrade prebuilt rules.
• Users can see the details of a rule being installed.
• Users can see the details of a rule being upgraded:
  • values of all the rule's parameters from the latest version from Elastic
  • a comparison between the currently installed version of the rule and the latest version from Elastic in two forms: a full side-by-side JSON diff and a per-field comparison.

The team is already working on the next milestone: #174168.