Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.16] Updates docs for and related to the excludedDataTiersForRuleExecution advanced setting #5962

Merged
merged 38 commits into from
Nov 14, 2024

Conversation

nastasha-solomon
Copy link
Contributor

@nastasha-solomon nastasha-solomon commented Oct 19, 2024

Fixes #5925 and https://github.com/elastic/security-docs-internal/issues/47 by updating the explanation for filtering out cold and frozen documents during rule executions and adding the disclaimer about certain rule types and cold/frozen shards.

Previews:

@nastasha-solomon nastasha-solomon added Feature: Rules Team: Detection Engine Priority: High Issues that are time-sensitive and/or are of high customer importance Effort: Small Issues that can be resolved quickly v8.16.0 v8.17.0 labels Oct 19, 2024
@nastasha-solomon nastasha-solomon self-assigned this Oct 19, 2024
Copy link

A documentation preview will be available soon.

Request a new doc build by commenting
  • Rebuild this PR: run docs-build
  • Rebuild this PR and all Elastic docs: run docs-build rebuild

run docs-build is much faster than run docs-build rebuild. A rebuild should only be needed in rare situations.

If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here.

@nastasha-solomon nastasha-solomon changed the title [8.16] Updates docs about excluding cold and frozen data from rule executions [8.16] Updates docs for and related to the excludedDataTiersForRuleExecution advanced setting Nov 12, 2024
@nastasha-solomon nastasha-solomon marked this pull request as ready for review November 12, 2024 19:06
@nastasha-solomon nastasha-solomon requested a review from a team as a code owner November 12, 2024 19:06
Copy link
Contributor

@marshallmain marshallmain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the phrase The best path forward continues to be modifying the index patterns to only use hot tier data., are we specifically intending to reference guidance that we've provided before? An alternative might be The best path forward is to modify the index patterns to only use hot tier data..

Also @yctercero do we have a specific modification we can provide that works across the board, e.g. -partial* to exclude frozen indices? Do we know if that would work everywhere? Users might read "just modify your index patterns" and think that's a monumental task unless we have an easy specific change they can make.

docs/detections/detection-engine-intro.asciidoc Outdated Show resolved Hide resolved
docs/detections/detection-engine-intro.asciidoc Outdated Show resolved Hide resolved
docs/getting-started/advanced-setting.asciidoc Outdated Show resolved Hide resolved
docs/release-notes/8.16.asciidoc Outdated Show resolved Hide resolved
docs/whats-new.asciidoc Outdated Show resolved Hide resolved
docs/release-notes/8.16.asciidoc Outdated Show resolved Hide resolved
docs/getting-started/advanced-setting.asciidoc Outdated Show resolved Hide resolved
docs/detections/detection-engine-intro.asciidoc Outdated Show resolved Hide resolved
nastasha-solomon and others added 4 commits November 14, 2024 09:28
Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Copy link
Contributor

@benironside benironside left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few minor comments which may or may not be helpful! Lmk when you're ready for an approval

nastasha-solomon and others added 5 commits November 14, 2024 13:07
…idoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
…idoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
…idoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
@nastasha-solomon nastasha-solomon added Effort: Large Issues that require significant planning, research, writing, and testing and removed Effort: Small Issues that can be resolved quickly labels Nov 14, 2024
@nastasha-solomon nastasha-solomon merged commit cd4f12b into main Nov 14, 2024
3 checks passed
mergify bot pushed a commit that referenced this pull request Nov 14, 2024
…xecution` advanced setting (#5962)

* First draft

* Updating IM rules

* disclaimer about certain rule types and shards

* Minor tweak to dsl query docs

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Marshall's suggestion

* Update docs/detections/detection-engine-intro.asciidoc

* Removes note that's no longer needed

* Moves file back to remove this change from the PR

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Updates what's new

* Fixed title

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/release-notes/8.16.asciidoc

* Fixes a typo

* Minor wording adjustments

* Update docs/whats-new.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>

---------

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
(cherry picked from commit cd4f12b)
mergify bot pushed a commit that referenced this pull request Nov 14, 2024
…xecution` advanced setting (#5962)

* First draft

* Updating IM rules

* disclaimer about certain rule types and shards

* Minor tweak to dsl query docs

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Marshall's suggestion

* Update docs/detections/detection-engine-intro.asciidoc

* Removes note that's no longer needed

* Moves file back to remove this change from the PR

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Updates what's new

* Fixed title

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/release-notes/8.16.asciidoc

* Fixes a typo

* Minor wording adjustments

* Update docs/whats-new.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>

---------

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
(cherry picked from commit cd4f12b)
nastasha-solomon added a commit that referenced this pull request Nov 14, 2024
…orRuleExecution` advanced setting (backport #5962) (#6174)

* First draft

* Updating IM rules

* disclaimer about certain rule types and shards

* Minor tweak to dsl query docs

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Marshall's suggestion

* Update docs/detections/detection-engine-intro.asciidoc

* Removes note that's no longer needed

* Moves file back to remove this change from the PR

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Updates what's new

* Fixed title

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/release-notes/8.16.asciidoc

* Fixes a typo

* Minor wording adjustments

* Update docs/whats-new.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>

---------

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
(cherry picked from commit cd4f12b)

Co-authored-by: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com>
nastasha-solomon added a commit that referenced this pull request Nov 14, 2024
…rRuleExecution` advanced setting (backport #5962) (#6173)

* First draft

* Updating IM rules

* disclaimer about certain rule types and shards

* Minor tweak to dsl query docs

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Marshall's suggestion

* Update docs/detections/detection-engine-intro.asciidoc

* Removes note that's no longer needed

* Moves file back to remove this change from the PR

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Updates what's new

* Fixed title

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/release-notes/8.16.asciidoc

* Fixes a typo

* Minor wording adjustments

* Update docs/whats-new.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>

---------

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
(cherry picked from commit cd4f12b)

Co-authored-by: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com>
acorretti pushed a commit that referenced this pull request Nov 19, 2024
…orRuleExecution` advanced setting (backport #5962) (#6174)

* First draft

* Updating IM rules

* disclaimer about certain rule types and shards

* Minor tweak to dsl query docs

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Marshall's suggestion

* Update docs/detections/detection-engine-intro.asciidoc

* Removes note that's no longer needed

* Moves file back to remove this change from the PR

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Updates what's new

* Fixed title

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/whats-new.asciidoc

* Update docs/release-notes/8.16.asciidoc

* Fixes a typo

* Minor wording adjustments

* Update docs/whats-new.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/release-notes/8.16.asciidoc

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>

* Update docs/getting-started/advanced-setting.asciidoc

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detections-exclude-cold-frozen-data-tiers.asciidoc

Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>

* Update docs/detections/detection-engine-intro.asciidoc

Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>

---------

Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com>
Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
(cherry picked from commit e6d6ec9)

Co-authored-by: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Effort: Large Issues that require significant planning, research, writing, and testing Feature: Rules Priority: High Issues that are time-sensitive and/or are of high customer importance Team: Detection Engine v8.16.0 v8.17.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Suggestion][Detection Engine] Cold/frozen filter
6 participants