Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Alerting][Docs] Adds Alerting & Task Manager Scalability Guidance & Health Monitoring #91171

Merged
merged 113 commits into from
Mar 4, 2021
Merged

[Alerting][Docs] Adds Alerting & Task Manager Scalability Guidance & Health Monitoring #91171

merged 113 commits into from
Mar 4, 2021

Conversation

gmmorris
Copy link
Contributor

@gmmorris gmmorris commented Feb 11, 2021 •
edited
Loading

Summary

Closes #89997
Closes #89120

Documentation for scaling Kibana alerting, what configurations can change, what impacts they have, etc.
Scaling Alerting relies heavily on scaling Task Manager, so these docs also document Task manager Health Monitoring and scaling.

These docs are designed like an onion - the user can unpeel layer after layer, revealing deeper details as needed.

For example, we'd expect the user to follow a flow such as:

Alerts and Actions => Alerting Production Considerations => Task Manager Production Considerations
Alerts and Actions => Alerting Scaling => Task Manager Scaling
Alerts and Actions => Alerting Troubleshooting => Task Manager Troubleshooting
Production Considerations => Task Manager Production Considerations => Task Manager Scaling

This breakdown is designed to spare the user the need of understanding the underlying implementations until they need to... but ensures that level of details is available if they need it.

After working with @gchaps on the location of specific docs we decided to group all Production Considerations together.
We now have Kibana in general, Alerting and TM production considerations all together under "Production Considerations" which appears right after "Set up".

Preview generated at:
https://kibana_91171.docs-preview.app.elstc.co/guide/en/kibana/master/alerting-production-considerations.html

Checklist

Delete any items that are not applicable to this PR.

  • Documentation was added for features that require explanation or tutorials

For maintainers

@gmmorris gmmorris changed the title [Alerting][Docs] scalability guidance [Alerting][Docs] Task Manager Scalability Guidance & Health Monitoring Feb 16, 2021
@gmmorris
added tm long running task diagnosis
ab5379c
@gmmorris
fixed headers
ecd2f33
@gmmorris
Merge branch 'master' into task-manager/docs-monitoring
61d7256 
* master: (157 commits)
  [DOCS] Adds machine learning to the security section of alerting (elastic#91501)
  [Uptime] Ping list step screenshot caption formatting (elastic#91403)
  [Vislib] Use timestamp on brush event instead of iso dates (elastic#91483)
  [Application Usage] Remove deprecated & unused legacy.appChanged API (elastic#91464)
  Migrate logstash, monitoring, url_drilldowns, xpack_legacy to ts projects (elastic#91194)
  [APM] Wrap Elasticsearch client errors (elastic#91125)
  [APM] Fix optimize-tsconfig script (elastic#91487)
  [Discover][docs] Add searchFieldsFromSource description (elastic#90980)
  Adds support for 'ip' data type (elastic#85087)
  [Detection Rules] Add updates from 7.11.2 rules (elastic#91553)
  [SECURITY SOLUTION] Eql in timeline (elastic#90816)
  [APM] Correlations Beta (elastic#86477) (elastic#89952)
  [Security Solutions][Detection Engine] Adds a warning banner when the alerts data has not been migrated yet. (elastic#90258)
  [Security Solution] [Timeline] Endpoint row renderers (2nd batch) (elastic#91446)
  skip flaky suite (elastic#91450)
  skip flaky suite (elastic#91592)
  [Security Solution][Endpoint][Admin] Endpoint Details UX Enhancements (elastic#90870)
  [ML] Add better UI support for runtime fields Transforms  (elastic#90363)
  [Security Solution] [Detections] Replace 'partial failure' with 'warning' for rule statuses (elastic#91167)
  [Security Solution][Detections] Adds Indicator path config for indicator match rules (elastic#91260)
  ...
@gmmorris
fixed merge conflict
c6549c8
@gmmorris
split tm docs
3d92701
@gmmorris
added workload
cc2f8be
@gmmorris
move TM into own section
da1834c
@gmmorris
formating
a3d577e
@gmmorris
remvoetm from alerting
239384b
@gmmorris
added tm section
ceca142
@gmmorris
tm getting started page
2bc26aa
@gmmorris
tm prod cons
5c94c32
@gmmorris
scaling
2f487dc
@gmmorris
tweaked tm
385d305
@gmmorris
added scaling
78cec2f
@gmmorris
typo
b243248
@gmmorris
tweaked alerting troiubleshooting
2d03471
@gmmorris
fixed grammer and typos
2113afe
@gmmorris gmmorris mentioned this pull request Feb 19, 2021
Closed
4 tasks
@gmmorris gmmorris added Feature:Alerting Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Team:Docs v7.11.2 labels Feb 19, 2021
gmmorris and others added 11 commits March 3, 2021 10:29
@gmmorris @gchaps
Update docs/user/production-considerations/task-manager-troubleshooti…
cc3d776 
…ng.asciidoc

Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
@gmmorris @gchaps
Update docs/user/production-considerations/task-manager-troubleshooti…
6a169fc 
…ng.asciidoc

Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
@gmmorris
applied PR review comments
467ba46
@gmmorris
Merge branch 'task-manager/docs-monitoring' of github.com:gmmorris/ki…
a324f66 
…bana into task-manager/docs-monitoring

* 'task-manager/docs-monitoring' of github.com:gmmorris/kibana:
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Apply suggestions from code review
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
  Update docs/user/production-considerations/task-manager-troubleshooting.asciidoc
@gmmorris @gchaps
Apply suggestions from code review
4ea2c9e 
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
@gmmorris
applied PR review formatting comments
3ea0a86
@gmmorris
Merge branch 'task-manager/docs-monitoring' of github.com:gmmorris/ki…
deb8160 
…bana into task-manager/docs-monitoring

* 'task-manager/docs-monitoring' of github.com:gmmorris/kibana:
  Apply suggestions from code review
@gmmorris
Merge branch 'master' into task-manager/docs-monitoring
b0c15d5 
* master: (45 commits)
  Add outcome of node scripts/build_api_docs (elastic#93399)
  [Lens] fix long field name on field stats panel doesn't wrap (elastic#93279)
  [Bug] Fix filter creation for numeric scripted fields in Discover (elastic#93224)
  [uptime] Fix anomaly alert edit (elastic#93025)
  Consolidate @babel/* packages and use latest compatible version (elastic#93264)
  [Search Embeddable] Add highlighting when searching (elastic#93178)
  [APM] Add missing bottom border to header (elastic#93179)
  [CI] No longer collect APM span stack traces (elastic#93263)
  [XY Chart] Fix "No data to display" error when using IP range aggregation to split series (elastic#93024)
  update generated public api docs
  API DOCS Step 3/3 (elastic#92929)
  chore(NA): look for bazel packages on npm_module folder during distributable build (elastic#93262)
  rename advanced setting ml:fileDataVisualizerMaxFileSize to fileUpload:maxFileSize and increase max geojson upload size to 1GB (elastic#92620)
  [kbn/optimizer] allow customizing the limits path from the script (elastic#93153)
  [Alerting][Docs] Adding template for documenting alert and action types (elastic#92830)
  [jenkins] convert baseline capture job to use tasks (elastic#93288)
  removing the linked issue in comments from PR (elastic#93303)
  chore(NA): do not include fs within a storybook build (elastic#93294)
  [Maps] Update Map extent queries to use bounding box logic for both point and shape queries (elastic#93156)
  Add searchDuration to EQL and Threshold rules (elastic#93149)
  ...
@gmmorris
boken down to bulltets
f96cebe
@gmmorris
change question to statement
92fe188
@gmmorris
indent troubleshooting
bde118c
gchaps
gchaps reviewed Mar 3, 2021
View reviewed changes
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
docs/user/production-considerations/task-manager-troubleshooting.asciidoc Outdated Show resolved Hide resolved
gmmorris and others added 4 commits March 4, 2021 11:48
@gmmorris @gchaps
Apply suggestions from code review
dc7ce41 
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
@gmmorris
fixed space
f1b92ef
@gmmorris
:wq!rge branch 'task-manager/docs-monitoring' of github.com:gmmorris/…
8fe9986 
…kibana into task-manager/docs-monitoring

* 'task-manager/docs-monitoring' of github.com:gmmorris/kibana:
  Apply suggestions from code review
@gmmorris
Merge branch 'master' into task-manager/docs-monitoring
6a3b8b3 
* master: (48 commits)
  Fix wrong import in data plugin causing 100kB bundle increase (elastic#93448)
  [Fleet] Correctly track install status of an integration (elastic#93464)
  Reviews data frame analytics UI text (elastic#93033)
  Display multiple copyable fields for process.args in resolver node detail panel (elastic#93280)
  [Security Solution][Detections] ML Popover overflow fix (elastic#93525)
  chore(NA): do not use execa on bazel workspace status update script (elastic#93532)
  Bump dependencies (elastic#93511)
  [dev/build_ts_refs] support disabling the ts-refs build completely (elastic#93529)
  [Security Solution] fix data provider cypress test (elastic#93465)
  Fix service map for All environment single service (elastic#93517)
  [Fleet] Fix package version comparaison in the UI (elastic#93498)
  [alerting] adds doc on JSON-expanded action variables and task manager max_workers (elastic#92720)
  [dev/build_ts_refs] ignore type checking failures when building ts refs (elastic#93473)
  [core-new-docs] Adds a dev-doc for core documentation (elastic#92976)
  remove opacity from maps legacy style (elastic#93456)
  [Security Solution][Lists] Escape quotes in list ids and quote the id in KQL query (elastic#93176)
  Revert "Make tests deterministic by providing unique timestamps (elastic#93350)"
  [Discover] Fix link from dashboard saved search to Discover (elastic#92937)
  update public api docs
  App Search - Polishing Analytics Views (elastic#92939)
  ...
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@gmmorris gmmorris merged commit 79134b3 into elastic:master Mar 4, 2021
gmmorris added a commit to gmmorris/kibana that referenced this pull request Mar 4, 2021
@gmmorris
[Alerting][Docs] Adds Alerting & Task Manager Scalability Guidance & …
b64d0ff 
…Health Monitoring (elastic#91171)

Documentation for scaling Kibana alerting, what configurations can change, what impacts they have, etc.
Scaling Alerting relies heavily on scaling Task Manager, so these docs also document Task manager Health Monitoring and scaling.
@gmmorris gmmorris mentioned this pull request Mar 4, 2021
Merged
@gmmorris gmmorris removed the v7.11.2 label Mar 4, 2021
@gmmorris gmmorris mentioned this pull request Mar 4, 2021
Merged
gmmorris added a commit to gmmorris/kibana that referenced this pull request Mar 4, 2021
@gmmorris
[Alerting][Docs] Adds Alerting & Task Manager Scalability Guidance & …
404532b 
…Health Monitoring (elastic#91171)

Documentation for scaling Kibana alerting, what configurations can change, what impacts they have, etc.
Scaling Alerting relies heavily on scaling Task Manager, so these docs also document Task manager Health Monitoring and scaling.
gmmorris added a commit that referenced this pull request Mar 4, 2021
@gmmorris @kibanamachine
[Alerting][Docs] Adds Alerting & Task Manager Scalability Guidance & …
ba917ae 
…Health Monitoring (#91171) (#93603)

Documentation for scaling Kibana alerting, what configurations can change, what impacts they have, etc.
Scaling Alerting relies heavily on scaling Task Manager, so these docs also document Task manager Health Monitoring and scaling.

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
gmmorris added a commit that referenced this pull request Mar 4, 2021
@gmmorris @kibanamachine
[Alerting][Docs] Adds Alerting & Task Manager Scalability Guidance & …
1d032ef 
…Health Monitoring (#91171) (#93604)

Documentation for scaling Kibana alerting, what configurations can change, what impacts they have, etc.
Scaling Alerting relies heavily on scaling Task Manager, so these docs also document Task manager Health Monitoring and scaling.

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@mikecote
Copy link
Contributor

@gmmorris, should this be backported to 7.11 as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gchaps gchaps gchaps approved these changes

@YulNaumenko YulNaumenko YulNaumenko approved these changes

@ymao1 ymao1 ymao1 approved these changes

Assignees
No one assigned
Labels
Feature:Alerting Feature:Task Manager release_note:skip Skip the PR/issue when compiling release notes Team:Docs Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.12.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Alerting GA - Task manager documentation Kibana alerting scalability guidance
7 participants
@gmmorris @elasticmachine @kibanamachine @mikecote @ymao1 @gchaps @YulNaumenko