[Reporting] Ensure reports:monitor
task is never "unrecognized"
#118477
Labels
bug
Fixes for quality problems that affect the customer experience
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
loe:small
Small Level of Effort
Kibana version: 7.14+
When the reporting queue system was replaced with ESQueue to use Task Manager, a
reports:monitor
task was created as a way to add self-healing when Kibana crashes or restarts in the process of executing a report.If this happens, the status of the report would stay in "processing" indefinitely if it weren't for the
report:execute
task which looks for these kinds of stuck jobs.Problem: If thereport:execute
task returns an error, Task Manager reschedules it with a time buffer. These buffers can become very large. When this happens, Reporting is not able to self-heal by rescheduling stuck jobs.Problem: the
reports:monitor
task is vulnerable to Kibana instances joining the cluster with the Reporting plugin disabled.Having a mismatch of plugins enabled in different Kibana instances is not supported: it causes problems especially for plugins that are responsible for server background tasks.
When this scenario happens, the instance with Reporting disabled will not recognize the
reports:monitor
task. Internal logic in Task Manager updates the status of the task tounrecognized
, which dictates that this task should stop running.Workaround
Make sure that in all of the instances, the same plugins are enabled - Reporting should be enabled on every instance. If you need to ensure that only one instance is able to claim jobs, set
xpack.reporting.queue.pollEnabled: false
in the configuration of all instances except that one. You can disable the Reporting UI features on a per-user and per-space basis by using application feature controls. See: https://www.elastic.co/guide/en/kibana/current/secure-reporting.html#grant-user-accessRequirements:
xpack.reporting.enabled: false
with minimal custom logic to register thereports:monitor
task.reports:monitor
task.The text was updated successfully, but these errors were encountered: