[ML] Force time shift to adjust for daylight saving event #2690
Assignees
Labels
Comments
valeriy42
changed the title
valeriy42
added a commit
to elastic/elasticsearch
that referenced
this issue
Jul 25, 2024
This PR extends the schema of the detection rule API by adding a new parametrizable action `force_time_shift` and a new property `params` to hold the parameter `total_shift_amount` as signed long in seconds to control the behavior of the time shift.
A valid schema looks like this:
```http
POST _ml/anomaly_detectors/my_job/_update
{
"detectors": {
"detector_index": 0,
"custom_rules": [
// update the detector with a custom rule that forces a time shift of 1 hour back starting now
{
"actions": [
"force_time_shift"
],
"params": {
"force_time_shift": {
"time_shift_amount": 3600}
}
},
"conditions": [{
"applies_to": "time",
"operator": "gt",
"value": "now"
},
{
"applies_to": "time",
"operator": "lt",
"value": "now+bucket_span"
},
]
},
...
```
Execution of the detection rule action `force_time_shift` will shift the time inside the anomaly detector by a specified amount. This is useful, e.g. to quickly adapt to the daylight saving time events that are known beforehand.
This PR is accompanied by changes in Kibana elastic/kibana#188710 and the native ml-cpp code elastic/ml-cpp#2690.
The integration tests between Java and C++ parts of the force-time shift action will be implemented in a separate PR.
|
C++ part was implemented in #2695 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Problem
In many countries around the world, a time shift related to Daylight Saving Time (DST) causes users to change their behavior with respect to UTC. While we continue to improve the machine learning algorithms associated with detecting these shifts, it still takes some time to gather evidence that supports the time shift hypothesis. The anomaly detection job may take a few days to adjust to the change. During this time, anomaly detection may produce false positive or false negative alerts.
Enhancement
We introduce a new detection rule that enforces a time shift in the anomaly model, directly overriding the training and hypothesis testing functionality.
The text was updated successfully, but these errors were encountered: