Opni Pretrained DRAIN Inferencing Service
This service serves as a cache for control-plane, Rancher and Longhorn log messages. It is based on the DRAIN3 log templatization algorithm. If a log message comes to this service and it is not currently a part of the pre-trained model's cache, it will be sent to the Deep Learning service to be inferred on. Once the anomaly level of the log has been determined, the inferencing service will send back this log to the DRAIN workload service and the log message with the corresponding anomaly level will be added to a secondary cache. The result will then be sent to the Opensearch Updating Service
- Python
- Create log templates for any log messages that are not part of the pretrained model's cache.
- Serve as a caching mechanism with the log template and anomaly level which is determined via the pre-trained Deep Learning model.
- Persist cache into S3 every hour if cache has been modified in any way for additional control-plane, Rancher or Longhorn logs.
- Send log message insights including log template and anomaly level to the Opensearch Updating Service.
| Component | Type | Description |
|---|---|---|
| preprocessed_logs_pretrained_model | Nats subject | The logs received from the raw_logs Nats subject come from the ingest plugin. |
| model_inferenced_pretrained_logs | Nats subject | Once the logs have been inferred by the pre-trained Deep Learning model, the DRAIN service will receive the inferred results and add the results to its cache. |
| batch_processed | Nats subject | Anytime the DRAIN3 model cache has been updated in any form, it will send over payload to the batch_processed Nats subject. If it has been at least an hour since the DRAIN cache model was last persisted into S3, then the updated cache will be saved into S3. |
| Component | Type | Description |
|---|---|---|
| inferenced_logs | Nats subject | For any log messages which are stored within the DRAIN cache, obtain the anomaly level and then publish the updated payload to the inferenced_logs Nats subject which will be received by the Opensearch Updating Service. |
| templates_index | Nats subject | When a new log template has been created or a log template has been modified, publish to the templates_index Nats subject so that it can be updated by the Opensearch Updating service. |
| opnilog_cp_logs | Nats subject | For any log messages that are not currently stored in the DRAIN cache, send over those logs to the Deep Learning model for inferencing by publishing to the opnilog_workload_logs Nats subject. |
| opnilog_rancher_logs | Nats subject | For any Rancher log messages that are not currently stored in the DRAIN cache, send over those logs to the Deep Learning model for inferencing by publishing to the opnilog_workload_logs Nats subject. |
| opnilog_longhorn_logs | Nats subject | For any Longhorn log messages that are not currently stored in the DRAIN cache, send over those logs to the Deep Learning model for inferencing by publishing to the opnilog_workload_logs Nats subject. |
| batch_processed | Nats subject | Anytime the DRAIN3 model cache has been updated in any form, it will send over payload to the batch_processed Nats subject. |
| Model Version | Description |
|---|---|
| v0.8.2 | Contains 699 log templates including some updated templates for Longhorn and control-plane/etcd logs. |
- Unit tests
- Integration tests
- e2e tests
- Manual testing