Skip to content

Commit

Permalink
Add Milvus skeleton
Browse files Browse the repository at this point in the history
  • Loading branch information
dkirov-dd committed Nov 14, 2024
1 parent 66d9280 commit 9c7f4db
Show file tree
Hide file tree
Showing 24 changed files with 593 additions and 0 deletions.
4 changes: 4 additions & 0 deletions milvus/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# CHANGELOG - Milvus

<!-- towncrier release notes start -->

60 changes: 60 additions & 0 deletions milvus/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Agent Check: Milvus

## Overview

This check monitors [Milvus][1] through the Datadog Agent.

Include a high level overview of what this integration does:
- What does your product do (in 1-2 sentences)?
- What value will customers get from this integration, and why is it valuable to them?
- What specific data will your integration monitor, and what's the value of that data?

## Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions.

### Installation

The Milvus check is included in the [Datadog Agent][2] package.
No additional installation is needed on your server.

### Configuration

1. Edit the `milvus.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your milvus performance data. See the [sample milvus.d/conf.yaml][4] for all available configuration options.

2. [Restart the Agent][5].

### Validation

[Run the Agent's status subcommand][6] and look for `milvus` under the Checks section.

## Data Collected

### Metrics

See [metadata.csv][7] for a list of metrics provided by this integration.

### Events

The Milvus integration does not include any events.

### Service Checks

The Milvus integration does not include any service checks.

See [service_checks.json][8] for a list of service checks provided by this integration.

## Troubleshooting

Need help? Contact [Datadog support][9].


[1]: **LINK_TO_INTEGRATION_SITE**
[2]: https://app.datadoghq.com/account/settings/agent/latest
[3]: https://docs.datadoghq.com/agent/kubernetes/integrations/
[4]: https://github.com/DataDog/integrations-core/blob/master/milvus/datadog_checks/milvus/data/conf.yaml.example
[5]: https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent
[6]: https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information
[7]: https://github.com/DataDog/integrations-core/blob/master/milvus/metadata.csv
[8]: https://github.com/DataDog/integrations-core/blob/master/milvus/assets/service_checks.json
[9]: https://docs.datadoghq.com/help/
10 changes: 10 additions & 0 deletions milvus/assets/configuration/spec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
name: Milvus
files:
- name: milvus.yaml
options:
- template: init_config
options:
- template: init_config/default
- template: instances
options:
- template: instances/default
1 change: 1 addition & 0 deletions milvus/assets/dashboards/milvus_overview.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Please build an out-of-the-box dashboard for your integration following our best practices here: https://datadoghq.dev/integrations-core/guidelines/dashboards/#best-practices
1 change: 1 addition & 0 deletions milvus/assets/service_checks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[]
1 change: 1 addition & 0 deletions milvus/changelog.d/1.added
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Initial Release

Check failure on line 1 in milvus/changelog.d/1.added

View workflow job for this annotation

GitHub Actions / run / Check PR

Your changelog entry has the wrong PR number. To fix this please run: mv milvus/changelog.d/1.added milvus/changelog.d/19055.added
4 changes: 4 additions & 0 deletions milvus/datadog_checks/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)
__path__ = __import__('pkgutil').extend_path(__path__, __name__) # type: ignore
4 changes: 4 additions & 0 deletions milvus/datadog_checks/milvus/__about__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)
__version__ = '0.0.1'
7 changes: 7 additions & 0 deletions milvus/datadog_checks/milvus/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)
from .__about__ import __version__
from .check import MilvusCheck

__all__ = ['__version__', 'MilvusCheck']
98 changes: 98 additions & 0 deletions milvus/datadog_checks/milvus/check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)
from typing import Any # noqa: F401

from datadog_checks.base import AgentCheck # noqa: F401

# from datadog_checks.base.utils.db import QueryManager
# from requests.exceptions import ConnectionError, HTTPError, InvalidURL, Timeout
# from json import JSONDecodeError


class MilvusCheck(AgentCheck):

# This will be the prefix of every metric and service check the integration sends
__NAMESPACE__ = 'milvus'

def __init__(self, name, init_config, instances):
super(MilvusCheck, self).__init__(name, init_config, instances)

# Use self.instance to read the check configuration
# self.url = self.instance.get("url")

# If the check is going to perform SQL queries you should define a query manager here.
# More info at
# https://datadoghq.dev/integrations-core/base/databases/#datadog_checks.base.utils.db.core.QueryManager
# sample_query = {
# "name": "sample",
# "query": "SELECT * FROM sample_table",
# "columns": [
# {"name": "metric", "type": "gauge"}
# ],
# }
# self._query_manager = QueryManager(self, self.execute_query, queries=[sample_query])
# self.check_initializations.append(self._query_manager.compile_queries)

def check(self, _):
# type: (Any) -> None
# The following are useful bits of code to help new users get started.

# Perform HTTP Requests with our HTTP wrapper.
# More info at https://datadoghq.dev/integrations-core/base/http/
# try:
# response = self.http.get(self.url)
# response.raise_for_status()
# response_json = response.json()

# except Timeout as e:
# self.service_check(
# "can_connect",
# AgentCheck.CRITICAL,
# message="Request timeout: {}, {}".format(self.url, e),
# )
# raise

# except (HTTPError, InvalidURL, ConnectionError) as e:
# self.service_check(
# "can_connect",
# AgentCheck.CRITICAL,
# message="Request failed: {}, {}".format(self.url, e),
# )
# raise

# except JSONDecodeError as e:
# self.service_check(
# "can_connect",
# AgentCheck.CRITICAL,
# message="JSON Parse failed: {}, {}".format(self.url, e),
# )
# raise

# except ValueError as e:
# self.service_check(
# "can_connect", AgentCheck.CRITICAL, message=str(e)
# )
# raise

# This is how you submit metrics
# There are different types of metrics that you can submit (gauge, event).
# More info at https://datadoghq.dev/integrations-core/base/api/#datadog_checks.base.checks.base.AgentCheck
# self.gauge("test", 1.23, tags=['foo:bar'])

# Perform database queries using the Query Manager
# self._query_manager.execute()

# This is how you use the persistent cache. This cache file based and persists across agent restarts.
# If you need an in-memory cache that is persisted across runs
# You can define a dictionary in the __init__ method.
# self.write_persistent_cache("key", "value")
# value = self.read_persistent_cache("key")

# If your check ran successfully, you can send the status.
# More info at
# https://datadoghq.dev/integrations-core/base/api/#datadog_checks.base.checks.base.AgentCheck.service_check
# self.service_check("can_connect", AgentCheck.OK)

# If it didn't then it should send a critical service check
self.service_check("can_connect", AgentCheck.CRITICAL)
25 changes: 25 additions & 0 deletions milvus/datadog_checks/milvus/config_models/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)

# This file is autogenerated.
# To change this file you should edit assets/configuration/spec.yaml and then run the following commands:
# ddev -x validate config -s <INTEGRATION_NAME>
# ddev -x validate models -s <INTEGRATION_NAME>


from .instance import InstanceConfig
from .shared import SharedConfig


class ConfigMixin:
_config_model_instance: InstanceConfig
_config_model_shared: SharedConfig

@property
def config(self) -> InstanceConfig:
return self._config_model_instance

@property
def shared_config(self) -> SharedConfig:
return self._config_model_shared
16 changes: 16 additions & 0 deletions milvus/datadog_checks/milvus/config_models/defaults.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)

# This file is autogenerated.
# To change this file you should edit assets/configuration/spec.yaml and then run the following commands:
# ddev -x validate config -s <INTEGRATION_NAME>
# ddev -x validate models -s <INTEGRATION_NAME>


def instance_empty_default_hostname():
return False


def instance_min_collection_interval():
return 15
51 changes: 51 additions & 0 deletions milvus/datadog_checks/milvus/config_models/instance.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)

# This file is autogenerated.
# To change this file you should edit assets/configuration/spec.yaml and then run the following commands:
# ddev -x validate config -s <INTEGRATION_NAME>
# ddev -x validate models -s <INTEGRATION_NAME>


from __future__ import annotations

from typing import Optional

from pydantic import BaseModel, ConfigDict, field_validator, model_validator

from datadog_checks.base.utils.functions import identity
from datadog_checks.base.utils.models import validation

from . import defaults, validators


class InstanceConfig(BaseModel):
model_config = ConfigDict(
validate_default=True,
arbitrary_types_allowed=True,
frozen=True,
)
empty_default_hostname: Optional[bool] = None
min_collection_interval: Optional[float] = None
service: Optional[str] = None
tags: Optional[tuple[str, ...]] = None

@model_validator(mode='before')
def _initial_validation(cls, values):
return validation.core.initialize_config(getattr(validators, 'initialize_instance', identity)(values))

@field_validator('*', mode='before')
def _validate(cls, value, info):
field = cls.model_fields[info.field_name]
field_name = field.alias or info.field_name
if field_name in info.context['configured_fields']:
value = getattr(validators, f'instance_{info.field_name}', identity)(value, field=field)
else:
value = getattr(defaults, f'instance_{info.field_name}', lambda: value)()

return validation.utils.make_immutable(value)

@model_validator(mode='after')
def _final_validation(cls, model):
return validation.core.check_model(getattr(validators, 'check_instance', identity)(model))
48 changes: 48 additions & 0 deletions milvus/datadog_checks/milvus/config_models/shared.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)

# This file is autogenerated.
# To change this file you should edit assets/configuration/spec.yaml and then run the following commands:
# ddev -x validate config -s <INTEGRATION_NAME>
# ddev -x validate models -s <INTEGRATION_NAME>


from __future__ import annotations

from typing import Optional

from pydantic import BaseModel, ConfigDict, field_validator, model_validator

from datadog_checks.base.utils.functions import identity
from datadog_checks.base.utils.models import validation

from . import defaults, validators


class SharedConfig(BaseModel):
model_config = ConfigDict(
validate_default=True,
arbitrary_types_allowed=True,
frozen=True,
)
service: Optional[str] = None

@model_validator(mode='before')
def _initial_validation(cls, values):
return validation.core.initialize_config(getattr(validators, 'initialize_shared', identity)(values))

@field_validator('*', mode='before')
def _validate(cls, value, info):
field = cls.model_fields[info.field_name]
field_name = field.alias or info.field_name
if field_name in info.context['configured_fields']:
value = getattr(validators, f'shared_{info.field_name}', identity)(value, field=field)
else:
value = getattr(defaults, f'shared_{info.field_name}', lambda: value)()

return validation.utils.make_immutable(value)

@model_validator(mode='after')
def _final_validation(cls, model):
return validation.core.check_model(getattr(validators, 'check_shared', identity)(model))
13 changes: 13 additions & 0 deletions milvus/datadog_checks/milvus/config_models/validators.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# (C) Datadog, Inc. 2024-present
# All rights reserved
# Licensed under a 3-clause BSD style license (see LICENSE)

# Here you can include additional config validators or transformers
#
# def initialize_instance(values, **kwargs):
# if 'my_option' not in values and 'my_legacy_option' in values:
# values['my_option'] = values['my_legacy_option']
# if values.get('my_number') > 10:
# raise ValueError('my_number max value is 10, got %s' % str(values.get('my_number')))
#
# return values
Loading

0 comments on commit 9c7f4db

Please sign in to comment.