-
Notifications
You must be signed in to change notification settings - Fork 11
SONATA Monitor
The SONATA SDK has a set of monitoring tools available that can gather and centralize monitored metrics into a local database (Prometheus Database). Metrics can be queried from either VNFs deployed in the emulator (son-emu) or in the SONATA platform itself. After the metric data is stored in the local Prometheus Database, further analysis can take place (eg. using son-analyze) to debug or optimize the performance of the monitored VNF or service.
The README file with example usage and source code can be found in the GitHub repo of son-monitor.
CLI command: son-monitor query
Next to the compute, storage and network related metrics gathered by cAdvisor in son-emu, additional network metrics are gathered and exported by starting a query loop. This loop is using an SDN controller to query via the OpenFlow protocol the packet and byte counters of the (virtual) network interfaces that are installed in the emulated service. Next to the interface counters, also specific flow counters can be installed to allow a more finer-grained network traffic monitoring. The set of interface and flow counters to be monitored can be configured. All metrics gathered in son-emu are pushed to a Prometheus Push Gateway where they await to be pulled by the external Prometheus database in the SDK.
The SONATA emulator, son-emu, has a REST API to control the export of metrics to the Prometheus Gateway.
This can also be controlled from a Monitoring Service Descriptor (MSD) file, that lists all needed metrics.
On the wiki we have a page dedicated to the format of the MSD file.
CLI command: son-monitor stream
The SONATA Service Platform (SP) has a similar monitoring framework based on Prometheus. In addition, the SP can be instructed to export several metrics from this SP database.
As reaction to an alert, or just to check, the service developer using the SDK can also request monitoring data from the SP. This way, the tools in the SDK can help to further analyze or debug the service performance.
By using credentials, access control is implemented by the Service Platform Gatekeeper. This way, an SDK user gets only access to an allowed set of deployed services and VNFs (eg. only the services uploaded and instantiated by this specific SDK user).
From the SDK, son-monitor will first use son-access to get authorized access to the SP's Gatekeeper API.
This ensures that the SDK can only access monitoring data it is allowed to see (eg. only of the services that were instantiated by this SDK user).
The SONATA SDK has two possibilities to retrieve monitoring data from the SP:
- Asynchronous request: Using the REST API of the Gatekeeper, a request can be made to the Monitoring Manager to return a list of monitored metrics stored in the metrics database (eg. metric gathered during the last 5 mins). This is a single query to the SP's monitoring framework of historically monitored data.
- Synchronous stream: This is implemented by streaming (during a limited time-frame) real-time monitored data over a websocket. By connecting to this websocket, opened by the SP's monitoring framework, data of real-time monitored metrics can be stored in the SDK's local database.
These commands are a wrapper for the Gatekeeper API documented here
CLI command: son-monitor init + son-monitor msd
An easy graphical presentation of the gathered metrics can be achieved using the Grafana web GUI. This visualizes the queried metrics from the Prometheus database through a web-based GUI. The Grafana framework is started as a Docker container. A Monitoring Service Descriptor (MSD). file describes which metrics need to be visualized and how they should be combined on different graphs. The son-monitor msd command parses this MSD file and translates this to a Grafana Dashboard. Grafana is then configured to display this dashboard, using its standard REST API. By default the Grafana REST API and web GUI are available at http://localhost:3000 on the SDK.