This repository provides a Grafana dashboard, exported as JSON, for monitoring GOLang runtime of applications in a DC based setup where a given application is deployed on a cluster of servers in a given Data center (on prem or cloud). This dashboard groups together servers by the services being run on them, and services by the DC they are being served from.
It is possible to select one or multiple DC, and one or multiple hosts for a given service to compare metrics cross hosts and cross DC.
This requires runtime metrics to be exported by the application using prometheus client library for GOLang
Given below is a simple, tiny code snippet for enabling your application to expose prometheus metrics at /metrics
endpoint
package main
import (
"net/http"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
func main() {
http.Handle("/metrics/", promhttp.Handler())
http.ListenAndServe(":8888", nil)
}
This will expose Go runtime metrics at http://127.0.0.1:8888/metrics
You can send custom metrics to. You can find out more about instrumenting your GOLang app with Prometheus at ther offician documentation.
Your target must have 3 labels attached to them. These are as follows
-
dc - The dc where this service and its hosts are deployed. This is just a nomenclature, you can group your service by cluster, team, or any other nomenclature.
-
service - Name of the service
-
hostname - Name of the server or anything that will uniquely identify the server.
Example target configuration:
- job_name: 'mygoapp-1-dc-1'
static_configs:
- targets: ['127.0.0.1:2112']
labels:
dc: "dc1"
hostname: "my-goapp-1-server-1"
service: "mygoapp-1"
- targets: ['127.0.0.1:2113']
labels:
dc: "dc1"
hostname: "mygoapp-1-server-2"
service: "mygoapp-1"
- job_name: 'mygoapp-2-dc-1'
static_configs:
- targets: ['127.0.0.1:2114']
labels:
dc: "dc1"
hostname: "my-goapp-2-server-1"
service: "mygoapp-2"
- targets: ['127.0.0.1:2114']
labels:
dc: "dc1"
hostname: "mygoapp-2-server-2"
service: "mygoapp-2"
Some of the runtime metrics this dashboard visualises :
- Process resident memory
- Process virtual memory
- Stack memory
- Heap memory
- Allocs
- Open file descriptors
- Pointer dereference
- Number of Goroutines
- Number of OS threads
- GC time
 |
 |
 |
 |
 |
 |