Add support for custom metrics in /metrics endpoint and Prometheus

[chauhang]

The support for getting the custom metrics via the /metrics endpoint for viewing via in dashboards using Prometheus + Grafana.

[msaroufim]

Summarizing @maaquib comments here as to how to support this. The issue right now is custom metrics are only supported in the frontend and they need to be passed to the backend.

These are the metrics supported in the frontend: https://github.com/pytorch/serve/blob/master/frontend/server/src/main/java/org/pytorch/serve/metrics/format/prometheous/PrometheusMetricManager.java which is called by

serve/frontend/server/src/main/java/org/pytorch/serve/metrics/api/MetricAggregator.java

Line 24 in 6d5ca96

metrics.incInferLatency(inferTime, modelName, modelVersion);

with the metrics computed here

serve/frontend/server/src/main/java/org/pytorch/serve/job/RestJob.java

Line 74 in 6d5ca96

MetricAggregator.handleInferenceMetric(

This is where the request is sent to the backend (and where the custom metrics need to be sent):

serve/frontend/server/src/main/java/org/pytorch/serve/wlm/WorkerThread.java

Line 185 in 8903ca1

backendChannel.writeAndFlush(req).sync();

Encoding/Decoding happens on the frontend here: https://github.com/pytorch/serve/tree/master/frontend/server/src/main/java/org/pytorch/serve/util/codec

Encoding/Decoding happens on the backend here: https://github.com/pytorch/serve/blob/master/ts/protocol/otf_message_handler.py

To test the frontend: https://github.com/pytorch/serve/tree/master/frontend

To test the backend: Python debugger with breakpoints

[msaroufim]

Duplicate of #1492