Add job queue status to describe API

[namannandan]

Description

Include job queue status in the describe API response:

• Remaining capacity
• Pending requests
• Consecutive failed requests

curl http://localhost:8081/models/noop
[
    {
      "modelName": "noop",
      "modelVersion": "1.0",
      "modelUrl": "noop.mar",
      "engine": "Torch",
      "runtime": "python",
      "minWorkers": 1,
      "maxWorkers": 1,
      "batchSize": 1,
      "maxBatchDelay": 100,
      "workers": [
        {
          "id": "9000",
          "startTime": "2018-10-02T13:44:53.034Z",
          "status": "READY",
          "gpu": false,
          "memoryUsage": 89247744
        }
      ],
      "jobQueueStatus": {
        "remainingCapacity": 100,
        "pendingRequests": 0
      }
    }
]

Fixes #2101, #2412

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

• Unit test added in this PR
• Sanity and regression tests in CI

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[codecov]

Codecov Report

Merging #2464 (2320505) into master (fdcc223) will not change coverage.
The diff coverage is n/a.

❗ Current head 2320505 differs from pull request most recent head c40005e. Consider uploading reports for the commit c40005e to get more accurate results

@@           Coverage Diff           @@
##           master    #2464   +/-   ##
=======================================
  Coverage   72.66%   72.66%           
=======================================
  Files          78       78           
  Lines        3669     3669           
  Branches       58       58           
=======================================
  Hits         2666     2666           
  Misses        999      999           
  Partials        4        4           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[lxning]

Please don't break existing description attributes.

[namannandan]

Please don't break existing description attributes.

@lxning currently we don't publish any data/metrics related to job queue in the describe API endpoint. Now, I've included an additional field called jobQueueStatus. Also, I changed the implementation name from metrics to jobQueueStatus since this may be confusing given we already have a separate metrics endpoint. Do you suggest creating a separate endpoint to fetch jobQueueStatus?

Existing implementation:

curl http://localhost:8081/models/noop
[
    {
      "modelName": "noop",
      "modelVersion": "1.0",
      "modelUrl": "noop.mar",
      "engine": "Torch",
      "runtime": "python",
      "minWorkers": 1,
      "maxWorkers": 1,
      "batchSize": 1,
      "maxBatchDelay": 100,
      "workers": [
        {
          "id": "9000",
          "startTime": "2018-10-02T13:44:53.034Z",
          "status": "READY",
          "gpu": false,
          "memoryUsage": 89247744
        }
      ]
    }
]

With the changes in this PR:

curl http://localhost:8081/models/noop
[
    {
      "modelName": "noop",
      "modelVersion": "1.0",
      "modelUrl": "noop.mar",
      "engine": "Torch",
      "runtime": "python",
      "minWorkers": 1,
      "maxWorkers": 1,
      "batchSize": 1,
      "maxBatchDelay": 100,
      "workers": [
        {
          "id": "9000",
          "startTime": "2018-10-02T13:44:53.034Z",
          "status": "READY",
          "gpu": false,
          "memoryUsage": 89247744
        }
      ],
      "jobQueueStatus": {
        "remainingCapacity": 100,
        "pendingRequests": 0,
        "consecutiveFailedRequests": 0
      }
    }
]

[lxning]

TS provide SDK to allow cx to customize the endpoint response. It will break cx's customized plugin when class Metrics and func getMetrics/etMetrics are moved.

[namannandan]

Updated implementation to retain the Metrics class definition as is.