Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include missing model configuration values in describe model API response #3122

Merged
merged 4 commits into from
May 9, 2024

Conversation

namannandan
Copy link
Collaborator

@namannandan namannandan commented Apr 30, 2024

Description

Since configuration options can be set at various levels, i.e command line, TorchServe configuration file, model configuration file etc.., it is useful to include what these configuration values finally resolve to in the describe model API response.

This PR includes some of the missing configuration values in the describe model API response.

Fixes #3037

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Feature/Issue validation/testing

  • CI
  • Manual test
$ torchserve --ncs --start --model-store ./model_store
$ curl -X POST "http://127.0.0.1:8081/models?url=resnet-18.mar&initial_workers=1"
$ curl "http://127.0.0.1:8081/models/resnet-18"
[                                                                                                                                                                                   
  {                                                                                                                                                                                 
    "modelName": "resnet-18",                                                                                                                                                       
    "modelVersion": "1.0",                                                                                                                                                          
    "modelUrl": "resnet-18",                                                                                                                                                        
    "runtime": "python",                                                                                                                                                            
    "minWorkers": 1,
    "maxWorkers": 1,
    "batchSize": 1,
    "maxBatchDelay": 100,
    "responseTimeout": 120,
    "maxRetryTimeoutInSec": 300,
    "clientTimeoutInMills": 0,
    "parallelType": "pp",
    "parallelLevel": 1,
    "deviceType": "gpu",
    "deviceIds": [
      0
    ],
    "continuousBatching": false,
    "useJobTicket": false,
    "useVenv": false,
    "stateful": false,
    "sequenceMaxIdleMSec": 0,
    "maxNumSequence": 0,
    "maxSequenceJobQueueSize": 0,
    "loadedAtStartup": false,
    "workers": [
      {
        "id": "29500",
        "startTime": "2024-04-30T22:58:40.635Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 5719,
        "gpu": true,
        "gpuUsage": "gpuId::0 utilization.gpu [%]::0 % utilization.memory [%]::0 % memory.used [MiB]::320 MiB"
      }
    ],
    "jobQueueStatus": {
      "remainingCapacity": 100,
      "pendingRequests": 0
    }
  }
]

Copy link
Collaborator

@mreso mreso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mreso mreso enabled auto-merge May 9, 2024 17:56
@mreso mreso added this pull request to the merge queue May 9, 2024
Merged via the queue into pytorch:master with commit e332949 May 9, 2024
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update describe model management API to show all model configuration options
2 participants