XGBoost classifier example

[agunapal]

Description

This example shows how to use a XGBoost classifier model with TorchServe

Fixes #(issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

• Start TorchServe

torchserve --start --ncs --model-store model_store --models xgb_iris=xgb_iris.mar
(torchserve) ubuntu@ip-172-31-12-209:~/serve/examples/xgboost_classfication$ WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2024-04-15T21:36:07,834 [WARN ] main org.pytorch.serve.util.ConfigManager - Your torchserve instance can access any URL to load models. When deploying to production, make sure to limit the set of allowed_urls in config.properties
2024-04-15T21:36:07,836 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager...
2024-04-15T21:36:07,884 [INFO ] main org.pytorch.serve.metrics.configuration.MetricConfiguration - Successfully loaded metrics configuration from /home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages/ts/configs/metrics.yaml
2024-04-15T21:36:07,955 [INFO ] main org.pytorch.serve.ModelServer - 
Torchserve version: 0.10.0
TS Home: /home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages
Current directory: /home/ubuntu/serve/examples/xgboost_classfication
Temp directory: /tmp
Metrics config path: /home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages/ts/configs/metrics.yaml
Number of GPUs: 1
Number of CPUs: 8
Max heap size: 7936 M
Python executable: /home/ubuntu/anaconda3/envs/torchserve/bin/python
Config file: N/A
Inference address: http://127.0.0.1:8080
Management address: http://127.0.0.1:8081
Metrics address: http://127.0.0.1:8082
Model Store: /home/ubuntu/serve/examples/xgboost_classfication/model_store
Initial Models: xgb_iris=xgb_iris.mar
Log dir: /home/ubuntu/serve/examples/xgboost_classfication/logs
Metrics dir: /home/ubuntu/serve/examples/xgboost_classfication/logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 1
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Limit Maximum Image Pixels: true
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Enable metrics API: true
Metrics mode: LOG
Disable system metrics: false
Workflow Store: /home/ubuntu/serve/examples/xgboost_classfication/model_store
CPP log config: N/A
Model config: N/A
System metrics command: default
2024-04-15T21:36:07,961 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager -  Loading snapshot serializer plugin...
2024-04-15T21:36:07,976 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: xgb_iris.mar
2024-04-15T21:36:08,000 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model xgb_iris
2024-04-15T21:36:08,000 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model xgb_iris
2024-04-15T21:36:08,001 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model xgb_iris loaded.
2024-04-15T21:36:08,001 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: xgb_iris, count: 1
2024-04-15T21:36:08,007 [DEBUG] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/home/ubuntu/anaconda3/envs/torchserve/bin/python, /home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /tmp/.ts.sock.9000, --metrics-config, /home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages/ts/configs/metrics.yaml]
2024-04-15T21:36:08,007 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2024-04-15T21:36:08,057 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://127.0.0.1:8080
2024-04-15T21:36:08,058 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: EpollServerSocketChannel.
2024-04-15T21:36:08,059 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://127.0.0.1:8081
2024-04-15T21:36:08,059 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2024-04-15T21:36:08,060 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
Model server started.
2024-04-15T21:36:08,235 [WARN ] pool-3-thread-1 org.pytorch.serve.metrics.MetricCollector - worker pid is not available yet.

(torchserve) ubuntu@ip-172-31-12-209:~/serve/examples/xgboost_classfication$ 
(torchserve) ubuntu@ip-172-31-12-209:~/serve/examples/xgboost_classfication$ 
(torchserve) ubuntu@ip-172-31-12-209:~/serve/examples/xgboost_classfication$ 2024-04-15T21:36:09,037 [INFO ] pool-3-thread-1 TS_METRICS - CPUUtilization.Percent:50.0|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,038 [INFO ] pool-3-thread-1 TS_METRICS - DiskAvailable.Gigabytes:83.6930046081543|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,038 [INFO ] pool-3-thread-1 TS_METRICS - DiskUsage.Gigabytes:206.85794067382812|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,038 [INFO ] pool-3-thread-1 TS_METRICS - DiskUtilization.Percent:71.2|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,039 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUtilization.Percent:0.0|#Level:Host,DeviceId:0|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,039 [INFO ] pool-3-thread-1 TS_METRICS - GPUMemoryUsed.Megabytes:0.0|#Level:Host,DeviceId:0|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,039 [INFO ] pool-3-thread-1 TS_METRICS - GPUUtilization.Percent:0.0|#Level:Host,DeviceId:0|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,040 [INFO ] pool-3-thread-1 TS_METRICS - MemoryAvailable.Megabytes:30219.88671875|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,040 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUsed.Megabytes:1072.98046875|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,040 [INFO ] pool-3-thread-1 TS_METRICS - MemoryUtilization.Percent:4.8|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216969
2024-04-15T21:36:09,218 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - s_name_part0=/tmp/.ts.sock, s_name_part1=9000, pid=17143
2024-04-15T21:36:09,219 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Listening on port: /tmp/.ts.sock.9000
2024-04-15T21:36:09,228 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Successfully loaded /home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages/ts/configs/metrics.yaml.
2024-04-15T21:36:09,228 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - [PID]17143
2024-04-15T21:36:09,229 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Torch worker started.
2024-04-15T21:36:09,229 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Python runtime: 3.10.0
2024-04-15T21:36:09,229 [DEBUG] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-xgb_iris_1.0 State change null -> WORKER_STARTED
2024-04-15T21:36:09,232 [INFO ] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /tmp/.ts.sock.9000
2024-04-15T21:36:09,238 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Connection accepted: /tmp/.ts.sock.9000.
2024-04-15T21:36:09,240 [DEBUG] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd LOAD repeats 1 to backend at: 1713216969240
2024-04-15T21:36:09,242 [INFO ] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - Looping backend response at: 1713216969242
2024-04-15T21:36:09,267 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - model_name: xgb_iris, batchSize: 2
2024-04-15T21:36:10,669 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Enabled tensor cores
2024-04-15T21:36:10,705 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - ONNX enabled
2024-04-15T21:36:10,706 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Torch TensorRT not enabled
2024-04-15T21:36:10,717 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - XGBoost Classifier for iris dataset with weights /tmp/models/f1d4979037994257a7a88b70e7653592/iris_model.json loaded successfully
2024-04-15T21:36:10,721 [INFO ] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1479
2024-04-15T21:36:10,722 [DEBUG] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - W-9000-xgb_iris_1.0 State change WORKER_STARTED -> WORKER_MODEL_LOADED

• Inference

2024-04-15T21:36:10,722 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - WorkerLoadTime.Milliseconds:2717.0|#WorkerName:W-9000-xgb_iris_1.0,Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216970
2024-04-15T21:36:10,722 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:3.0|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216970
2024-04-15T21:36:12,712 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - ts_inference_requests_total.Count:1.0|#model_name:xgb_iris,model_version:default|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,712 [INFO ] epollEventLoopGroup-3-1 TS_METRICS - ts_inference_requests_total.Count:1.0|#model_name:xgb_iris,model_version:default|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,713 [DEBUG] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req.cmd PREDICT repeats 1 to backend at: 1713216972713
2024-04-15T21:36:12,714 [INFO ] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - Looping backend response at: 1713216972714
2024-04-15T21:36:12,715 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_LOG - Backend received inference at: 1713216972
2024-04-15T21:36:12,716 [INFO ] W-9000-xgb_iris_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - result=[METRICS]HandlerTime.Milliseconds:0.9|#ModelName:xgb_iris,Level:Model|#type:GAUGE|#hostname:ip-172-31-12-209,1713216972,63448b2f-8f51-44e2-8ae6-b32484ea166d,7353efc8-4a20-4d5d-9c3b-80026fcc9706, pattern=[METRICS]
2024-04-15T21:36:12,716 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_METRICS - HandlerTime.ms:0.9|#ModelName:xgb_iris,Level:Model|#hostname:ip-172-31-12-209,requestID:63448b2f-8f51-44e2-8ae6-b32484ea166d,7353efc8-4a20-4d5d-9c3b-80026fcc9706,timestamp:1713216972
2024-04-15T21:36:12,717 [INFO ] W-9000-xgb_iris_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - result=[METRICS]PredictionTime.Milliseconds:1.01|#ModelName:xgb_iris,Level:Model|#type:GAUGE|#hostname:ip-172-31-12-209,1713216972,63448b2f-8f51-44e2-8ae6-b32484ea166d,7353efc8-4a20-4d5d-9c3b-80026fcc9706, pattern=[METRICS]
2024-04-15T21:36:12,717 [INFO ] W-9000-xgb_iris_1.0-stdout MODEL_METRICS - PredictionTime.ms:1.01|#ModelName:xgb_iris,Level:Model|#hostname:ip-172-31-12-209,requestID:63448b2f-8f51-44e2-8ae6-b32484ea166d,7353efc8-4a20-4d5d-9c3b-80026fcc9706,timestamp:1713216972
2024-04-15T21:36:12,717 [INFO ] W-9000-xgb_iris_1.0 ACCESS_LOG - /127.0.0.1:47072 "POST /predictions/xgb_iris HTTP/1.1" 200 7
2024-04-15T21:36:12,718 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - Requests2XX.Count:1.0|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,718 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - ts_inference_latency_microseconds.Microseconds:3307.88|#model_name:xgb_iris,model_version:default|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,718 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - ts_queue_latency_microseconds.Microseconds:144.753|#model_name:xgb_iris,model_version:default|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,719 [DEBUG] W-9000-xgb_iris_1.0 org.pytorch.serve.job.RestJob - Waiting time ns: 144753, Backend time ns: 5253408
2024-04-15T21:36:12,719 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - QueueTime.Milliseconds:0.0|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,719 [INFO ] W-9000-xgb_iris_1.0 ACCESS_LOG - /127.0.0.1:47074 "POST /predictions/xgb_iris HTTP/1.1" 200 9
2024-04-15T21:36:12,719 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - Requests2XX.Count:1.0|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,720 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - ts_inference_latency_microseconds.Microseconds:5926.099|#model_name:xgb_iris,model_version:default|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,720 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - ts_queue_latency_microseconds.Microseconds:150.173|#model_name:xgb_iris,model_version:default|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,720 [DEBUG] W-9000-xgb_iris_1.0 org.pytorch.serve.job.RestJob - Waiting time ns: 150173, Backend time ns: 7092946
2024-04-15T21:36:12,721 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - QueueTime.Milliseconds:0.0|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216972
2024-04-15T21:36:12,721 [INFO ] W-9000-xgb_iris_1.0 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2024-04-15T21:36:12,721 [INFO ] W-9000-xgb_iris_1.0 TS_METRICS - WorkerThreadTime.Milliseconds:6.0|#Level:Host|#hostname:ip-172-31-12-209,timestamp:1713216972

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[mreso]

LGTM