Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix tcp port in model_service_worker.py #2377

Merged
merged 4 commits into from
Jun 1, 2023
Merged

fix tcp port in model_service_worker.py #2377

merged 4 commits into from
Jun 1, 2023

Conversation

lxning
Copy link
Collaborator

@lxning lxning commented May 31, 2023

Description

Please read our CONTRIBUTING.md prior to creating your first pull request.

Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #(issue)
#2368 #2349

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test
    torchserve --ncs --start --model-store model_store/ --models opt-30b-pippy.mar --ts-config benchmarks/config.properties
inference_address=http://0.0.0.0:8080
management_address=http://0.0.0.0:8081

number_of_netty_threads=32
job_queue_size=1000

vmargs=-Xmx4g -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError
prefer_direct_buffer=True

default_response_timeout=300
unregister_model_timeout=300
install_py_dep_per_model=true
default_workers_per_model=1
use_native_io=false
2023-06-01T06:16:40,278 [DEBUG] W-29500-opt-30b-pippy_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [torchrun, --nnodes, 1, --nproc-per-node, 4, --log-dir, /home/ubuntu/serve/logs/torchelastic_ts, --rdzv-backend, c10d, --rdzv-id, opt-30b-pippy_29500, --max-restarts, 1, /opt/conda/envs/py38/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, tcp, --port, 29500, --metrics-config, /opt/conda/envs/py38/lib/python3.8/site-packages/ts/configs/metrics.yaml]

curl "http://localhost:8081/models"
2023-06-01T06:22:13,177 [INFO ] nioEventLoopGroup-3-1 ACCESS_LOG - /127.0.0.1:52714 "GET /models HTTP/1.1" 200 6
2023-06-01T06:22:13,177 [INFO ] nioEventLoopGroup-3-1 TS_METRICS - Requests2XX.Count:1.0|#Level:Host|#hostname:ip-172-31-6-222,timestamp:1685600533
{
"models": [
{
"modelName": "opt-30b-pippy",
"modelUrl": "opt-30b-pippy.mar"
}
]
}

Checklist:

  • Did you have fun?
  • Have you added tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

@lxning lxning added the bug Something isn't working label May 31, 2023
@lxning lxning added this to the v0.8.1 milestone May 31, 2023
@lxning lxning self-assigned this May 31, 2023
@codecov
Copy link

codecov bot commented May 31, 2023

Codecov Report

Merging #2377 (ed0b74d) into master (c685b5c) will decrease coverage by 0.02%.
The diff coverage is 33.33%.

❗ Current head ed0b74d differs from pull request most recent head e53b65c. Consider uploading reports for the commit e53b65c to get more accurate results

@@            Coverage Diff             @@
##           master    #2377      +/-   ##
==========================================
- Coverage   72.15%   72.13%   -0.02%     
==========================================
  Files          78       78              
  Lines        3641     3642       +1     
  Branches       58       58              
==========================================
  Hits         2627     2627              
- Misses       1010     1011       +1     
  Partials        4        4              
Impacted Files Coverage Δ
ts/model_service_worker.py 63.88% <33.33%> (-0.45%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Collaborator

@HamidShojanazeri HamidShojanazeri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks @lxning

Copy link
Collaborator

@mreso mreso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lxning lxning merged commit a81fc52 into master Jun 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants