Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add KServe gRPC v2 support #2176

Merged
merged 16 commits into from
Aug 24, 2023

Conversation

jagadeeshi2i
Copy link
Collaborator

@jagadeeshi2i jagadeeshi2i commented Mar 13, 2023

Description

Add gRPC support for KServe v2 protocol

Depends on #2175
Fixes #1859

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A
    Logs for Test A

  • Test B
    Logs for Test B

Checklist:

  • Did you have fun?
  • Have you added tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

@codecov
Copy link

codecov bot commented Mar 14, 2023

Codecov Report

Merging #2176 (e092b6c) into master (2ff5020) will increase coverage by 0.01%.
The diff coverage is 85.71%.

❗ Current head e092b6c differs from pull request most recent head 374f958. Consider uploading reports for the commit 374f958 to get more accurate results

@@            Coverage Diff             @@
##           master    #2176      +/-   ##
==========================================
+ Coverage   72.85%   72.86%   +0.01%     
==========================================
  Files          78       78              
  Lines        3695     3697       +2     
  Branches       58       58              
==========================================
+ Hits         2692     2694       +2     
  Misses        999      999              
  Partials        4        4              
Files Changed Coverage Δ
ts/torch_handler/request_envelope/kservev2.py 90.27% <85.71%> (+0.27%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

kubernetes/kserve/kserve_wrapper/TorchserveModel.py Outdated Show resolved Hide resolved
kubernetes/kserve/kserve_wrapper/__main__.py Outdated Show resolved Hide resolved
ts/torch_handler/request_envelope/kservev2.py Show resolved Hide resolved
kubernetes/kserve/kserve_wrapper/inference_pb2.py Outdated Show resolved Hide resolved
kubernetes/kserve/kserve_wrapper/inference_pb2_grpc.py Outdated Show resolved Hide resolved
async_result = await grpc_stub.Predictions(payload)
return async_result

def postprocess(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this function is called. Could you add some comments (eg. which function call this one)?

)
return self.grpc_client_stub

async def _grpc_predict(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this function is called. Could you add some comments (eg. which function call this one)?

@fsatka
Copy link

fsatka commented May 2, 2023

Somebody, please! Get this man approve 🙂

jagadeesh added 5 commits August 4, 2023 12:17
Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
 - add logs

Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
@jagadeeshi2i jagadeeshi2i changed the title [WIP] feat: add KServe gRPC v2 support feat: add KServe gRPC v2 support Aug 4, 2023
Copy link
Collaborator

@agunapal agunapal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jagadeeshi2i , Please check the regression failures.

Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com>
@jagadeeshi2i jagadeeshi2i added the enhancement New feature or request label Aug 16, 2023
@msaroufim msaroufim merged commit 39e715d into pytorch:master Aug 24, 2023
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for KServe V2 gRPC Prediction API?
6 participants