Enable token authorization and model control for gRPC

[namannandan]

Description

Enable and test token authorization, model control and allowed_urls for gRPC

• Token Authorization

# Start torchserve with token auth enabled
$ torchserve --ncs --start --enable-model-api --model-store ./model_store

# Register model without auth token
$ python ts_scripts/torchserve_grpc_client.py register densenet161                      
## Check densenet161.mar in mar_set : set()
## Register marfile: https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar

Failed to register model densenet161.
Token Authorization failed. Token either incorrect, expired, or not provided correctly

# Register model with incorrect management token
$ python ts_scripts/torchserve_grpc_client.py register densenet161 --auth-token abcd123
## Check densenet161.mar in mar_set : set()
## Register marfile: https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar

Failed to register model densenet161.
Token Authorization failed. Token either incorrect, expired, or not provided correctly

# Register model with correct management token
$ python ts_scripts/torchserve_grpc_client.py register densenet161 --auth-token 5YbFPWBG
## Check densenet161.mar in mar_set : set()
## Register marfile: https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar

Model densenet161 registered successfully

# Model inference with no inference token
$ python ts_scripts/torchserve_grpc_client.py infer densenet161 ./examples/image_classifier/kitten.jpg                   
Traceback (most recent call last):
  File "/Volumes/workplace/pytorch/serve/ts_scripts/torchserve_grpc_client.py", line 363, in <module>
    infer(get_inference_stub(), args.model_name, args.model_input, metadata)
  File "/Volumes/workplace/pytorch/serve/ts_scripts/torchserve_grpc_client.py", line 30, in infer
    response = stub.Predictions(
  File "/Volumes/workplace/pytorch/serve/venvs/ts_dev/lib/python3.9/site-packages/grpc/_channel.py", line 1176, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/Volumes/workplace/pytorch/serve/venvs/ts_dev/lib/python3.9/site-packages/grpc/_channel.py", line 1005, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.PERMISSION_DENIED
        details = "Token Authorization failed. Token either incorrect, expired, or not provided correctly"
        debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-07-15T10:00:31.12596-07:00", grpc_status:7, grpc_message:"Token Authorization failed. Token either incorrect, expired, or not provided correctly"}"
>

# Model inference with incorrect inference token
$ python ts_scripts/torchserve_grpc_client.py infer densenet161 ./examples/image_classifier/kitten.jpg --auth-token abcd12
3
Traceback (most recent call last):
  File "/Volumes/workplace/pytorch/serve/ts_scripts/torchserve_grpc_client.py", line 363, in <module>
    infer(get_inference_stub(), args.model_name, args.model_input, metadata)
  File "/Volumes/workplace/pytorch/serve/ts_scripts/torchserve_grpc_client.py", line 30, in infer
    response = stub.Predictions(
  File "/Volumes/workplace/pytorch/serve/venvs/ts_dev/lib/python3.9/site-packages/grpc/_channel.py", line 1176, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/Volumes/workplace/pytorch/serve/venvs/ts_dev/lib/python3.9/site-packages/grpc/_channel.py", line 1005, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.PERMISSION_DENIED
        details = "Token Authorization failed. Token either incorrect, expired, or not provided correctly"
        debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-07-15T10:02:38.320565-07:00", grpc_status:7, grpc_message:"Token Authorization failed. Token either incorrect, expired, or not provided correctly"}"
>

# Model Inference with correct inference token
$ python ts_scripts/torchserve_grpc_client.py infer densenet161 ./examples/image_classifier/kitten.jpg --auth-token P1AjuhEV
{
  "tabby": 0.46661901473999023,
  "tiger_cat": 0.46449053287506104,
  "Egyptian_cat": 0.06614045053720474,
  "lynx": 0.001292440458200872,
  "plastic_bag": 0.00022909720428287983
}

# Unregister model without auth  token
$ python ts_scripts/torchserve_grpc_client.py unregister densenet161                                   
Failed to unregister model densenet161.
Token Authorization failed. Token either incorrect, expired, or not provided correctly

# Unregister model with incorrect management token
$ python ts_scripts/torchserve_grpc_client.py unregister densenet161 --auth-token abcd123
Failed to unregister model densenet161.
Token Authorization failed. Token either incorrect, expired, or not provided correctly

# Unregister model with correct management token
$ python ts_scripts/torchserve_grpc_client.py unregister densenet161 --auth-token 5YbFPWBG
Model densenet161 unregistered successfully

# Stop torchserve
$ torchserve --stop

• Model API Control

# Start torchserve with model API disabled
$ torchserve --ncs --start --disable-token-auth --model-store ./model_store

# Register model
$ python ts_scripts/torchserve_grpc_client.py register densenet161                                     
## Check densenet161.mar in mar_set : set()
## Register marfile: https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar

Failed to register model densenet161.
Model API disabled
org.pytorch.serve.archive.model.ModelException

# Stop torchserve
$ torchserve --stop

# Start torchserve with model API disabled and default model
$ torchserve --ncs --start --disable-token-auth --model-store ./model_store --models densenet161=https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar

# Unregister model
$ python ts_scripts/torchserve_grpc_client.py unregister densenet161
Failed to unregister model densenet161.
Model API disabled
org.pytorch.serve.archive.model.ModelException

# Stop torchserve
$ torchserve --stop

• allowed urls configuration

# Setup allowed_urls
$ cat custom_config.properties    
allowed_urls=https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar

# Start torchserve
$ torchserve --ncs --start --disable-token-auth --enable-model-api --model-store ./model_store --ts-config custom_config.properties

# Attempt registration of model from allowed url
$ python ts_scripts/torchserve_grpc_client.py register densenet161                        
## Check densenet161.mar in mar_set : set()
## Register marfile: https://torchserve.s3.amazonaws.com/mar_files/densenet161.mar

Model densenet161 registered successfully

# Attempt registration of model from unallowed url
$ python ts_scripts/torchserve_grpc_client.py register resnet-18
## Check resnet-18.mar in mar_set : set()
## Register marfile: https://torchserve.s3.amazonaws.com/mar_files/resnet-18.mar

Failed to register model resnet-18.
Given URL https://torchserve.s3.amazonaws.com/mar_files/resnet-18.mar does not match any allowed URL(s)
org.pytorch.serve.archive.model.ModelNotFoundException

# Stop torchserve
$ torchserve --stop

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

• test_gRPC_token_authorization.py
• test_gRPC_model_control.py
• test_gRPC_allowed_urls.py

[udaij12]

Why is gRPC unable to use the original token authorization handler?

[namannandan]

Why is gRPC unable to use the original token authorization handler?

The token authorization implementation is protocol agnostic and can be shared by both the HTTP and gRPC handlers. Since TokenAuthorizationHandler.java contains the token authorization implementation for HTTP and we ideally don't want the gRPC handler depending on the HTTP handler, I moved the core token authorization implementation to its own separate file TokenAuthorization.java which is shared by HTTP TokenAuthorizationHandler.java and gRPC GRPCInterceptor.java

[udaij12]

Passing the pytests so LGTM

[agunapal]

LGTM