Security documentation update (#3183)

* adding security update * adding security update * doc updates * changes to model mode * adding more update * adding token auth paragraph * doc changes and logging addition * doc changes and logging addition * fix formatting * changing flag name and adding env for model api * flag fixes * fixing doc * changing misaligned name * change to variable name * change name * changing config name * spellcheck test * testing docker change * test * fixing test_util * changes to llm * adding model api flag * fixes to llm update * launcher fix * testing token * testing token * fixing docker * change branch name' * change branch name' * final changes * Doc changes Co-authored-by: Naman Nandan <namankt55@gmail.com> * adding key name --------- Co-authored-by: Matthias Reso <13337103+mreso@users.noreply.github.com> Co-authored-by: Naman Nandan <namankt55@gmail.com>
pytorch · Jul 3, 2024 · dc08344 · dc08344
1 parent b4bbc29
commit dc08344
Show file tree

Hide file tree

Showing 44 changed files with 171 additions and 75 deletions.
diff --git a/.github/workflows/docker-ci.yaml b/.github/workflows/docker-ci.yaml
@@ -29,7 +29,7 @@ jobs:
         working-directory: docker
         run: |
           IMAGE_TAG=test-image-${{ matrix.python-version }}
-          ./build_image.sh -py "${{ matrix.python-version }}" -t "${IMAGE_TAG}" -s -r
+          ./build_image.sh -py "${{ matrix.python-version }}" -t "${IMAGE_TAG}" -b ${GITHUB_HEAD_REF} -s
           echo "IMAGE_TAG=${IMAGE_TAG}" >> $GITHUB_OUTPUT
 
       - name: Container Healthcheck

diff --git a/README.md b/README.md
@@ -1,5 +1,9 @@
+# ❗ANNOUNCEMENT: Security Changes❗
+TorchServe now enforces token authorization enabled and model API control disabled by default. These security features are intended to address the concern of unauthorized API calls and to prevent potential malicious code from being introduced to the model server. Refer the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 # TorchServe
 
+
 ![Nightly build](https://github.com/pytorch/serve/actions/workflows/torchserve-nightly-build.yml/badge.svg)
 ![Docker Nightly build](https://github.com/pytorch/serve/actions/workflows/docker-nightly-build.yml/badge.svg)
 ![Benchmark Nightly](https://github.com/pytorch/serve/actions/workflows/benchmark_nightly.yml/badge.svg)

diff --git a/benchmarks/utils/system_under_test.py b/benchmarks/utils/system_under_test.py
@@ -116,7 +116,7 @@ def start(self):
         click.secho("*Starting local Torchserve instance...", fg="green")
 
         ts_cmd = (
-            f"torchserve --start --model-store {self.execution_params['tmp_dir']}/model_store --model-api-enabled --disable-token "
+            f"torchserve --start --model-store {self.execution_params['tmp_dir']}/model_store --enable-model-api --disable-token-auth "
             f"--workflow-store {self.execution_params['tmp_dir']}/wf_store "
             f"--ts-config {self.execution_params['tmp_dir']}/benchmark/conf/{self.execution_params['config_properties_name']} "
             f" > {self.execution_params['tmp_dir']}/benchmark/logs/model_metrics.log"
@@ -195,7 +195,7 @@ def start(self):
             f"docker run {self.execution_params['docker_runtime']} {backend_profiling} --name ts --user root -p "
             f"127.0.0.1:{inference_port}:{inference_port} -p 127.0.0.1:{management_port}:{management_port} "
             f"-v {self.execution_params['tmp_dir']}:/tmp {enable_gpu} -itd {docker_image} "
-            f'"torchserve --start --model-store /home/model-server/model-store --model-api-enabled --disable-token '
+            f'"torchserve --start --model-store /home/model-server/model-store --enable-model-api --disable-token-auth '
             f"\--workflow-store /home/model-server/wf-store "
             f"--ts-config /tmp/benchmark/conf/{self.execution_params['config_properties_name']} > "
             f'/tmp/benchmark/logs/model_metrics.log"'

diff --git a/docker/README.md b/docker/README.md
@@ -1,3 +1,6 @@
+## Security Changes
+TorchServe now enforces token authorization enabled and model API control disabled by default. Refer the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 ### Deprecation notice:
 [Dockerfile.neuron.dev](https://github.com/pytorch/serve/blob/master/docker/Dockerfile.neuron.dev) has been deprecated. Please refer to [deep learning containers](https://github.com/aws/deep-learning-containers/blob/master/available_images.md) repository for neuron torchserve containers.
 

diff --git a/docker/dockerd-entrypoint.sh b/docker/dockerd-entrypoint.sh
@@ -1,9 +1,10 @@
 #!/bin/bash
 set -e
 
+
 if [[ "$1" = "serve" ]]; then
     shift 1
-    torchserve --start --ts-config /home/model-server/config.properties --disable-token
+    torchserve --start --ts-config /home/model-server/config.properties --disable-token-auth
 else
     eval "$@"
 fi

diff --git a/docker/test_container_model_prediction.sh b/docker/test_container_model_prediction.sh
@@ -19,7 +19,7 @@ torch-model-archiver \
     --handler=/home/model-server/mnist_handler.py \
     --export-path=/home/model-server/model-store
 
-torchserve --start --ts-config=/home/model-server/config.properties --models mnist=mnist.mar --disable-token
+torchserve --start --ts-config=/home/model-server/config.properties --models mnist=mnist.mar --disable-token-auth
 EOF
 
 echo "Starting container ${CONTAINER}"

diff --git a/docs/README.md b/docs/README.md
@@ -1,3 +1,6 @@
+# ❗ANNOUNCEMENT: Security Changes❗
+TorchServe now enforces token authorization enabled and model API control disabled by default. These security features are intended to address the concern of unauthorized API calls and to prevent potential malicious code from being introduced to the model server. Refer the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 # TorchServe
 
 TorchServe is a performant, flexible and easy to use tool for serving PyTorch eager mode and torchscripted models.

diff --git a/docs/inference_api.md b/docs/inference_api.md
@@ -2,6 +2,8 @@
 
 Inference API is listening on port 8080 and only accessible from localhost by default. To change the default setting, see [TorchServe Configuration](configuration.md).
 
+For all Inference API requests, TorchServe requires the correct Inference token to be included or token authorization must be disable. For more details see [token authorization documentation](./token_authorization_api.md)
+
 The TorchServe server supports the following APIs:
 
 * [API Description](#api-description) - Gets a list of available APIs and options

diff --git a/docs/management_api.md b/docs/management_api.md
@@ -8,9 +8,14 @@ TorchServe provides the following APIs that allows you to manage models at runti
 4. [Unregister a model](#unregister-a-model)
 5. [List registered models](#list-models)
 6. [Set default version of a model](#set-default-version)
+7. [Refresh tokens for token authorization](#token-authorization-api)
 
 The Management API listens on port 8081 and is only accessible from localhost by default. To change the default setting, see [TorchServe Configuration](./configuration.md).
 
+Management API for registering and deleting models is disabled by default. Add `--enable-model-api` to command line when running TorchServe to enable the use of these APIs. For more details and ways to enable see [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
+For all Management API requests, TorchServe requires the correct Management token to be included or token authorization must be disabled. For more details see [token authorization documentation](./token_authorization_api.md)
+
 Similar to the [Inference API](inference_api.md), the Management API provides a [API description](#api-description) to describe management APIs with the OpenAPI 3.0 specification.
 
 Alternatively, if you want to use KServe, TorchServe supports both v1 and v2 API. For more details please look into this [kserve documentation](https://github.com/pytorch/serve/tree/master/kubernetes/kserve)
@@ -19,6 +24,8 @@ Alternatively, if you want to use KServe, TorchServe supports both v1 and v2 API
 
 This API follows the [ManagementAPIsService.RegisterModel](https://github.com/pytorch/serve/blob/master/frontend/server/src/main/resources/proto/management.proto) gRPC API.
 
+To use this API after TorchServe starts, model API control has to be enabled. Add `--enable-model-api` to command line when starting TorchServe to enable the use of this API. For more details see [model API control](./model_api_control.md)
+
 `POST /models`
 
 * `url` - Model archive download url. Supports the following locations:
@@ -441,6 +448,8 @@ print(customizedMetadata)
 
 This API follows the [ManagementAPIsService.UnregisterModel](https://github.com/pytorch/serve/blob/master/frontend/server/src/main/resources/proto/management.proto) gRPC API. It returns the status of a model in the ModelServer.
 
+To use this API after TorchServe starts, model API control has to be enabled. Add `--enable-model-api` to command line when starting TorchServe to enable the use of this API. For more details see [model API control](./model_api_control.md)
+
 `DELETE /models/{model_name}/{version}`
 
 Use the Unregister Model API to free up system resources by unregistering specific version of a model from TorchServe:
@@ -522,3 +531,21 @@ curl -v -X PUT http://localhost:8081/models/noop/2.0/set-default
 ```
 
 The out is OpenAPI 3.0.1 json format. You use it to generate client code, see [swagger codegen](https://swagger.io/swagger-codegen/) for detail.
+
+## Token Authorization API
+
+TorchServe now enforces token authorization by default. Check the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md).
+
+This API is used in order to generate a new key to replace either the management or inference key.
+
+Management Example:
+```
+curl localhost:8081/token?type=management -H "Authorization: Bearer {API Token}"
+```
+will replace the current management key in the key_file with a new one and will update the expiration time.
+
+Inference example:
+```
+curl localhost:8081/token?type=inference -H "Authorization: Bearer {API Token}"
+```
+will replace the current inference key in the key_file with a new one and will update the expiration time.
diff --git a/docs/model_control_mode.md → docs/model_api_control.md b/docs/model_control_mode.md → docs/model_api_control.md
@@ -1,28 +1,31 @@
-# Model Control Mode
+# Model API Control
 
-TorchServe now supports model control mode with two settings "none"(default) and "enabled"
+TorchServe now disables the use of model API (specifically registering and deleting models) by default. The use of these APIs can be enabled through command line or config.properties file.
 
-## Two ways to set Model Control
-1. Add `--model-api-enabled` to command line when running TorchServe to switch from none to enabled mode. Command line cannot be used to set mode to none, can only be used to set to enabled
-2. Add `model_api_enabled=false` or `model_api_enabled=true` to config.properties file
-    * `model_api_enabled=false` is default and prevents users from registering or deleting models once TorchServe is running
-    * `model_api_enabled=true` is not default and allows users to register and delete models using the TorchServe model load APIs
+TorchServe disables the ability to register and delete models using API calls by default once TorchServe is running. This is a security feature which addresses the concern of unintended registration and deletion of models once TorchServe has started. This is applicable in the scenario where a user may upload malicious code to the model server in the form of a model or where a user may delete a model that is being used. The default behavior prevents users from registering or deleting models once TorchServe is running. Model API control can be enabled to allow users to register and delete models using the TorchServe model load and delete APIs.
 
-Priority between cmd and config file follows the following [TorchServer standard](https://github.com/pytorch/serve/blob/c74a29e8144bc12b84196775076b0e8cf3c5a6fc/docs/configuration.md#advanced-configuration)
+## Three ways to set Model API Control
+1. Environment variable: use `TS_ENABLE_MODEL_API` and set to `true` to enable and `false` to disable model API use. Note that `enable_envvars_config=true` must be set in config.properties to use environment variables configuration
+2. Add `--enable-model-api` to command line when starting TorchServe to switch from disabled to enabled. Command line cannot be used to disable, can only be used to enable
+3. Add `enable_model_api=false` or `enable_model_api=true` to config.properties file
+    * `enable_model_api=false` is default and prevents users from registering or deleting models once TorchServe is running
+    * `enable_model_api=true` is not default and allows users to register and delete models using the TorchServe model APIs
+
+Priority follows the following [TorchServe standard](https://github.com/pytorch/serve/blob/c74a29e8144bc12b84196775076b0e8cf3c5a6fc/docs/configuration.md#advanced-configuration)
 * Example 1:
-  * Config file: `model_api_enabled=false`
+  * Config file: `enable_model_api=false`
 
-    cmd line: `torchserve --start --ncs --model-store model_store --model-api-enabled`
+    cmd line: `torchserve --start --ncs --model-store model_store --enable-model-api`
 
     Result: Model api mode enabled
 * Example 2:
-  * Config file: `model_api_enabled=true`
+  * Config file: `enable_model_api=true`
 
     cmd line: `torchserve --start --ncs --model-store model_store`
 
     Result: Mode is enabled (no way to disable api mode through cmd)
 
-## Model Control Mode Default
+## Model API Control Default
 At startup TorchServe loads only those models specified explicitly with the `--models` command-line option. After startup users will be unable to register or delete models in this mode.
 
 ### Example default
@@ -40,11 +43,11 @@ ubuntu@ip-172-31-11-32:~/serve$ curl -X POST  "http://localhost:8081/models?url=
 ```
 
 ## Model Control API Enabled
-Setting model control to `enabled` allows users to load and unload models using the model load APIs.
+Setting model API to `enabled` allows users to load and unload models using the model load APIs.
 
 ### Example using cmd line to set mode to enabled
 ```
-ubuntu@ip-172-31-11-32:~/serve$ torchserve --start --ncs --model-store model_store --models resnet-18=resnet-18.mar --ts-config config.properties --model-api-enabled
+ubuntu@ip-172-31-11-32:~/serve$ torchserve --start --ncs --model-store model_store --models resnet-18=resnet-18.mar --ts-config config.properties --enable-model-api
 
 ubuntu@ip-172-31-11-32:~/serve$ curl -X POST  "http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/squeezenet1_1.mar"
 {

diff --git a/docs/token_authorization_api.md b/docs/token_authorization_api.md
@@ -2,18 +2,19 @@
 
 TorchServe now enforces token authorization by default
 
+TorchServe enforces token authorization by default which requires the correct token to be provided when calling an API. This is a security feature which addresses the concern of unauthorized API calls. This is applicable in the scenario where an unauthorized user may try to access a running TorchServe instance. The default behavior is to enable this feature which creates a key file with the appropriate tokens to be used for API calls. Users can disable this feature to prevent token authorization from being required for API calls ([how to disable](#how-to-set-and-disable-token-authorization)), however users are warned that this will open up TorchServe to potential unauthorized API calls.
 
 ## How to set and disable Token Authorization
 * Global environment variable: use `TS_DISABLE_TOKEN_AUTHORIZATION` and set to `true` to disable and `false` to enable token authorization. Note that `enable_envvars_config=true` must be set in config.properties for global environment variables to be used
-* Command line: Command line can only be used to disable token authorization by adding the `--disable-token` flag.
+* Command line: Command line can only be used to disable token authorization by adding the `--disable-token-auth` flag.
 * Config properties file: use `disable_token_authorization` and set to `true` to disable and `false` to enable token authorization.
 
 Priority between env variables, cmd, and config file follows the following [TorchServer standard](https://github.com/pytorch/serve/blob/master/docs/configuration.md)
 
 * Example 1:
   * Config file: `disable_token_authorization=false`
 
-    cmd line: `torchserve --start --ncs --model-store model_store --disable-token`
+    cmd line: `torchserve --start --ncs --model-store model_store --disable-token-auth`
 
     Result: Token authorization disabled through command line but enabled through config file, resulting in token authorization being disabled. Command line takes precedence
 * Example 2:

diff --git a/examples/README.md b/examples/README.md
@@ -1,5 +1,8 @@
 # [Examples showcasing TorchServe Features and Integrations](#torchserve-internals)
 
+## Security Changes
+TorchServe now enforces token authorization and model API control by default. This change will impact the current examples so please check the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 ## TorchServe Internals
 
 * [Creating mar file for an eager mode model](#creating-mar-file-for-eager-mode-model)

diff --git a/frontend/server/src/main/java/org/pytorch/serve/http/TokenAuthorizationHandler.java b/frontend/server/src/main/java/org/pytorch/serve/http/TokenAuthorizationHandler.java
@@ -85,7 +85,16 @@ public static void setupToken() {
             try {
                 token = new Token();
                 if (token.generateKeyFile("token")) {
-                    logger.info("Token Authorization Enabled");
+                    String loggingMessage =
+                            "\n######\n"
+                                    + "TorchServe now enforces token authorization by default.\n"
+                                    + "This requires the correct token to be provided when calling an API.\n"
+                                    + "Key file located at "
+                                    + ConfigManager.getInstance().getModelServerHome()
+                                    + "/key_file.json\n"
+                                    + "Check token authorization documenation for information: https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md \n"
+                                    + "######\n";
+                    logger.info(loggingMessage);
                 }
             } catch (IOException e) {
                 e.printStackTrace();