Api change #2888

udaij12 · 2024-01-03T19:06:36Z

Adding plugin and associated code that generate three tokens (1 management 1 inference 1 api) that will then be mandated for all subsequent curl commands. Check the readme for complete information

… to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

namannandan

Initial review. Added a few questions about the implementation.

namannandan · 2024-01-05T01:28:21Z

frontend/server/src/main/java/org/pytorch/serve/util/ConfigManager.java

+        return true;
+    }
+
+    public List<String> parseFile(File tsTokenFile) {


parseFile is invoked by checkTokenAuthorization which in turn will be called during all inference and management API calls. Reading from file during each of these calls is going to be expensive. Could we potentially maintain the required data, i.e the tokens and expiration time in memory so that we can readily check against them. Also, we can flush this data to the file using say updateKeyFile when it is initially generated and when any of the data changes.

Changed to maintain the data in memory and no longer need to parse the key file.

namannandan · 2024-01-05T01:33:16Z

frontend/server/src/main/java/org/pytorch/serve/util/ConfigManager.java

+                if (tokenBearer == null) {
+                    throw new InvalidKeyException("NO TOKEN PROVIDED");
+                }
+                String[] arrOfStr = tokenBearer.split(" ", 2);


We could make parsing the token from the header more robust using a regex pattern.

No longer need to parse the key file.

lxning · 2024-01-05T17:36:34Z

frontend/server/src/main/java/org/pytorch/serve/ModelServer.java

+            Map<String, ModelServerEndpoint> plugins =
+                    PluginsManager.getInstance().getInferenceEndpoints();
+            if (plugins.containsKey("token")) {
+                configManager.generateKeyFile();


Should a new key be generated as long as model server is started?

Yes, when model server is started the key file is generated with the three keys.

lxning · 2024-01-05T17:42:48Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+
+@Endpoint(
+        urlPattern = "token",
+        endpointType = EndpointTypes.INFERENCE,


this should belong to management endpoint.

lxning · 2024-01-05T17:49:38Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+    private String managementToken;
+
+    @Override
+    public void doGet(Request req, Response rsp, Context ctx) throws IOException {


it seems everyone has permission to get/set the server's token by accessing this endpoint. Then, what is the security protection from this PR?

The api can only be accessed with the api key which is held by the owner thus maintaining the security of the api.

lxning · 2024-01-05T17:54:24Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+        }
+
+        public String findManagementKey() {
+            String userDirectory = System.getProperty("user.dir");


does "user.dir" work for linux, mac, and windows?

Yes it does, "user.dir" returns the current working directory regardless of the operating system.

lxning · 2024-01-05T17:55:00Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+
+        public boolean keyFile(String managementToken) throws IOException {
+            String fileData = " ";
+            String userDirectory = System.getProperty("user.dir");


lxning · 2024-01-05T19:01:13Z

frontend/server/src/main/java/org/pytorch/serve/http/api/rest/InferenceRequestHandler.java

+        ConfigManager configManager = ConfigManager.getInstance();
        if (isInferenceReq(segments)) {
            if (endpointMap.getOrDefault(segments[1], null) != null) {
+                configManager.checkTokenAuthorization(req, false);
                handleCustomEndpoint(ctx, req, segments, decoder);
            } else {
+                configManager.checkTokenAuthorization(req, true);


the implementation token authentication should not be spreaded in each endpoint. It should be handled by InvalidRequestHandler.

lxning · 2024-01-05T19:01:47Z

frontend/server/src/main/java/org/pytorch/serve/http/api/rest/ManagementRequestHandler.java

+        ConfigManager configManager = ConfigManager.getInstance();
+        configManager.checkTokenAuthorization(req, false);


lxning · 2024-01-05T19:01:57Z

...nd/server/src/main/java/org/pytorch/serve/http/api/rest/PrometheusMetricsRequestHandler.java

+        ConfigManager configManager = ConfigManager.getInstance();
+        configManager.checkTokenAuthorization(req, true);


lxning · 2024-01-05T19:02:10Z

...erver/src/main/java/org/pytorch/serve/workflow/api/http/WorkflowInferenceRequestHandler.java

+
+        ConfigManager configManager = ConfigManager.getInstance();
+        configManager.checkTokenAuthorization(req, true);


lxning · 2024-01-05T19:02:22Z

...end/server/src/main/java/org/pytorch/serve/workflow/api/http/WorkflowMgmtRequestHandler.java

+
+        ConfigManager configManager = ConfigManager.getInstance();
+        configManager.checkTokenAuthorization(req, false);


…i_change

mreso

Overall great work. Left some comments.

mreso · 2024-01-19T05:54:23Z

docs/token_authorization_api.md

+2. Torchserve will enable token authorization if the plugin is provided. In the model server home folder a file `key_file.txt` will be generated.
+    1. Example key file:
+
+    `Management Key: aadJv_R6 --- Expiration time: 2024-01-16T22:23:32.952499Z`


Better to make this a json to make is easier machine readable.

Is the key file example still accurate now?

Also why use an array to store the keys in the json file? Better to use a more structured approach like a dict:

{"management":{"key":"some_key", "expiration_time":"some_timestamp"},"inference":{}, ...etc}

mreso · 2024-01-19T05:55:30Z

docs/token_authorization_api.md

@@ -0,0 +1,39 @@
+# TorchServe token authorization API
+
+## Customer Use


How about "Basic Configuration" or "Basics"

mreso · 2024-01-19T05:57:16Z

docs/token_authorization_api.md

+    2. Inference key: Used for inference apis. Example:
+    `curl http://127.0.0.1:8080/predictions/densenet161 -T examples/image_classifier/kitten.jpg -H "Authorization: Bearer poZXAlqe"`
+    3. API key: Used for the token authorization api. Check section 4 for api use.
+    4. 3 tokens allow the owner with the most flexibility in use and enables them to adapt the tokens to their use. Owners of the server can provide users with the inference token if users should not mess with models. The owner can also provide owners with the management key if owners want users to add and remove models.


We don't need to sell it here. Its also pretty self-explanatory in my opinion.

I think it is better to have the full explanation, does not change anything.

In that case it should be part of the leading description as it does not fit into the list of keys.

mreso · 2024-01-19T05:59:32Z

docs/token_authorization_api.md

+    `curl http://localhost:8081/models/densenet161 -H "Authorization: Bearer aadJv_R6"`
+    2. Inference key: Used for inference apis. Example:
+    `curl http://127.0.0.1:8080/predictions/densenet161 -T examples/image_classifier/kitten.jpg -H "Authorization: Bearer poZXAlqe"`
+    3. API key: Used for the token authorization api. Check section 4 for api use.


nit: forall: api->API

mreso · 2024-01-19T06:03:30Z

frontend/archive/src/main/java/org/pytorch/serve/archive/model/s3/InvalidKeyException.java

@@ -0,0 +1,32 @@
+package org.pytorch.serve.archive.model;


Why is this file in model/s3?

Changed to model folder

mreso · 2024-01-19T06:25:12Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+        } else {
+            test = "{\n\t\"Error\": " + queryResponse + "\n}\n";
+        }
+        rsp.getOutputStream().write(test.getBytes(StandardCharsets.UTF_8));


Can we ensure https is enforced and send the new key through this secured channel? That way the caller would already poses the new key and can distribute to users.

mreso · 2024-01-19T06:28:56Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+    }
+
+    public String generateKey() {
+        byte[] randomBytes = new byte[6];


Why only 6 bytes here?

To keep the key length short

And we keep the key length short for saving on bandwidth?

mreso · 2024-01-19T06:30:31Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+    // 2: generates inference key and keeps other 2 the same
+    public boolean generateKeyFile(Integer keyCase) throws IOException {
+        String fileData = " ";
+        String userDirectory = System.getProperty("user.dir") + "/key_file.txt";


Better to restrict rights for this file to 600 like ssh key files

mreso · 2024-01-19T06:33:28Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+        switch (keyCase) {
+            case 1:
+                managementKey = generateKey();
+                managementExpirationTime = generateTokenExpiration(timeToExpiration);


timeToExpiration is class variable, why give it as a parameter?

mreso · 2024-01-19T06:34:06Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+                        + "\nAPI Key: "
+                        + apiKey
+                        + "\n";
+        Files.write(Paths.get("key_file.txt"), fileData.getBytes());


See comment about machine readable format

lxning

Please add test in modelservertest

lxning · 2024-01-19T06:26:53Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+    }
+
+    // checks the token provided in the http with the saved keys depening on parameters
+    public boolean checkTokenAuthorization(FullHttpRequest req, Integer keyCase) {


it's better to set keyCase as enum.

lxning · 2024-01-19T07:35:05Z

frontend/server/src/main/java/org/pytorch/serve/http/api/rest/InferenceRequestHandler.java

+        ConfigManager configManager = ConfigManager.getInstance();
+        configManager.checkTokenAuthorization(req, 2);


can you create a TokenAuthenticationHandler and add it into the handlerchain so that such call will not be spread everywhere?

lxning · 2024-01-19T07:35:33Z

frontend/server/src/main/java/org/pytorch/serve/http/api/rest/ManagementRequestHandler.java

+                if (req.toString().contains("/token")) {
+                    configManager.checkTokenAuthorization(req, 0);
+                }


lxning · 2024-01-19T07:35:44Z

frontend/server/src/main/java/org/pytorch/serve/http/api/rest/ManagementRequestHandler.java

                handleCustomEndpoint(ctx, req, segments, decoder);
            } else {
+                configManager.checkTokenAuthorization(req, 1);


lxning · 2024-01-19T07:42:03Z

docs/token_authorization_api.md

+
+## Customer Use
+1. Enable token authorization by adding the provided plugin at start using the `--plugin-path` command.
+2. Torchserve will enable token authorization if the plugin is provided. In the model server home folder a file `key_file.txt` will be generated.


Token.jar set the folder is "user.dir" which can be not model server home folder.

lxning · 2024-01-19T07:56:05Z

frontend/server/src/main/java/org/pytorch/serve/util/ConfigManager.java

+        tokenAuthorizationEnabled = true;
+    }
+
+    public void dumpKeyLogs() {


According to the doc, it seems that dumping key in log file will trigger security concern. Please get security engineer's feedback.

Reached out. As of now was told not to include log dumps for the key but will update if any changes.

lxning · 2024-01-19T08:23:05Z

frontend/server/src/main/java/org/pytorch/serve/util/ConfigManager.java

+    // 0: token api
+    // 1: management api
+    // 2: inference api
+    public void checkTokenAuthorization(FullHttpRequest req, Integer requestType)


It's better to move this func to TokenAuthenticationHandler

lxning · 2024-01-19T08:27:46Z

frontend/server/src/main/java/org/pytorch/serve/util/ConfigManager.java

+        } catch (ClassNotFoundException e) {
+            e.printStackTrace();
+        } catch (NoSuchMethodException
+                | IllegalAccessException
+                | InstantiationException
+                | InvocationTargetException e) {
+            e.printStackTrace();
+        }
+        tokenAuthorizationEnabled = true;


please add log to explicitly notify user that token is not enabled successfully.

lxning · 2024-01-19T08:31:39Z

frontend/server/src/main/java/org/pytorch/serve/util/ConfigManager.java

+    private boolean tokenAuthorizationEnabled;
+    private Class<?> tokenClass;
+    private Object tokenObject;
+    private Integer timeToExpiration = 60;


please explicitly add unit in var name. eg. timeToExpirationSec.

lxning · 2024-01-19T08:34:36Z

frontend/server/src/main/java/org/pytorch/serve/ModelServer.java

+            Map<String, ModelServerEndpoint> plugins =
+                    PluginsManager.getInstance().getManagementEndpoints();
+            if (plugins.containsKey("token")) {
+                configManager.setupTokenClass();
+            }


Can you move this plugin check in PluginManager?

lxning · 2024-02-13T07:29:04Z

docs/token_authorization_api.md

+
+## Customization
+Torchserve offers various ways to customize the token authorization to allow owners to reach the desired result.
+1. Time to expiration is set to default at 60 minutes but can be changed in the config.properties by adding `token_expiration`. Ex:`token_expiration=30`


Please specify the unit is min in variable (eg, token_expiration_min). It's better to give an example of setting expiration.

Noted will change the variable to 'token_expiration_min'. The example on how to set the time is in the docs.

lxning · 2024-02-13T07:46:01Z

frontend/server/src/main/java/org/pytorch/serve/http/TokenAuthorizationHandler.java

+            String[] segments)
+            throws ModelException, DownloadArchiveException, WorkflowException,
+                    WorkerInitializationException {
+        ConfigManager configManager = ConfigManager.getInstance();


"if (tokenEnabled" should happen at the beginning to skip immediately if it is disabled.

lxning · 2024-02-13T07:57:53Z

test/pytest/test_token_authorization.py

+
+
+# Set up token plugin
+def get_plugin_jar():


You can create a jar file in test/resource, and then remove this function to reduce the time for regression test.

I left the setup in the tests in case future changes to the plugin are made, makes it easier for new tests and changes to be made in the future.

lxning · 2024-02-13T08:03:31Z

ts/model_server.py

+                try:
+                    os.remove(os.getcwd() + "/key_file.json")
+                except FileNotFoundError:
+                    print("Token authorization not enabled")


You can print delete the file if the file exists.

lxning · 2024-02-13T08:05:07Z

test/pytest/test_utils.py

+    if plugin_folder:
+        cmd.extend(["--plugins-path", plugin_folder])


did you extend torchserve cmd's option?

I added plugin option to be passed in as a parameter to the torchserve start command in the test util.

mreso

Left some comments. Especially, the silently failing of the token authentication should be changed into a fatal error.

mreso · 2024-02-14T21:30:02Z

frontend/server/src/main/java/org/pytorch/serve/http/TokenAuthorizationHandler.java

-            } else {
-                checkTokenAuthorization(req, "management");
+        if (tokenEnabled) {
+            ConfigManager configManager = ConfigManager.getInstance();


Can you clarify what this line does?
If we're leveraging a side effect of this call we should make it either explicit or at least leave a comment.

No use so removed it.

mreso · 2024-02-14T21:46:22Z

ts/model_server.py

@@ -51,7 +51,7 @@ def start() -> None:
                try:
                    os.remove(os.getcwd() + "/key_file.json")
                except FileNotFoundError:
-                    print("Token authorization not enabled")
+                    print("Delete key file if it exists")


For a user this message will be very cryptic as he has no context on this. Also the file should not be there on FileNotFoundError. Better to check silently for the file and delete if present.

The server should delete the file if present but that is assuming that the owner doesnt change directories from where the file was created in the first place. Can change the print output to "If token authorization was enabled, delete key file if still present"

If the user moved the files this should not be our concern. In that case we should silently move on. Not seeing the value in the print here. If you leave it its better to use logging.debug instead of print.

mreso · 2024-02-14T21:49:53Z

docs/token_authorization_api.md

+    2. Inference key: Used for inference apis. Example:
+    `curl http://127.0.0.1:8080/predictions/densenet161 -T examples/image_classifier/kitten.jpg -H "Authorization: Bearer poZXAlqe"`
+    3. API key: Used for the token authorization api. Check section 4 for api use.
+    4. 3 tokens allow the owner with the most flexibility in use and enables them to adapt the tokens to their use. Owners of the server can provide users with the inference token if users should not mess with models. The owner can also provide owners with the management key if owners want users to add and remove models.


In that case it should be part of the leading description as it does not fit into the list of keys.

mreso · 2024-02-14T21:56:45Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+    }
+
+    public String generateKey() {
+        byte[] randomBytes = new byte[6];


And we keep the key length short for saving on bandwidth?

mreso · 2024-02-14T21:58:11Z

docs/token_authorization_api.md

+2. Torchserve will enable token authorization if the plugin is provided. In the model server home folder a file `key_file.txt` will be generated.
+    1. Example key file:
+
+    `Management Key: aadJv_R6 --- Expiration time: 2024-01-16T22:23:32.952499Z`


Is the key file example still accurate now?

mreso · 2024-02-14T22:08:07Z

docs/token_authorization_api.md

+2. Torchserve will enable token authorization if the plugin is provided. In the model server home folder a file `key_file.txt` will be generated.
+    1. Example key file:
+
+    `Management Key: aadJv_R6 --- Expiration time: 2024-01-16T22:23:32.952499Z`


Also why use an array to store the keys in the json file? Better to use a more structured approach like a dict:

{"management":{"key":"some_key", "expiration_time":"some_timestamp"},"inference":{}, ...etc}

mreso · 2024-02-14T22:10:35Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+
+        if (!setFilePermissions()) {
+            try {
+                Files.delete(Paths.get("key_file.txt"));


Filename has changed. Better to store the name in a constant somewhere to provide central edit access.

Noted, but will still be needed to be changed in multiple places. Will make a variable for the plugin but the frontend will not use it.

mreso · 2024-02-14T22:15:19Z

plugins/endpoints/src/main/java/org/pytorch/serve/plugins/endpoint/Token.java

+            } catch (IOException e) {
+                return false;
+            }
+            return false;


Looks like we're silently (only some messages in the log) failing if something goes wrong with the token setup. We should not let torchserve start at all if user requests token auth and this happens as we leave the server (unknowingly) unprotected otherwise.

mreso

Left some comment especially regarding testing. Let me know if you have questions.

mreso · 2024-02-16T18:44:22Z

ts/model_server.py

@@ -51,7 +51,7 @@ def start() -> None:
                try:
                    os.remove(os.getcwd() + "/key_file.json")
                except FileNotFoundError:
-                    print("Token authorization not enabled")
+                    print("Delete key file if it exists")


If the user moved the files this should not be our concern. In that case we should silently move on. Not seeing the value in the print here. If you leave it its better to use logging.debug instead of print.

mreso · 2024-02-16T18:47:22Z

docs/token_authorization_api.md

+
+```python
+    [
+        {


I would prefer a dict instead of a list as the main structure. Otherwise you have to look into the object to see what key you have. (see test where you check if "managemen_key in iterm" etc). Why not {"management": {"key":"..."} etc}? In that case you can easily access the values with

json_content.get("management", {}).get("key", "NOT_PRESENT")

mreso · 2024-02-16T18:48:28Z

test/pytest/test_token_authorization.py

+    )
+    time.sleep(5)
+    print("register reponse")
+    print(response.text)


Better to remove the print statements or use logging here.

mreso · 2024-02-16T18:50:02Z

test/pytest/test_token_authorization.py

+    print("register reponse")
+    print(response.text)
+
+    result = subprocess.run(


Why do we use cat + stdout here instead of Path(file).read_text() or similar?

mreso · 2024-02-16T18:53:01Z

test/pytest/test_token_authorization.py

+    PLUGIN_STORE = os.path.join(ROOT_DIR, "plugins-path")
+
+    test_utils.start_torchserve(no_config_snapshots=True, plugin_folder=PLUGIN_STORE)
+    time.sleep(10)


No need to wait 10 seconds as start_torchserve already waits for torchserve to start. These fixed waits just slows down our tests.

mreso · 2024-02-16T18:53:58Z

test/pytest/test_token_authorization.py

+    response = requests.post(
+        "http://localhost:8081/models", params=params, headers=header
+    )
+    time.sleep(5)


What is this 5 seconds wait for?

mreso · 2024-02-16T18:56:03Z

test/pytest/test_token_authorization.py

+# Test describe model API with token enabled
+def test_managament_api_with_token():
+    test_utils.stop_torchserve()
+    setup_torchserve()


better to make this a module wide fixture that starts on module load and stops torchserve on cleanup. You can not rely on this test always being run.

mreso · 2024-02-16T18:56:42Z

test/pytest/test_token_authorization.py

+    header = {"Authorization": "Bearer abcd1234"}
+
+    response = requests.get(f"http://localhost:8081/models/resnet18", headers=header)
+    time.sleep(5)


No need to wait here. requests already does this for us.

mreso · 2024-02-16T18:58:59Z

test/pytest/test_token_authorization.py

+    key = read_key_file("management")
+    header = {"Authorization": f"Bearer {key}"}
+    print(key)
+    response = requests.get("http://localhost:8081/models/resnet18", headers=header)


Should be part of the fixture as well.

See this example for module scope fixtures:

serve/test/pytest/test_example_micro_batching.py

Line 152 in cd52683

def register_model(mar_file_path, model_store, torchserve):

You should still test management api but this should be independent from setup for the other tests.

mreso · 2024-02-16T19:03:01Z

test/pytest/test_token_authorization.py

+
+    assert management_key != read_key_file("management"), "Key file not updated"
+    assert response.status_code == 200, "Token check failed"
+    test_utils.stop_torchserve()


See comment about module scope fixture for this.

Would be good to actually test token expiration as well. Just setup expiration to a low value and run two requests while waiting in-between for token to expire.

mreso

Left some comments

mreso · 2024-02-16T22:47:42Z

test/pytest/test_token_authorization.py

+
+
+@pytest.fixture(scope="module")
+def setup_torchserve_expiration():


This potentially tries to start 2 torchserves at a time. Better to move the second one into its own module.

mreso · 2024-02-16T22:49:31Z

test/pytest/test_token_authorization.py

+    response = requests.get("http://localhost:8081/models/resnet18", headers=header)
+    assert response.status_code == 200, "Token check failed"
+
+    time.sleep(60)


This adds one minute to the total execution of our tests which is pretty inefficient. Better to wait for less. If we can not give fractional minutes to the frontend we should use seconds instead of minutes.

mreso · 2024-02-16T22:52:21Z

ts/model_server.py

-                    os.remove(os.getcwd() + "/key_file.json")
-                except FileNotFoundError:
-                    print("Delete key file if it exists")
+                os.remove(os.getcwd() + "/key_file.json")


This fails now if user moved the file. Better use: pathlib.Path(filename).unlink(missing_ok=True)

mreso · 2024-02-16T22:55:46Z

test/pytest/test_token_authorization.py

+        )
+    os.chdir(REPO_ROOT)
+    result = subprocess.run(
+        f"python ts_scripts/install_from_source",


Don't like to install_from_source in a test. Better to assume that we're testing the currently installed version (Otherwise each test would have to make sure that the newest version is installed etc.)

mreso · 2024-02-16T22:56:01Z

test/pytest/test_token_authorization.py

+    plugin_folder = os.path.join(REPO_ROOT, "plugins")
+    os.makedirs(new_folder_path, exist_ok=True)
+    os.chdir(plugin_folder)
+    subprocess.run(["./gradlew", "formatJava"])


Is the format necessary here?

mreso · 2024-02-16T22:58:32Z

test/pytest/test_token_authorization.py

+    os.makedirs(new_folder_path, exist_ok=True)
+    os.chdir(plugin_folder)
+    subprocess.run(["./gradlew", "formatJava"])
+    result = subprocess.run(["./gradlew", "build"])


building plugin is okay

mreso · 2024-02-16T22:59:12Z

test/pytest/test_token_authorization.py

+    jar_path = os.path.join(plugin_folder, "endpoints/build/libs")
+    jar_file = [file for file in os.listdir(jar_path) if file.endswith(".jar")]
+    if jar_file:
+        shutil.move(


Probably better to just copy as otherwise users have to recompile plugin after they run a test.

mreso · 2024-02-16T23:37:01Z

test/pytest/test_token_authorization.py

+    MODEL_STORE = os.path.join(ROOT_DIR, "model_store/")
+    PLUGIN_STORE = os.path.join(ROOT_DIR, "plugins-path")
+
+    test_utils.start_torchserve(no_config_snapshots=True, plugin_folder=PLUGIN_STORE)


Better to set gen_mar=False to avoid unnecessary recreation.

mreso · 2024-02-16T23:57:03Z

test/pytest/test_token_authorization.py

+
+    params = (
+        ("model_name", "resnet18"),
+        ("url", "resnet-18.mar"),


Better use mnist model instead of resnet18 to keep lower test runtime on cpu

…i_change

mreso

LGTM now

The requested changes were addressed.

* token authorization update * update token * token authorization plugin test * fix format * add key file generation at default * fix format * updated token plugin * fixed file delete * fixed imports * added custom expection * fix format * token handler * fix doc * fixed handler * added integration tests * Added integration tests * updating token auth * small changes to token auth * fixing changes * changed keyfile to dictionary and updated readme and tests * remove comments * changes to tests * added config file * reduce time for expiration test * change test to mnist * removing install from src * final test change * Fix spellcheck --------- Co-authored-by: Ubuntu <ubuntu@ip-172-31-11-32.us-west-2.compute.internal> Co-authored-by: Matthias Reso <13337103+mreso@users.noreply.github.com>

Ubuntu and others added 7 commits December 5, 2023 18:10

token authorization update

4ddb74e

update token

1304ca3

token authorization plugin test

fde5f4d

fix format

8ec3c40

add key file generation at default

d50beae

fix format

d5c25e6

Merge branch 'master' into api_change # Please enter a commit message…

1b35372

… to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

namannandan previously requested changes Jan 5, 2024

View reviewed changes

lxning requested a review from mreso January 5, 2024 17:27

lxning reviewed Jan 5, 2024

View reviewed changes

udaij12 added 7 commits January 17, 2024 13:02

Merge branch 'master' into api_change

0010c86

updated token plugin

62cadc8

Merge branch 'api_change' of https://github.com/pytorch/serve into ap…

51f5889

…i_change

fixed file delete

129b652

fixed imports

cecc086

added custom expection

6876781

fix format

4848114

mreso requested changes Jan 19, 2024

View reviewed changes

lxning reviewed Jan 19, 2024

View reviewed changes

udaij12 added 6 commits January 30, 2024 14:20

token handler

fa32c28

fix doc

c5c0f77

merging master

84343c4

fixed handler

3b7fe29

added integration tests

0d7afe3

Added integration tests

13caa27

udaij12 force-pushed the api_change branch from f4e7366 to 13caa27 Compare February 5, 2024 19:50

lxning reviewed Feb 13, 2024

View reviewed changes

updating token auth

a49acf5

mreso requested changes Feb 14, 2024

View reviewed changes

small changes to token auth

6b329b2

udaij12 added 3 commits February 15, 2024 15:12

fixing changes

0690e01

changed keyfile to dictionary and updated readme and tests

eb37eff

remove comments

71bc7f5

mreso reviewed Feb 16, 2024

View reviewed changes

udaij12 and others added 3 commits February 16, 2024 13:22

changes to tests

3e18230

added config file

f69c632

Merge branch 'master' into api_change

599c635

mreso reviewed Feb 16, 2024

View reviewed changes

udaij12 added 5 commits February 16, 2024 16:21

reduce time for expiration test

55cedd5

Merge branch 'api_change' of https://github.com/pytorch/serve into ap…

4c7080d

…i_change

change test to mnist

54841d9

removing install from src

4e1c1ea

final test change

698b95a

udaij12 marked this pull request as ready for review February 17, 2024 04:59

Fix spellcheck

68b3b04

mreso approved these changes Feb 19, 2024

View reviewed changes

mreso enabled auto-merge February 19, 2024 22:20

mreso requested a review from namannandan February 20, 2024 17:21

mreso added this pull request to the merge queue Feb 20, 2024

Merged via the queue into master with commit 18057f3 Feb 20, 2024
15 checks passed

mreso mentioned this pull request Feb 21, 2024

support inf2 neuronx transformer continuous batching #2803

Merged

8 tasks

udaij12 deleted the api_change branch February 21, 2024 19:42

chauhang added this to the v0.10.0 milestone Feb 27, 2024

udaij12 mentioned this pull request Mar 6, 2024

token based management url #2612

Closed

		ConfigManager configManager = ConfigManager.getInstance();
		configManager.checkTokenAuthorization(req, false);

		@@ -0,0 +1,39 @@
		# TorchServe token authorization API

		## Customer Use

		if plugin_folder:
		cmd.extend(["--plugins-path", plugin_folder])



		@pytest.fixture(scope="module")
		def setup_torchserve_expiration():

Api change #2888

Api change #2888

Conversation

udaij12 commented Jan 3, 2024 • edited Loading

namannandan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mreso left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lxning left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mreso left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

udaij12 commented Jan 3, 2024 •

edited

Loading