A guide for loading models in TorchServe

[agunapal]

Description

This is a guide for loading models in TorchServe

Main motivation was from the note by Hamel Husain https://hamel.dev/notes/serving/torchserve/
Fixes #(issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

N/A

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[msaroufim]

On whether handler has an initialize method i think it's more nuanced, someone could also be calling super()
The model type node makes a lot of sense
in the self contained package node i didn't really understand what you were trying to say because serialized file node intersects with torchscript and onnx. The righthand bottom node should be is your model large and do you care about model initialization time

[codecov]

Codecov Report

Merging #2592 (d857eae) into master (f1c22a5) will not change coverage.
The diff coverage is n/a.

❗ Current head d857eae differs from pull request most recent head f9a53e8. Consider uploading reports for the commit f9a53e8 to get more accurate results

@@           Coverage Diff           @@
##           master    #2592   +/-   ##
=======================================
  Coverage   70.87%   70.87%           
=======================================
  Files          83       83           
  Lines        3839     3839           
  Branches       58       58           
=======================================
  Hits         2721     2721           
  Misses       1114     1114           
  Partials        4        4           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[agunapal]

@msaroufim Thanks. Agree with both. Changed both

[msaroufim]

top part looks much clearer

The first node in the photo i think deciision should be do you want a self contained artifact or do you want fast model loads

Also in the bottom row theres arrows going both ways with no labels

also there's too much text in the large model thing

[agunapal]

top part looks much clearer

The first node in the photo i think deciision should be do you want a self contained artifact or do you want fast model loads

Also in the bottom row theres arrows going both ways with no labels

also there's too much text in the large model thing

Got it. Let me think about how to address it

[agunapal]

@msaroufim Changed the bottom part a bit. If the model is large, we want users to not package the model at all. So, I think that should be the starting point