Skip to content

Commit

Permalink
Add ModelRegistry custom storage intializer example
Browse files Browse the repository at this point in the history
Signed-off-by: Andrea Lamparelli <a.lamparelli95@gmail.com>
  • Loading branch information
lampajr committed Feb 26, 2024
1 parent 84fd59e commit d4492a5
Showing 1 changed file with 58 additions and 4 deletions.
62 changes: 58 additions & 4 deletions docs/modelserving/storage/storagecontainers.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,45 @@ In a _ClusterStorageContainer_ spec, you can specify container resource requests

If a storage URI is supported by two or more _ClusterStorageContainer_ CRs, there is no guarantee which one will be used. **Please make sure that the URI format is only supported by one ClusterStorageContainer CR**.

## Custom Protocol Example

If you would like to use a custom protocol `model-registry://`, for example, you can create a custom image and add a new ClusterStorageContainer CR like this:
If you would like to use a custom protocol `model-registry://`, for example, you can create a custom image and add a new `ClusterStorageContainer` CR to make it available to KServe.

```yaml
### Create the `Custom Storage Initializer` Image

The first step is to create a custom container image that will be injected into the KServe deployment, as init container, and that will be in charge to download the model.

The only requirement is that the `Entrypoint` of this container image should take (and properly manage) 2 positional arguments:
1. __Source URI__: identifies the `storageUri` set in the `InferenceService`
2. __Deestination Path__: the location where the model should be stored, e.g., `/mnt/models`

!!! note
KServe controller will take care of properly injecting your container image and invoking it with those proper arguments.

A more concrete example can be found [here](https://github.com/lampajr/model-registry-storage-initializer), where the storage initializer query an existing `model registry` service in order to retrieve the original location of the model that the user requested to deploy.

### Create the `ClusterStorageContainer` CR

Once the Custom Storage Initializer image is ready, you just need to create a new `ClusterStorageContainer` CR to make it available in the cluster. You just need to provide 2 essential information:
1. The _container spec definition_, this is strictly dependent on your own custom storage initializer image.
2. The _supported uri formats_ for which your custom storage initializer should be injected, in this case just `model-registry://`.

=== "kubectl"
```bash
kubectl apply -f - <<EOF
apiVersion: "serving.kserve.io/v1alpha1"
kind: ClusterStorageContainer
metadata:
name: abc
spec:
container:
name: storage-initializer
image: abc/custom-storage-initializer:latest
image: abc/model-registry-storage-initializer:latest
env:
- name: MODEL_REGISTRY_BASE_URL
value: "$MODEL_REGISTRY_SERVICE.model-registry.svc.cluster.local:$MODEL_REGISTRY_REST_PORT"
- name: MODEL_REGISTRY_SCHEME
value: "http"
resources:
requests:
memory: 100Mi
Expand All @@ -57,8 +84,35 @@ spec:
cpu: "1"
supportedUriFormats:
- prefix: model-registry://
EOF
```

### Deploy the Model with `InferenceService`

Create the `InferenceService` with the `model-registry` specific URI format.

=== "kubectl"
```bash
kubectl apply -n kserve-test -f - <<EOF
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
name: "iris-model"
spec:
predictor:
model:
modelFormat:
name: sklearn
storageUri: "model-registry://iris/v1"
EOF
```

!!! note
The only assumption here is that the ML model you are going to deploy has been already registered in the Model Registry, more information can be found in the [kubeflow/model-registry](https://github.com/kubeflow/model-registry) repository.

In this specific example the `model-registry://iris/v1` model is referring to a registered model pointing to `gs://kfserving-examples/models/sklearn/1.0/model`. The crucial point here is that this information needs to be provided just during the registration process, whereas during every deployment action you just need to provide the `model-registry` specific URI that identifies that model (in this case `model-registry://${MODEL_NAME}/${MODEL_VERSION}`).

## Spec Attributes

Spec attributes are in [API Reference](/website/reference/api/#serving.kserve.io/v1alpha1.ClusterStorageContainer) doc.
Spec attributes are in [API Reference](/website/reference/api/#serving.kserve.io/v1alpha1.ClusterStorageContainer) doc.

0 comments on commit d4492a5

Please sign in to comment.