-
Install Kind (Kubernetes in Docker) I am using kind here in place of minikube because it will enable me install kserve as a standalone.
-
Install kubectl
It is required that you have Anaconda(python) and tensorflow installed.
-
python model.py
To train the model and save it in SavedModel format for the tensorflow serving service.
-
docker build -t tfserving_classifier:v01 .
-
REST is a communication “protocol” used by web applications. It defines a communication style on how clients communicate with web services. REST clients communicate with the server using the standard HTTP methods like GET, POST, DELETE, etc. The payloads of the requests are mostly encoded in JSON format.
-
gRPC on the other hand is a communication protocol initially developed at Google. The standard data format used with gRPC is called the protocol buffer. gRPC provides low- latency communication and smaller payloads than REST and is preferred when working with extremely large files during inference.
-
docker run -it --rm -p 8501:8501 tfserving_classifier:v01
-
python predict.py
http://{HOST}:{PORT}/v1/models/{MODEL_NAME}:{VERB}
To load images from your local into the cluster -
kind load docker-image tfserving_classifier:v01
-
kubectl apply -f kubeconfig/deployment.yaml
-
kubectl apply -f kubeconfig/service.yaml
-
kubectl port-forward svc/tfserving-classifier 8080:80
-
Install Kserve standalone (requires kind and kubectl)
-
Run your first InferenceService
kubectl get svc istio-ingressgateway -n istio-system
kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80
INGRESS_HOST="localhost"
INGRESS_PORT="8080"
DOMAIN="example.com"
NAMESPACE="kserve-test"
SERVICE="sklearn-iris"
SERVICE_HOSTNAME="${SERVICE}.${NAMESPACE}.${DOMAIN}"
curl -v -H "Host: ${SERVICE_HOSTNAME}" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/sklearn-iris:predict -d @./iris-input.json
Download the model
wget https://github.com/alexeygrigorev/mlbookcamp-code/releases/download/chapter7-model/xception_v4_large_08_0.894.h5
Convert it to saved_model:
python convert.py
Instead of using an s3 bucket or Google cloud storage
cd clothing-model/
tar -cvf artifacts.tar 1/
gzip < artifacts.tar > artifacts.tgz
Host the file on your local machine
python -m http.server
kubectl apply -f tensorflow.yaml
curl -v -H "Host: clothes.default.example.com" http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/clothes:predict -d $INPUT_PATH
Create custom image transformer link
-
Create a python script with ImageTransformer Class inheriting from kserve.Model
-
Build Transformer docker image and push to dockerhub
docker build -t nerdward/image-transformer:v02 .
docker build -t <hub-user>/<repo-name>[:<tag>]
docker push nerdward/image-transformer:v02
docker push <hub-user>/<repo-name>:<tag>
- Create the InferenceService and Apply.
kubectl apply -f tensorflow.yaml
- Run a prediction.