docs/samples/client/kfserving_sdk_sample.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sample for KFServing SDK "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a sample for KFServing SDK. \n",
    "\n",
    "The notebook shows how to use KFServing SDK to create, get, rollout_canary, promote and delete InferenceService."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from kubernetes import client\n",
    "\n",
    "from kfserving import KFServingClient\n",
    "from kfserving import constants\n",
    "from kfserving import utils\n",
    "from kfserving import V1alpha2EndpointSpec\n",
    "from kfserving import V1alpha2PredictorSpec\n",
    "from kfserving import V1alpha2TensorflowSpec\n",
    "from kfserving import V1alpha2InferenceServiceSpec\n",
    "from kfserving import V1alpha2InferenceService\n",
    "from kubernetes.client import V1ResourceRequirements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define namespace where InferenceService needs to be deployed to. If not specified, below function defines namespace to the current one where SDK is running in the cluster, otherwise it will deploy to default namespace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "namespace = utils.get_default_target_namespace()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define InferenceService"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Firstly define default endpoint spec, and then define the inferenceservice basic on the endpoint spec."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "api_version = constants.KFSERVING_GROUP + '/' + constants.KFSERVING_VERSION\n",
    "default_endpoint_spec = V1alpha2EndpointSpec(\n",
    "                          predictor=V1alpha2PredictorSpec(\n",
    "                            tensorflow=V1alpha2TensorflowSpec(\n",
    "                              storage_uri='gs://kfserving-samples/models/tensorflow/flowers',\n",
    "                              resources=V1ResourceRequirements(\n",
    "                                  requests={'cpu':'100m','memory':'1Gi'},\n",
    "                                  limits={'cpu':'100m', 'memory':'1Gi'}))))\n",
    "    \n",
    "isvc = V1alpha2InferenceService(api_version=api_version,\n",
    "                          kind=constants.KFSERVING_KIND,\n",
    "                          metadata=client.V1ObjectMeta(\n",
    "                              name='flower-sample', namespace=namespace),\n",
    "                          spec=V1alpha2InferenceServiceSpec(default=default_endpoint_spec))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create InferenceService"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Call KFServingClient to create InferenceService."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "KFServing = KFServingClient()\n",
    "KFServing.create(isvc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check the InferenceService"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "KFServing.get('flower-sample', namespace=namespace, watch=True, timeout_seconds=120)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add Canary to InferenceService"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Firstly define canary endpoint spec, and then rollout 10% traffic to the canary version, watch the rollout process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "canary_endpoint_spec = V1alpha2EndpointSpec(\n",
    "                         predictor=V1alpha2PredictorSpec(\n",
    "                           tensorflow=V1alpha2TensorflowSpec(\n",
    "                             storage_uri='gs://kfserving-samples/models/tensorflow/flowers-2',\n",
    "                             resources=V1ResourceRequirements(\n",
    "                                 requests={'cpu':'100m','memory':'1Gi'},\n",
    "                                 limits={'cpu':'100m', 'memory':'1Gi'}))))\n",
    "\n",
    "KFServing.rollout_canary('flower-sample', canary=canary_endpoint_spec, percent=10,\n",
    "                         namespace=namespace, watch=True, timeout_seconds=120)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rollout more traffic to canary of the InferenceService"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rollout traffice percent to 50% to canary version."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "KFServing.rollout_canary('flower-sample', percent=50, namespace=namespace,\n",
    "                         watch=True, timeout_seconds=120)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Promote Canary to Default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "KFServing.promote('flower-sample', namespace=namespace, watch=True, timeout_seconds=120)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Delete the InferenceService"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "KFServing.delete('flower-sample', namespace=namespace)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}