README.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# End-to-end MLOps with Seldon Core and Jenkins X\n",
    "\n",
    "This tutorial provides an end-to-end hands-on tutorial that shows you how to build your own re-usable MLOps pipelines leveraging Seldon Core and Jenkins X. \n",
    "\n",
    "By the end of this tutorial, you will be able to:\n",
    "\n",
    "* Quickly spin up a project based on the MLOps quickstart\n",
    "* Leverage Seldon's prepackaged model servers\n",
    "* Leverage Seldon's language wrapper for custom model servers\n",
    "* Run unit tests using Jenkins X \n",
    "* Run end-to-end tests for your model with KIND (Kubernetes in Docker)\n",
    "* Promote your model as a Jenkins X application across multiple (staging / prod) environments\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intuitive explanation\n",
    "\n",
    "In this project, we will be building an MLOps workflow to deploy your production machine learning models by buiding a re-usable pre-packaged model server through CI, and then deploying individual models using CD.\n",
    "\n",
    "![](images/jenkins-x-full-diagram.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requirements\n",
    "\n",
    "* A Kubernetes cluster running v1.13+ (this was run using GKE)\n",
    "* The [jx CLI](https://github.com/jenkins-x/jx/) version 2.0.916\n",
    "* Jenkins-X installed in your cluster (you can set it up with the [jx boot tutorial](https://jenkins-x.io/docs/getting-started/setup/boot/))\n",
    "* Seldon Core [v0.5.0 installed]() in your cluster\n",
    "\n",
    "Once you set everything up, we'll be ready to kick off 🚀"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setting up repo\n",
    "\n",
    "Now we want to start setting up our repo. For this we'll just leverage the MLOps quickstart by running:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jx create quickstart --org \"SeldonIO\" --project-name \"mlops-deployment\" --filter \"mlops-quickstart\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What this command does is basically the following:\n",
    "\n",
    "* Find the quickstarts in the organisation \"SeldonIO\"\n",
    "* Find the quickstart named \"mlops-quickstart\"\n",
    "* Build the project with name \"mlops-deployment\"\n",
    "\n",
    "You now have a repo where you'll be able to leverage [Seldon's pre-packaged model servers](https://docs.seldon.io/projects/seldon-core/en/latest/servers/overview.html).\n",
    "\n",
    "Let's have a look at what was created:\n",
    "\n",
    "* `jenkins-x.yml` - File specifying the CI / CD steps \n",
    "* `Makefile` - Commands to build and test model\n",
    "* `README.(md|ipynb)` - This file!\n",
    "* `VERSION` - A file containing the version which is updated upon each release\n",
    "* `charts/` \n",
    "    * `mlops-server/` - Folder containing helm charts to deploy your model\n",
    "    * `preview/` - Folder containing reference to helm charts to create preview environments\n",
    "* `integration/`\n",
    "    * `kind_test_all.sh` - File that spins up KIND cluster and runs your model\n",
    "    * `test_e2e_model_server.py` - End-to-end tests to run on your model\n",
    "    * `requirements-dev.py` - Requirements for your end to end tests\n",
    "* `src/` \n",
    "    * `model.joblib` - Sample trained model that is deployed when importing project\n",
    "    * `train_model.py` - Sample code to train your model and output a model.pickle\n",
    "    * `test_model.py` - Sample code to unit test your model \n",
    "    * `requirements.txt` - Example requirements file with supported versions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's train a model locally\n",
    "\n",
    "First we will train a machine learning model, which will help us classify news across multiple categories.\n",
    "\n",
    "### Install dependencies \n",
    "\n",
    "We will need the following dependencies in order to run the Python code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# You need the right versions for your model server:\r\n",
      "# Model servers: https://docs.seldon.io/projects/seldon-core/en/latest/servers/overview.html\r\n",
      "\r\n",
      "# For SKLearn you need a pickle and the following:\r\n",
      "scikit-learn==0.20.3 # See https://docs.seldon.io/projects/seldon-core/en/latest/servers/sklearn.html\r\n",
      "joblib==0.13.2\r\n",
      "\r\n",
      "# For XGBoost you need v 0.82 and an xgboost export (not a pickle)\r\n",
      "#xgboost==0.82\r\n",
      "\r\n",
      "# For MLFlow you need the following, and a link to the built model:\r\n",
      "#mlflow==1.1.0\r\n",
      "#pandas==0.25\r\n",
      "\r\n",
      "# For tensorflow, any models supported by tensorflow serving (less than v2.0)\r\n"
     ]
    }
   ],
   "source": [
    "!cat src/requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now install the dependencies using the make command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!make install_dev"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the ML data\n",
    "\n",
    "Now that we have all the dependencies we can proceed to download the data.\n",
    "\n",
    "We will download the news stories dataset, and we'll be attempting to classify across the four classes below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "From: sd345@city.ac.uk (Michael Collier)\n",
      "Subject: Converting images to HP LaserJet III?\n",
      "Nntp-Posting-Host: hampton\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import fetch_20newsgroups\n",
    "categories = ['alt.atheism', 'soc.religion.christian',\n",
    "              'comp.graphics', 'sci.med']\n",
    "\n",
    "twenty_train = fetch_20newsgroups(\n",
    "    subset='train', categories=categories, shuffle=True, random_state=42)\n",
    "\n",
    "twenty_test = fetch_20newsgroups(\n",
    "    subset='test', categories=categories, shuffle=True, random_state=42)\n",
    "\n",
    "# Printing the top 3 newstories\n",
    "print(\"\\n\".join(twenty_train.data[0].split(\"\\n\")[:3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train a model\n",
    "\n",
    "Now that we've downloaded the data, we can train the ML model using a simple pipeline with basic text pre-processors and a Multiclass naive bayes classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Pipeline(memory=None,\n",
       "         steps=[('vect',\n",
       "                 CountVectorizer(analyzer='word', binary=False,\n",
       "                                 decode_error='strict',\n",
       "                                 dtype=<class 'numpy.int64'>, encoding='utf-8',\n",
       "                                 input='content', lowercase=True, max_df=1.0,\n",
       "                                 max_features=None, min_df=1,\n",
       "                                 ngram_range=(1, 1), preprocessor=None,\n",
       "                                 stop_words=None, strip_accents=None,\n",
       "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "                                 tokenizer=None, vocabulary=None)),\n",
       "                ('tfidf',\n",
       "                 TfidfTransformer(norm='l2', smooth_idf=True,\n",
       "                                  sublinear_tf=False, use_idf=True)),\n",
       "                ('clf',\n",
       "                 MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True))],\n",
       "         verbose=False)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "\n",
    "text_clf = Pipeline([\n",
    "    ('vect', CountVectorizer()),\n",
    "    ('tfidf', TfidfTransformer()),\n",
    "    ('clf', MultinomialNB()),\n",
    "])\n",
    "\n",
    "text_clf.fit(twenty_train.data, twenty_train.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test single prediction\n",
    "\n",
    "Now that we've trained our model we can use it to predict from un-seen data.\n",
    "\n",
    "We can see below that the model is able to predict the first datapoint in the dataset correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTENT:\n",
      "Subject: Re: HELP for Kidney Stones ..............\n",
      "Organization: The Avant-Garde of the Now, Ltd.\n",
      "Lines: 12\n",
      "NNTP-Posting-Host: ucsd.edu\n",
      "\n",
      "As I recall from my bout with kidney stones, there isn't \n",
      "\n",
      "-----------\n",
      "\n",
      "PREDICTED CLASS: comp.graphics\n"
     ]
    }
   ],
   "source": [
    "idx = 0\n",
    "print(f\"CONTENT:{twenty_test.data[idx][35:230]}\\n\\n-----------\\n\")\n",
    "print(f\"PREDICTED CLASS: {categories[twenty_test.target[idx]]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Print accuracy\n",
    "\n",
    "We can print the accuracy of the model by running the test data and counting the number of correct classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.83\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "predicted = text_clf.predict(twenty_test.data)\n",
    "print(f\"Accuracy: {np.mean(predicted == twenty_test.target):.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy the model\n",
    "\n",
    "Now we want to be able to deploy the model we just trained. This will just be as simple as updated the model binary.\n",
    "\n",
    "### Save the trained model\n",
    "\n",
    "First we have to save the trained model in the `src/` folder, which our wrapper will load"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['src/model.joblib']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import joblib\n",
    "joblib.dump(text_clf, \"src/model.joblib\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Update your unit test\n",
    "\n",
    "We'll write a very simple unit test that make sure that the model loads and runs as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting src/test_model.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile src/test_model.py\n",
    "\n",
    "import numpy as np\n",
    "from unittest import mock\n",
    "import joblib\n",
    "\n",
    "EXPECTED_RESPONSE = np.array([3, 3])\n",
    "\n",
    "def test_model(*args, **kwargs):\n",
    "    data = [\"text 1\", \"text 2\"]\n",
    "\n",
    "    m = joblib.load(\"model.joblib\")\n",
    "    result = m.predict(data)\n",
    "    assert all(result == EXPECTED_RESPONSE)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cat: VERSION: No such file or directory\n",
      "Makefile:12: warning: overriding recipe for target 'make'\n",
      "Makefile:9: warning: ignoring old recipe for target 'make'\n",
      "Makefile:15: warning: overriding recipe for target 'make'\n",
      "Makefile:12: warning: ignoring old recipe for target 'make'\n",
      "(cd src && pytest -s --verbose -W ignore 2>&1)\n",
      "\u001b[1m============================= test session starts ==============================\u001b[0m\n",
      "platform linux -- Python 3.7.3, pytest-5.1.1, py-1.8.0, pluggy-0.12.0 -- /home/alejandro/miniconda3/envs/reddit-classification/bin/python\n",
      "cachedir: .pytest_cache\n",
      "rootdir: /home/alejandro/Programming/kubernetes/seldon/sig-mlops-example/src\n",
      "plugins: cov-2.7.1, forked-1.0.2, localserver-0.5.0\n",
      "collected 1 item                                                               \u001b[0m\n",
      "\n",
      "test_model.py::test_model \u001b[32mPASSED\u001b[0m\n",
      "\n",
      "\u001b[32m\u001b[1m============================== 1 passed in 2.21s ===============================\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!make test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Updating Integration Tests\n",
    "\n",
    "We can also now update the integration tests. This is another very simple step, where we'll want to test this model specifically.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting integration/test_e2e_sklearn_server.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile integration/test_e2e_sklearn_server.py\n",
    "from seldon_core.seldon_client import SeldonClient\n",
    "import numpy as np\n",
    "\n",
    "API_AMBASSADOR = \"localhost:8003\"\n",
    "\n",
    "def test_sklearn_server():\n",
    "    data = [\"From: brian@ucsd.edu (Brian Kantor)\\nSubject: Re: HELP for Kidney Stones ..............\\nOrganization: The Avant-Garde of the Now, Ltd.\\nLines: 12\\nNNTP-Posting-Host: ucsd.edu\\n\\nAs I recall from my bout with kidney stones, there isn't any\\nmedication that can do anything about them except relieve the pain.\\n\\nEither they pass, or they have to be broken up with sound, or they have\\nto be extracted surgically.\\n\\nWhen I was in, the X-ray tech happened to mention that she'd had kidney\\nstones and children, and the childbirth hurt less.\\n\\nDemerol worked, although I nearly got arrested on my way home when I barfed\\nall over the police car parked just outside the ER.\\n\\t- Brian\\n\",\n",
    "            'From: rind@enterprise.bih.harvard.edu (David Rind)\\nSubject: Re: Candida(yeast) Bloom, Fact or Fiction\\nOrganization: Beth Israel Hospital, Harvard Medical School, Boston Mass., USA\\nLines: 37\\nNNTP-Posting-Host: enterprise.bih.harvard.edu\\n\\nIn article <1993Apr26.103242.1@vms.ocom.okstate.edu>\\n banschbach@vms.ocom.okstate.edu writes:\\n>are in a different class.  The big question seems to be is it reasonable to \\n>use them in patients with GI distress or sinus problems that *could* be due \\n>to candida blooms following the use of broad-spectrum antibiotics?\\n\\nI guess I\\'m still not clear on what the term \"candida bloom\" means,\\nbut certainly it is well known that thrush (superficial candidal\\ninfections on mucous membranes) can occur after antibiotic use.\\nThis has nothing to do with systemic yeast syndrome, the \"quack\"\\ndiagnosis that has been being discussed.\\n\\n\\n>found in the sinus mucus membranes than is candida.  Women have been known \\n>for a very long time to suffer from candida blooms in the vagina and a \\n>women is lucky to find a physician who is willing to treat the cause and \\n>not give give her advise to use the OTC anti-fungal creams.\\n\\nLucky how?  Since a recent article (randomized controlled trial) of\\noral yogurt on reducing vaginal candidiasis, I\\'ve mentioned to a \\nnumber of patients with frequent vaginal yeast infections that they\\ncould try eating 6 ounces of yogurt daily.  It turns out most would\\nrather just use anti-fungal creams when they get yeast infections.\\n\\n>yogurt dangerous).  If this were a standard part of medical practice, as \\n>Gordon R. says it is, then the incidence of GI distress and vaginal yeast \\n>infections should decline.\\n\\nAgain, this just isn\\'t what the systemic yeast syndrome is about, and\\nhas nothing to do with the quack therapies that were being discussed.\\nThere is some evidence that attempts to reinoculate the GI tract with\\nbacteria after antibiotic therapy don\\'t seem to be very helpful in\\nreducing diarrhea, but I don\\'t think anyone would view this as a\\nquack therapy.\\n-- \\nDavid Rind\\nrind@enterprise.bih.harvard.edu\\n']\n",
    "    labels = [2, 2]\n",
    "    \n",
    "    sc = SeldonClient(\n",
    "        gateway=\"ambassador\", \n",
    "        gateway_endpoint=API_AMBASSADOR,\n",
    "        deployment_name=\"news-classifier-server\",\n",
    "        payload_type=\"ndarray\",\n",
    "        namespace=\"default\",\n",
    "        transport=\"rest\")\n",
    "\n",
    "    result = sc.predict(np.array(data))\n",
    "    assert all(result.response.data.ndarray.values == labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now push your changes to trigger the pipeline\n",
    "Because Jenkins X has created a CI GitOps pipeline for our repo we just need to push our changes to run all the tests\n",
    "\n",
    "We can do this by running our good old git commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "git add .\n",
    "git push origin master"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now see that the pipeline has been triggered by viewing our activities:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Create Effective Pipeline                          11h28m57s       7s Succeeded \r\n",
      "    Create Tekton Crds                                 11h28m50s      11s Succeeded \r\n",
      "  test and deploy sklearn server                       11h28m38s    1m54s Succeeded \r\n",
      "    Credential Initializer 59hx6                       11h28m38s       0s Succeeded \r\n",
      "    Working Dir Initializer Fslpm                      11h28m38s       1s Succeeded \r\n",
      "    Place Tools                                        11h28m37s       1s Succeeded \r\n",
      "    Git Source Seldonio Sig Mlops Seldon Jenki Ftjtn   11h28m36s       6s Succeeded https://github.com/SeldonIO/sig-mlops-seldon-jenkins-x.git\r\n",
      "    Git Merge                                          11h28m30s       1s Succeeded \r\n",
      "    Run Tests                                          11h28m29s      13s Succeeded \r\n",
      "    Build And Push Images                              11h28m16s    1m32s Succeeded \r\n"
     ]
    }
   ],
   "source": [
    "!jx get activity -f sig-mlops-seldon-jenkins-x | tail"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Similarly we can actually see the logs of our running job:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "error: Failed to parse docker reference ELDON_BASE_WRAPPER\n",
      "ERROR: An error occurred: unable to get metadata for ELDON_BASE_WRAPPER:latest\n",
      "ERROR: Suggested solution: check image name\n",
      "ERROR: If the problem persists consult the docs at https://github.com/openshift/source-to-image/tree/master/docs. Eventually reach us on freenode #openshift or file an issue at https://github.com/openshift/source-to-image/issues providing us with a log from your build using log output level 3.\n",
      "Makefile:8: recipe for target 'build' failed\n",
      "make: *** [build] Error 1\n",
      "Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.\n",
      "\u001b[31m\n",
      "Pipeline failed on stage 'test-and-deploy-sklearn-server' : container 'step-build-and-push-images'. The execution of the pipeline has stopped.\u001b[0m\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "wrote: /tmp/086bfe4e-d4ac-46e6-baa1-71d4ef7abca4095596018\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "YOUR_GIT_USERNAME=SeldonIO\n",
    "jx get build logs \"$YOUR_GIT_USERNAME/sig-mlops-seldon-jenkins-x/master #7 release\" | tail"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Managing your Jenkins X Application\n",
    "\n",
    "Now that we've deployed our MLOps repo, Jenkins X now has created an application from our charts.\n",
    "\n",
    "This application gets automatically syncd into the Jenkins X staging environment, which you can see:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!kubectl get pods -n jx-staging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test your application in the staging environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ndarray {\n",
       "  values {\n",
       "    number_value: 2.0\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "import numpy as np\n",
    "\n",
    "url = !kubectl get svc ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'\n",
    "\n",
    "sc = SeldonClient(\n",
    "    gateway=\"ambassador\", \n",
    "    gateway_endpoint=\"localhost:80\",\n",
    "    deployment_name=\"mlops-server\",\n",
    "    payload_type=\"ndarray\",\n",
    "    namespace=\"jx-staging\",\n",
    "    transport=\"rest\")\n",
    "\n",
    "response = sc.predict(data=np.array([twenty_test.data[0]]))\n",
    "\n",
    "response.response.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"so6n21pkf70fm66eka28lc63cr\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    },\n",
      "    \"requestPath\": {\n",
      "      \"news-classifier-server-processor\": \"axsauze/sklearn-server:0.1\"\n",
      "    },\n",
      "    \"metrics\": []\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [],\n",
      "    \"ndarray\": [2.0]\n",
      "  }\n",
      "}"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "\r",
      "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
      "100   350  100   278  100    72   7942   2057 --:--:-- --:--:-- --:--:-- 10294\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "curl -X POST -H 'Content-Type: application/json' \\\n",
    "     -d \"{'data': {'names': ['text'], 'ndarray': ['Hello world this is a test']}}\" \\\n",
    "    http://localhost/seldon/jx-staging/news-classifier-server/api/v0.1/predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Diving into our continuous integration\n",
    "\n",
    "We have now separated our model development into two chunks: \n",
    "\n",
    "* The first one involves the creation of a model serve, and the second one involves the CI of the model server, and the second involves the deployment of models that create the model.\n",
    "\n",
    "\n",
    "## Using the Jenkins X pipeline\n",
    "\n",
    "In order to do this we will be able to first run some tests and the push to the docker repo.\n",
    "\n",
    "For this we will be leveraging the Jenkins X file, we'll first start with a simple file that just runs the tests:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting jenkins-x.yml\n"
     ]
    }
   ],
   "source": [
    "%%writefile jenkins-x.yml\n",
    "buildPack: none\n",
    "pipelineConfig:\n",
    "  pipelines:\n",
    "    release:\n",
    "      pipeline:\n",
    "        agent:\n",
    "          image: seldonio/core-builder:0.4\n",
    "        stages:\n",
    "          - name: test-sklearn-server\n",
    "            steps:\n",
    "            - name: run-tests\n",
    "              command: make\n",
    "              args:\n",
    "              - install_dev\n",
    "              - test\n",
    "    pullRequest:\n",
    "      pipeline:\n",
    "        agent:\n",
    "          image: seldonio/core-builder:0.4\n",
    "        stages:\n",
    "          - name: test-sklearn-server\n",
    "            steps:\n",
    "            - name: run-tests\n",
    "              command: make\n",
    "              args:\n",
    "              - install_dev\n",
    "              - test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `jenkins-x.yml` file is pretty easy to understand if we read through the different steps.\n",
    "\n",
    "Basically we can define the steps of what happens upon `release` - i.e. when a PR / Commit is added to master - and what happens upon `pullRequest` - whenever someone opens a pull request.\n",
    "\n",
    "You can see that the steps are exactly the same for both release and PR for now - namely, we run `make install_dev test` which basically installs all the dependencies and runs all the tests."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Integration tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a model that we want to be able to deploy, we want to make sure that we run end-to-end tests on that model to make sure everything works as expected.\n",
    "\n",
    "For this we will leverage the same framework that the Kubernetes team uses to test Kubernetes itself: KIND.\n",
    "\n",
    "KIND stands for Kubernetes in Docker, and is used to isolate a Kubernetes environent for end-to-end tests.\n",
    "\n",
    "In our case, we will be able to leverage to create an isolated environment, where we'll be able to test our model.\n",
    "\n",
    "For this, the steps we'll have to carry out include:\n",
    "\n",
    "1. Authenticate your docker with the jx CLI\n",
    "2. Add the steps in the `Jenkins-X.yml` to run this in the production cluster\n",
    "3. Leverage the `kind_run_all.sh` script that creates a KIND cluster and runs the tests\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add docker auth to your cluster\n",
    "\n",
    "Adding a docker authentication with Jenkins X can be done through a JX CLI command, which is the following:\n",
    "\n",
    "* `jx create docker auth --host https://index.docker.io/v1/ --user $YOUR_DOCKER_USERNAME --secret $YOUR_DOCKER_KEY_SECRET --email $YOUR_DOCKER_EMAIL`\n",
    "\n",
    "This comamnd will use these credentials to authenticate with Docker and create an auth token (which expires)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extend JenkinsX file for integration\n",
    "\n",
    "Now that we have the test that would run for the integration tests, we need to extend the JX pipeline to run this.\n",
    "\n",
    "This extension is quite simple, and only requires adding the following line:\n",
    "    \n",
    "```\n",
    "            - name: run-end-to-end-tests\n",
    "              command: bash\n",
    "              args:\n",
    "              - integration/kind_test_all.sh\n",
    "```\n",
    "\n",
    "This line would be added in both the PR and release pipelines so that we can run integration tests then.\n",
    "\n",
    "It is also possible to move the integration tests into a separate jenkins-x file such as `jenkins-x-integration.yml` by leveraging [Contexts & Schedules]() which basically allow us to extend the functionality of Prow by writing our own triggers, however this is outside the scope of this tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Config to provide docker authentication\n",
    "\n",
    "This piece is slightly more extensive, as we will need to use Docker to build out containers due to the dependency on `s2i` to build the model wrappers.\n",
    "\n",
    "First we need to define the volumes that we'll be mounting to the container.\n",
    "\n",
    "The first few volumes before basically consist of the core components that docker will need to be able to run.\n",
    "```\n",
    "          volumes:\n",
    "            - name: modules\n",
    "              hostPath:\n",
    "                path: /lib/modules\n",
    "                type: Directory\n",
    "            - name: cgroup\n",
    "              hostPath:\n",
    "                path: /sys/fs/cgroup\n",
    "                type: Directory\n",
    "            - name: dind-storage\n",
    "              emptyDir: {}\n",
    "```\n",
    "We also want to mount the docker credentials which we will generate in the next step.\n",
    "```\n",
    "            - name: jenkins-docker-config-volume\n",
    "              secret:\n",
    "                items:\n",
    "                - key: config.json\n",
    "                  path: config.json\n",
    "                secretName: jenkins-docker-cfg\n",
    "```\n",
    "Once we've created the volumes, now we just need to mount them. This can be done as follows:\n",
    "```\n",
    "        options:\n",
    "          containerOptions:\n",
    "            volumeMounts:\n",
    "              - mountPath: /lib/modules\n",
    "                name: modules\n",
    "                readOnly: true\n",
    "              - mountPath: /sys/fs/cgroup\n",
    "                name: cgroup\n",
    "              - name: dind-storage\n",
    "                mountPath: /var/lib/docker                 \n",
    "```\n",
    "And finally we also mount the docker auth configuration so we don't have to run `docker login`:\n",
    "```\n",
    "              - mountPath: /builder/home/.docker\n",
    "                name: jenkins-docker-config-volume\n",
    "```\n",
    "\n",
    "And to finalise, we need to make sure that the pod can run with privileged context.\n",
    "\n",
    "The reason why this is required is in order to be able to run the docker daemon:\n",
    "```\n",
    "            securityContext:\n",
    "              privileged: true\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kind run all integration tests script\n",
    "\n",
    "The kind_run_all may seem complicated at first, but it's actually quite simple. \n",
    "\n",
    "All the script does is set-up a kind cluster with all dependencies, deploy the model and clean everything up.\n",
    "\n",
    "Let's break down each of the components within the script.\n",
    "\n",
    "#### Start docker\n",
    "\n",
    "We first start the docker daemon and wait until Docker is running (using `docker ps q` for guidance.\n",
    "\n",
    "```\n",
    "# FIRST WE START THE DOCKER DAEMON\n",
    "service docker start\n",
    "# the service can be started but the docker socket not ready, wait for ready\n",
    "WAIT_N=0\n",
    "while true; do\n",
    "    # docker ps -q should only work if the daemon is ready\n",
    "    docker ps -q > /dev/null 2>&1 && break\n",
    "    if [[ ${WAIT_N} -lt 5 ]]; then\n",
    "        WAIT_N=$((WAIT_N+1))\n",
    "        echo \"[SETUP] Waiting for Docker to be ready, sleeping for ${WAIT_N} seconds ...\"\n",
    "        sleep ${WAIT_N}\n",
    "    else\n",
    "        echo \"[SETUP] Reached maximum attempts, not waiting any longer ...\"\n",
    "        break\n",
    "    fi\n",
    "done\n",
    "```\n",
    "\n",
    "#### Create and set-up KIND cluster\n",
    "\n",
    "Once we're running a docker daemon, we can run the command to create our KIND cluster, and install all the components.\n",
    "\n",
    "This will set up a Kubnernetes cluster using the docker daemon (using containers as Nodes), and then install Ambassador + Seldon Core.\n",
    "\n",
    "```\n",
    "#######################################\n",
    "# AVOID EXIT ON ERROR FOR FOLLOWING CMDS\n",
    "set +o errexit\n",
    "\n",
    "# START CLUSTER \n",
    "make kind_create_cluster\n",
    "KIND_EXIT_VALUE=$?\n",
    "\n",
    "# Ensure we reach the kubeconfig path\n",
    "export KUBECONFIG=$(kind get kubeconfig-path)\n",
    "\n",
    "# ONLY RUN THE FOLLOWING IF SUCCESS\n",
    "if [[ ${KIND_EXIT_VALUE} -eq 0 ]]; then\n",
    "    # KIND CLUSTER SETUP\n",
    "    make kind_setup\n",
    "    SETUP_EXIT_VALUE=$?\n",
    "```\n",
    "\n",
    "#### Run python tests\n",
    "\n",
    "We can now run the tests; for this we run all the dev installations and kick off our tests (which we'll add inside of the integration folder).\n",
    "\n",
    "```\n",
    "    # BUILD S2I BASE IMAGES\n",
    "    make build\n",
    "    S2I_EXIT_VALUE=$?\n",
    "\n",
    "    ## INSTALL ALL REQUIRED DEPENDENCIES\n",
    "    make install_integration_dev\n",
    "    INSTALL_EXIT_VALUE=$?\n",
    "    \n",
    "    ## RUNNING TESTS AND CAPTURING ERROR\n",
    "    make test\n",
    "    TEST_EXIT_VALUE=$?\n",
    "fi\n",
    "```\n",
    "\n",
    "#### Clean up\n",
    "\n",
    "Finally we just clean everything, including the cluster, the containers and the docker daemon.\n",
    "\n",
    "```\n",
    "# DELETE KIND CLUSTER\n",
    "make kind_delete_cluster\n",
    "DELETE_EXIT_VALUE=$?\n",
    "\n",
    "#######################################\n",
    "# EXIT STOPS COMMANDS FROM HERE ONWARDS\n",
    "set -o errexit\n",
    "\n",
    "# CLEANING DOCKER\n",
    "docker ps -aq | xargs -r docker rm -f || true\n",
    "service docker stop || true\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Promote your application\n",
    "Now that we've verified that our CI pipeline is working, we want to promote our application to production\n",
    "\n",
    "This can be done with our JX CLI:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!jx promote application --..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test your production application\n",
    "\n",
    "Once your production application is deployed, you can test it using the same script, but in the `jx-production` namespace:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "import numpy as np\n",
    "\n",
    "url = !kubectl get svc ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'\n",
    "\n",
    "sc = SeldonClient(\n",
    "    gateway=\"ambassador\", \n",
    "    gateway_endpoint=\"localhost:80\",\n",
    "    deployment_name=\"mlops-server\",\n",
    "    payload_type=\"ndarray\",\n",
    "    namespace=\"jx-production\",\n",
    "    transport=\"rest\")\n",
    "\n",
    "response = sc.predict(data=np.array([twenty_test.data[0]]))\n",
    "\n",
    "response.response.data"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}