notebooks/Watson OpenScale Explanation for Image Multiclass Classification Model CP4D.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://github.com/pmservice/ai-openscale-tutorials/raw/master/notebooks/images/banner.png\" align=\"left\" alt=\"banner\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial on generating an explanation for an image-based model on Watson OpenScale (IBM Cloud Pak for Data 2.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook includes steps for creating an image-based watson-machine-learning model, creating a subscription, configuring explainability, and finally generating an explanation for a transaction.\n",
    "\n",
    "It has been adapted for IBM Cloud Pak for Data 2.5; the original version for IBM Cloud is found here https://github.com/pmservice/ai-openscale-tutorials/blob/master/notebooks/Watson%20OpenScale%20Explanation%20for%20Image%20Multiclass%20Classification%20Model.ipynb "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Contents\n",
    "- [1. Setup](#setup)\n",
    "- [2. Creating and deploying an image-based model](#deployment)\n",
    "- [3. Subscriptions](#subscription)\n",
    "- [4. Explainability](#explainability)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"setup\"></a>\n",
    "## 1. Setup\n",
    "\n",
    "### 1.1 Install Watson OpenScale and WML packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade ibm-ai-openscale --no-cache | tail -n 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade watson-machine-learning-client --no-cache | tail -n 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: Restart the kernel to assure the new libraries are being used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Configure credentials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For Watson OpenScale: Add URL from your IBM Cloud Pak for Data instance, and user name and password that are used for login to the platform. \n",
    "\n",
    "<font color='red'>Remember to remove your credentials when sharing the notebook! </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CP4D specifics: credentials for Watson OpenScale\n",
    "\n",
    "AIOS_CREDENTIALS = {\n",
    "    \"url\": \"<url for CP4D>\",\n",
    "    \"username\": \"<username>\",\n",
    "    \"password\": \"<password>\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For Watson Machine Learning: Do not change the credentials below! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CP4D specifics: credentials for Watson Machine Learning \n",
    "\n",
    "import os\n",
    "token = os.environ['USER_ACCESS_TOKEN']\n",
    "WML_CREDENTIALS = {\n",
    "                   \"token\": token,\n",
    "                   \"instance_id\" : \"wml_local\",\n",
    "                   \"url\": os.environ['RUNTIME_ENV_APSX_URL'],\n",
    "                   \"version\": \"2.5.0\"\n",
    "                }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"deployment\"></a>\n",
    "## 2. Creating and deploying an image-based model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset used is MNIST dataset of handwritten digits. It consists of 60,000 28x28 grayscale images of the 10 digits, along with a test set of 10,000 images. More information about the dataset can be found here: https://keras.io/datasets/#mnist-database-of-handwritten-digits\n",
    "\n",
    "Note: Keras and TensorFlow versions supported by WML are: Keras 2.1.6 with TensorFlow 1.13 backend and Keras 2.2.4 with TensorFlow 1.14 backend. The latter combination is used in this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Creating a model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install keras==2.2.4\n",
    "!pip install tensorflow==1.14.0\n",
    "!pip install keras_sequential_ascii\n",
    "\n",
    "import keras\n",
    "from keras.datasets import mnist\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Flatten\n",
    "from keras.layers import Conv2D, MaxPooling2D\n",
    "from keras_sequential_ascii import sequential_model_to_ascii_printout\n",
    "from keras import backend as keras_backend\n",
    "print(keras.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 128\n",
    "num_classes = 10\n",
    "epochs = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# input image dimensions\n",
    "img_rows, img_cols = 28, 28\n",
    "\n",
    "# the data, split between train and test sets\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "if keras_backend.image_data_format() == 'channels_first':\n",
    "    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)\n",
    "    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)\n",
    "    input_shape = (1, img_rows, img_cols)\n",
    "else:\n",
    "    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n",
    "    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n",
    "    input_shape = (img_rows, img_cols, 1)\n",
    "\n",
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "\n",
    "print('x_train shape:', x_train.shape)\n",
    "print(x_train.shape[0], 'train samples')\n",
    "print(x_test.shape[0], 'test samples')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert class vectors to binary class matrices\n",
    "y_train = keras.utils.to_categorical(y_train, num_classes)\n",
    "y_test = keras.utils.to_categorical(y_test, num_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define Model\n",
    "\n",
    "def base_model():\n",
    "    model = Sequential()\n",
    "    model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape))\n",
    "    model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "    model.add(Dropout(0.25))\n",
    "    model.add(Flatten())\n",
    "    model.add(Dense(128, activation='relu'))\n",
    "    model.add(Dropout(0.5))\n",
    "    model.add(Dense(num_classes, activation='softmax'))\n",
    "\n",
    "    model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "                  optimizer=keras.optimizers.Adadelta(),\n",
    "                  metrics=['accuracy'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnn_n = base_model()\n",
    "cnn_n.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Vizualizing model structure\n",
    "sequential_model_to_ascii_printout(cnn_n)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fit model\n",
    "print(y_train.shape)\n",
    "cnn = cnn_n.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scores = cnn_n.evaluate(x_test, y_test, verbose=0)\n",
    "print(scores)\n",
    "print(\"Accuracy: %.2f%%\" % (scores[1]*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Storing the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from watson_machine_learning_client import WatsonMachineLearningAPIClient\n",
    "\n",
    "wml_client = WatsonMachineLearningAPIClient(WML_CREDENTIALS)\n",
    "cnn_n.save(\"mnist_cnn.h5\")\n",
    "!rm mnist_cnn.tar*\n",
    "!tar -czvf mnist_cnn.tar.gz mnist_cnn.h5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm mnist_cnn.h5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add the name of an existing deployment space or a name for a new space to be created: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CP4D specifics: deployment space\n",
    "\n",
    "DEFAULT_DEPLOYMENT_SPACE=\"<deployment space name>\"\n",
    "\n",
    "filtered_spaces=[ space['metadata']['guid'] for space in wml_client.spaces.get_details()['resources'] if space['entity']['name']== DEFAULT_DEPLOYMENT_SPACE]\n",
    "if len(filtered_spaces)>0:\n",
    "    space_id=filtered_spaces[0]\n",
    "else:\n",
    "    space_metadata = {\n",
    "        wml_client.spaces.ConfigurationMetaNames.NAME: DEFAULT_DEPLOYMENT_SPACE,\n",
    "        wml_client.spaces.ConfigurationMetaNames.DESCRIPTION: DEFAULT_DEPLOYMENT_SPACE,\n",
    "    }\n",
    "    space_details=wml_client.spaces.store(meta_props=space_metadata);\n",
    "    space_id=space_details['metadata']['guid']\n",
    "    \n",
    "print(space_id)\n",
    "\n",
    "# setting default space\n",
    "wml_client.set.default_space(space_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure that the existing or new deployment space contains the collaborator \"admin\", which is required for the Watson OpenScale integration later on. Refer to https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_current/wsj/analyze-data/collaborator-permissions-wml.html on how to add collaborators. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CP4D specifics: project integration\n",
    "\n",
    "from project_lib import Project\n",
    "project = Project.access()\n",
    "project_id=project.get_metadata()['metadata']['guid']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CP4D specifics: ModelMetaNames properties, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_current/wsj/wmls/wmls-deploy-python-types.html \n",
    "\n",
    "model_name = \"MNIST Model\"\n",
    "model_meta = {\n",
    "    wml_client.repository.ModelMetaNames.NAME: model_name,\n",
    "    wml_client.repository.ModelMetaNames.RUNTIME_UID: \"tensorflow_1.14-py3.6\",\n",
    "    wml_client.repository.ModelMetaNames.TYPE: \"keras_2.2.4\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "published_model_details = wml_client.repository.store_model(model='mnist_cnn.tar.gz', meta_props=model_meta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_uid = wml_client.repository.get_model_uid(published_model_details)\n",
    "model_uid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Deploying the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deploy_meta = {\n",
    "     wml_client.deployments.ConfigurationMetaNames.NAME: \"MNIST deploy\",\n",
    "     wml_client.deployments.ConfigurationMetaNames.ONLINE: {}\n",
    " }\n",
    "deployment = wml_client.deployments.create(model_uid, meta_props=deploy_meta)\n",
    "\n",
    "# CP4D specifics: storing the deployment_uid for later\n",
    "deployment_uid = wml_client.deployments.get_uid(deployment)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CP4D specifics: scoring URL(href)\n",
    "for deployment in  wml_client.deployments.get_details()['resources']:\n",
    "    print(deployment['entity']['name'],'\\t',wml_client.deployments.get_scoring_href(deployment))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Subscriptions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Configuring OpenScale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#CP4D specifics: APIClient4ICP\n",
    "from ibm_ai_openscale import APIClient4ICP\n",
    "from ibm_ai_openscale.engines import WatsonMachineLearningAsset\n",
    "\n",
    "aios_client = APIClient4ICP(AIOS_CREDENTIALS)\n",
    "aios_client.version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Subscribe the asset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ibm_ai_openscale.supporting_classes import *\n",
    "\n",
    "subscription = aios_client.data_mart.subscriptions.add(WatsonMachineLearningAsset(\n",
    "    model_uid,\n",
    "    problem_type=ProblemType.MULTICLASS_CLASSIFICATION,\n",
    "    input_data_type=InputDataType.UNSTRUCTURED_IMAGE,\n",
    "    probability_column='probability'\n",
    "))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aios_client.data_mart.subscriptions.list()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subscription.get_details()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Score the model and get transaction-id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install numpy\n",
    "!pip install matplotlib\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline \n",
    "img = np.array(x_test[999], dtype='float')\n",
    "pixels = img.reshape((28, 28))\n",
    "plt.imshow(pixels, cmap='gray')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CP4D specifics: calling the endpoint, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_current/wsj/wmls/wmls-deploy-python.html\n",
    "job_payload = {\n",
    "   wml_client.deployments.ScoringMetaNames.INPUT_DATA: [{\n",
    "     'values': [x_test[990].tolist()]\n",
    "   }]\n",
    " }\n",
    "\n",
    "job_details = wml_client.deployments.score(deployment_uid,\n",
    "                                        job_payload)\n",
    "print(job_details)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: Please wait for a few seconds before running the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transaction_id = subscription.payload_logging.get_table_content().scoring_id[0]\n",
    "print(transaction_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"explainability\"></a>\n",
    "## 4. Explainability"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Configure Explainability"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subscription.explainability.enable()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subscription.explainability.get_details()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Get explanation for the transaction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explanation = subscription.explainability.run(transaction_id, background_mode=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explanation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The explanation images can be obtained using the cells below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install Pillow\n",
    "from PIL import Image\n",
    "import base64\n",
    "import io\n",
    "\n",
    "img = explanation[\"entity\"][\"predictions\"][0][\"explanation\"][0][\"full_image\"]\n",
    "img_data = base64.b64decode(img)\n",
    "Image.open(io.BytesIO(img_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "img = explanation[\"entity\"][\"predictions\"][1][\"explanation\"][0][\"full_image\"]\n",
    "img_data = base64.b64decode(img)\n",
    "Image.open(io.BytesIO(img_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}