Copy_of_Assignment_6_practical_machine_learning_Estimators.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of Assignment 6_practical machine learning -  Estimators.ipynb",
      "provenance": [],
      "private_outputs": true,
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vipulgote1999/practical-machine-learining-with-tensorflow/blob/master/Copy_of_Assignment_6_practical_machine_learning_Estimators.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ltd9Ar449-cl",
        "colab_type": "text"
      },
      "source": [
        "### **Setup**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "TtRIELH_97Ro"
      },
      "source": [
        "Install and import all the necessary libraries for the assignment."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "NQ83YONE97Rs",
        "colab": {}
      },
      "source": [
        "!pip install tensorflow==2.0.0-rc0\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.datasets import load_boston\n",
        "import tensorflow as tf\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "sns_colors = sns.color_palette('colorblind')\n",
        "\n",
        "tf.random.set_seed(1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "n2h0X-5v97R3"
      },
      "source": [
        "### **Importing the dataset**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "L25L1OIK97R6",
        "colab": {}
      },
      "source": [
        "boston_dataset = load_boston()\n",
        "\n",
        "data_X = pd.DataFrame(boston_dataset.data, columns=boston_dataset.feature_names)\n",
        "data_Y = pd.DataFrame(boston_dataset.target, columns=[\"target\"])\n",
        "data = pd.concat([data_X, data_Y], axis=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "6wOjFmUJ97SD",
        "colab": {}
      },
      "source": [
        "train, test = train_test_split(data, test_size=0.2, random_state=1)\n",
        "print(len(train), \"train examples\")\n",
        "print(len(test), \"test examples\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "J9bNY19D97SP"
      },
      "source": [
        "Converting the Pandas DataFrames into Tensorflow Datasets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Zbejh2Wp97SS",
        "colab": {}
      },
      "source": [
        "def df_to_dataset(dataframe, shuffle_and_repeat=True, batch_size=32):\n",
        "  dataframe = dataframe.copy()\n",
        "  labels = dataframe.pop('target')\n",
        "  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))\n",
        "  if shuffle_and_repeat:\n",
        "    ds = ds.shuffle(buffer_size=len(dataframe))\n",
        "    ds = ds.repeat()\n",
        "    \n",
        "  ds = ds.batch(batch_size)\n",
        "  \n",
        "  return ds"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "G49Lkx5497Sd"
      },
      "source": [
        "### Defining Feature Columns"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "_N-7Cuak97Sf",
        "colab": {}
      },
      "source": [
        "# define feature_columns as a list of all the features as numeric columns using functions from tf.feature_column\n",
        "feature_columns1 = []\n",
        "l1=train.keys()\n",
        "for key in l1:\n",
        "  feature_columns1.append(tf.feature_column.numeric_column(key=key))\n",
        "feature_columns1"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Z585fT0N97Sm"
      },
      "source": [
        "### Building the model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "vEDD1ZAh97So",
        "colab": {}
      },
      "source": [
        "config1 = tf.estimator.RunConfig(tf_random_seed=1)\n",
        "\n",
        "esti=tf.estimator.LinearRegressor(feature_columns=feature_columns1,config=config1,)\n",
        "\n",
        "# Build the model using tf.estimator and pass as config variable to the config parameter of the estimator\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "a7kUZnP997Sv"
      },
      "source": [
        "Train the model by using the df_to_dataset function. Batch size should be 32. Note that while training the model, the shuffle_and_repeat parameter of the df_to_dataset function should be set to True."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IjF6Rq7-Cxv7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Train the model\n",
        "\n",
        "esti.train(input_fn=lambda:df_to_dataset(train),steps=2500)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cEKmcvBQEDRO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Evaluate the model using the df_to_dataset function and the test dataframe\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j_dyVU8-POff",
        "colab_type": "text"
      },
      "source": [
        "### **Boosted Trees Interpretation**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0UUEX6UyPDIy",
        "colab_type": "text"
      },
      "source": [
        "You will need this section only for questions 5-7."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "TIL93B4sDRqE",
        "colab": {}
      },
      "source": [
        "pred_dicts = list(model.experimental_predict_with_explanations(lambda: df_to_dataset(test, False, 32)))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "tDPoRx_ZaY1E",
        "colab": {}
      },
      "source": [
        "# Create DFC Pandas dataframe.\n",
        "predictions = pd.Series([pred['predictions'][0] for pred in pred_dicts])\n",
        "df_dfc = pd.DataFrame([pred['dfc'] for pred in pred_dicts])\n",
        "df_dfc.describe().T"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Hd9VuizRaY1H",
        "colab": {}
      },
      "source": [
        "# Sum of DFCs + bias == probabality.\n",
        "bias = pred_dicts[0]['bias']\n",
        "dfc_pred = df_dfc.sum(axis=1) + bias\n",
        "np.testing.assert_almost_equal(dfc_pred.values,\n",
        "                               predictions.values)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "6z_Tq1Pquczj",
        "colab": {}
      },
      "source": [
        "# Boilerplate code for plotting :)\n",
        "def _get_color(value):\n",
        "    \"\"\"To make positive DFCs plot green, negative DFCs plot red.\"\"\"\n",
        "    green, red = sns.color_palette()[2:4]\n",
        "    if value >= 0: return green\n",
        "    return red\n",
        "\n",
        "def _add_feature_values(feature_values, ax):\n",
        "    \"\"\"Display feature's values on left of plot.\"\"\"\n",
        "    x_coord = ax.get_xlim()[0]\n",
        "    OFFSET = 0.15\n",
        "    for y_coord, (feat_name, feat_val) in enumerate(feature_values.items()):\n",
        "        t = plt.text(x_coord, y_coord - OFFSET, '{}'.format(feat_val), size=12)\n",
        "        t.set_bbox(dict(facecolor='white', alpha=0.5))\n",
        "    from matplotlib.font_manager import FontProperties\n",
        "    font = FontProperties()\n",
        "    font.set_weight('bold')\n",
        "    t = plt.text(x_coord, y_coord + 1 - OFFSET, 'feature\\nvalue',\n",
        "    fontproperties=font, size=12)\n",
        "\n",
        "def plot_example(example):\n",
        "  TOP_N = 8 # View top 8 features.\n",
        "  sorted_ix = example.abs().sort_values()[-TOP_N:].index  # Sort by magnitude.\n",
        "  example = example[sorted_ix]\n",
        "  colors = example.map(_get_color).tolist()\n",
        "  ax = example.to_frame().plot(kind='barh',\n",
        "                          color=[colors],\n",
        "                          legend=None,\n",
        "                          alpha=0.75,\n",
        "                          figsize=(10,6))\n",
        "  ax.grid(False, axis='y')\n",
        "  ax.set_yticklabels(ax.get_yticklabels(), size=14)\n",
        "\n",
        "  # Add feature values.\n",
        "  _add_feature_values(test.iloc[ID][sorted_ix], ax)\n",
        "  return ax"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Ht1P2-1euczk",
        "colab": {}
      },
      "source": [
        "# Plot results.\n",
        "ID = 10\n",
        "example = df_dfc.iloc[ID]  # Choose ith example from evaluation set.\n",
        "TOP_N = 8  # View top 8 features.\n",
        "sorted_ix = example.abs().sort_values()[-TOP_N:].index\n",
        "ax = plot_example(example)\n",
        "ax.set_title('Feature contributions for example {}\\n pred: {:1.2f};'.format(ID, predictions[ID]))\n",
        "ax.set_xlabel('Contribution to prediction', size=14)\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "zo7rNd1v_5e2",
        "colab": {}
      },
      "source": [
        "# Boilerplate plotting code.\n",
        "def dist_violin_plot(df_dfc, ID):\n",
        "  # Initialize plot.\n",
        "  fig, ax = plt.subplots(1, 1, figsize=(10, 6))\n",
        "\n",
        "  # Create example dataframe.\n",
        "  TOP_N = 8  # View top 8 features.\n",
        "  example = df_dfc.iloc[ID]\n",
        "  ix = example.abs().sort_values()[-TOP_N:].index\n",
        "  example = example[ix]\n",
        "  example_df = example.to_frame(name='dfc')\n",
        "\n",
        "  # Add contributions of entire distribution.\n",
        "  parts=ax.violinplot([df_dfc[w] for w in ix],\n",
        "                 vert=False,\n",
        "                 showextrema=False,\n",
        "                 widths=0.7,\n",
        "                 positions=np.arange(len(ix)))\n",
        "  face_color = sns_colors[0]\n",
        "  alpha = 0.15\n",
        "  for pc in parts['bodies']:\n",
        "      pc.set_facecolor(face_color)\n",
        "      pc.set_alpha(alpha)\n",
        "\n",
        "  # Add feature values.\n",
        "  _add_feature_values(test.iloc[ID][sorted_ix], ax)\n",
        "\n",
        "  # Add local contributions.\n",
        "  ax.scatter(example,\n",
        "              np.arange(example.shape[0]),\n",
        "              color=sns.color_palette()[2],\n",
        "              s=100,\n",
        "              marker=\"s\",\n",
        "              label='contributions for example')\n",
        "\n",
        "  # Legend\n",
        "  # Proxy plot, to show violinplot dist on legend.\n",
        "  ax.plot([0,0], [1,1], label='eval set contributions\\ndistributions',\n",
        "          color=face_color, alpha=alpha, linewidth=10)\n",
        "  legend = ax.legend(loc='lower right', shadow=True, fontsize='x-large',\n",
        "                     frameon=True)\n",
        "  legend.get_frame().set_facecolor('white')\n",
        "\n",
        "  # Format plot.\n",
        "  ax.set_yticks(np.arange(example.shape[0]))\n",
        "  ax.set_yticklabels(example.index)\n",
        "  ax.grid(False, axis='y')\n",
        "  ax.set_xlabel('Contribution to prediction', size=14)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "VkCqraA2uczm",
        "colab": {}
      },
      "source": [
        "dist_violin_plot(df_dfc, ID)\n",
        "plt.title('Feature contributions for example {}\\n pred: {:1.2f}'.format(ID, predictions[ID]))\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "3ocBcMatuczs"
      },
      "source": [
        "### Gain-based feature importances"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "pPTxbAaeuczt",
        "colab": {}
      },
      "source": [
        "importances = model.experimental_feature_importances(normalize=True)\n",
        "df_imp = pd.Series(importances)\n",
        "\n",
        "# Visualize importances.\n",
        "N = 8\n",
        "ax = (df_imp.iloc[0:N][::-1]\n",
        "    .plot(kind='barh',\n",
        "          color=sns_colors[0],\n",
        "          title='Gain feature importances',\n",
        "          figsize=(10, 6)))\n",
        "ax.grid(False, axis='y')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Z0k_DvPLaY1o"
      },
      "source": [
        "You can also see how DFCs vary as a feature value varies."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ZcIfN1IpaY1o",
        "colab": {}
      },
      "source": [
        "FEATURE = 'LSTAT'\n",
        "feature = pd.Series(df_dfc[FEATURE].values, index=test[FEATURE].values).sort_index()\n",
        "ax = sns.regplot(feature.index.values, feature.values, lowess=True)\n",
        "ax.set_ylabel('contribution')\n",
        "ax.set_xlabel(FEATURE)\n",
        "ax.set_xlim(0, 35)\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}