TensorFlow_Audiobooks_DL2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Practical example. Audiobooks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem\n",
    "\n",
    "You are given data from an Audiobook app. Logically, it relates only to the audio versions of books. Each customer in the database has made a purchase at least once, that's why he/she is in the database. We want to create a machine learning algorithm based on our available data that can predict if a customer will buy again from the Audiobook company.\n",
    "\n",
    "The main idea is that if a customer has a low probability of coming back, there is no reason to spend any money on advertizing to him/her. If we can focus our efforts ONLY on customers that are likely to convert again, we can make great savings. Moreover, this model can identify the most important metrics for a customer to come back again. Identifying new customers creates value and growth opportunities.\n",
    "\n",
    "You have a .csv summarizing the data. There are several variables: Customer ID, Book length in mins_avg (average of all purchases), Book length in minutes_sum (sum of all purchases), Price Paid_avg (average of all purchases), Price paid_sum (sum of all purchases), Review (a Boolean variable), Review (out of 10), Total minutes listened, Completion (from 0 to 1), Support requests (number), and Last visited minus purchase date (in days).\n",
    "\n",
    "So these are the inputs (excluding customer ID, as it is completely arbitrary. It's more like a name, than a number).\n",
    "\n",
    "The targets are a Boolean variable (so 0, or 1). We are taking a period of 2 years in our inputs, and the next 6 months as targets. So, in fact, we are predicting if: based on the last 2 years of activity and engagement, a customer will convert in the next 6 months. 6 months sounds like a reasonable time. If they don't convert after 6 months, chances are they've gone to a competitor or didn't like the Audiobook way of digesting information. \n",
    "\n",
    "The task is simple: create a machine learning algorithm, which is able to predict if a customer will buy again. \n",
    "\n",
    "This is a classification problem with two classes: won't buy and will buy, represented by 0s and 1s. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the machine learning algorithm\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import the relevant libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we must import the libraries once again since we haven't imported them in this file\n",
    "import numpy as np\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's create a temporary variable npz, where we will store each of the three Audiobooks datasets\n",
    "npz = np.load('Audiobooks_data_train.npz')\n",
    "\n",
    "# we extract the inputs using the keyword under which we saved them\n",
    "# to ensure that they are all floats, let's also take care of that\n",
    "train_inputs = npz['inputs'].astype(np.float)\n",
    "# targets must be int because of sparse_categorical_crossentropy (we want to be able to smoothly one-hot encode them)\n",
    "train_targets = npz['targets'].astype(np.int)\n",
    "\n",
    "# we load the validation data in the temporary variable\n",
    "npz = np.load('Audiobooks_data_validation.npz')\n",
    "# we can load the inputs and the targets in the same line\n",
    "validation_inputs, validation_targets = npz['inputs'].astype(np.float), npz['targets'].astype(np.int)\n",
    "\n",
    "# we load the test data in the temporary variable\n",
    "npz = np.load('Audiobooks_data_test.npz')\n",
    "# we create 2 variables that will contain the test inputs and the test targets\n",
    "test_inputs, test_targets = npz['inputs'].astype(np.float), npz['targets'].astype(np.int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model\n",
    "Outline, optimizers, loss, early stopping and training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 3579 samples, validate on 447 samples\n",
      "Epoch 1/100\n",
      "3579/3579 - 0s - loss: 0.5406 - accuracy: 0.7868 - val_loss: 0.4206 - val_accuracy: 0.8479\n",
      "Epoch 2/100\n",
      "3579/3579 - 0s - loss: 0.3729 - accuracy: 0.8720 - val_loss: 0.3309 - val_accuracy: 0.8680\n",
      "Epoch 3/100\n",
      "3579/3579 - 0s - loss: 0.3252 - accuracy: 0.8824 - val_loss: 0.3018 - val_accuracy: 0.8904\n",
      "Epoch 4/100\n",
      "3579/3579 - 0s - loss: 0.3036 - accuracy: 0.8863 - val_loss: 0.2882 - val_accuracy: 0.8881\n",
      "Epoch 5/100\n",
      "3579/3579 - 0s - loss: 0.2902 - accuracy: 0.8908 - val_loss: 0.2800 - val_accuracy: 0.8993\n",
      "Epoch 6/100\n",
      "3579/3579 - 0s - loss: 0.2813 - accuracy: 0.8961 - val_loss: 0.2663 - val_accuracy: 0.9016\n",
      "Epoch 7/100\n",
      "3579/3579 - 0s - loss: 0.2706 - accuracy: 0.9019 - val_loss: 0.2619 - val_accuracy: 0.8971\n",
      "Epoch 8/100\n",
      "3579/3579 - 0s - loss: 0.2652 - accuracy: 0.9008 - val_loss: 0.2587 - val_accuracy: 0.9060\n",
      "Epoch 9/100\n",
      "3579/3579 - 0s - loss: 0.2612 - accuracy: 0.9014 - val_loss: 0.2567 - val_accuracy: 0.9016\n",
      "Epoch 10/100\n",
      "3579/3579 - 0s - loss: 0.2562 - accuracy: 0.9036 - val_loss: 0.2577 - val_accuracy: 0.9038\n",
      "Epoch 11/100\n",
      "3579/3579 - 0s - loss: 0.2525 - accuracy: 0.9056 - val_loss: 0.2507 - val_accuracy: 0.9083\n",
      "Epoch 12/100\n",
      "3579/3579 - 0s - loss: 0.2481 - accuracy: 0.9044 - val_loss: 0.2494 - val_accuracy: 0.9105\n",
      "Epoch 13/100\n",
      "3579/3579 - 0s - loss: 0.2462 - accuracy: 0.9098 - val_loss: 0.2610 - val_accuracy: 0.8949\n",
      "Epoch 14/100\n",
      "3579/3579 - 0s - loss: 0.2453 - accuracy: 0.9072 - val_loss: 0.2480 - val_accuracy: 0.9016\n",
      "Epoch 15/100\n",
      "3579/3579 - 0s - loss: 0.2477 - accuracy: 0.9089 - val_loss: 0.2455 - val_accuracy: 0.9038\n",
      "Epoch 16/100\n",
      "3579/3579 - 0s - loss: 0.2423 - accuracy: 0.9092 - val_loss: 0.2459 - val_accuracy: 0.9172\n",
      "Epoch 17/100\n",
      "3579/3579 - 0s - loss: 0.2399 - accuracy: 0.9098 - val_loss: 0.2403 - val_accuracy: 0.9150\n",
      "Epoch 18/100\n",
      "3579/3579 - 0s - loss: 0.2378 - accuracy: 0.9114 - val_loss: 0.2472 - val_accuracy: 0.9016\n",
      "Epoch 19/100\n",
      "3579/3579 - 0s - loss: 0.2366 - accuracy: 0.9120 - val_loss: 0.2437 - val_accuracy: 0.9083\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x1b9933850b8>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set the input and output sizes\n",
    "input_size = 10\n",
    "output_size = 2\n",
    "# Use same hidden layer size for both hidden layers. Not a necessity.\n",
    "hidden_layer_size = 50\n",
    "    \n",
    "# define how the model will look like\n",
    "model = tf.keras.Sequential([\n",
    "    # tf.keras.layers.Dense is basically implementing: output = activation(dot(input, weight) + bias)\n",
    "    # it takes several arguments, but the most important ones for us are the hidden_layer_size and the activation function\n",
    "    tf.keras.layers.Dense(hidden_layer_size, activation='relu'), # 1st hidden layer\n",
    "    tf.keras.layers.Dense(hidden_layer_size, activation='relu'), # 2nd hidden layer\n",
    "    # the final layer is no different, we just make sure to activate it with softmax\n",
    "    tf.keras.layers.Dense(output_size, activation='softmax') # output layer\n",
    "])\n",
    "\n",
    "\n",
    "### Choose the optimizer and the loss function\n",
    "\n",
    "# we define the optimizer we'd like to use, \n",
    "# the loss function, \n",
    "# and the metrics we are interested in obtaining at each iteration\n",
    "model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
    "\n",
    "### Training\n",
    "# That's where we train the model we have built.\n",
    "\n",
    "# set the batch size\n",
    "batch_size = 100\n",
    "\n",
    "# set a maximum number of training epochs\n",
    "max_epochs = 100\n",
    "\n",
    "# set an early stopping mechanism\n",
    "# let's set patience=2, to be a bit tolerant against random validation loss increases\n",
    "early_stopping = tf.keras.callbacks.EarlyStopping(patience=2)\n",
    "\n",
    "# fit the model\n",
    "# note that this time the train, validation and test data are not iterable\n",
    "model.fit(train_inputs, # train inputs\n",
    "          train_targets, # train targets\n",
    "          batch_size=batch_size, # batch size\n",
    "          epochs=max_epochs, # epochs that we will train for (assuming early stopping doesn't kick in)\n",
    "          # callbacks are functions called by a task when a task is completed\n",
    "          # task here is to check if val_loss is increasing\n",
    "          callbacks=[early_stopping], # early stopping\n",
    "          validation_data=(validation_inputs, validation_targets), # validation data\n",
    "          verbose = 2 # making sure we get enough information about the training process\n",
    "          )  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test the model\n",
    "\n",
    "It is very important to realize that fiddling with the hyperparameters overfits the validation dataset. \n",
    "\n",
    "The test is the absolute final instance. You should not test before you are completely done with adjusting your model.\n",
    "\n",
    "If you adjust your model after testing, you will start overfitting the test dataset, which will defeat its purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "448/448 [==============================] - 0s 41us/sample - loss: 0.2391 - accuracy: 0.9107\n"
     ]
    }
   ],
   "source": [
    "test_loss, test_accuracy = model.evaluate(test_inputs, test_targets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Test loss: 0.24. Test accuracy: 91.07%\n"
     ]
    }
   ],
   "source": [
    "print('\\nTest loss: {0:.2f}. Test accuracy: {1:.2f}%'.format(test_loss, test_accuracy*100.))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the initial model and hyperparameters given in this notebook, the final test accuracy should be roughly around 91%.\n",
    "\n",
    "Note that each time the code is rerun, we get a different accuracy because each training is different. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:root] *",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}