TFTake1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building Victory  Net"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, I am reading in matches from Destiny's Curcible -- the Player Versus Player arena.\n",
    "The dataset consists of individual matches between team's Alpha and Bravo. The matches are zero-sum therefore in predicting victory -- the alphaVictory column of the data -- and there are only two outcomes. Alpha wins (0) or alpha losses (1). The complementary outcomes can be generated since bravoVictory = 1 - alphaVictory.\n",
    "\n",
    "The data has already been preprocessed and flattened prior to building the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from sklearn.preprocessing import LabelBinarizer\n",
    "import math\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading and Batching"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def split_preprocessed_training_batch(batch_size, npdf, nplabels):\n",
    "    \"\"\"\n",
    "    Load the Preprocessed Training data and return them in batches of <batch_size> or less\n",
    "    \"\"\"\n",
    "    \n",
    "#     df = pd.read_csv(\"indexByMatch_batchUdacity_205_CLEANED_NORM.csv\", index_col = \"Unnamed: 0\")\n",
    "#     df.drop(\"Unnamed: 0\", axis=1, inplace=True)\n",
    "#     labels = df[\"alphaVictory\"]\n",
    "#     df.drop(\"alphaVictory\", axis=1, inplace=True)\n",
    "#     #labels.head(n=10)\n",
    "    \n",
    "#     nplabels = labels.values\n",
    "#     npdf = df.values\n",
    "\n",
    "\n",
    "#     X_train = pd.read_csv(\"X_Train_indexByMatch_103.csv\", index_col=\"Unnamed: 0\")\n",
    "#     y_train = pd.read_csv(\"y_train_indexByMatch_103.csv\")\n",
    "\n",
    "    # Return the training data in batches of size <batch_size> or less\n",
    "    return batch_features_labels(npdf, nplabels, batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def batch_features_labels(features, labels, batch_size):\n",
    "    \"\"\"\n",
    "    Split features and labels into batches\n",
    "    \"\"\"\n",
    "    for start in range(0, len(features), batch_size):\n",
    "        end = min(start + batch_size, len(features))\n",
    "        #print(features[start:end], labels[start:end])\n",
    "        yield features[start:end], labels[start:end]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# x, y = load_preprocessed_training_batch(batch_size=128)\n",
    "# print(x.shape, y.shape)\n",
    "# y.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def normalize(df):\n",
    "    result = df.copy()\n",
    "    for feature_name in df.columns:\n",
    "        max_value = df[feature_name].max()\n",
    "        min_value = df[feature_name].min()\n",
    "        result[feature_name] = (df[feature_name] - min_value) / (max_value - min_value)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: The normalization has already been done at the earlier step in the pipeline. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def OneHotEncode(x):\n",
    "    \n",
    "    lb = LabelBinarizer()\n",
    "    lb.fit(np.array(range(0, 2)))\n",
    "#    lb.classes_\n",
    "    x1 = lb.transform(x)\n",
    "    for i in range(20):\n",
    "        print(x[i], x1[i])\n",
    "    return x1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Input\n",
    "\n",
    "The neural network needs to read the match data, one-hot encoded labels, and dropout keep probability. Implement the following functions\n",
    "* Implement neural_net_image_input\n",
    "  * Return a TF Placeholder\n",
    "  * Set the shape using image_shape with batch size set to None.\n",
    "  * Name the TensorFlow placeholder \"x\" using the TensorFlow name parameter in the TF Placeholder.\n",
    "* Implement neural_net_label_input\n",
    "  * Return a TF Placeholder\n",
    "  * Set the shape using n_classes with batch size set to None.\n",
    "  * Name the TensorFlow placeholder \"y\" using the TensorFlow name parameter in the TF Placeholder.\n",
    "* Implement neural_net_keep_prob_input\n",
    "  * Return a TF Placeholder for dropout keep probability.\n",
    "  * Name the TensorFlow placeholder \"keep_prob\" using the TensorFlow name parameter in the TF Placeholder.\n",
    "* Implement learningRate\n",
    "  * return a TF Placeholder\n",
    "  * name the TF placeholder \"learningRate\" using the name parameter in the TF Placeholder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def neural_net_match_input(input_shape):\n",
    "    \"\"\"\n",
    "    Return a Tensor for a batch of match input\n",
    "    : image_shape: Shape of the the data for a given match -- flattened\n",
    "    : return: Tensor for match input.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    x = tf.placeholder(tf.float32, shape=[None, input_shape], name=\"x\")\n",
    "    return x\n",
    "\n",
    "\n",
    "def neural_net_label_input(n_classes):\n",
    "    \"\"\"\n",
    "    Return a Tensor for a batch of label input\n",
    "    : n_classes: Number of classes\n",
    "    : return: Tensor for label input.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    y = tf.placeholder(tf.float32, shape=[None, n_classes], name=\"y\")\n",
    "    return y\n",
    "\n",
    "\n",
    "def neural_net_keep_prob_input():\n",
    "    \"\"\"\n",
    "    Return a Tensor for keep probability\n",
    "    : return: Tensor for keep probability.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    keep_prob = tf.placeholder(tf.float32, shape=None, name=\"keep_prob\")\n",
    "    return keep_prob\n",
    "\n",
    "def learningRate():\n",
    "    \"\"\"\n",
    "    Return a Tensor for learning Rate\n",
    "    : return: Tensor for learning Rate.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    learningRate = tf.placeholder(tf.float32, shape=None, name=\"learningRate\")\n",
    "    return learningRate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building the network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Output Layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def output(x_tensor, num_outputs):\n",
    "    \n",
    "    \"\"\"\n",
    "    Apply an output layer to x_tensor using weight and bias\n",
    "    : x_tensor: A 2-D tensor where the first dimension is batch size.\n",
    "    : num_outputs: The number of output that the new tensor should be.\n",
    "    : return: A 2-D tensor where the second dimension is num_outputs.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    input_dims = x_tensor.get_shape().as_list()\n",
    "    #print(input_dims)\n",
    "    filter_shape = [input_dims[1], num_outputs]\n",
    "    #print(filter_shape)\n",
    "    \n",
    "    std = math.sqrt( 2/ (filter_shape[0] * filter_shape[1]) ) #sqrt(2/(filterHeight*filterWidth #*filterDepth))    \n",
    "    \n",
    "    weight = tf.Variable(tf.truncated_normal(filter_shape, stddev=std))\n",
    "    #print(weight)\n",
    "    bias = tf.Variable(tf.zeros(num_outputs))\n",
    "    #print(bias)\n",
    "        \n",
    "    fc2 = tf.nn.xw_plus_b(x_tensor, weights=weight, biases=bias)\n",
    "    return fc2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Fully connected Layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fully_conn(x_tensor, num_outputs):\n",
    "    \n",
    "    fc1 = tf.nn.relu(output(x_tensor, num_outputs))\n",
    "    \n",
    "    return fc1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating the Multi_Layer_Perceptron"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "New Structure according to Huang et. al. formulas:\n",
    "(equations 1, 2, and 3: http://dstath.users.uth.gr/papers/IJRS2009_Stathakis.pdf )\n",
    "\n",
    "m = num_class = 1\n",
    "\n",
    "n = num_examples = 86965\n",
    "\n",
    "total_num_hidden_nodes ~= 2(sqrt((m + 1)(n))) = 834\n",
    "\n",
    "Nodes in HL1 = sqrt((m+2)n) + 2(sqrt(n/(m+2))) = 511\n",
    "\n",
    "Nodes in HL2 = m(sqrt(n/(m+2))) = 170"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def MLP(x, keep_prob):\n",
    "    \n",
    "    flattened = x\n",
    "    \n",
    "    # TODO: Apply 1, 2, or 3 Fully Connected Layers\n",
    "    #    Play around with different number of outputs\n",
    "    # Function Definition from Above:\n",
    "    #   fully_conn(x_tensor, num_outputs)\n",
    "    \n",
    "    fc1 = fully_conn(flattened, num_outputs=511)\n",
    "    drop1 = tf.nn.dropout(fc1, keep_prob)\n",
    "    fc2 = fully_conn(drop1, num_outputs=170)\n",
    "\n",
    "#     drop2 = tf.nn.dropout(fc2, keep_prob)\n",
    "#     fc3 = fully_conn(drop2, num_outputs=128)\n",
    "#     drop3 = tf.nn.dropout(fc3, keep_prob)\n",
    "#     fc4 = fully_conn(drop3, num_outputs=64)\n",
    "#     drop4 = tf.nn.dropout(fc4, keep_prob)\n",
    "#     fc5 = fully_conn(drop4, num_outputs=32)\n",
    "    \n",
    "    # TODO: Apply an Output Layer\n",
    "    #    Set this to the number of classes\n",
    "    # Function Definition from Above:\n",
    "    #   output(x_tensor, num_outputs)\n",
    "    \n",
    "    logits = output(fc2, num_outputs=1)\n",
    "    \n",
    "    # TODO: return output\n",
    "    return logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build the Neural Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "##############################\n",
    "## Build the Neural Network ##\n",
    "##############################\n",
    "\n",
    "# Remove previous weights, bias, inputs, etc..\n",
    "tf.reset_default_graph()\n",
    "\n",
    "# Inputs\n",
    "x = neural_net_match_input(input_shape = (12*17))\n",
    "y = neural_net_label_input(n_classes = 1)\n",
    "keep_prob = neural_net_keep_prob_input()\n",
    "#learningRate = learningRate()\n",
    "\n",
    "# Model\n",
    "logits = MLP(x, keep_prob)\n",
    "\n",
    "# Name logits Tensor, so that is can be loaded from disk after training\n",
    "logits = tf.identity(logits, name='logits')\n",
    "\n",
    "# Loss and Optimizer\n",
    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "\n",
    "#Adagrad optimizer.\n",
    "#adaGradOpt = tf.train.AdagradOptimizer(learning_rate = learningRate, name=adaGradOpt).minimize(cost)\n",
    "\n",
    "optimizer = tf.train.AdamOptimizer().minimize(cost)\n",
    "\n",
    "# Accuracy\n",
    "correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')\n",
    "\n",
    "#tests.test_conv_net(conv_net)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bringing in the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0 [0]\n",
      "0.0 [0]\n",
      "0.0 [0]\n",
      "0.0 [0]\n",
      "0.0 [0]\n",
      "0.0 [0]\n",
      "1.0 [1]\n",
      "1.0 [1]\n",
      "1.0 [1]\n",
      "0.0 [0]\n",
      "0.0 [0]\n",
      "1.0 [1]\n",
      "1.0 [1]\n",
      "0.0 [0]\n",
      "1.0 [1]\n",
      "0.0 [0]\n",
      "1.0 [1]\n",
      "1.0 [1]\n",
      "1.0 [1]\n",
      "1.0 [1]\n"
     ]
    }
   ],
   "source": [
    "df = pd.read_csv(\"indexByMatch_batchUdacity_205_CLEANED_NORM.csv\", index_col=\"Unnamed: 0\")\n",
    "#df.drop(\"Unnamed: 0\", axis=1, inplace=True)\n",
    "labels = df[\"alphaVictory\"]\n",
    "#labels = OneHotEncode(labels)\n",
    "df.drop(\"alphaVictory\", axis=1, inplace=True)\n",
    "#print(df.head(n=10))\n",
    "\n",
    "# vars = [\"kills\", \"averageKillDistance\", \"deaths\", \"score\", \"averageScorePerKill\", \"averageScorePerLife\", \"weaponKillsPrimary\", \"weaponKillsSecondary\", \"weaponKillsHeavy\", \"weaponKillsOther\", \"assists\", \"completed\", \"killsDeathsRatio\", \"killsDeathsAssists\", \"activityDurationSeconds\", \"combatRating\", \"team\"]\n",
    "# #print(df.columns)\n",
    "# #print(len(vars))\n",
    "# vars1 = []\n",
    "# for each_element in vars:\n",
    "#     for each_person in range(12):\n",
    "#         if each_element in [\"kills\", \"averageKillDistance\", \"deaths\", \"score\", \"averageScorePerKill\", \"averageScorePerLife\", \"weaponKillsPrimary\", \"weaponKillsSecondary\", \"weaponKillsHeavy\", \"weaponKillsOther\", \"assists\", \"completed\", \"killsDeathsRatio\", \"killsDeathsAssists\", \"activityDurationSeconds\", \"combatRating\", \"team\"]:\n",
    "#             vars1.append(each_element + str(each_person))\n",
    "    \n",
    "# #print(vars1)\n",
    "\n",
    "# df.drop(vars1, axis=1, inplace=True)\n",
    "\n",
    "#print(df.head(n=10), labels.head(n=10))\n",
    "\n",
    "nplabels = OneHotEncode(labels.values)\n",
    "npdf = df.values\n",
    "\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(npdf, nplabels, test_size=0.3, random_state=444)\n",
    "\n",
    "newX_train = np.reshape(X_train, (-1, (12*17)))\n",
    "newy_train = np.reshape(y_train, (-1, 1))\n",
    "newX_test = np.reshape(X_test, (-1, (12*17)))\n",
    "newy_test = np.reshape(y_test, (-1, 1))\n",
    "\n",
    "X_TEST, X_VALID, y_TEST, y_VALID = train_test_split(newX_test, newy_test, test_size=0.33, random_state=120)\n",
    "\n",
    "X_TEST = np.reshape(X_TEST, (-1, (12*17)))\n",
    "y_TEST = np.reshape(y_TEST, (-1, 1))\n",
    "X_VALID = np.reshape(X_VALID, (-1, (12*17)))\n",
    "y_VALID = np.reshape(y_VALID, (-1, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(124236, 204)\n",
      "(124236, 1)\n",
      "(86965, 204)\n",
      "(86965, 1)\n",
      "(37271, 204)\n",
      "(37271, 1)\n"
     ]
    }
   ],
   "source": [
    "print(npdf.shape)\n",
    "print(nplabels.shape)\n",
    "print(newX_train.shape)\n",
    "print(newy_train.shape)\n",
    "print(newX_test.shape)\n",
    "print(newy_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):\n",
    "    \"\"\"\n",
    "    Optimize the session on a batch of matches and alphaVictory labels\n",
    "    : session: Current TensorFlow session\n",
    "    : optimizer: TensorFlow optimizer function\n",
    "    : keep_probability: keep probability\n",
    "    : feature_batch: Batch of Numpy match data\n",
    "    : label_batch: Batch of Numpy label data\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    \n",
    "    session.run(optimizer, feed_dict={x:feature_batch, y:label_batch, keep_prob:keep_probability})\n",
    "    \n",
    "    return\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Showing stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: newX_test and newy_test are global variables. They're the testing set and are instantiated when the data is read into the program and split into training/testing sets. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def print_stats(session, feature_batch, label_batch, cost, accuracy):\n",
    "    \"\"\"\n",
    "    Print information about loss and validation accuracy\n",
    "    : session: Current TensorFlow session\n",
    "    : feature_batch: Batch of Numpy match data\n",
    "    : label_batch: Batch of Numpy label data\n",
    "    : cost: TensorFlow cost function\n",
    "    : accuracy: TensorFlow accuracy function\n",
    "    \"\"\"\n",
    "    \n",
    "    # TODO: Implement Function\n",
    "\n",
    "    loss = session.run(cost, feed_dict={x: feature_batch, y:label_batch,  keep_prob: 1.00})\n",
    "    acc_ans = session.run(accuracy, feed_dict={x:X_VALID, y:y_VALID, keep_prob: 1.00})\n",
    "    print(\"\")\n",
    "    print(\"cost/loss:  \", loss)\n",
    "    print(\"accuracy:   \", acc_ans)\n",
    "       \n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Set epochs to the number of iterations until the network stops learning or start overfitting\n",
    "* Set batch_size to the highest number that your machine has memory for. Most people set them to common sizes of memory:\n",
    "  * 64\n",
    "  * 128\n",
    "  * 256\n",
    "  * ...\n",
    "* Set keep_probability to the probability of keeping a node using dropout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 250,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO: Tune Parameters\n",
    "epochs = 10\n",
    "batch_size = 128\n",
    "keep_probability = 1.00\n",
    "#learningRate = 1e-6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training on single (and smaller) testing batch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of training the neural network on all the matches of data, let's use a single small test batch. This should save time while you iterate on the model to get a better accuracy. Once the final validation accuracy is 50% or greater, run the model on all the data in the next section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 251,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking the Training on the testing batch...\n",
      "Epoch  1, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  2, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  3, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  4, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  5, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  6, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  7, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  8, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch  9, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n",
      "Epoch 10, Destiny Matches Batch Full dataset:  \n",
      "cost/loss:   0.0\n",
      "accuracy:    1.0\n"
     ]
    }
   ],
   "source": [
    "print('Checking the Training on the testing batch...')\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    # Initializing the variables\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    \n",
    "    # Training cycle\n",
    "    for epoch in range(epochs):\n",
    "        batch_i = \"Full dataset\"\n",
    "        for batch_features, batch_labels in split_preprocessed_training_batch(batch_size, newX_train, newy_train):\n",
    "            \n",
    "            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)\n",
    "        print('Epoch {:>2}, Destiny Matches Batch {}:  '.format(epoch + 1, batch_i), end='')\n",
    "        print_stats(sess, batch_features, batch_labels, cost, accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}