notebooks/09_native_ftrl_solutions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b6f0ce57",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "papermill": {
     "duration": 0.021178,
     "end_time": "2021-08-25T09:34:17.990975",
     "exception": false,
     "start_time": "2021-08-25T09:34:17.969797",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<center>\n",
    "    <h1>DatatableTon</h1>\n",
    "    💯 datatable exercises\n",
    "    <br>\n",
    "    <br>\n",
    "    <a href='https://github.com/vopani/datatableton/blob/master/LICENSE'>\n",
    "        <img src='https://img.shields.io/badge/license-Apache%202.0-blue.svg?logo=apache'>\n",
    "    </a>\n",
    "    <a href='https://github.com/vopani/datatableton'>\n",
    "        <img src='https://img.shields.io/github/stars/vopani/datatableton?color=yellowgreen&logo=github'>\n",
    "    </a>\n",
    "    <a href='https://twitter.com/vopani'>\n",
    "        <img src='https://img.shields.io/twitter/follow/vopani'>\n",
    "    </a>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7213e068",
   "metadata": {
    "papermill": {
     "duration": 0.019664,
     "end_time": "2021-08-25T09:34:18.031038",
     "exception": false,
     "start_time": "2021-08-25T09:34:18.011374",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<center>\n",
    "    This is Set 9: Native Models (Exercises 81-90) of <b>DatatableTon</b>: <i>💯 datatable exercises</i>\n",
    "    <br>\n",
    "    You can find all the exercises and solutions on <a href=\"https://github.com/vopani/datatableton#exercises-\">GitHub</a>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b8357ee",
   "metadata": {
    "papermill": {
     "duration": 0.019797,
     "end_time": "2021-08-25T09:34:18.071289",
     "exception": false,
     "start_time": "2021-08-25T09:34:18.051492",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Prerequisites**\n",
    "* The `datatable` package should be upgraded to the latest version (or v1.0.0+).\n",
    "* The sample dataset [1% sample of KDD Cup 2012 (CTR Prediction)](https://www.openml.org/d/1219) from openml will be used for the exercises."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "28d8cc63",
   "metadata": {
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:18.126612Z",
     "iopub.status.busy": "2021-08-25T09:34:18.125875Z",
     "iopub.status.idle": "2021-08-25T09:34:53.564736Z",
     "shell.execute_reply": "2021-08-25T09:34:53.563936Z",
     "shell.execute_reply.started": "2021-07-26T10:03:10.473916Z"
    },
    "papermill": {
     "duration": 35.473996,
     "end_time": "2021-08-25T09:34:53.564923",
     "exception": false,
     "start_time": "2021-08-25T09:34:18.090927",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pip in /opt/conda/lib/python3.7/site-packages (21.1.2)\r\n",
      "Collecting pip\r\n",
      "  Downloading pip-21.2.4-py3-none-any.whl (1.6 MB)\r\n",
      "\u001b[K     |████████████████████████████████| 1.6 MB 864 kB/s \r\n",
      "\u001b[?25hInstalling collected packages: pip\r\n",
      "  Attempting uninstall: pip\r\n",
      "    Found existing installation: pip 21.1.2\r\n",
      "    Uninstalling pip-21.1.2:\r\n",
      "      Successfully uninstalled pip-21.1.2\r\n",
      "Successfully installed pip-21.2.4\r\n",
      "\u001b[33mWARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv\u001b[0m\r\n",
      "Requirement already satisfied: datatable in /opt/conda/lib/python3.7/site-packages (1.0.0)\r\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\r\n",
      "--2021-08-25 09:34:50--  https://www.openml.org/data/get_csv/183151/kdd_ctr.csv\r\n",
      "Resolving www.openml.org (www.openml.org)... 131.155.11.11\r\n",
      "Connecting to www.openml.org (www.openml.org)|131.155.11.11|:443... connected.\r\n",
      "HTTP request sent, awaiting response... 200 OK\r\n",
      "Length: unspecified [text/plain]\r\n",
      "Saving to: ‘kdd_ctr.csv’\r\n",
      "\r\n",
      "kdd_ctr.csv             [          <=>       ]  27.50M  12.1MB/s    in 2.3s    \r\n",
      "\r\n",
      "2021-08-25 09:34:53 (12.1 MB/s) - ‘kdd_ctr.csv’ saved [28837031]\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!python3 -m pip install -U pip\n",
    "!python3 -m pip install -U datatable\n",
    "!wget https://www.openml.org/data/get_csv/183151/kdd_ctr.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f65ac0fb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:53.630511Z",
     "iopub.status.busy": "2021-08-25T09:34:53.629537Z",
     "iopub.status.idle": "2021-08-25T09:34:53.888363Z",
     "shell.execute_reply": "2021-08-25T09:34:53.888864Z",
     "shell.execute_reply.started": "2021-07-26T10:03:53.38345Z"
    },
    "papermill": {
     "duration": 0.294355,
     "end_time": "2021-08-25T09:34:53.889066",
     "exception": false,
     "start_time": "2021-08-25T09:34:53.594711",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type='text/css'>\n",
       ".datatable table.frame { margin-bottom: 0; }\n",
       ".datatable table.frame thead { border-bottom: none; }\n",
       ".datatable table.frame tr.coltypes td {  color: #FFFFFF;  line-height: 6px;  padding: 0 0.5em;}\n",
       ".datatable .bool    { background: #DDDD99; }\n",
       ".datatable .object  { background: #565656; }\n",
       ".datatable .int     { background: #5D9E5D; }\n",
       ".datatable .float   { background: #4040CC; }\n",
       ".datatable .str     { background: #CC4040; }\n",
       ".datatable .time    { background: #40CC40; }\n",
       ".datatable .row_index {  background: var(--jp-border-color3);  border-right: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  font-size: 9px;}\n",
       ".datatable .frame tbody td { text-align: left; }\n",
       ".datatable .frame tr.coltypes .row_index {  background: var(--jp-border-color0);}\n",
       ".datatable th:nth-child(2) { padding-left: 12px; }\n",
       ".datatable .hellipsis {  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .vellipsis {  background: var(--jp-layout-color0);  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .na {  color: var(--jp-cell-editor-border-color);  font-size: 80%;}\n",
       ".datatable .sp {  opacity: 0.25;}\n",
       ".datatable .footer { font-size: 9px; }\n",
       ".datatable .frame_dimensions {  background: var(--jp-border-color3);  border-top: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  display: inline-block;  opacity: 0.6;  padding: 1px 10px 1px 5px;}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>click</th><th>impression</th><th>url_hash</th><th>ad_id</th><th>advertiser_id</th><th>depth</th><th>position</th><th>query_id</th><th>keyword_id</th><th>title_id</th><th>description_id</th><th>user_id</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='bool' title='bool8'>&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='float' title='float64'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>0</td><td>1</td><td>1.071e+19</td><td>8343295</td><td>11700</td><td>3</td><td>3</td><td>7702266</td><td>21264</td><td>27892</td><td>1559</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>1</td><td>1</td><td>1.73639e+19</td><td>20017077</td><td>23798</td><td>1</td><td>1</td><td>93079</td><td>35498</td><td>4</td><td>36476</td><td>562934</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>0</td><td>1</td><td>8.91547e+18</td><td>21348354</td><td>36654</td><td>1</td><td>1</td><td>10981</td><td>19975</td><td>36105</td><td>33292</td><td>11621116</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>0</td><td>1</td><td>4.42669e+18</td><td>20366086</td><td>33280</td><td>3</td><td>3</td><td>0</td><td>5942</td><td>4057</td><td>4390</td><td>8778348</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>0</td><td>1</td><td>1.15726e+19</td><td>6803526</td><td>10790</td><td>2</td><td>1</td><td>9881978</td><td>60593</td><td>25242</td><td>1679</td><td>12118311</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>1</td><td>1</td><td>2.82758e+17</td><td>21186478</td><td>35793</td><td>2</td><td>1</td><td>163315</td><td>4871</td><td>3257</td><td>1153</td><td>2886008</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>0</td><td>1</td><td>8.8139e+18</td><td>20886690</td><td>34840</td><td>2</td><td>2</td><td>316</td><td>543</td><td>2206</td><td>2888</td><td>7589739</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>0</td><td>1</td><td>3.81103e+18</td><td>21367376</td><td>20667</td><td>3</td><td>2</td><td>2601439</td><td>118</td><td>9594</td><td>9705</td><td>579253</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>0</td><td>1</td><td>9.80684e+18</td><td>21811752</td><td>37737</td><td>3</td><td>2</td><td>1631</td><td>333</td><td>841</td><td>2175</td><td>5277279</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>1</td><td>1</td><td>1.43404e+19</td><td>9027213</td><td>23808</td><td>2</td><td>1</td><td>5</td><td>1</td><td>0</td><td>0</td><td>11808635</td></tr>\n",
       "    <tr><td class='row_index'>10</td><td>0</td><td>1</td><td>1.15485e+19</td><td>21188630</td><td>17432</td><td>3</td><td>3</td><td>54485</td><td>20526</td><td>680977</td><td>621442</td><td>329865</td></tr>\n",
       "    <tr><td class='row_index'>11</td><td>1</td><td>3</td><td>1.20579e+19</td><td>20170434</td><td>27961</td><td>1</td><td>1</td><td>16554377</td><td>63355</td><td>106189</td><td>100521</td><td>1378623</td></tr>\n",
       "    <tr><td class='row_index'>12</td><td>0</td><td>1</td><td>2.35282e+18</td><td>10362063</td><td>28873</td><td>2</td><td>1</td><td>3203</td><td>38</td><td>1150</td><td>1129</td><td>5864198</td></tr>\n",
       "    <tr><td class='row_index'>13</td><td>0</td><td>1</td><td>5.12068e+18</td><td>8676724</td><td>1268</td><td>1</td><td>1</td><td>1330</td><td>40</td><td>45</td><td>13</td><td>16760230</td></tr>\n",
       "    <tr><td class='row_index'>14</td><td>0</td><td>1</td><td>5.58965e+18</td><td>20950936</td><td>35067</td><td>1</td><td>1</td><td>637</td><td>9563</td><td>8908</td><td>42</td><td>19015059</td></tr>\n",
       "    <tr><td class='row_index'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td></tr>\n",
       "    <tr><td class='row_index'>399,477</td><td>0</td><td>1</td><td>2.95046e+18</td><td>3405979</td><td>4613</td><td>2</td><td>1</td><td>1914</td><td>1681</td><td>65853</td><td>77289</td><td>11811980</td></tr>\n",
       "    <tr><td class='row_index'>399,478</td><td>0</td><td>1</td><td>1.73639e+19</td><td>20017083</td><td>23798</td><td>1</td><td>1</td><td>172673</td><td>1645</td><td>1418</td><td>1678</td><td>10027694</td></tr>\n",
       "    <tr><td class='row_index'>399,479</td><td>1</td><td>2</td><td>5.85125e+18</td><td>10421215</td><td>28698</td><td>2</td><td>2</td><td>1238</td><td>3230</td><td>512144</td><td>37399</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>399,480</td><td>0</td><td>1</td><td>5.88003e+18</td><td>4832693</td><td>15994</td><td>3</td><td>3</td><td>2242779</td><td>3054</td><td>167005</td><td>55548</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>399,481</td><td>0</td><td>1</td><td>1.43404e+19</td><td>4757453</td><td>23777</td><td>3</td><td>3</td><td>40982</td><td>10394</td><td>179968</td><td>152955</td><td>75532</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>399,482 rows &times; 12 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff8845fe270 399482x12>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import datatable as dt\n",
    "\n",
    "data = dt.fread('kdd_ctr.csv', fill=True)[1:,:]\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a71333df",
   "metadata": {
    "papermill": {
     "duration": 0.034908,
     "end_time": "2021-08-25T09:34:53.963801",
     "exception": false,
     "start_time": "2021-08-25T09:34:53.928893",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 81: Initialize a FTRL model `model_ftrl`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f6c299b8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.031979Z",
     "iopub.status.busy": "2021-08-25T09:34:54.030926Z",
     "iopub.status.idle": "2021-08-25T09:34:54.036707Z",
     "shell.execute_reply": "2021-08-25T09:34:54.037314Z",
     "shell.execute_reply.started": "2021-07-26T10:07:59.418265Z"
    },
    "papermill": {
     "duration": 0.042861,
     "end_time": "2021-08-25T09:34:54.037476",
     "exception": false,
     "start_time": "2021-08-25T09:34:53.994615",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<datatable.models.Ftrl at 0x7ff8845fe720>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datatable.models import Ftrl\n",
    "\n",
    "model_ftrl = Ftrl()\n",
    "model_ftrl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1ddaa46",
   "metadata": {
    "papermill": {
     "duration": 0.03192,
     "end_time": "2021-08-25T09:34:54.100646",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.068726",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 82: Display the hyperparameters of `model_ftrl`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4cc279f4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.165900Z",
     "iopub.status.busy": "2021-08-25T09:34:54.165024Z",
     "iopub.status.idle": "2021-08-25T09:34:54.170760Z",
     "shell.execute_reply": "2021-08-25T09:34:54.171354Z",
     "shell.execute_reply.started": "2021-07-26T10:12:53.662442Z"
    },
    "papermill": {
     "duration": 0.03993,
     "end_time": "2021-08-25T09:34:54.171513",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.131583",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FtrlParams(alpha=0.005, beta=1.0, lambda1=0.0, lambda2=0.0, nbins=1000000, mantissa_nbits=10, nepochs=1.0, double_precision=False, negative_class=False, interactions=None, model_type='auto')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_ftrl.params"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c04e077",
   "metadata": {
    "papermill": {
     "duration": 0.030955,
     "end_time": "2021-08-25T09:34:54.233887",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.202932",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 83: Split `data` into `train_data` consisting of the first 75% of rows and `valid_data` consisting of the last 25% of rows**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4290a222",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.299665Z",
     "iopub.status.busy": "2021-08-25T09:34:54.298710Z",
     "iopub.status.idle": "2021-08-25T09:34:54.304860Z",
     "shell.execute_reply": "2021-08-25T09:34:54.305423Z",
     "shell.execute_reply.started": "2021-07-26T10:08:05.202565Z"
    },
    "papermill": {
     "duration": 0.04065,
     "end_time": "2021-08-25T09:34:54.305577",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.264927",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(299612, 12)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data = data[:round(data.nrows * 0.75), :]\n",
    "train_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4d23dab0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.372054Z",
     "iopub.status.busy": "2021-08-25T09:34:54.371070Z",
     "iopub.status.idle": "2021-08-25T09:34:54.377086Z",
     "shell.execute_reply": "2021-08-25T09:34:54.377528Z",
     "shell.execute_reply.started": "2021-07-26T10:08:05.810837Z"
    },
    "papermill": {
     "duration": 0.040972,
     "end_time": "2021-08-25T09:34:54.377688",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.336716",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(99870, 12)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "valid_data = data[round(data.nrows * 0.75):, :]\n",
    "valid_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdab38ab",
   "metadata": {
    "papermill": {
     "duration": 0.031921,
     "end_time": "2021-08-25T09:34:54.441281",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.409360",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 84: Assign the target variable `click` to `train_target` from `train_data` and `valid_target` from `valid_data` and drop the columns `url_hash` and `click` from `train_data` and `valid_data`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3a9d2ba6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.513690Z",
     "iopub.status.busy": "2021-08-25T09:34:54.512738Z",
     "iopub.status.idle": "2021-08-25T09:34:54.520198Z",
     "shell.execute_reply": "2021-08-25T09:34:54.519579Z",
     "shell.execute_reply.started": "2021-07-26T10:08:28.654474Z"
    },
    "papermill": {
     "duration": 0.04347,
     "end_time": "2021-08-25T09:34:54.520350",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.476880",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>click</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='bool' title='bool8'>&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>1</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>10 rows &times; 1 column</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff88462b750 10x1>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_target = train_data['click']\n",
    "train_target.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f8ec1163",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.604194Z",
     "iopub.status.busy": "2021-08-25T09:34:54.603530Z",
     "iopub.status.idle": "2021-08-25T09:34:54.606246Z",
     "shell.execute_reply": "2021-08-25T09:34:54.606708Z",
     "shell.execute_reply.started": "2021-07-26T10:08:29.181764Z"
    },
    "papermill": {
     "duration": 0.041197,
     "end_time": "2021-08-25T09:34:54.606871",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.565674",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>click</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='bool' title='bool8'>&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>1</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>10 rows &times; 1 column</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff88462bfc0 10x1>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "valid_target = valid_data['click']\n",
    "valid_target.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "083bf6f6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.675943Z",
     "iopub.status.busy": "2021-08-25T09:34:54.675335Z",
     "iopub.status.idle": "2021-08-25T09:34:54.681682Z",
     "shell.execute_reply": "2021-08-25T09:34:54.681203Z",
     "shell.execute_reply.started": "2021-07-26T10:08:34.458014Z"
    },
    "papermill": {
     "duration": 0.042002,
     "end_time": "2021-08-25T09:34:54.681813",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.639811",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>impression</th><th>ad_id</th><th>advertiser_id</th><th>depth</th><th>position</th><th>query_id</th><th>keyword_id</th><th>title_id</th><th>description_id</th><th>user_id</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>1</td><td>8343295</td><td>11700</td><td>3</td><td>3</td><td>7702266</td><td>21264</td><td>27892</td><td>1559</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>1</td><td>20017077</td><td>23798</td><td>1</td><td>1</td><td>93079</td><td>35498</td><td>4</td><td>36476</td><td>562934</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>1</td><td>21348354</td><td>36654</td><td>1</td><td>1</td><td>10981</td><td>19975</td><td>36105</td><td>33292</td><td>11621116</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>1</td><td>20366086</td><td>33280</td><td>3</td><td>3</td><td>0</td><td>5942</td><td>4057</td><td>4390</td><td>8778348</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>1</td><td>6803526</td><td>10790</td><td>2</td><td>1</td><td>9881978</td><td>60593</td><td>25242</td><td>1679</td><td>12118311</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>1</td><td>21186478</td><td>35793</td><td>2</td><td>1</td><td>163315</td><td>4871</td><td>3257</td><td>1153</td><td>2886008</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>1</td><td>20886690</td><td>34840</td><td>2</td><td>2</td><td>316</td><td>543</td><td>2206</td><td>2888</td><td>7589739</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>1</td><td>21367376</td><td>20667</td><td>3</td><td>2</td><td>2601439</td><td>118</td><td>9594</td><td>9705</td><td>579253</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>1</td><td>21811752</td><td>37737</td><td>3</td><td>2</td><td>1631</td><td>333</td><td>841</td><td>2175</td><td>5277279</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>1</td><td>9027213</td><td>23808</td><td>2</td><td>1</td><td>5</td><td>1</td><td>0</td><td>0</td><td>11808635</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>10 rows &times; 10 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff88462be70 10x10>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del train_data[:, ['url_hash', 'click']]\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9c361c3f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.754273Z",
     "iopub.status.busy": "2021-08-25T09:34:54.753344Z",
     "iopub.status.idle": "2021-08-25T09:34:54.756860Z",
     "shell.execute_reply": "2021-08-25T09:34:54.757409Z",
     "shell.execute_reply.started": "2021-07-26T10:08:39.298171Z"
    },
    "papermill": {
     "duration": 0.042376,
     "end_time": "2021-08-25T09:34:54.757568",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.715192",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>impression</th><th>ad_id</th><th>advertiser_id</th><th>depth</th><th>position</th><th>query_id</th><th>keyword_id</th><th>title_id</th><th>description_id</th><th>user_id</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='int' title='int32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>1</td><td>21104237</td><td>29683</td><td>2</td><td>2</td><td>10689741</td><td>11495</td><td>43449</td><td>1704</td><td>11631003</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>1</td><td>21162526</td><td>1325</td><td>2</td><td>1</td><td>15</td><td>10</td><td>24</td><td>25</td><td>2220103</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>1</td><td>9027613</td><td>23808</td><td>2</td><td>1</td><td>2940</td><td>692</td><td>138</td><td>198</td><td>11428948</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>1</td><td>21866030</td><td>385</td><td>3</td><td>1</td><td>3801572</td><td>742504</td><td>2551939</td><td>1725632</td><td>1927769</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>1</td><td>10397715</td><td>27354</td><td>3</td><td>1</td><td>41805</td><td>15602</td><td>675726</td><td>315253</td><td>340926</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>1</td><td>21264083</td><td>36307</td><td>2</td><td>1</td><td>2491</td><td>2723</td><td>502995</td><td>228073</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>1</td><td>1000515</td><td>2714</td><td>3</td><td>3</td><td>349332</td><td>4075</td><td>3106</td><td>3835</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>1</td><td>21362188</td><td>36729</td><td>1</td><td>1</td><td>14198118</td><td>647</td><td>2046</td><td>2265</td><td>376316</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>2</td><td>3048011</td><td>23782</td><td>1</td><td>1</td><td>28487</td><td>484</td><td>322</td><td>463</td><td>31285</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>1</td><td>7098654</td><td>19581</td><td>2</td><td>1</td><td>2974680</td><td>60887</td><td>512601</td><td>576993</td><td>4984469</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>10 rows &times; 10 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff88462bdb0 10x10>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del valid_data[:, ['url_hash', 'click']]\n",
    "valid_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a23a6ab1",
   "metadata": {
    "papermill": {
     "duration": 0.033638,
     "end_time": "2021-08-25T09:34:54.825336",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.791698",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 85: Train `model_ftrl` using `train_data` and `train_target`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d427e1d0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:54.896273Z",
     "iopub.status.busy": "2021-08-25T09:34:54.895430Z",
     "iopub.status.idle": "2021-08-25T09:34:55.031498Z",
     "shell.execute_reply": "2021-08-25T09:34:55.032387Z",
     "shell.execute_reply.started": "2021-07-26T10:08:52.346923Z"
    },
    "papermill": {
     "duration": 0.173452,
     "end_time": "2021-08-25T09:34:55.032585",
     "exception": false,
     "start_time": "2021-08-25T09:34:54.859133",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<datatable.models.Ftrl at 0x7ff8845fe720>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_ftrl.fit(train_data, train_target)\n",
    "model_ftrl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5062845a",
   "metadata": {
    "papermill": {
     "duration": 0.036015,
     "end_time": "2021-08-25T09:34:55.112357",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.076342",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 86: Assign the feature importances of `model_ftrl` to `model_ftrl_fimp`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c2bd3e39",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:55.189207Z",
     "iopub.status.busy": "2021-08-25T09:34:55.188365Z",
     "iopub.status.idle": "2021-08-25T09:34:55.191779Z",
     "shell.execute_reply": "2021-08-25T09:34:55.189832Z",
     "shell.execute_reply.started": "2021-07-26T10:09:59.090216Z"
    },
    "papermill": {
     "duration": 0.045708,
     "end_time": "2021-08-25T09:34:55.191938",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.146230",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>feature_name</th><th>feature_importance</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='str' title='str32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='float' title='float32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>impression</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>ad_id</td><td>0.0141565</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>advertiser_id</td><td>0.0777534</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>depth</td><td>0.678434</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>position</td><td>0.721739</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>query_id</td><td>0.0126874</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>keyword_id</td><td>0.0173265</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>title_id</td><td>0.0101844</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>description_id</td><td>0.0129936</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>user_id</td><td>0.182643</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>10 rows &times; 2 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff88462bde0 10x2>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_ftrl_fimp = model_ftrl.feature_importances\n",
    "model_ftrl_fimp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b7c2b30",
   "metadata": {
    "papermill": {
     "duration": 0.035709,
     "end_time": "2021-08-25T09:34:55.270087",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.234378",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 87: Use `model_ftrl` to predict on `valid_data` and assign the predictions to `preds_ftrl`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e64f1aed",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:55.377422Z",
     "iopub.status.busy": "2021-08-25T09:34:55.376410Z",
     "iopub.status.idle": "2021-08-25T09:34:55.381825Z",
     "shell.execute_reply": "2021-08-25T09:34:55.382386Z",
     "shell.execute_reply.started": "2021-07-26T10:10:52.660901Z"
    },
    "papermill": {
     "duration": 0.077952,
     "end_time": "2021-08-25T09:34:55.382573",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.304621",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>False</th><th>True</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='float' title='float32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='float' title='float32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>0.84105</td><td>0.15895</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>0.755434</td><td>0.244566</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>0.781626</td><td>0.218374</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>0.824817</td><td>0.175183</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>0.823578</td><td>0.176422</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>0.859613</td><td>0.140387</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>0.90428</td><td>0.0957198</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>0.807028</td><td>0.192972</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>0.645712</td><td>0.354288</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>0.792004</td><td>0.207996</td></tr>\n",
       "    <tr><td class='row_index'>10</td><td>0.836483</td><td>0.163517</td></tr>\n",
       "    <tr><td class='row_index'>11</td><td>0.907333</td><td>0.092667</td></tr>\n",
       "    <tr><td class='row_index'>12</td><td>0.893126</td><td>0.106874</td></tr>\n",
       "    <tr><td class='row_index'>13</td><td>0.813899</td><td>0.186101</td></tr>\n",
       "    <tr><td class='row_index'>14</td><td>0.786944</td><td>0.213056</td></tr>\n",
       "    <tr><td class='row_index'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td></tr>\n",
       "    <tr><td class='row_index'>99,865</td><td>0.79014</td><td>0.20986</td></tr>\n",
       "    <tr><td class='row_index'>99,866</td><td>0.805463</td><td>0.194537</td></tr>\n",
       "    <tr><td class='row_index'>99,867</td><td>0.842936</td><td>0.157064</td></tr>\n",
       "    <tr><td class='row_index'>99,868</td><td>0.905877</td><td>0.0941228</td></tr>\n",
       "    <tr><td class='row_index'>99,869</td><td>0.874907</td><td>0.125093</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>99,870 rows &times; 2 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff88462bd80 99870x2>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preds_ftrl = model_ftrl.predict(valid_data)\n",
    "preds_ftrl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73124f6b",
   "metadata": {
    "papermill": {
     "duration": 0.0359,
     "end_time": "2021-08-25T09:34:55.461866",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.425966",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 88: Calculate the logloss of `preds_ftrl` with `valid_target`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "a0fee14a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:55.571247Z",
     "iopub.status.busy": "2021-08-25T09:34:55.570233Z",
     "iopub.status.idle": "2021-08-25T09:34:55.574629Z",
     "shell.execute_reply": "2021-08-25T09:34:55.573958Z",
     "shell.execute_reply.started": "2021-07-26T10:11:42.131083Z"
    },
    "papermill": {
     "duration": 0.078384,
     "end_time": "2021-08-25T09:34:55.574765",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.496381",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4300276041030884"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preds = dt.cbind(preds_ftrl, valid_target)\n",
    "preds[:, -dt.mean(dt.f.click * dt.math.log(dt.f['True']) + (1-dt.f.click) * dt.math.log(dt.f['False']))][0, 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1cb38a2",
   "metadata": {
    "papermill": {
     "duration": 0.035166,
     "end_time": "2021-08-25T09:34:55.645774",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.610608",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 89: Train a new FTRL model `model_ftrl_2` with `lambda2=1.2`, `nepochs=5` and `double_precision=True` using `train_data` and `train_target`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f6dd39cb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:56.351966Z",
     "iopub.status.busy": "2021-08-25T09:34:56.351212Z",
     "iopub.status.idle": "2021-08-25T09:34:56.355339Z",
     "shell.execute_reply": "2021-08-25T09:34:56.356217Z",
     "shell.execute_reply.started": "2021-07-26T10:12:00.913762Z"
    },
    "papermill": {
     "duration": 0.67538,
     "end_time": "2021-08-25T09:34:56.356416",
     "exception": false,
     "start_time": "2021-08-25T09:34:55.681036",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<datatable.models.Ftrl at 0x7ff8841b3450>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_ftrl_2 = Ftrl(lambda2=1.2, nepochs=5, double_precision=True)\n",
    "model_ftrl_2.fit(train_data, train_target)\n",
    "model_ftrl_2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ee500df",
   "metadata": {
    "papermill": {
     "duration": 0.036341,
     "end_time": "2021-08-25T09:34:56.438768",
     "exception": false,
     "start_time": "2021-08-25T09:34:56.402427",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Exercise 90: Train a new FTRL model `model_ftrl_cv` with `alpha=0.01` and `nepochs=3` over 5-fold cross-validation using all features in `data` and assign the cross-validated predictions along with the target variable `click` to `preds_ftrl_cv`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "55ce3e0c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-25T09:34:56.920083Z",
     "iopub.status.busy": "2021-08-25T09:34:56.919071Z",
     "iopub.status.idle": "2021-08-25T09:34:58.693334Z",
     "shell.execute_reply": "2021-08-25T09:34:58.694280Z",
     "shell.execute_reply.started": "2021-07-19T09:03:14.58146Z"
    },
    "papermill": {
     "duration": 2.220225,
     "end_time": "2021-08-25T09:34:58.694484",
     "exception": false,
     "start_time": "2021-08-25T09:34:56.474259",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div class='datatable'>\n",
       "  <table class='frame'>\n",
       "  <thead>\n",
       "    <tr class='colnames'><td class='row_index'></td><th>False</th><th>True</th><th>click</th></tr>\n",
       "    <tr class='coltypes'><td class='row_index'></td><td class='float' title='float32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='float' title='float32'>&#x25AA;&#x25AA;&#x25AA;&#x25AA;</td><td class='bool' title='bool8'>&#x25AA;</td></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><td class='row_index'>0</td><td>0.945976</td><td>0.0540244</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>1</td><td>0.805362</td><td>0.194638</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>2</td><td>0.815628</td><td>0.184372</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>3</td><td>0.955324</td><td>0.0446757</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>4</td><td>0.785547</td><td>0.214453</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>5</td><td>0.756429</td><td>0.243571</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>6</td><td>0.842474</td><td>0.157526</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>7</td><td>0.91301</td><td>0.0869895</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>8</td><td>0.901192</td><td>0.0988076</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>9</td><td>0.726925</td><td>0.273075</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>10</td><td>0.9064</td><td>0.0936005</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>11</td><td>0.763865</td><td>0.236135</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>12</td><td>0.783399</td><td>0.216601</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>13</td><td>0.88228</td><td>0.11772</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>14</td><td>0.807331</td><td>0.192669</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td><td class='hellipsis'>&#x22EE;</td></tr>\n",
       "    <tr><td class='row_index'>399,477</td><td>0.763977</td><td>0.236023</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>399,478</td><td>0.808232</td><td>0.191768</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>399,479</td><td>0.85208</td><td>0.14792</td><td>1</td></tr>\n",
       "    <tr><td class='row_index'>399,480</td><td>0.942647</td><td>0.0573533</td><td>0</td></tr>\n",
       "    <tr><td class='row_index'>399,481</td><td>0.917701</td><td>0.0822989</td><td>0</td></tr>\n",
       "  </tbody>\n",
       "  </table>\n",
       "  <div class='footer'>\n",
       "    <div class='frame_dimensions'>399,482 rows &times; 3 columns</div>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<Frame#7ff8841b8090 399482x3>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datatable.models import kfold\n",
    "\n",
    "preds_ftrl_cv = dt.Frame()\n",
    "\n",
    "for (train_idx, valid_idx) in kfold(nrows=data.nrows, nsplits=5):\n",
    "    train_data = data[train_idx, [col for col in data.names if col != 'click']]\n",
    "    valid_data = data[valid_idx, [col for col in data.names if col != 'click']]\n",
    "    train_target = data[train_idx, 'click']\n",
    "    valid_target = data[valid_idx, 'click']\n",
    "\n",
    "    model_ftrl_cv = Ftrl(alpha=0.01, nepochs=3)\n",
    "    model_ftrl_cv.fit(train_data, train_target)\n",
    "    preds_ftrl = model_ftrl_cv.predict(valid_data)\n",
    "    preds_ftrl.cbind(valid_target)\n",
    "\n",
    "    preds_ftrl_cv.rbind(preds_ftrl)\n",
    "\n",
    "preds_ftrl_cv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d11a11c8",
   "metadata": {
    "papermill": {
     "duration": 0.036799,
     "end_time": "2021-08-25T09:34:58.775544",
     "exception": false,
     "start_time": "2021-08-25T09:34:58.738745",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "✅ This completes Set 9: Native Models (Exercises 81-90) of **DatatableTon**: *💯 datatable exercises*\n",
    "\n",
    "#### Set 10 • Capstone Projects • Expert • Exercises 91-100\n",
    "\n",
    "| Style | Colab | Kaggle | Binder | GitHub |\n",
    "| ----- | ----- | ------ | ------ | ------ |\n",
    "| Exercises | [![Open in Colab](https://img.shields.io/static/v1?label=&message=Open%20in%20Colab&labelColor=grey&color=blue&logo=google-colab)](https://colab.research.google.com/github/vopani/datatableton/blob/main/notebooks/10_capstone_projects_exercises.ipynb) | [![Open in Kaggle](https://img.shields.io/static/v1?label=&message=Open%20in%20Kaggle&labelColor=grey&color=blue&logo=kaggle)](https://www.kaggle.com/rohanrao/datatableton-capstone-projects-exercises) | [![Open in Binder](https://img.shields.io/static/v1?label=&message=Open%20in%20Binder&labelColor=grey&color=blue&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFkAAABZCAMAAABi1XidAAAB8lBMVEX///9XmsrmZYH1olJXmsr1olJXmsrmZYH1olJXmsr1olJXmsrmZYH1olL1olJXmsr1olJXmsrmZYH1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olJXmsrmZYH1olL1olL0nFf1olJXmsrmZYH1olJXmsq8dZb1olJXmsrmZYH1olJXmspXmspXmsr1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olLeaIVXmsrmZYH1olL1olL1olJXmsrmZYH1olLna31Xmsr1olJXmsr1olJXmsrmZYH1olLqoVr1olJXmsr1olJXmsrmZYH1olL1olKkfaPobXvviGabgadXmsqThKuofKHmZ4Dobnr1olJXmsr1olJXmspXmsr1olJXmsrfZ4TuhWn1olL1olJXmsqBi7X1olJXmspZmslbmMhbmsdemsVfl8ZgmsNim8Jpk8F0m7R4m7F5nLB6jbh7jbiDirOEibOGnKaMhq+PnaCVg6qWg6qegKaff6WhnpKofKGtnomxeZy3noG6dZi+n3vCcpPDcpPGn3bLb4/Mb47UbIrVa4rYoGjdaIbeaIXhoWHmZYHobXvpcHjqdHXreHLroVrsfG/uhGnuh2bwj2Hxk17yl1vzmljzm1j0nlX1olL3AJXWAAAAbXRSTlMAEBAQHx8gICAuLjAwMDw9PUBAQEpQUFBXV1hgYGBkcHBwcXl8gICAgoiIkJCQlJicnJ2goKCmqK+wsLC4usDAwMjP0NDQ1NbW3Nzg4ODi5+3v8PDw8/T09PX29vb39/f5+fr7+/z8/Pz9/v7+zczCxgAABC5JREFUeAHN1ul3k0UUBvCb1CTVpmpaitAGSLSpSuKCLWpbTKNJFGlcSMAFF63iUmRccNG6gLbuxkXU66JAUef/9LSpmXnyLr3T5AO/rzl5zj137p136BISy44fKJXuGN/d19PUfYeO67Znqtf2KH33Id1psXoFdW30sPZ1sMvs2D060AHqws4FHeJojLZqnw53cmfvg+XR8mC0OEjuxrXEkX5ydeVJLVIlV0e10PXk5k7dYeHu7Cj1j+49uKg7uLU61tGLw1lq27ugQYlclHC4bgv7VQ+TAyj5Zc/UjsPvs1sd5cWryWObtvWT2EPa4rtnWW3JkpjggEpbOsPr7F7EyNewtpBIslA7p43HCsnwooXTEc3UmPmCNn5lrqTJxy6nRmcavGZVt/3Da2pD5NHvsOHJCrdc1G2r3DITpU7yic7w/7Rxnjc0kt5GC4djiv2Sz3Fb2iEZg41/ddsFDoyuYrIkmFehz0HR2thPgQqMyQYb2OtB0WxsZ3BeG3+wpRb1vzl2UYBog8FfGhttFKjtAclnZYrRo9ryG9uG/FZQU4AEg8ZE9LjGMzTmqKXPLnlWVnIlQQTvxJf8ip7VgjZjyVPrjw1te5otM7RmP7xm+sK2Gv9I8Gi++BRbEkR9EBw8zRUcKxwp73xkaLiqQb+kGduJTNHG72zcW9LoJgqQxpP3/Tj//c3yB0tqzaml05/+orHLksVO+95kX7/7qgJvnjlrfr2Ggsyx0eoy9uPzN5SPd86aXggOsEKW2Prz7du3VID3/tzs/sSRs2w7ovVHKtjrX2pd7ZMlTxAYfBAL9jiDwfLkq55Tm7ifhMlTGPyCAs7RFRhn47JnlcB9RM5T97ASuZXIcVNuUDIndpDbdsfrqsOppeXl5Y+XVKdjFCTh+zGaVuj0d9zy05PPK3QzBamxdwtTCrzyg/2Rvf2EstUjordGwa/kx9mSJLr8mLLtCW8HHGJc2R5hS219IiF6PnTusOqcMl57gm0Z8kanKMAQg0qSyuZfn7zItsbGyO9QlnxY0eCuD1XL2ys/MsrQhltE7Ug0uFOzufJFE2PxBo/YAx8XPPdDwWN0MrDRYIZF0mSMKCNHgaIVFoBbNoLJ7tEQDKxGF0kcLQimojCZopv0OkNOyWCCg9XMVAi7ARJzQdM2QUh0gmBozjc3Skg6dSBRqDGYSUOu66Zg+I2fNZs/M3/f/Grl/XnyF1Gw3VKCez0PN5IUfFLqvgUN4C0qNqYs5YhPL+aVZYDE4IpUk57oSFnJm4FyCqqOE0jhY2SMyLFoo56zyo6becOS5UVDdj7Vih0zp+tcMhwRpBeLyqtIjlJKAIZSbI8SGSF3k0pA3mR5tHuwPFoa7N7reoq2bqCsAk1HqCu5uvI1n6JuRXI+S1Mco54YmYTwcn6Aeic+kssXi8XpXC4V3t7/ADuTNKaQJdScAAAAAElFTkSuQmCC)](https://mybinder.org/v2/gh/vopani/datatableton/main?filepath=notebooks%2F10_capstone_projects_exercises.ipynb) | [![Open in GitHub](https://img.shields.io/static/v1?label=&message=Open%20in%20GitHub&labelColor=grey&color=blue&logo=github)](https://github.com/vopani/datatableton/blob/main/notebooks/10_capstone_projects_exercises.ipynb) |\n",
    "| Solutions | [![Open in Colab](https://img.shields.io/static/v1?label=&message=Open%20in%20Colab&labelColor=grey&color=blue&logo=google-colab)](https://colab.research.google.com/github/vopani/datatableton/blob/main/notebooks/10_capstone_projects_solutions.ipynb) | [![Open in Kaggle](https://img.shields.io/static/v1?label=&message=Open%20in%20Kaggle&labelColor=grey&color=blue&logo=kaggle)](https://www.kaggle.com/rohanrao/datatableton-capstone-projects-solutions) | [![Open in Binder](https://img.shields.io/static/v1?label=&message=Open%20in%20Binder&labelColor=grey&color=blue&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAFkAAABZCAMAAABi1XidAAAB8lBMVEX///9XmsrmZYH1olJXmsr1olJXmsrmZYH1olJXmsr1olJXmsrmZYH1olL1olJXmsr1olJXmsrmZYH1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olJXmsrmZYH1olL1olL0nFf1olJXmsrmZYH1olJXmsq8dZb1olJXmsrmZYH1olJXmspXmspXmsr1olL1olJXmsrmZYH1olJXmsr1olL1olJXmsrmZYH1olL1olLeaIVXmsrmZYH1olL1olL1olJXmsrmZYH1olLna31Xmsr1olJXmsr1olJXmsrmZYH1olLqoVr1olJXmsr1olJXmsrmZYH1olL1olKkfaPobXvviGabgadXmsqThKuofKHmZ4Dobnr1olJXmsr1olJXmspXmsr1olJXmsrfZ4TuhWn1olL1olJXmsqBi7X1olJXmspZmslbmMhbmsdemsVfl8ZgmsNim8Jpk8F0m7R4m7F5nLB6jbh7jbiDirOEibOGnKaMhq+PnaCVg6qWg6qegKaff6WhnpKofKGtnomxeZy3noG6dZi+n3vCcpPDcpPGn3bLb4/Mb47UbIrVa4rYoGjdaIbeaIXhoWHmZYHobXvpcHjqdHXreHLroVrsfG/uhGnuh2bwj2Hxk17yl1vzmljzm1j0nlX1olL3AJXWAAAAbXRSTlMAEBAQHx8gICAuLjAwMDw9PUBAQEpQUFBXV1hgYGBkcHBwcXl8gICAgoiIkJCQlJicnJ2goKCmqK+wsLC4usDAwMjP0NDQ1NbW3Nzg4ODi5+3v8PDw8/T09PX29vb39/f5+fr7+/z8/Pz9/v7+zczCxgAABC5JREFUeAHN1ul3k0UUBvCb1CTVpmpaitAGSLSpSuKCLWpbTKNJFGlcSMAFF63iUmRccNG6gLbuxkXU66JAUef/9LSpmXnyLr3T5AO/rzl5zj137p136BISy44fKJXuGN/d19PUfYeO67Znqtf2KH33Id1psXoFdW30sPZ1sMvs2D060AHqws4FHeJojLZqnw53cmfvg+XR8mC0OEjuxrXEkX5ydeVJLVIlV0e10PXk5k7dYeHu7Cj1j+49uKg7uLU61tGLw1lq27ugQYlclHC4bgv7VQ+TAyj5Zc/UjsPvs1sd5cWryWObtvWT2EPa4rtnWW3JkpjggEpbOsPr7F7EyNewtpBIslA7p43HCsnwooXTEc3UmPmCNn5lrqTJxy6nRmcavGZVt/3Da2pD5NHvsOHJCrdc1G2r3DITpU7yic7w/7Rxnjc0kt5GC4djiv2Sz3Fb2iEZg41/ddsFDoyuYrIkmFehz0HR2thPgQqMyQYb2OtB0WxsZ3BeG3+wpRb1vzl2UYBog8FfGhttFKjtAclnZYrRo9ryG9uG/FZQU4AEg8ZE9LjGMzTmqKXPLnlWVnIlQQTvxJf8ip7VgjZjyVPrjw1te5otM7RmP7xm+sK2Gv9I8Gi++BRbEkR9EBw8zRUcKxwp73xkaLiqQb+kGduJTNHG72zcW9LoJgqQxpP3/Tj//c3yB0tqzaml05/+orHLksVO+95kX7/7qgJvnjlrfr2Ggsyx0eoy9uPzN5SPd86aXggOsEKW2Prz7du3VID3/tzs/sSRs2w7ovVHKtjrX2pd7ZMlTxAYfBAL9jiDwfLkq55Tm7ifhMlTGPyCAs7RFRhn47JnlcB9RM5T97ASuZXIcVNuUDIndpDbdsfrqsOppeXl5Y+XVKdjFCTh+zGaVuj0d9zy05PPK3QzBamxdwtTCrzyg/2Rvf2EstUjordGwa/kx9mSJLr8mLLtCW8HHGJc2R5hS219IiF6PnTusOqcMl57gm0Z8kanKMAQg0qSyuZfn7zItsbGyO9QlnxY0eCuD1XL2ys/MsrQhltE7Ug0uFOzufJFE2PxBo/YAx8XPPdDwWN0MrDRYIZF0mSMKCNHgaIVFoBbNoLJ7tEQDKxGF0kcLQimojCZopv0OkNOyWCCg9XMVAi7ARJzQdM2QUh0gmBozjc3Skg6dSBRqDGYSUOu66Zg+I2fNZs/M3/f/Grl/XnyF1Gw3VKCez0PN5IUfFLqvgUN4C0qNqYs5YhPL+aVZYDE4IpUk57oSFnJm4FyCqqOE0jhY2SMyLFoo56zyo6becOS5UVDdj7Vih0zp+tcMhwRpBeLyqtIjlJKAIZSbI8SGSF3k0pA3mR5tHuwPFoa7N7reoq2bqCsAk1HqCu5uvI1n6JuRXI+S1Mco54YmYTwcn6Aeic+kssXi8XpXC4V3t7/ADuTNKaQJdScAAAAAElFTkSuQmCC)](https://mybinder.org/v2/gh/vopani/datatableton/main?filepath=notebooks%2F10_capstone_projects_solutions.ipynb) | [![Open in GitHub](https://img.shields.io/static/v1?label=&message=Open%20in%20GitHub&labelColor=grey&color=blue&logo=github)](https://github.com/vopani/datatableton/blob/main/notebooks/10_capstone_projects_solutions.ipynb) |\n",
    "\n",
    "You can find all the exercises and solutions on [GitHub](https://github.com/vopani/datatableton#exercises-)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 49.286294,
   "end_time": "2021-08-25T09:34:59.420702",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2021-08-25T09:34:10.134408",
   "version": "2.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}