Day4_Supervised Learning I.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Supervised Learning I\n",
    "\n",
    "Supervised Learning은 input과 정답이 있는 데이터 집합을 학습시키는 방법이다. input과 output의 관계를 유추하여 올바른 해답을 제시한다. 사람이 직접 labeling을 하여 정확도가 높은 데이터를 사용할 수 있다.\n",
    "\n",
    "지도학습에는 Regression과 Classification이 있다. \n",
    "Regression은 input과 output을 분석하여 잘 표현하는 연속 함수를 찾는 과정이며, Classification은 input에 따른 이산적인 output값을 찾는다. \n",
    "\n",
    "실제 raw data를 가공하여 특징을 찾아낸 후, 데이터의 특징을 잘 표현하는 Model생성, 생성된 Model로 부터 결과를 예측하고 이를 분석하는 과정을 Feature Engineering이라고 한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>InvoiceNo</th>\n",
       "      <th>StockCode</th>\n",
       "      <th>Description</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>InvoiceDate</th>\n",
       "      <th>UnitPrice</th>\n",
       "      <th>CustomerID</th>\n",
       "      <th>Country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>546729</td>\n",
       "      <td>22775</td>\n",
       "      <td>PURPLE DRAWERKNOB ACRYLIC EDWARDIAN</td>\n",
       "      <td>12</td>\n",
       "      <td>2011-03-16 11:36:00</td>\n",
       "      <td>1.25</td>\n",
       "      <td>18231.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>559898</td>\n",
       "      <td>21868</td>\n",
       "      <td>POTTING SHED TEA MUG</td>\n",
       "      <td>6</td>\n",
       "      <td>2011-07-13 12:18:00</td>\n",
       "      <td>1.25</td>\n",
       "      <td>16225.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>548648</td>\n",
       "      <td>71459</td>\n",
       "      <td>HANGING JAM JAR T-LIGHT HOLDER</td>\n",
       "      <td>24</td>\n",
       "      <td>2011-04-01 13:20:00</td>\n",
       "      <td>0.85</td>\n",
       "      <td>12949.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>540543</td>\n",
       "      <td>22173</td>\n",
       "      <td>METAL 4 HOOK HANGER FRENCH CHATEAU</td>\n",
       "      <td>4</td>\n",
       "      <td>2011-01-09 15:23:00</td>\n",
       "      <td>2.95</td>\n",
       "      <td>14395.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>561390</td>\n",
       "      <td>20726</td>\n",
       "      <td>LUNCH BAG WOODLAND</td>\n",
       "      <td>10</td>\n",
       "      <td>2011-07-27 09:52:00</td>\n",
       "      <td>1.65</td>\n",
       "      <td>17068.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>565836</td>\n",
       "      <td>85152</td>\n",
       "      <td>HAND OVER THE CHOCOLATE   SIGN</td>\n",
       "      <td>48</td>\n",
       "      <td>2011-09-07 11:41:00</td>\n",
       "      <td>1.85</td>\n",
       "      <td>17511.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>570672</td>\n",
       "      <td>22866</td>\n",
       "      <td>HAND WARMER SCOTTY DOG DESIGN</td>\n",
       "      <td>12</td>\n",
       "      <td>2011-10-11 14:52:00</td>\n",
       "      <td>2.10</td>\n",
       "      <td>12536.0</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>557950</td>\n",
       "      <td>23118</td>\n",
       "      <td>PARISIENNE JEWELLERY DRAWER</td>\n",
       "      <td>2</td>\n",
       "      <td>2011-06-23 17:35:00</td>\n",
       "      <td>7.50</td>\n",
       "      <td>17848.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>575728</td>\n",
       "      <td>23264</td>\n",
       "      <td>SET OF 3 WOODEN SLEIGH DECORATIONS</td>\n",
       "      <td>12</td>\n",
       "      <td>2011-11-10 19:40:00</td>\n",
       "      <td>1.25</td>\n",
       "      <td>14911.0</td>\n",
       "      <td>EIRE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>575195</td>\n",
       "      <td>84006</td>\n",
       "      <td>MAGIC TREE -PAPER FLOWERS</td>\n",
       "      <td>12</td>\n",
       "      <td>2011-11-09 09:40:00</td>\n",
       "      <td>0.85</td>\n",
       "      <td>16767.0</td>\n",
       "      <td>United Kingdom</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   InvoiceNo StockCode                          Description  Quantity  \\\n",
       "0     546729     22775  PURPLE DRAWERKNOB ACRYLIC EDWARDIAN        12   \n",
       "1     559898     21868                 POTTING SHED TEA MUG         6   \n",
       "2     548648     71459       HANGING JAM JAR T-LIGHT HOLDER        24   \n",
       "3     540543     22173   METAL 4 HOOK HANGER FRENCH CHATEAU         4   \n",
       "4     561390     20726                   LUNCH BAG WOODLAND        10   \n",
       "5     565836     85152      HAND OVER THE CHOCOLATE   SIGN         48   \n",
       "6     570672     22866        HAND WARMER SCOTTY DOG DESIGN        12   \n",
       "7     557950     23118         PARISIENNE JEWELLERY DRAWER          2   \n",
       "8     575728     23264   SET OF 3 WOODEN SLEIGH DECORATIONS        12   \n",
       "9     575195     84006            MAGIC TREE -PAPER FLOWERS        12   \n",
       "\n",
       "           InvoiceDate  UnitPrice  CustomerID         Country  \n",
       "0  2011-03-16 11:36:00       1.25     18231.0  United Kingdom  \n",
       "1  2011-07-13 12:18:00       1.25     16225.0  United Kingdom  \n",
       "2  2011-04-01 13:20:00       0.85     12949.0  United Kingdom  \n",
       "3  2011-01-09 15:23:00       2.95     14395.0  United Kingdom  \n",
       "4  2011-07-27 09:52:00       1.65     17068.0  United Kingdom  \n",
       "5  2011-09-07 11:41:00       1.85     17511.0  United Kingdom  \n",
       "6  2011-10-11 14:52:00       2.10     12536.0          France  \n",
       "7  2011-06-23 17:35:00       7.50     17848.0  United Kingdom  \n",
       "8  2011-11-10 19:40:00       1.25     14911.0            EIRE  \n",
       "9  2011-11-09 09:40:00       0.85     16767.0  United Kingdom  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('data/retail_transaction_data.csv')\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(397884, 8)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['InvoiceDate'] = pd.to_datetime(df['InvoiceDate'])\n",
    "df['revenue'] = df['UnitPrice']*df['Quantity']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "operations = {'revenue':'sum','InvoiceDate':'first', 'CustomerID':'first'}\n",
    "df = df.groupby('InvoiceNo').agg(operations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>revenue</th>\n",
       "      <th>InvoiceDate</th>\n",
       "      <th>CustomerID</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>InvoiceNo</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>536365</th>\n",
       "      <td>139.12</td>\n",
       "      <td>2010-12-01 08:26:00</td>\n",
       "      <td>17850.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536366</th>\n",
       "      <td>22.20</td>\n",
       "      <td>2010-12-01 08:28:00</td>\n",
       "      <td>17850.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536367</th>\n",
       "      <td>278.73</td>\n",
       "      <td>2010-12-01 08:34:00</td>\n",
       "      <td>13047.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536368</th>\n",
       "      <td>70.05</td>\n",
       "      <td>2010-12-01 08:34:00</td>\n",
       "      <td>13047.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536369</th>\n",
       "      <td>17.85</td>\n",
       "      <td>2010-12-01 08:35:00</td>\n",
       "      <td>13047.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536370</th>\n",
       "      <td>855.86</td>\n",
       "      <td>2010-12-01 08:45:00</td>\n",
       "      <td>12583.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536371</th>\n",
       "      <td>204.00</td>\n",
       "      <td>2010-12-01 09:00:00</td>\n",
       "      <td>13748.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536372</th>\n",
       "      <td>22.20</td>\n",
       "      <td>2010-12-01 09:01:00</td>\n",
       "      <td>17850.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536373</th>\n",
       "      <td>259.86</td>\n",
       "      <td>2010-12-01 09:02:00</td>\n",
       "      <td>17850.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536374</th>\n",
       "      <td>350.40</td>\n",
       "      <td>2010-12-01 09:09:00</td>\n",
       "      <td>15100.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           revenue         InvoiceDate  CustomerID\n",
       "InvoiceNo                                         \n",
       "536365      139.12 2010-12-01 08:26:00     17850.0\n",
       "536366       22.20 2010-12-01 08:28:00     17850.0\n",
       "536367      278.73 2010-12-01 08:34:00     13047.0\n",
       "536368       70.05 2010-12-01 08:34:00     13047.0\n",
       "536369       17.85 2010-12-01 08:35:00     13047.0\n",
       "536370      855.86 2010-12-01 08:45:00     12583.0\n",
       "536371      204.00 2010-12-01 09:00:00     13748.0\n",
       "536372       22.20 2010-12-01 09:01:00     17850.0\n",
       "536373      259.86 2010-12-01 09:02:00     17850.0\n",
       "536374      350.40 2010-12-01 09:09:00     15100.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>revenue</th>\n",
       "      <th>InvoiceDate</th>\n",
       "      <th>CustomerID</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>InvoiceNo</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>536365</th>\n",
       "      <td>139.12</td>\n",
       "      <td>2010-12-01 08:26:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536366</th>\n",
       "      <td>22.20</td>\n",
       "      <td>2010-12-01 08:28:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536367</th>\n",
       "      <td>278.73</td>\n",
       "      <td>2010-12-01 08:34:00</td>\n",
       "      <td>13047.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536368</th>\n",
       "      <td>70.05</td>\n",
       "      <td>2010-12-01 08:34:00</td>\n",
       "      <td>13047.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536369</th>\n",
       "      <td>17.85</td>\n",
       "      <td>2010-12-01 08:35:00</td>\n",
       "      <td>13047.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536370</th>\n",
       "      <td>855.86</td>\n",
       "      <td>2010-12-01 08:45:00</td>\n",
       "      <td>12583.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536371</th>\n",
       "      <td>204.00</td>\n",
       "      <td>2010-12-01 09:00:00</td>\n",
       "      <td>13748.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536372</th>\n",
       "      <td>22.20</td>\n",
       "      <td>2010-12-01 09:01:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536373</th>\n",
       "      <td>259.86</td>\n",
       "      <td>2010-12-01 09:02:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536374</th>\n",
       "      <td>350.40</td>\n",
       "      <td>2010-12-01 09:09:00</td>\n",
       "      <td>15100.0</td>\n",
       "      <td>2010</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           revenue         InvoiceDate  CustomerID  year\n",
       "InvoiceNo                                               \n",
       "536365      139.12 2010-12-01 08:26:00     17850.0  2010\n",
       "536366       22.20 2010-12-01 08:28:00     17850.0  2010\n",
       "536367      278.73 2010-12-01 08:34:00     13047.0  2010\n",
       "536368       70.05 2010-12-01 08:34:00     13047.0  2010\n",
       "536369       17.85 2010-12-01 08:35:00     13047.0  2010\n",
       "536370      855.86 2010-12-01 08:45:00     12583.0  2010\n",
       "536371      204.00 2010-12-01 09:00:00     13748.0  2010\n",
       "536372       22.20 2010-12-01 09:01:00     17850.0  2010\n",
       "536373      259.86 2010-12-01 09:02:00     17850.0  2010\n",
       "536374      350.40 2010-12-01 09:09:00     15100.0  2010"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['year'] = df['InvoiceDate'].apply(lambda x: x.year)\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/barunpuri/Library/Python/3.7/lib/python/site-packages/ipykernel_launcher.py:1: FutureWarning: The pandas.datetime class is deprecated and will be removed from pandas in a future version. Import from datetime module instead.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>revenue</th>\n",
       "      <th>InvoiceDate</th>\n",
       "      <th>CustomerID</th>\n",
       "      <th>year</th>\n",
       "      <th>days_since</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>InvoiceNo</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>536365</th>\n",
       "      <td>139.12</td>\n",
       "      <td>2010-12-01 08:26:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536366</th>\n",
       "      <td>22.20</td>\n",
       "      <td>2010-12-01 08:28:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536367</th>\n",
       "      <td>278.73</td>\n",
       "      <td>2010-12-01 08:34:00</td>\n",
       "      <td>13047.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536368</th>\n",
       "      <td>70.05</td>\n",
       "      <td>2010-12-01 08:34:00</td>\n",
       "      <td>13047.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536369</th>\n",
       "      <td>17.85</td>\n",
       "      <td>2010-12-01 08:35:00</td>\n",
       "      <td>13047.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536370</th>\n",
       "      <td>855.86</td>\n",
       "      <td>2010-12-01 08:45:00</td>\n",
       "      <td>12583.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536371</th>\n",
       "      <td>204.00</td>\n",
       "      <td>2010-12-01 09:00:00</td>\n",
       "      <td>13748.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536372</th>\n",
       "      <td>22.20</td>\n",
       "      <td>2010-12-01 09:01:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536373</th>\n",
       "      <td>259.86</td>\n",
       "      <td>2010-12-01 09:02:00</td>\n",
       "      <td>17850.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>536374</th>\n",
       "      <td>350.40</td>\n",
       "      <td>2010-12-01 09:09:00</td>\n",
       "      <td>15100.0</td>\n",
       "      <td>2010</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           revenue         InvoiceDate  CustomerID  year  days_since\n",
       "InvoiceNo                                                           \n",
       "536365      139.12 2010-12-01 08:26:00     17850.0  2010          29\n",
       "536366       22.20 2010-12-01 08:28:00     17850.0  2010          29\n",
       "536367      278.73 2010-12-01 08:34:00     13047.0  2010          29\n",
       "536368       70.05 2010-12-01 08:34:00     13047.0  2010          29\n",
       "536369       17.85 2010-12-01 08:35:00     13047.0  2010          29\n",
       "536370      855.86 2010-12-01 08:45:00     12583.0  2010          29\n",
       "536371      204.00 2010-12-01 09:00:00     13748.0  2010          29\n",
       "536372       22.20 2010-12-01 09:01:00     17850.0  2010          29\n",
       "536373      259.86 2010-12-01 09:02:00     17850.0  2010          29\n",
       "536374      350.40 2010-12-01 09:09:00     15100.0  2010          29"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['days_since'] = (pd.datetime(year=2010, month=12, day=31) - df['InvoiceDate']).apply(lambda x: x.days)\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>revenue</th>\n",
       "      <th colspan=\"3\" halign=\"left\">days_since</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>sum</th>\n",
       "      <th>max</th>\n",
       "      <th>min</th>\n",
       "      <th>nunique</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CustomerID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12347.0</th>\n",
       "      <td>711.79</td>\n",
       "      <td>23</td>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12348.0</th>\n",
       "      <td>892.80</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12370.0</th>\n",
       "      <td>1868.02</td>\n",
       "      <td>16</td>\n",
       "      <td>13</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12377.0</th>\n",
       "      <td>1001.52</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12383.0</th>\n",
       "      <td>600.72</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            revenue days_since            \n",
       "                sum        max min nunique\n",
       "CustomerID                                \n",
       "12347.0      711.79         23  23       1\n",
       "12348.0      892.80         14  14       1\n",
       "12370.0     1868.02         16  13       2\n",
       "12377.0     1001.52         10  10       1\n",
       "12383.0      600.72          8   8       1"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "operations = {'revenue':'sum',\n",
    "              'days_since':['max','min', 'nunique'],\n",
    "             }\n",
    "X = df[df['year'] == 2010].groupby('CustomerID').agg(operations)\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>revenue sum</th>\n",
       "      <th>days_since max</th>\n",
       "      <th>days_since min</th>\n",
       "      <th>days_since nunique</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CustomerID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12347.0</th>\n",
       "      <td>711.79</td>\n",
       "      <td>23</td>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12348.0</th>\n",
       "      <td>892.80</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12370.0</th>\n",
       "      <td>1868.02</td>\n",
       "      <td>16</td>\n",
       "      <td>13</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12377.0</th>\n",
       "      <td>1001.52</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12383.0</th>\n",
       "      <td>600.72</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            revenue sum  days_since max  days_since min  days_since nunique\n",
       "CustomerID                                                                 \n",
       "12347.0          711.79              23              23                   1\n",
       "12348.0          892.80              14              14                   1\n",
       "12370.0         1868.02              16              13                   2\n",
       "12377.0         1001.52              10              10                   1\n",
       "12383.0          600.72               8               8                   1"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.columns = [' '.join(col).strip() for col in X.columns.values]\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>revenue sum</th>\n",
       "      <th>days_since max</th>\n",
       "      <th>days_since min</th>\n",
       "      <th>days_since nunique</th>\n",
       "      <th>average_order_cost</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CustomerID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12347.0</th>\n",
       "      <td>711.79</td>\n",
       "      <td>23</td>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "      <td>711.79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12348.0</th>\n",
       "      <td>892.80</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "      <td>1</td>\n",
       "      <td>892.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12370.0</th>\n",
       "      <td>1868.02</td>\n",
       "      <td>16</td>\n",
       "      <td>13</td>\n",
       "      <td>2</td>\n",
       "      <td>934.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12377.0</th>\n",
       "      <td>1001.52</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>1001.52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12383.0</th>\n",
       "      <td>600.72</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>600.72</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            revenue sum  days_since max  days_since min  days_since nunique  \\\n",
       "CustomerID                                                                    \n",
       "12347.0          711.79              23              23                   1   \n",
       "12348.0          892.80              14              14                   1   \n",
       "12370.0         1868.02              16              13                   2   \n",
       "12377.0         1001.52              10              10                   1   \n",
       "12383.0          600.72               8               8                   1   \n",
       "\n",
       "            average_order_cost  \n",
       "CustomerID                      \n",
       "12347.0                 711.79  \n",
       "12348.0                 892.80  \n",
       "12370.0                 934.01  \n",
       "12377.0                1001.52  \n",
       "12383.0                 600.72  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X['average_order_cost'] = X['revenue sum']/X['days_since nunique']\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CustomerID\n",
       "12346.0    77183.60\n",
       "12347.0     3598.21\n",
       "12348.0      904.44\n",
       "12349.0     1757.55\n",
       "12350.0      334.40\n",
       "Name: revenue, dtype: float64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y = df[df['year'] == 2011].groupby('CustomerID')['revenue'].sum()\n",
    "Y.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>2010 revenue</th>\n",
       "      <th>days_since_first_purchase</th>\n",
       "      <th>days_since_last_purchase</th>\n",
       "      <th>number_of_purchases</th>\n",
       "      <th>avg_order_cost</th>\n",
       "      <th>2011 revenue</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CustomerID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12346.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>77183.60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12347.0</th>\n",
       "      <td>711.79</td>\n",
       "      <td>23.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>711.79</td>\n",
       "      <td>3598.21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12348.0</th>\n",
       "      <td>892.80</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>892.80</td>\n",
       "      <td>904.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12349.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1757.55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12350.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>334.40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12352.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2506.04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12353.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>89.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12354.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1079.40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12355.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>459.40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12356.0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2811.43</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            2010 revenue  days_since_first_purchase  days_since_last_purchase  \\\n",
       "CustomerID                                                                      \n",
       "12346.0              NaN                        NaN                       NaN   \n",
       "12347.0           711.79                       23.0                      23.0   \n",
       "12348.0           892.80                       14.0                      14.0   \n",
       "12349.0              NaN                        NaN                       NaN   \n",
       "12350.0              NaN                        NaN                       NaN   \n",
       "12352.0              NaN                        NaN                       NaN   \n",
       "12353.0              NaN                        NaN                       NaN   \n",
       "12354.0              NaN                        NaN                       NaN   \n",
       "12355.0              NaN                        NaN                       NaN   \n",
       "12356.0              NaN                        NaN                       NaN   \n",
       "\n",
       "            number_of_purchases  avg_order_cost  2011 revenue  \n",
       "CustomerID                                                     \n",
       "12346.0                     NaN             NaN      77183.60  \n",
       "12347.0                     1.0          711.79       3598.21  \n",
       "12348.0                     1.0          892.80        904.44  \n",
       "12349.0                     NaN             NaN       1757.55  \n",
       "12350.0                     NaN             NaN        334.40  \n",
       "12352.0                     NaN             NaN       2506.04  \n",
       "12353.0                     NaN             NaN         89.00  \n",
       "12354.0                     NaN             NaN       1079.40  \n",
       "12355.0                     NaN             NaN        459.40  \n",
       "12356.0                     NaN             NaN       2811.43  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wrangled_df = pd.concat([X,Y], axis=1)\n",
    "wrangled_df.columns = ['2010 revenue',\n",
    "                       'days_since_first_purchase',\n",
    "                       'days_since_last_purchase',\n",
    "                       'number_of_purchases',\n",
    "                       'avg_order_cost',\n",
    "                       '2011 revenue']\n",
    "wrangled_df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>2010 revenue</th>\n",
       "      <th>days_since_first_purchase</th>\n",
       "      <th>days_since_last_purchase</th>\n",
       "      <th>number_of_purchases</th>\n",
       "      <th>avg_order_cost</th>\n",
       "      <th>2011 revenue</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CustomerID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12347.0</th>\n",
       "      <td>711.79</td>\n",
       "      <td>23.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>711.79</td>\n",
       "      <td>3598.21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12348.0</th>\n",
       "      <td>892.80</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>892.80</td>\n",
       "      <td>904.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12370.0</th>\n",
       "      <td>1868.02</td>\n",
       "      <td>16.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>934.01</td>\n",
       "      <td>1677.67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12377.0</th>\n",
       "      <td>1001.52</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1001.52</td>\n",
       "      <td>626.60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12383.0</th>\n",
       "      <td>600.72</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>600.72</td>\n",
       "      <td>1249.84</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            2010 revenue  days_since_first_purchase  days_since_last_purchase  \\\n",
       "CustomerID                                                                      \n",
       "12347.0           711.79                       23.0                      23.0   \n",
       "12348.0           892.80                       14.0                      14.0   \n",
       "12370.0          1868.02                       16.0                      13.0   \n",
       "12377.0          1001.52                       10.0                      10.0   \n",
       "12383.0           600.72                        8.0                       8.0   \n",
       "\n",
       "            number_of_purchases  avg_order_cost  2011 revenue  \n",
       "CustomerID                                                     \n",
       "12347.0                     1.0          711.79       3598.21  \n",
       "12348.0                     1.0          892.80        904.44  \n",
       "12370.0                     2.0          934.01       1677.67  \n",
       "12377.0                     1.0         1001.52        626.60  \n",
       "12383.0                     1.0          600.72       1249.84  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wrangled_df = wrangled_df[~wrangled_df['2010 revenue'].isnull()]\n",
    "wrangled_df = wrangled_df[~wrangled_df['2011 revenue'].isnull()]\n",
    "wrangled_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9265.701776633614\n",
      "1278.3019646470884\n"
     ]
    }
   ],
   "source": [
    "print(wrangled_df['2011 revenue'].median() + wrangled_df['2011 revenue'].std()*3)\n",
    "print(wrangled_df['2010 revenue'].median() + wrangled_df['2010 revenue'].std()*3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>2010 revenue</th>\n",
       "      <th>days_since_first_purchase</th>\n",
       "      <th>days_since_last_purchase</th>\n",
       "      <th>number_of_purchases</th>\n",
       "      <th>avg_order_cost</th>\n",
       "      <th>2011 revenue</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CustomerID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12347.0</th>\n",
       "      <td>711.79</td>\n",
       "      <td>23.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>711.79</td>\n",
       "      <td>3598.21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12348.0</th>\n",
       "      <td>892.80</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>892.80</td>\n",
       "      <td>904.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12377.0</th>\n",
       "      <td>1001.52</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1001.52</td>\n",
       "      <td>626.60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12383.0</th>\n",
       "      <td>600.72</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>600.72</td>\n",
       "      <td>1249.84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12386.0</th>\n",
       "      <td>258.90</td>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>258.90</td>\n",
       "      <td>143.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            2010 revenue  days_since_first_purchase  days_since_last_purchase  \\\n",
       "CustomerID                                                                      \n",
       "12347.0           711.79                       23.0                      23.0   \n",
       "12348.0           892.80                       14.0                      14.0   \n",
       "12377.0          1001.52                       10.0                      10.0   \n",
       "12383.0           600.72                        8.0                       8.0   \n",
       "12386.0           258.90                       22.0                      22.0   \n",
       "\n",
       "            number_of_purchases  avg_order_cost  2011 revenue  \n",
       "CustomerID                                                     \n",
       "12347.0                     1.0          711.79       3598.21  \n",
       "12348.0                     1.0          892.80        904.44  \n",
       "12377.0                     1.0         1001.52        626.60  \n",
       "12383.0                     1.0          600.72       1249.84  \n",
       "12386.0                     1.0          258.90        143.00  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wrangled_df = wrangled_df[wrangled_df['2011 revenue'] < ((wrangled_df['2011 revenue'].median()) + wrangled_df['2011 revenue'].std()*3)]\n",
    "wrangled_df = wrangled_df[wrangled_df['2010 revenue'] < ((wrangled_df['2010 revenue'].median()) + wrangled_df['2010 revenue'].std()*3)]\n",
    "wrangled_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>2010 revenue</th>\n",
       "      <th>days_since_first_purchase</th>\n",
       "      <th>days_since_last_purchase</th>\n",
       "      <th>number_of_purchases</th>\n",
       "      <th>avg_order_cost</th>\n",
       "      <th>2011 revenue</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CustomerID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>12347.0</th>\n",
       "      <td>711.79</td>\n",
       "      <td>23.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>711.79</td>\n",
       "      <td>3598.21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12348.0</th>\n",
       "      <td>892.80</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>892.80</td>\n",
       "      <td>904.44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12377.0</th>\n",
       "      <td>1001.52</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1001.52</td>\n",
       "      <td>626.60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12383.0</th>\n",
       "      <td>600.72</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>600.72</td>\n",
       "      <td>1249.84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12386.0</th>\n",
       "      <td>258.90</td>\n",
       "      <td>22.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>258.90</td>\n",
       "      <td>143.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13199.0</th>\n",
       "      <td>711.80</td>\n",
       "      <td>22.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>355.90</td>\n",
       "      <td>9105.32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13209.0</th>\n",
       "      <td>318.85</td>\n",
       "      <td>17.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>318.85</td>\n",
       "      <td>5572.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13211.0</th>\n",
       "      <td>110.70</td>\n",
       "      <td>17.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>110.70</td>\n",
       "      <td>1086.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13221.0</th>\n",
       "      <td>498.16</td>\n",
       "      <td>15.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>498.16</td>\n",
       "      <td>349.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13225.0</th>\n",
       "      <td>1206.72</td>\n",
       "      <td>17.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1206.72</td>\n",
       "      <td>5257.12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            2010 revenue  days_since_first_purchase  days_since_last_purchase  \\\n",
       "CustomerID                                                                      \n",
       "12347.0           711.79                       23.0                      23.0   \n",
       "12348.0           892.80                       14.0                      14.0   \n",
       "12377.0          1001.52                       10.0                      10.0   \n",
       "12383.0           600.72                        8.0                       8.0   \n",
       "12386.0           258.90                       22.0                      22.0   \n",
       "...                  ...                        ...                       ...   \n",
       "13199.0           711.80                       22.0                      14.0   \n",
       "13209.0           318.85                       17.0                      17.0   \n",
       "13211.0           110.70                       17.0                      17.0   \n",
       "13221.0           498.16                       15.0                      15.0   \n",
       "13225.0          1206.72                       17.0                      17.0   \n",
       "\n",
       "            number_of_purchases  avg_order_cost  2011 revenue  \n",
       "CustomerID                                                     \n",
       "12347.0                     1.0          711.79       3598.21  \n",
       "12348.0                     1.0          892.80        904.44  \n",
       "12377.0                     1.0         1001.52        626.60  \n",
       "12383.0                     1.0          600.72       1249.84  \n",
       "12386.0                     1.0          258.90        143.00  \n",
       "...                         ...             ...           ...  \n",
       "13199.0                     2.0          355.90       9105.32  \n",
       "13209.0                     1.0          318.85       5572.76  \n",
       "13211.0                     1.0          110.70       1086.72  \n",
       "13221.0                     1.0          498.16        349.25  \n",
       "13225.0                     1.0         1206.72       5257.12  \n",
       "\n",
       "[100 rows x 6 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wrangled_df.to_csv('Data/wrangled_transaction_data_result.csv')\n",
    "wrangled_df.head(100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(701, 6)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wrangled_df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}