【飞桨学习赛：MarTech Challenge 用户购买预测】第6名方案.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "![](https://ai-studio-static-online.cdn.bcebos.com/ef9f89df07cb4c22a11086a488af62ddefb6399fa97d467da9e0471035265f32)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "\t在本次赛题中，虽然赛题是一个二分类任务（用户购买、未购买），但从赛题数据看，属于比较典型的时间序列数据，也可以参照以往的线性回归任务的做法处理。 反映了一个客观事实——在真实场景应用机器学习/深度学习技术时，通常是没有已经整理好的训练集、验证集、测试集，需要自己设计。\n",
    "\tXGBoost（eXtreme Gradient Boosting）是一种流行的机器学习算法，用于分类和回归问题。它是一种集成学习方法，即通过训练多个弱学习器（例如决策树）并将它们组合在一起，形成一个更强大的模型。XGBoost的优势在于它能够快速地学习特征重要性，并使用决策树算法构建高质量的模型。它还使用了一些技术来提高训练效率，包括基于决策树的正则化、直方图算法和并行训练。XGBoost通常用于竞赛和实际应用中，因为它能够在大型数据集上快速构建高质量的模型，能够较好地处理缺失数据并在高维度空间中执行优秀的预测。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "数据如下图所示\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/d997a498b4da47b49178d7f0c0f22581e1941200d9b44863af66131611a29d6b)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "\t部分数据，如会员id、用户id等数据，应该剔除，所以要对原始数据进行预处理，进行特征的提取与剔除。由于数据集中的数据属于时间序列数据，可以引入时间滑窗模型。时间滑窗在业务应用上被称为RFM模型，RFM（Recency, Frequency, Monetary Value）模型是一种客户分析方法，用于衡量客户的价值和潜在收益。它通过对客户的最近一次交易时间（Recency）、交易频率（Frequency）以及交易金额（Monetary Value）进行评估，并将客户分为不同的类别，以便更好地了解客户的购买行为，并采取相应的营销策略。在RFM模型中，分数越高的客户被认为是更有价值的客户。因此，对于分数较低的客户，企业可以采取一些措施来提高他们的分数，并将他们转变为更有价值的客户。例如，企业可以通过给予他们优惠或促销活动来吸引他们的注意，或者通过提供优质的产品和服务来提高他们的满意度，从而增加他们的交易频率和金额。\n",
    "\t本实验采用RFM模型，考虑到按周、按月的订单规律性变化，时间滑窗往往是14和30的倍数，一般选取的特征是滑窗中的基本统计值，最大、最小、均值、中位数、求和等等。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 引入工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np  # linear algebra\n",
    "import pandas as pd  #\n",
    "from datetime import datetime, date, timedelta\n",
    "from scipy.stats import skew  # for some statistics\n",
    "from scipy.special import boxcox1p\n",
    "from scipy.stats import boxcox_normmax\n",
    "from sklearn.linear_model import ElasticNetCV, LassoCV, RidgeCV,Ridge,Lasso,ElasticNet\n",
    "from sklearn.ensemble import GradientBoostingRegressor, RandomForestRegressor,RandomForestClassifier\n",
    "from sklearn.feature_selection import mutual_info_regression\n",
    "from sklearn.svm import SVR, LinearSVC\n",
    "from sklearn.pipeline import make_pipeline,Pipeline\n",
    "from sklearn.preprocessing import RobustScaler, LabelEncoder\n",
    "from sklearn.model_selection import KFold, cross_val_score, train_test_split\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import train_test_split\n",
    "import os\n",
    "import re\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "from itertools import product\n",
    "import datetime as dt\n",
    "import calendar\n",
    "import gc\n",
    "\n",
    "RANDOM_SEED = 42\n",
    "# 引入中文字体\n",
    "from matplotlib.font_manager import FontProperties\n",
    "myfont = FontProperties(fname=\"/home/aistudio/NotoSansCJKsc-Light.otf\", size=12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 加载数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "PATH = './data/data19383/'\n",
    "train = pd.read_csv(PATH + 'train.csv')\n",
    "# train = pd.read_csv('./data/data19383/train.csv', usecols=[2, 3, 4, 6, 7, 18])\n",
    "# set index to ID to avoid droping it later\n",
    "# 把测试集的id列作为索引，防止误删\n",
    "test  = pd.read_csv(PATH + 'submission.csv').set_index('customer_id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集使用内容 510.40096282958984 MB\n",
      "测试集使用内存 24.200225830078125 MB\n"
     ]
    }
   ],
   "source": [
    "# 对于特别大的文件，我们需要做一些内存检查\n",
    "mem_train = train.memory_usage(index=True).sum()\n",
    "mem_test=test.memory_usage(index=True).sum()\n",
    "print(u\"训练集使用内容 \"+ str(mem_train/ 1024**2)+\" MB\")\n",
    "print(u\"测试集使用内存 \"+ str(mem_test/ 1024**2)+\" MB\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 内存优化脚本\n",
    "- 参考[缓解pandas中DataFrame占用内存过大的问题](https://blog.csdn.net/wj1066/article/details/81124959)\n",
    "- 效果非常显著，有效避免内存溢出"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# @from: https://www.kaggle.com/arjanso/reducing-dataframe-memory-size-by-65/code\r\n",
    "# @liscense: Apache 2.0\r\n",
    "# @author: weijian\r\n",
    "def reduce_mem_usage(props):\r\n",
    "    # 计算当前内存\r\n",
    "    start_mem_usg = props.memory_usage().sum() / 1024 ** 2\r\n",
    "    print(\"Memory usage of the dataframe is :\", start_mem_usg, \"MB\")\r\n",
    "    \r\n",
    "    # 哪些列包含空值，空值用-999填充。why：因为np.nan当做float处理\r\n",
    "    NAlist = []\r\n",
    "    for col in props.columns:\r\n",
    "        # 这里只过滤了objectd格式，如果你的代码中还包含其他类型，请一并过滤\r\n",
    "        if (props[col].dtypes != object):\r\n",
    "            \r\n",
    "            # print(\"**************************\")\r\n",
    "            # print(\"columns: \", col)\r\n",
    "            # print(\"dtype before\", props[col].dtype)\r\n",
    "            \r\n",
    "            # 判断是否是int类型\r\n",
    "            isInt = False\r\n",
    "            mmax = props[col].max()\r\n",
    "            mmin = props[col].min()\r\n",
    "            \r\n",
    "            # Integer does not support NA, therefore Na needs to be filled\r\n",
    "            if not np.isfinite(props[col]).all():\r\n",
    "                NAlist.append(col)\r\n",
    "                props[col].fillna(-999, inplace=True) # 用-999填充\r\n",
    "                \r\n",
    "            # test if column can be converted to an integer\r\n",
    "            asint = props[col].fillna(0).astype(np.int64)\r\n",
    "            result = np.fabs(props[col] - asint)\r\n",
    "            result = result.sum()\r\n",
    "            if result < 0.01: # 绝对误差和小于0.01认为可以转换的，要根据task修改\r\n",
    "                isInt = True\r\n",
    "            \r\n",
    "            # make interger / unsigned Integer datatypes\r\n",
    "            if isInt:\r\n",
    "                if mmin >= 0: # 最小值大于0，转换成无符号整型\r\n",
    "                    if mmax <= 255:\r\n",
    "                        props[col] = props[col].astype(np.uint8)\r\n",
    "                    elif mmax <= 65535:\r\n",
    "                        props[col] = props[col].astype(np.uint16)\r\n",
    "                    elif mmax <= 4294967295:\r\n",
    "                        props[col] = props[col].astype(np.uint32)\r\n",
    "                    else:\r\n",
    "                        props[col] = props[col].astype(np.uint64)\r\n",
    "                else: # 转换成有符号整型\r\n",
    "                    if mmin > np.iinfo(np.int8).min and mmax < np.iinfo(np.int8).max:\r\n",
    "                        props[col] = props[col].astype(np.int8)\r\n",
    "                    elif mmin > np.iinfo(np.int16).min and mmax < np.iinfo(np.int16).max:\r\n",
    "                        props[col] = props[col].astype(np.int16)\r\n",
    "                    elif mmin > np.iinfo(np.int32).min and mmax < np.iinfo(np.int32).max:\r\n",
    "                        props[col] = props[col].astype(np.int32)\r\n",
    "                    elif mmin > np.iinfo(np.int64).min and mmax < np.iinfo(np.int64).max:\r\n",
    "                        props[col] = props[col].astype(np.int64)  \r\n",
    "            else: # 注意：这里对于float都转换成float16，需要根据你的情况自己更改\r\n",
    "                props[col] = props[col].astype(np.float16)\r\n",
    "            \r\n",
    "            # print(\"dtype after\", props[col].dtype)\r\n",
    "            # print(\"********************************\")\r\n",
    "    print(\"___MEMORY USAGE AFTER COMPLETION:___\")\r\n",
    "    mem_usg = props.memory_usage().sum() / 1024**2 \r\n",
    "    print(\"Memory usage is: \",mem_usg,\" MB\")\r\n",
    "    print(\"This is \",100*mem_usg/start_mem_usg,\"% of the initial size\")\r\n",
    "    return props, NAlist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集使用内容 349.8006982803345 MB\n",
      "测试集使用内存 24.200225830078125 MB\n"
     ]
    }
   ],
   "source": [
    "# 处理id字段\n",
    "train['order_detail_id'] = train['order_detail_id'].astype(np.uint32)\n",
    "train['order_id'] = train['order_id'].astype(np.uint32)\n",
    "train['customer_id'] = train['customer_id'].astype(np.uint32)\n",
    "train['goods_id'] = train['goods_id'].astype(np.uint32)\n",
    "train['goods_class_id'] = train['goods_class_id'].astype(np.uint32)\n",
    "train['member_id'] = train['member_id'].astype(np.uint32)\n",
    "# 处理状态字段，这里同时处理空值，将空值置为0\n",
    "train['order_status'] = train['order_status'].astype(np.uint8)\n",
    "train['goods_has_discount'] = train['goods_has_discount'].astype(np.uint8)\n",
    "train[\"is_member_actived\"].fillna(0, inplace=True)\n",
    "train[\"is_member_actived\"]=train[\"is_member_actived\"].astype(np.int8)\n",
    "train[\"member_status\"].fillna(0, inplace=True)\n",
    "train[\"member_status\"]=train[\"member_status\"].astype(np.int8)\n",
    "train[\"customer_gender\"].fillna(0, inplace=True)\n",
    "train[\"customer_gender\"]=train[\"customer_gender\"].astype(np.int8)\n",
    "train['is_customer_rate'] = train['is_customer_rate'].astype(np.uint8)\n",
    "train['order_detail_status'] = train['order_detail_status'].astype(np.uint8)\n",
    "# 处理日期\n",
    "train['goods_list_time']=pd.to_datetime(train['goods_list_time'],format=\"%Y-%m-%d\")\n",
    "train['order_pay_time']=pd.to_datetime(train['order_pay_time'],format=\"%Y-%m-%d\")\n",
    "train['goods_delist_time']=pd.to_datetime(train['goods_delist_time'],format=\"%Y-%m-%d\")\n",
    "# 检查内存使用\n",
    "mem_train = train.memory_usage(index=True).sum()\n",
    "mem_test=test.memory_usage(index=True).sum()\n",
    "print(u\"训练集使用内容 \"+ str(mem_train/ 1024**2)+\" MB\")\n",
    "print(u\"测试集使用内存 \"+ str(mem_test/ 1024**2)+\" MB\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train['customer_city_id'] = LabelEncoder().fit_transform(train['customer_city'].astype(str))\n",
    "train['customer_province_id'] = LabelEncoder().fit_transform(train['customer_province'].astype(str))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 纯时间序列滑窗\n",
    "训练集和测试集构造思路：\n",
    "- 训练集\n",
    "\t- 训练日期截止分别为2013.6.6、2013.6.20、2013.7.1、2013.7.15\n",
    "\t- 标签为训练截止日期开始的30天有没有下过单\n",
    "\t- 标签通过切片的形式构造\n",
    "    - 把4个部分训练集拼起来\n",
    "- 验证集\n",
    "\t- 验证数据到2013.8.1\n",
    "\t- 标签为2013.8.1开始的30天有没有下过单\n",
    "\t- 标签时间2013.8.1-2013.8.30\n",
    "- 测试集\n",
    "\t- 测试数据到2013.8.31\n",
    "\t- 没有标签"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>order_detail_id</th>\n",
       "      <th>order_id</th>\n",
       "      <th>order_total_num</th>\n",
       "      <th>order_amount</th>\n",
       "      <th>order_total_payment</th>\n",
       "      <th>order_total_discount</th>\n",
       "      <th>order_pay_time</th>\n",
       "      <th>order_status</th>\n",
       "      <th>order_count</th>\n",
       "      <th>is_customer_rate</th>\n",
       "      <th>...</th>\n",
       "      <th>is_member_actived</th>\n",
       "      <th>goods_id</th>\n",
       "      <th>goods_class_id</th>\n",
       "      <th>goods_price</th>\n",
       "      <th>goods_status</th>\n",
       "      <th>goods_has_discount</th>\n",
       "      <th>goods_list_time</th>\n",
       "      <th>goods_delist_time</th>\n",
       "      <th>customer_city_id</th>\n",
       "      <th>customer_province_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000000</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>239.9</td>\n",
       "      <td>96.9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2012-11-01 00:10:56</td>\n",
       "      <td>6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>998</td>\n",
       "      <td>998</td>\n",
       "      <td>54.909289</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2014-10-25 11:08:07</td>\n",
       "      <td>2014-11-01 11:08:07</td>\n",
       "      <td>53</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1001530</td>\n",
       "      <td>1001327</td>\n",
       "      <td>2.0</td>\n",
       "      <td>288.0</td>\n",
       "      <td>96.9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2013-08-31 23:14:42</td>\n",
       "      <td>6</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1953</td>\n",
       "      <td>1953</td>\n",
       "      <td>45.961352</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2013-08-28 17:27:50</td>\n",
       "      <td>2013-09-01 00:38:17</td>\n",
       "      <td>322</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1001531</td>\n",
       "      <td>1001327</td>\n",
       "      <td>2.0</td>\n",
       "      <td>288.0</td>\n",
       "      <td>96.9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2013-08-31 23:14:42</td>\n",
       "      <td>6</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1083</td>\n",
       "      <td>1083</td>\n",
       "      <td>53.035439</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2014-10-29 18:21:05</td>\n",
       "      <td>2014-11-05 18:21:05</td>\n",
       "      <td>322</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1001532</td>\n",
       "      <td>1001328</td>\n",
       "      <td>3.0</td>\n",
       "      <td>180.0</td>\n",
       "      <td>89.7</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2013-08-31 22:06:35</td>\n",
       "      <td>6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1013</td>\n",
       "      <td>1013</td>\n",
       "      <td>46.046917</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2014-10-25 11:00:00</td>\n",
       "      <td>2014-11-01 11:00:00</td>\n",
       "      <td>185</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1001533</td>\n",
       "      <td>1001329</td>\n",
       "      <td>1.0</td>\n",
       "      <td>159.9</td>\n",
       "      <td>65.9</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2013-08-31 21:33:36</td>\n",
       "      <td>6</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1628</td>\n",
       "      <td>1628</td>\n",
       "      <td>50.722161</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2014-10-23 15:35:33</td>\n",
       "      <td>2014-10-30 15:35:33</td>\n",
       "      <td>53</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   order_detail_id  order_id  order_total_num  order_amount  \\\n",
       "0          1000000   1000000              1.0         239.9   \n",
       "1          1001530   1001327              2.0         288.0   \n",
       "2          1001531   1001327              2.0         288.0   \n",
       "3          1001532   1001328              3.0         180.0   \n",
       "4          1001533   1001329              1.0         159.9   \n",
       "\n",
       "   order_total_payment  order_total_discount      order_pay_time  \\\n",
       "0                 96.9                   0.0 2012-11-01 00:10:56   \n",
       "1                 96.9                   0.0 2013-08-31 23:14:42   \n",
       "2                 96.9                   0.0 2013-08-31 23:14:42   \n",
       "3                 89.7                   0.0 2013-08-31 22:06:35   \n",
       "4                 65.9                   0.0 2013-08-31 21:33:36   \n",
       "\n",
       "   order_status  order_count  is_customer_rate  ...  is_member_actived  \\\n",
       "0             6          1.0                 0  ...                  0   \n",
       "1             6          2.0                 0  ...                  0   \n",
       "2             6          2.0                 0  ...                  0   \n",
       "3             6          1.0                 0  ...                  0   \n",
       "4             6          1.0                 0  ...                  0   \n",
       "\n",
       "   goods_id  goods_class_id  goods_price  goods_status goods_has_discount  \\\n",
       "0       998             998    54.909289           1.0                  0   \n",
       "1      1953            1953    45.961352           0.0                  1   \n",
       "2      1083            1083    53.035439           1.0                  0   \n",
       "3      1013            1013    46.046917           1.0                  1   \n",
       "4      1628            1628    50.722161           1.0                  0   \n",
       "\n",
       "      goods_list_time   goods_delist_time  customer_city_id  \\\n",
       "0 2014-10-25 11:08:07 2014-11-01 11:08:07                53   \n",
       "1 2013-08-28 17:27:50 2013-09-01 00:38:17               322   \n",
       "2 2014-10-29 18:21:05 2014-11-05 18:21:05               322   \n",
       "3 2014-10-25 11:00:00 2014-11-01 11:00:00               185   \n",
       "4 2014-10-23 15:35:33 2014-10-30 15:35:33                53   \n",
       "\n",
       "   customer_province_id  \n",
       "0                     4  \n",
       "1                     3  \n",
       "2                     3  \n",
       "3                    20  \n",
       "4                     4  \n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1585986"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train['customer_id'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1585986 entries, 1000000 to 2826574\n",
      "Data columns (total 1 columns):\n",
      " #   Column  Non-Null Count    Dtype  \n",
      "---  ------  --------------    -----  \n",
      " 0   result  1585986 non-null  float64\n",
      "dtypes: float64(1)\n",
      "memory usage: 24.2 MB\n"
     ]
    }
   ],
   "source": [
    "test.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>result</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>customer_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1000000</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000014</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000034</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000046</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000048</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             result\n",
       "customer_id        \n",
       "1000000         0.0\n",
       "1000014         0.0\n",
       "1000034         0.0\n",
       "1000046         0.0\n",
       "1000048         0.0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "从上面的比对可以看出，这里要预测的是全体用户未来一个月的购买情况，训练集和测试集的`id`完全重合，没有需要特别处理的地方"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 构造时间滑窗特征"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "时间滑窗构造\n",
    "- [已完成] 考虑到按周、按月的订单规律性变化，时间滑窗往往是7和30的倍数\n",
    "- [已完成] 一般选取的特征是滑窗中的基本统计值，最大、最小、均值、中位数、求和等等\n",
    "- [未完成] 节假日信息、影响周期的时间滑窗整合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 每日付款金额"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  after removing the cwd from sys.path.\n"
     ]
    }
   ],
   "source": [
    "# 将用户下单金额按天进行汇总\r\n",
    "# df = train[train.order_status<101][train.order_pay_time>'2013-02-01']\r\n",
    "df = train[train.order_pay_time>'2013-02-01']\r\n",
    "df['date'] = pd.DatetimeIndex(df['order_pay_time']).date\r\n",
    "df_payment = df[['customer_id','date','order_total_payment']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "685471"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df_payment['customer_id'].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "注意，成功交易的客户数量不等于全部客户数量，说明有相当一部分客户虽然下过单，但是没有成功的订单，那么这些客户自然应当算在训练集之外。\n",
    "数据合并时，由于`test.csv`中，已经设置了默认0值，只需要和训练后的预测标签做一个`left join`就可以了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "df_payment = df_payment.groupby(['date','customer_id']).agg({'order_total_payment': ['sum']})\n",
    "df_payment.columns = ['day_total_payment']\n",
    "df_payment.reset_index(inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "df_payment = df_payment.set_index(\n",
    "    [\"customer_id\", \"date\"])[[\"day_total_payment\"]].unstack(level=-1).fillna(0)\n",
    "df_payment.columns = df_payment.columns.get_level_values(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>date</th>\n",
       "      <th>2013-02-01</th>\n",
       "      <th>2013-02-02</th>\n",
       "      <th>2013-02-03</th>\n",
       "      <th>2013-02-04</th>\n",
       "      <th>2013-02-05</th>\n",
       "      <th>2013-02-06</th>\n",
       "      <th>2013-02-07</th>\n",
       "      <th>2013-02-08</th>\n",
       "      <th>2013-02-09</th>\n",
       "      <th>2013-02-10</th>\n",
       "      <th>...</th>\n",
       "      <th>2013-08-22</th>\n",
       "      <th>2013-08-23</th>\n",
       "      <th>2013-08-24</th>\n",
       "      <th>2013-08-25</th>\n",
       "      <th>2013-08-26</th>\n",
       "      <th>2013-08-27</th>\n",
       "      <th>2013-08-28</th>\n",
       "      <th>2013-08-29</th>\n",
       "      <th>2013-08-30</th>\n",
       "      <th>2013-08-31</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>customer_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1000014</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000034</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000046</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000069</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000105</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 212 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "date         2013-02-01  2013-02-02  2013-02-03  2013-02-04  2013-02-05  \\\n",
       "customer_id                                                               \n",
       "1000014             0.0         0.0         0.0         0.0         0.0   \n",
       "1000034             0.0         0.0         0.0         0.0         0.0   \n",
       "1000046             0.0         0.0         0.0         0.0         0.0   \n",
       "1000069             0.0         0.0         0.0         0.0         0.0   \n",
       "1000105             0.0         0.0         0.0         0.0         0.0   \n",
       "\n",
       "date         2013-02-06  2013-02-07  2013-02-08  2013-02-09  2013-02-10  ...  \\\n",
       "customer_id                                                              ...   \n",
       "1000014             0.0         0.0         0.0         0.0         0.0  ...   \n",
       "1000034             0.0         0.0         0.0         0.0         0.0  ...   \n",
       "1000046             0.0         0.0         0.0         0.0         0.0  ...   \n",
       "1000069             0.0         0.0         0.0         0.0         0.0  ...   \n",
       "1000105             0.0         0.0         0.0         0.0         0.0  ...   \n",
       "\n",
       "date         2013-08-22  2013-08-23  2013-08-24  2013-08-25  2013-08-26  \\\n",
       "customer_id                                                               \n",
       "1000014             0.0         0.0         0.0         0.0         0.0   \n",
       "1000034             0.0         0.0         0.0         0.0         0.0   \n",
       "1000046             0.0         0.0         0.0         0.0         0.0   \n",
       "1000069             0.0         0.0         0.0         0.0         0.0   \n",
       "1000105             0.0         0.0         0.0         0.0         0.0   \n",
       "\n",
       "date         2013-08-27  2013-08-28  2013-08-29  2013-08-30  2013-08-31  \n",
       "customer_id                                                              \n",
       "1000014             0.0         0.0         0.0         0.0         0.0  \n",
       "1000034             0.0         0.0         0.0         0.0         0.0  \n",
       "1000046             0.0         0.0         0.0         0.0         0.0  \n",
       "1000069             0.0         0.0         0.0         0.0         0.0  \n",
       "1000105             0.0         0.0         0.0         0.0         0.0  \n",
       "\n",
       "[5 rows x 212 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_payment.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 每日购买商品数量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "df_goods = df[['customer_id','date','order_total_num']]\r\n",
    "df_goods = df_goods.groupby(['date','customer_id']).agg({'order_total_num': ['sum']})\r\n",
    "df_goods.columns = ['day_total_num']\r\n",
    "df_goods.reset_index(inplace=True)\r\n",
    "df_goods = df_goods.set_index(\r\n",
    "    [\"customer_id\", \"date\"])[[\"day_total_num\"]].unstack(level=-1).fillna(0)\r\n",
    "df_goods.columns = df_goods.columns.get_level_values(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "该场景每天都有成交记录，这样就不需要考虑生成完整时间段填充的问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 这是一个时间滑窗函数，获得dt之前minus天以来periods的dataframe，以便进一步计算\n",
    "def get_timespan(df, dt, minus, periods, freq='D'):\n",
    "    return df[pd.date_range(dt - timedelta(days=minus), periods=periods, freq=freq)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "1. 构造dataset这里有个取巧的地方，因为要预测的9月份除了开学季以外不是非常特殊的月份，因此主要考虑近期的因素，数据集的开始时间也是2月1日，尽量避免了双十一、元旦假期的影响，当然春节假期继续保留。同时，构造数据集的时候保留了customer_id，主要为了与其它特征做整合。\n",
    "2. 通过一个函数整合付款金额和商品数量的时间滑窗，主要是因为分开做到时候合并占用内存更大，并且函数最后在返回值处做了内存优化，用时间代价尽可能避免内存溢出。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def prepare_dataset(df_payment, df_goods, t2018, is_train=True):\r\n",
    "    X = {}\r\n",
    "    # 整合用户id\r\n",
    "    tmp = df_payment.reset_index()\r\n",
    "    X['customer_id'] = tmp['customer_id']\r\n",
    "    # 消费特征\r\n",
    "    print('Preparing payment feature...')\r\n",
    "    for i in [7,14,30,49,60,91,120]:\r\n",
    "        tmp_1 = get_timespan(df_payment, t2018, i, i)\r\n",
    "        # X['diff_%s_mean' % i] = tmp_1.diff(axis=1).mean(axis=1).values\r\n",
    "        # X['mean_%s_decay' % i] = (tmp_1 * np.power(0.9, np.arange(i)[::-1])).sum(axis=1).values\r\n",
    "        X['mean_%s' % i] = tmp_1.mean(axis=1).values\r\n",
    "        X['median_%s' % i] = tmp_1.median(axis=1).values\r\n",
    "        # X['min_%s' % i] = tmp_1.min(axis=1).values\r\n",
    "        X['max_%s' % i] = tmp_1.max(axis=1).values\r\n",
    "        # X['std_%s' % i] = tmp_1.std(axis=1).values\r\n",
    "        X['sum_%s' % i] = tmp_1.sum(axis=1).values\r\n",
    "\r\n",
    "        tmp_2 = get_timespan(df_payment, t2018 + timedelta(weeks=-1), i, i)\r\n",
    "        # X['diff_%s_mean_2' % i] = tmp_2.diff(axis=1).mean(axis=1).values\r\n",
    "        # X['mean_%s_decay_2' % i] = (tmp_2 * np.power(0.9, np.arange(i)[::-1])).sum(axis=1).values\r\n",
    "        X['mean_%s_2' % i] = tmp_2.mean(axis=1).values\r\n",
    "        X['median_%s_2' % i] = tmp_2.median(axis=1).values\r\n",
    "        # X['min_%s_2' % i] = tmp_2.min(axis=1).values\r\n",
    "        X['max_%s_2' % i] = tmp_2.max(axis=1).values\r\n",
    "        # X['std_%s_2' % i] = tmp_2.std(axis=1).values\r\n",
    "\r\n",
    "        tmp_3 = get_timespan(df_payment, t2018, i, i)\r\n",
    "        X['has_sales_days_in_last_%s' % i] = (tmp_3 != 0).sum(axis=1).values\r\n",
    "        X['last_has_sales_day_in_last_%s' % i] = i - ((tmp_3 != 0) * np.arange(i)).max(axis=1).values\r\n",
    "        X['first_has_sales_day_in_last_%s' % i] = ((tmp_3 != 0) * np.arange(i, 0, -1)).max(axis=1).values\r\n",
    "\r\n",
    "    # 对此处进行微调，主要考虑近期因素\r\n",
    "    for i in range(1, 4):\r\n",
    "        X['day_%s_2018' % i] = get_timespan(df_payment, t2018, i*30, 30).sum(axis=1).values\r\n",
    "\r\n",
    "    for i in range(7):\r\n",
    "        X['mean_4_dow{}_2013'.format(i)] = get_timespan(df_payment, t2018, 56-i*2, 4, freq='14D').mean(axis=1).values\r\n",
    "        # X['mean_20_dow{}_2013'.format(i)] = get_timespan(df, t2018, 140-i, 20, freq='7D').mean(axis=1).values\r\n",
    "        X['mean_20_dow{}_2013'.format(i)] = get_timespan(df_payment, t2018, 140-i*2, 10, freq='14D').mean(axis=1).values\r\n",
    "    \r\n",
    "    # 商品数量特征，这里故意把时间和消费特征错开，提高时间滑窗的覆盖面\r\n",
    "    print('Preparing num feature...')\r\n",
    "    for i in [3,21,35,49,70,84,105]:\r\n",
    "            tmp_1 = get_timespan(df_goods, t2018, i, i)\r\n",
    "            # X['goods_diff_%s_mean' % i] = tmp_1.diff(axis=1).mean(axis=1).values\r\n",
    "            # X['goods_mean_%s_decay' % i] = (tmp_1 * np.power(0.9, np.arange(i)[::-1])).sum(axis=1).values\r\n",
    "            X['goods_mean_%s' % i] = tmp_1.mean(axis=1).values\r\n",
    "            X['goods_median_%s' % i] = tmp_1.median(axis=1).values\r\n",
    "            # X['goods_min_%s' % i] = tmp_1.min(axis=1).values\r\n",
    "            X['goods_max_%s' % i] = tmp_1.max(axis=1).values\r\n",
    "            # X['goods_std_%s' % i] = tmp_1.std(axis=1).values\r\n",
    "            X['goods_sum_%s' % i] = tmp_1.sum(axis=1).values\r\n",
    "    \r\n",
    "            tmp_2 = get_timespan(df_goods, t2018 + timedelta(weeks=-1), i, i)\r\n",
    "            # X['goods_diff_%s_mean_2' % i] = tmp_2.diff(axis=1).mean(axis=1).values\r\n",
    "            # X['goods_mean_%s_decay_2' % i] = (tmp_2 * np.power(0.9, np.arange(i)[::-1])).sum(axis=1).values\r\n",
    "            X['goods_mean_%s_2' % i] = tmp_2.mean(axis=1).values\r\n",
    "            X['goods_median_%s_2' % i] = tmp_2.median(axis=1).values\r\n",
    "            # X['goods_min_%s_2' % i] = tmp_2.min(axis=1).values\r\n",
    "            X['goods_max_%s_2' % i] = tmp_2.max(axis=1).values\r\n",
    "            X['goods_sum_%s_2' % i] = tmp_2.sum(axis=1).values\r\n",
    "    \r\n",
    "            tmp_3 = get_timespan(df_goods, t2018, i, i)\r\n",
    "            X['goods_has_sales_days_in_last_%s' % i] = (tmp_3 > 0).sum(axis=1).values\r\n",
    "            X['goods_last_has_sales_day_in_last_%s' % i] = i - ((tmp_3 > 0) * np.arange(i)).max(axis=1).values\r\n",
    "            X['goods_first_has_sales_day_in_last_%s' % i] = ((tmp_3 > 0) * np.arange(i, 0, -1)).max(axis=1).values\r\n",
    "\r\n",
    "\r\n",
    "    # 对此处进行微调，主要考虑近期因素\r\n",
    "    for i in range(1, 4):\r\n",
    "        X['goods_day_%s_2018' % i] = get_timespan(df_goods, t2018, i*28, 28).sum(axis=1).values\r\n",
    "\r\n",
    "    for i in range(7):\r\n",
    "        X['goods_mean_4_dow{}_2013'.format(i)] = get_timespan(df_goods, t2018, 56-i*2, 4, freq='14D').mean(axis=1).values\r\n",
    "        # X['mean_20_dow{}_2013'.format(i)] = get_timespan(df, t2018, 140-i, 20, freq='7D').mean(axis=1).values\r\n",
    "        X['goods_mean_20_dow{}_2013'.format(i)] = get_timespan(df_goods, t2018, 140-i*2, 10, freq='14D').mean(axis=1).values\r\n",
    "\r\n",
    "    X = pd.DataFrame(X)\r\n",
    "    \r\n",
    "    reduce_mem_usage(X)\r\n",
    "    \r\n",
    "    if is_train:\r\n",
    "        # 这样转换之后，打标签直接用numpy切片就可以了\r\n",
    "        # 当然这里前提是确认付款总额没有负数的问题\r\n",
    "        y = df_goods[pd.date_range(t2018, periods=30)].max(axis=1).values\r\n",
    "        y[y > 0] = 1\r\n",
    "        return X, y\r\n",
    "    return X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 生成滑窗数据\n",
    "此处对内存占用要求非常高，尽量分步操作避免crash，即使如此，内存溢出仍然频繁出现，通过下面的办法进一步压缩数据集大小"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 生成训练集\n",
    "对此处进行优化，把训练集不曾涉及到的新增用户剔除，也就是说，模型研究的是已有用户的购买行为变化趋势。如果一个用户在8月中旬才第一次下单，那么就不会被放到训练集/验证集中，只会出现在测试集中。\n",
    "\n",
    "内存限制的情况下，训练集和验证集时间的选取会比较纠结，既要考虑月份，也要考虑星期。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def make_slides(train,df_part,begin,end,column):\n",
    "    # 将用户下单金额按天进行汇总\n",
    "    # df = train[train.order_status <= 6][train.order_pay_time > '2013-02-01']\n",
    "    df = train[train.order_pay_time > begin][train.order_pay_time < end]\n",
    "    df = pd.merge(df,df_part,how='inner')\n",
    "    df['date'] = pd.DatetimeIndex(df['order_pay_time']).date\n",
    "    df = df[['customer_id', 'date', column]]\n",
    "    df = df.groupby(['date', 'customer_id']).agg({column: ['sum']})\n",
    "    df.columns = ['day_' + column]\n",
    "    df.reset_index(inplace=True)\n",
    "    df = df.set_index(\n",
    "        [\"customer_id\", \"date\"])[['day_' + column]].unstack(level=-1).fillna(0)\n",
    "    df.columns = df.columns.get_level_values(1)\n",
    "    return df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### part1\n",
    "2013.06.06之前的用户，在2013.06.06后30天的时间滑窗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:1: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:4: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing payment feature...\n",
      "Preparing num feature...\n",
      "Memory usage of the dataframe is : 1057.7543182373047 MB\n",
      "___MEMORY USAGE AFTER COMPLETION:___\n",
      "Memory usage is:  219.39694023132324  MB\n",
      "This is  20.741767388568775 % of the initial size\n"
     ]
    }
   ],
   "source": [
    "df_0606 = train[train.order_pay_time > '2013-01-01'][train.order_pay_time <= '2013-06-06'][['customer_id']]\n",
    "# 删除重复行\n",
    "df_0606 = df_0606.drop_duplicates(['customer_id'])\n",
    "df_part1_partment = make_slides(train,df_0606,'2013-01-01','2013-07-06','order_total_payment')\n",
    "df_part1_goods = make_slides(train,df_0606,'2013-01-01','2013-07-06','order_total_num')\n",
    "X_part1, y_part1 = prepare_dataset(df_part1_partment, df_part1_goods, date(2013, 6, 6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_part1.to_pickle('work/X_part1.pkl')\n",
    "np.save(\"work/y_part1.npy\", y_part1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del df_0606\n",
    "del df_part1_partment\n",
    "del df_part1_goods\n",
    "del X_part1\n",
    "del y_part1\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### part2\n",
    "2013.06.20之前的用户，在2013.06.20后30天的时间滑窗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:1: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:4: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing payment feature...\n",
      "Preparing num feature...\n",
      "Memory usage of the dataframe is : 1051.5419464111328 MB\n",
      "___MEMORY USAGE AFTER COMPLETION:___\n",
      "Memory usage is:  215.2195339202881  MB\n",
      "This is  20.46704219977368 % of the initial size\n"
     ]
    }
   ],
   "source": [
    "df_0620 = train[train.order_pay_time > '2013-01-15'][train.order_pay_time <= '2013-06-20'][['customer_id']]\n",
    "# 删除重复行\n",
    "df_0620 = df_0620.drop_duplicates(['customer_id'])\n",
    "df_part2_partment = make_slides(train,df_0620,'2013-01-15','2013-07-20','order_total_payment')\n",
    "df_part2_goods = make_slides(train,df_0620,'2013-01-15','2013-07-20','order_total_num')\n",
    "X_part2, y_part2 = prepare_dataset(df_part2_partment, df_part2_goods, date(2013, 6, 20))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_part2.to_pickle('work/X_part2.pkl')\n",
    "np.save(\"work/y_part2.npy\", y_part2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del df_0620\n",
    "del df_part2_partment\n",
    "del df_part2_goods\n",
    "del X_part2\n",
    "del y_part2\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### part3\n",
    "2013.07.01之前的用户，在2013.07.01后30天的时间滑窗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:1: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:4: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing payment feature...\n",
      "Preparing num feature...\n",
      "Memory usage of the dataframe is : 801.8140411376953 MB\n",
      "___MEMORY USAGE AFTER COMPLETION:___\n",
      "Memory usage is:  165.20903396606445  MB\n",
      "This is  20.604407691794606 % of the initial size\n"
     ]
    }
   ],
   "source": [
    "df_0701 = train[train.order_pay_time > '2013-01-26'][train.order_pay_time <= '2013-07-01'][['customer_id']]\n",
    "# 删除重复行\n",
    "df_0701 = df_0701.drop_duplicates(['customer_id'])\n",
    "df_part3_partment = make_slides(train,df_0701,'2013-01-26','2013-07-31','order_total_payment')\n",
    "df_part3_goods = make_slides(train,df_0701,'2013-01-26','2013-07-31','order_total_num')\n",
    "X_part3, y_part3 = prepare_dataset(df_part3_partment, df_part3_goods, date(2013, 7, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_part3.to_pickle('work/X_part3.pkl')\n",
    "np.save(\"work/y_part3.npy\", y_part3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del df_0701\n",
    "del df_part3_partment\n",
    "del df_part3_goods\n",
    "del X_part3\n",
    "del y_part3\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### part4\n",
    "2013.07.15之前的用户，在2013.07.15后30天的时间滑窗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:1: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:4: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing payment feature...\n",
      "Preparing num feature...\n",
      "Memory usage of the dataframe is : 526.6256866455078 MB\n",
      "___MEMORY USAGE AFTER COMPLETION:___\n",
      "Memory usage is:  107.06136322021484  MB\n",
      "This is  20.32968879702254 % of the initial size\n"
     ]
    }
   ],
   "source": [
    "df_0715 = train[train.order_pay_time > '2013-02-10'][train.order_pay_time <= '2013-07-15'][['customer_id']]\n",
    "# 删除重复行\n",
    "df_0715 = df_0715.drop_duplicates(['customer_id'])\n",
    "df_part4_partment = make_slides(train,df_0715,'2013-02-10','2013-08-16','order_total_payment')\n",
    "df_part4_goods = make_slides(train,df_0715,'2013-02-10','2013-08-16','order_total_num')\n",
    "X_part4, y_part4 = prepare_dataset(df_part4_partment, df_part4_goods, date(2013, 7, 15))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_part4.to_pickle('work/X_part4.pkl')\n",
    "np.save(\"work/y_part4.npy\", y_part4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del df_0715\n",
    "del df_part4_partment\n",
    "del df_part4_goods\n",
    "del X_part4\n",
    "del y_part4\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing Train dataset...\n",
      "done!\n"
     ]
    }
   ],
   "source": [
    "print(\"Preparing Train dataset...\")\n",
    "X_l, y_l = [], []\n",
    "X_part1 = pd.read_pickle('work/X_part1.pkl')\n",
    "y_part1 = np.load('work/y_part1.npy')\n",
    "X_l.append(X_part1)\n",
    "y_l.append(y_part1)\n",
    "X_part2 = pd.read_pickle('work/X_part2.pkl')\n",
    "y_part2 = np.load('work/y_part2.npy')\n",
    "X_l.append(X_part2)\n",
    "y_l.append(y_part2)\n",
    "X_part3 = pd.read_pickle('work/X_part3.pkl')\n",
    "y_part3 = np.load('work/y_part3.npy')\n",
    "X_l.append(X_part3)\n",
    "y_l.append(y_part3)\n",
    "X_part4 = pd.read_pickle('work/X_part4.pkl')\n",
    "y_part4 = np.load('work/y_part4.npy')\n",
    "X_l.append(X_part4)\n",
    "y_l.append(y_part4)\n",
    "X_train = pd.concat(X_l, axis=0)\n",
    "y_train = np.concatenate(y_l, axis=0)\n",
    "print('done!')\n",
    "del X_l\n",
    "del y_l\n",
    "del X_part1\n",
    "del y_part1\n",
    "del X_part2\n",
    "del y_part2\n",
    "del X_part3\n",
    "del y_part3\n",
    "del X_part4\n",
    "del y_part4\n",
    "gc.collect()\n",
    "X_train.to_pickle('work/X_train.pkl')\n",
    "np.save(\"work/y_train.npy\", y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 生成验证集\n",
    "2013.08.01之前的用户，在2013.08.01后30天的时间滑窗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing Validation dataset...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:2: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:4: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing payment feature...\n",
      "Preparing num feature...\n",
      "Memory usage of the dataframe is : 561.9365081787109 MB\n",
      "___MEMORY USAGE AFTER COMPLETION:___\n",
      "Memory usage is:  113.85399150848389  MB\n",
      "This is  20.26100633281425 % of the initial size\n"
     ]
    }
   ],
   "source": [
    "print(\"Preparing Validation dataset...\")\n",
    "df_0801 = train[train.order_pay_time > '2013-03-01'][train.order_pay_time <= '2013-08-01'][['customer_id']]\n",
    "# 删除重复行\n",
    "df_0801 = df_0801.drop_duplicates(['customer_id'])\n",
    "df_part5_partment = make_slides(train,df_0801,'2013-03-01','2013-08-31','order_total_payment')\n",
    "df_part5_goods = make_slides(train,df_0801,'2013-03-01','2013-08-31','order_total_num')\n",
    "X_val, y_val = prepare_dataset(df_part5_partment, df_part5_goods, date(2013, 8, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_val.to_pickle('work/X_val.pkl')\n",
    "np.save(\"work/y_val.npy\", y_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del df_0801\n",
    "del df_part5_partment\n",
    "del df_part5_goods\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 生成测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:1: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:4: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  after removing the cwd from sys.path.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing payment feature...\n",
      "Preparing num feature...\n",
      "Memory usage of the dataframe is : 672.3984222412109 MB\n",
      "___MEMORY USAGE AFTER COMPLETION:___\n",
      "Memory usage is:  137.15829181671143  MB\n",
      "This is  20.398366099602228 % of the initial size\n"
     ]
    }
   ],
   "source": [
    "df_0901 = train[train.order_pay_time > '2013-03-01'][train.order_pay_time < '2013-09-01'][['customer_id']]\n",
    "# 删除重复行\n",
    "df_0901 = df_0901.drop_duplicates(['customer_id'])\n",
    "df_test_partment = make_slides(train,df_0901,'2013-04-01','2013-09-01','order_total_payment')\n",
    "df_test_goods = make_slides(train,df_0901,'2013-04-01','2013-09-01','order_total_num')\n",
    "X_test = prepare_dataset(df_test_partment, df_test_goods, date(2013, 9, 1),is_train=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_test.to_pickle('work/X_test.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "至此，整合了用户id的消费时间滑窗就完整了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 用户基本特征\n",
    "\n",
    "把地域、会员这些基本信息整合到训练数据中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>member_id</th>\n",
       "      <th>customer_id</th>\n",
       "      <th>customer_gender</th>\n",
       "      <th>customer_province_id</th>\n",
       "      <th>customer_city_id</th>\n",
       "      <th>is_member_actived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1000000</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1001324</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>322</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1001325</td>\n",
       "      <td>0</td>\n",
       "      <td>20</td>\n",
       "      <td>185</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1001326</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>1001327</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   member_id  customer_id  customer_gender  customer_province_id  \\\n",
       "0          0      1000000                0                     4   \n",
       "1          0      1001324                0                     3   \n",
       "3          0      1001325                0                    20   \n",
       "4          0      1001326                0                     4   \n",
       "5          0      1001327                0                     4   \n",
       "\n",
       "   customer_city_id  is_member_actived  \n",
       "0                53                  0  \n",
       "1               322                  0  \n",
       "3               185                  0  \n",
       "4                53                  0  \n",
       "5                53                  0  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_customer = train[['member_id','customer_id','customer_gender','customer_province_id','customer_city_id','is_member_actived']]\n",
    "# 删除重复行\n",
    "df_customer = df_customer.drop_duplicates(['customer_id'])\n",
    "df_customer.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_train = pd.merge(X_train,df_customer,how='left')\n",
    "X_val = pd.merge(X_val,df_customer,how='left')\n",
    "X_test = pd.merge(X_test,df_customer,how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_train.to_pickle('X_train.pkl')\n",
    "X_val.to_pickle('X_val.pkl')\n",
    "X_test.to_pickle('X_test.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_train = pd.read_pickle('X_train.pkl')\r\n",
    "X_val = pd.read_pickle('X_val.pkl')\r\n",
    "X_test = pd.read_pickle('X_test.pkl')\r\n",
    "\r\n",
    "y_train = np.load(\"work/y_train.npy\")\r\n",
    "y_val = np.load(\"work/y_val.npy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7948"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_l, y_l = [], []\r\n",
    "xx_l,yy_l=[],[]\r\n",
    "X_l.append(X_train)\r\n",
    "y_l.append(y_train)\r\n",
    "\r\n",
    "xx_l.append(X_val)\r\n",
    "yy_l.append(y_val)\r\n",
    "\r\n",
    "X_train_cv = pd.concat(X_l, axis=0)\r\n",
    "y_train_cv = np.concatenate(y_l, axis=0)\r\n",
    "\r\n",
    "X_val_cv = pd.concat(xx_l, axis=0)\r\n",
    "y_val_cv = np.concatenate(yy_l, axis=0)\r\n",
    "\r\n",
    "\r\n",
    "del X_l\r\n",
    "del y_l\r\n",
    "del xx_l\r\n",
    "del yy_l\r\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "**由于正负样本不均衡，因此scale_pos_weight设置为 sum(negative cases) / sum(positive cases)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[16:09:23] WARNING: ../src/learner.cc:541: \n",
      "Parameters: { missing, silent } might not be used.\n",
      "\n",
      "  This may not be accurate due to some parameters are only used in language bindings but\n",
      "  passed down to XGBoost core.  Or some parameters are not used but slip through this\n",
      "  verification. Please open an issue if you find above cases.\n",
      "\n",
      "\n",
      "[0]\ttrain-auc:0.78205\tvalid-auc:0.73007\n",
      "[1]\ttrain-auc:0.78570\tvalid-auc:0.73243\n",
      "[2]\ttrain-auc:0.80639\tvalid-auc:0.75874\n",
      "[3]\ttrain-auc:0.80430\tvalid-auc:0.75572\n",
      "[4]\ttrain-auc:0.81058\tvalid-auc:0.76400\n",
      "[5]\ttrain-auc:0.80928\tvalid-auc:0.76143\n",
      "[6]\ttrain-auc:0.80795\tvalid-auc:0.76005\n",
      "[7]\ttrain-auc:0.81139\tvalid-auc:0.76408\n",
      "[8]\ttrain-auc:0.81016\tvalid-auc:0.76280\n",
      "[9]\ttrain-auc:0.81288\tvalid-auc:0.76653\n",
      "[10]\ttrain-auc:0.81188\tvalid-auc:0.76589\n",
      "[11]\ttrain-auc:0.81170\tvalid-auc:0.76568\n",
      "[12]\ttrain-auc:0.81375\tvalid-auc:0.76880\n",
      "[13]\ttrain-auc:0.81317\tvalid-auc:0.76809\n",
      "[14]\ttrain-auc:0.81505\tvalid-auc:0.77034\n",
      "[15]\ttrain-auc:0.81641\tvalid-auc:0.77228\n",
      "[16]\ttrain-auc:0.81605\tvalid-auc:0.77184\n",
      "[17]\ttrain-auc:0.81565\tvalid-auc:0.77123\n",
      "[18]\ttrain-auc:0.81526\tvalid-auc:0.77047\n",
      "[19]\ttrain-auc:0.81497\tvalid-auc:0.77037\n",
      "[20]\ttrain-auc:0.81635\tvalid-auc:0.77181\n",
      "[21]\ttrain-auc:0.81745\tvalid-auc:0.77325\n",
      "[22]\ttrain-auc:0.81854\tvalid-auc:0.77418\n",
      "[23]\ttrain-auc:0.81922\tvalid-auc:0.77514\n",
      "[24]\ttrain-auc:0.81895\tvalid-auc:0.77498\n",
      "[25]\ttrain-auc:0.81963\tvalid-auc:0.77613\n",
      "[26]\ttrain-auc:0.81941\tvalid-auc:0.77566\n",
      "[27]\ttrain-auc:0.81928\tvalid-auc:0.77535\n",
      "[28]\ttrain-auc:0.81912\tvalid-auc:0.77511\n",
      "[29]\ttrain-auc:0.81980\tvalid-auc:0.77607\n",
      "[30]\ttrain-auc:0.81978\tvalid-auc:0.77584\n",
      "[31]\ttrain-auc:0.82045\tvalid-auc:0.77630\n",
      "[32]\ttrain-auc:0.82035\tvalid-auc:0.77595\n",
      "[33]\ttrain-auc:0.82090\tvalid-auc:0.77673\n",
      "[34]\ttrain-auc:0.82165\tvalid-auc:0.77744\n",
      "[35]\ttrain-auc:0.82173\tvalid-auc:0.77714\n",
      "[36]\ttrain-auc:0.82172\tvalid-auc:0.77687\n",
      "[37]\ttrain-auc:0.82155\tvalid-auc:0.77655\n",
      "[38]\ttrain-auc:0.82156\tvalid-auc:0.77625\n",
      "[39]\ttrain-auc:0.82157\tvalid-auc:0.77608\n",
      "[40]\ttrain-auc:0.82163\tvalid-auc:0.77578\n",
      "[41]\ttrain-auc:0.82235\tvalid-auc:0.77663\n",
      "[42]\ttrain-auc:0.82241\tvalid-auc:0.77664\n",
      "[43]\ttrain-auc:0.82240\tvalid-auc:0.77643\n",
      "[44]\ttrain-auc:0.82309\tvalid-auc:0.77688\n",
      "[45]\ttrain-auc:0.82304\tvalid-auc:0.77682\n",
      "[46]\ttrain-auc:0.82316\tvalid-auc:0.77670\n",
      "[47]\ttrain-auc:0.82322\tvalid-auc:0.77659\n",
      "[48]\ttrain-auc:0.82327\tvalid-auc:0.77643\n",
      "[49]\ttrain-auc:0.82392\tvalid-auc:0.77695\n",
      "[50]\ttrain-auc:0.82381\tvalid-auc:0.77684\n",
      "[51]\ttrain-auc:0.82439\tvalid-auc:0.77740\n",
      "[52]\ttrain-auc:0.82444\tvalid-auc:0.77741\n",
      "[53]\ttrain-auc:0.82500\tvalid-auc:0.77798\n",
      "[54]\ttrain-auc:0.82503\tvalid-auc:0.77795\n",
      "[55]\ttrain-auc:0.82510\tvalid-auc:0.77784\n",
      "[56]\ttrain-auc:0.82511\tvalid-auc:0.77762\n",
      "[57]\ttrain-auc:0.82522\tvalid-auc:0.77768\n",
      "[58]\ttrain-auc:0.82586\tvalid-auc:0.77827\n",
      "[59]\ttrain-auc:0.82592\tvalid-auc:0.77816\n",
      "[60]\ttrain-auc:0.82650\tvalid-auc:0.77877\n",
      "[61]\ttrain-auc:0.82701\tvalid-auc:0.77930\n",
      "[62]\ttrain-auc:0.82710\tvalid-auc:0.77921\n",
      "[63]\ttrain-auc:0.82713\tvalid-auc:0.77914\n",
      "[64]\ttrain-auc:0.82760\tvalid-auc:0.77970\n",
      "[65]\ttrain-auc:0.82774\tvalid-auc:0.77967\n",
      "[66]\ttrain-auc:0.82775\tvalid-auc:0.77967\n",
      "[67]\ttrain-auc:0.82783\tvalid-auc:0.77959\n",
      "[68]\ttrain-auc:0.82792\tvalid-auc:0.77961\n",
      "[69]\ttrain-auc:0.82810\tvalid-auc:0.77963\n",
      "[70]\ttrain-auc:0.82868\tvalid-auc:0.78017\n",
      "[71]\ttrain-auc:0.82913\tvalid-auc:0.78058\n",
      "[72]\ttrain-auc:0.82924\tvalid-auc:0.78037\n",
      "[73]\ttrain-auc:0.82976\tvalid-auc:0.78083\n",
      "[74]\ttrain-auc:0.83033\tvalid-auc:0.78125\n",
      "[75]\ttrain-auc:0.83042\tvalid-auc:0.78121\n",
      "[76]\ttrain-auc:0.83086\tvalid-auc:0.78158\n",
      "[77]\ttrain-auc:0.83138\tvalid-auc:0.78200\n",
      "[78]\ttrain-auc:0.83178\tvalid-auc:0.78230\n",
      "[79]\ttrain-auc:0.83189\tvalid-auc:0.78198\n",
      "[80]\ttrain-auc:0.83234\tvalid-auc:0.78229\n",
      "[81]\ttrain-auc:0.83270\tvalid-auc:0.78263\n",
      "[82]\ttrain-auc:0.83307\tvalid-auc:0.78290\n",
      "[83]\ttrain-auc:0.83351\tvalid-auc:0.78328\n",
      "[84]\ttrain-auc:0.83383\tvalid-auc:0.78359\n",
      "[85]\ttrain-auc:0.83418\tvalid-auc:0.78383\n",
      "[86]\ttrain-auc:0.83424\tvalid-auc:0.78380\n",
      "[87]\ttrain-auc:0.83457\tvalid-auc:0.78406\n",
      "[88]\ttrain-auc:0.83495\tvalid-auc:0.78429\n",
      "[89]\ttrain-auc:0.83499\tvalid-auc:0.78410\n",
      "[90]\ttrain-auc:0.83538\tvalid-auc:0.78435\n",
      "[91]\ttrain-auc:0.83567\tvalid-auc:0.78450\n",
      "[92]\ttrain-auc:0.83583\tvalid-auc:0.78457\n",
      "[93]\ttrain-auc:0.83609\tvalid-auc:0.78477\n",
      "[94]\ttrain-auc:0.83646\tvalid-auc:0.78499\n",
      "[95]\ttrain-auc:0.83658\tvalid-auc:0.78501\n",
      "[96]\ttrain-auc:0.83694\tvalid-auc:0.78516\n",
      "[97]\ttrain-auc:0.83719\tvalid-auc:0.78532\n",
      "[98]\ttrain-auc:0.83727\tvalid-auc:0.78537\n",
      "[99]\ttrain-auc:0.83741\tvalid-auc:0.78542\n",
      "[100]\ttrain-auc:0.83761\tvalid-auc:0.78560\n",
      "[101]\ttrain-auc:0.83773\tvalid-auc:0.78562\n",
      "[102]\ttrain-auc:0.83782\tvalid-auc:0.78563\n",
      "[103]\ttrain-auc:0.83817\tvalid-auc:0.78588\n",
      "[104]\ttrain-auc:0.83843\tvalid-auc:0.78607\n",
      "[105]\ttrain-auc:0.83873\tvalid-auc:0.78623\n",
      "[106]\ttrain-auc:0.83901\tvalid-auc:0.78642\n",
      "[107]\ttrain-auc:0.83933\tvalid-auc:0.78660\n",
      "[108]\ttrain-auc:0.83958\tvalid-auc:0.78674\n",
      "[109]\ttrain-auc:0.84002\tvalid-auc:0.78694\n",
      "[110]\ttrain-auc:0.84031\tvalid-auc:0.78706\n",
      "[111]\ttrain-auc:0.84048\tvalid-auc:0.78703\n",
      "[112]\ttrain-auc:0.84068\tvalid-auc:0.78714\n",
      "[113]\ttrain-auc:0.84101\tvalid-auc:0.78732\n",
      "[114]\ttrain-auc:0.84120\tvalid-auc:0.78742\n",
      "[115]\ttrain-auc:0.84138\tvalid-auc:0.78752\n",
      "[116]\ttrain-auc:0.84159\tvalid-auc:0.78763\n",
      "[117]\ttrain-auc:0.84184\tvalid-auc:0.78776\n",
      "[118]\ttrain-auc:0.84208\tvalid-auc:0.78789\n",
      "[119]\ttrain-auc:0.84223\tvalid-auc:0.78783\n",
      "[120]\ttrain-auc:0.84246\tvalid-auc:0.78792\n",
      "[121]\ttrain-auc:0.84274\tvalid-auc:0.78804\n",
      "[122]\ttrain-auc:0.84301\tvalid-auc:0.78819\n",
      "[123]\ttrain-auc:0.84331\tvalid-auc:0.78824\n",
      "[124]\ttrain-auc:0.84345\tvalid-auc:0.78826\n",
      "[125]\ttrain-auc:0.84368\tvalid-auc:0.78837\n",
      "[126]\ttrain-auc:0.84394\tvalid-auc:0.78852\n",
      "[127]\ttrain-auc:0.84418\tvalid-auc:0.78866\n",
      "[128]\ttrain-auc:0.84434\tvalid-auc:0.78862\n",
      "[129]\ttrain-auc:0.84445\tvalid-auc:0.78862\n",
      "[130]\ttrain-auc:0.84452\tvalid-auc:0.78865\n",
      "[131]\ttrain-auc:0.84460\tvalid-auc:0.78871\n",
      "[132]\ttrain-auc:0.84470\tvalid-auc:0.78860\n",
      "[133]\ttrain-auc:0.84481\tvalid-auc:0.78856\n",
      "[134]\ttrain-auc:0.84504\tvalid-auc:0.78869\n",
      "[135]\ttrain-auc:0.84528\tvalid-auc:0.78881\n",
      "[136]\ttrain-auc:0.84552\tvalid-auc:0.78891\n",
      "[137]\ttrain-auc:0.84578\tvalid-auc:0.78900\n",
      "[138]\ttrain-auc:0.84594\tvalid-auc:0.78906\n",
      "[139]\ttrain-auc:0.84613\tvalid-auc:0.78916\n",
      "[140]\ttrain-auc:0.84630\tvalid-auc:0.78924\n",
      "[141]\ttrain-auc:0.84636\tvalid-auc:0.78924\n",
      "[142]\ttrain-auc:0.84662\tvalid-auc:0.78938\n",
      "[143]\ttrain-auc:0.84683\tvalid-auc:0.78944\n",
      "[144]\ttrain-auc:0.84693\tvalid-auc:0.78945\n",
      "[145]\ttrain-auc:0.84702\tvalid-auc:0.78942\n",
      "[146]\ttrain-auc:0.84722\tvalid-auc:0.78951\n",
      "[147]\ttrain-auc:0.84738\tvalid-auc:0.78955\n",
      "[148]\ttrain-auc:0.84755\tvalid-auc:0.78953\n",
      "[149]\ttrain-auc:0.84763\tvalid-auc:0.78954\n",
      "[150]\ttrain-auc:0.84785\tvalid-auc:0.78963\n",
      "[151]\ttrain-auc:0.84794\tvalid-auc:0.78963\n",
      "[152]\ttrain-auc:0.84801\tvalid-auc:0.78960\n",
      "[153]\ttrain-auc:0.84808\tvalid-auc:0.78960\n",
      "[154]\ttrain-auc:0.84818\tvalid-auc:0.78958\n",
      "[155]\ttrain-auc:0.84826\tvalid-auc:0.78959\n",
      "[156]\ttrain-auc:0.84860\tvalid-auc:0.78978\n",
      "[157]\ttrain-auc:0.84884\tvalid-auc:0.78987\n",
      "[158]\ttrain-auc:0.84900\tvalid-auc:0.78980\n",
      "[159]\ttrain-auc:0.84920\tvalid-auc:0.78988\n",
      "[160]\ttrain-auc:0.84945\tvalid-auc:0.78993\n",
      "[161]\ttrain-auc:0.84952\tvalid-auc:0.78996\n",
      "[162]\ttrain-auc:0.84985\tvalid-auc:0.79012\n",
      "[163]\ttrain-auc:0.85001\tvalid-auc:0.79021\n",
      "[164]\ttrain-auc:0.85014\tvalid-auc:0.79020\n",
      "[165]\ttrain-auc:0.85032\tvalid-auc:0.79029\n",
      "[166]\ttrain-auc:0.85056\tvalid-auc:0.79039\n",
      "[167]\ttrain-auc:0.85070\tvalid-auc:0.79042\n",
      "[168]\ttrain-auc:0.85087\tvalid-auc:0.79047\n",
      "[169]\ttrain-auc:0.85118\tvalid-auc:0.79062\n",
      "[170]\ttrain-auc:0.85129\tvalid-auc:0.79059\n",
      "[171]\ttrain-auc:0.85143\tvalid-auc:0.79061\n",
      "[172]\ttrain-auc:0.85159\tvalid-auc:0.79065\n",
      "[173]\ttrain-auc:0.85166\tvalid-auc:0.79065\n",
      "[174]\ttrain-auc:0.85173\tvalid-auc:0.79065\n",
      "[175]\ttrain-auc:0.85194\tvalid-auc:0.79071\n",
      "[176]\ttrain-auc:0.85205\tvalid-auc:0.79076\n",
      "[177]\ttrain-auc:0.85219\tvalid-auc:0.79081\n",
      "[178]\ttrain-auc:0.85223\tvalid-auc:0.79076\n",
      "[179]\ttrain-auc:0.85230\tvalid-auc:0.79072\n",
      "[180]\ttrain-auc:0.85247\tvalid-auc:0.79077\n",
      "[181]\ttrain-auc:0.85263\tvalid-auc:0.79080\n",
      "[182]\ttrain-auc:0.85272\tvalid-auc:0.79080\n",
      "[183]\ttrain-auc:0.85290\tvalid-auc:0.79085\n",
      "[184]\ttrain-auc:0.85298\tvalid-auc:0.79087\n",
      "[185]\ttrain-auc:0.85325\tvalid-auc:0.79098\n",
      "[186]\ttrain-auc:0.85338\tvalid-auc:0.79101\n",
      "[187]\ttrain-auc:0.85354\tvalid-auc:0.79096\n",
      "[188]\ttrain-auc:0.85365\tvalid-auc:0.79096\n",
      "[189]\ttrain-auc:0.85381\tvalid-auc:0.79104\n",
      "[190]\ttrain-auc:0.85402\tvalid-auc:0.79109\n",
      "[191]\ttrain-auc:0.85419\tvalid-auc:0.79113\n",
      "[192]\ttrain-auc:0.85451\tvalid-auc:0.79130\n",
      "[193]\ttrain-auc:0.85464\tvalid-auc:0.79136\n",
      "[194]\ttrain-auc:0.85473\tvalid-auc:0.79135\n",
      "[195]\ttrain-auc:0.85497\tvalid-auc:0.79148\n",
      "[196]\ttrain-auc:0.85513\tvalid-auc:0.79155\n",
      "[197]\ttrain-auc:0.85521\tvalid-auc:0.79155\n",
      "[198]\ttrain-auc:0.85551\tvalid-auc:0.79170\n",
      "[199]\ttrain-auc:0.85574\tvalid-auc:0.79182\n"
     ]
    }
   ],
   "source": [
    "import xgboost as xgb\r\n",
    "\r\n",
    "xgb_train = xgb.DMatrix(X_train_cv.values, label=y_train_cv)\r\n",
    "xgb_val = xgb.DMatrix(X_val_cv.values, label=y_val_cv)\r\n",
    "xgb_test = xgb.DMatrix(X_test.values)\r\n",
    "y=y_train_cv\r\n",
    "\r\n",
    "params = {\r\n",
    "    'booster': 'gbtree',\r\n",
    "    'objective': 'binary:logistic',\r\n",
    "    'eval_metric': 'logloss',\r\n",
    "    'gamma': 0.1,  # 用于控制是否后剪枝的参数,越大越保守，一般0.1、0.2这样子。\r\n",
    "    'max_depth': 8,  # 构建树的深度，越大越容易过拟合\r\n",
    "    'alpha': 0,   # L1正则化系数\r\n",
    "    'lambda': 10,  # 控制模型复杂度的权重值的L2正则化项参数，参数越大，模型越不容易过拟合。\r\n",
    "    'subsample': 0.7,  # 随机采样训练样本\r\n",
    "    'colsample_bytree': 0.5,  # 生成树时进行的列采样\r\n",
    "    'min_child_weight': 3,\r\n",
    "    'silent': 0,  \r\n",
    "    'eta': 0.03, \r\n",
    "    'seed': 1000,\r\n",
    "    'nthread': 8,  \r\n",
    "    'missing': 1,\r\n",
    "    'scale_pos_weight': (np.sum(y==0)/np.sum(y==1))  ,# 用来处理正负样本不均衡的问题,通常取：sum(negative cases) / sum(positive cases)\r\n",
    "    'eval_metric': \"auc\"\r\n",
    "}\r\n",
    "\r\n",
    "plst = list(params.items())\r\n",
    "num_rounds = 200  # 迭代次数\r\n",
    "watchlist = [(xgb_train, 'train'),(xgb_val,'valid')]\r\n",
    "model = xgb.train(plst, xgb_train, num_rounds, watchlist)\r\n",
    "model.save_model('model.xgb')\r\n",
    "pred_value = model.predict(xgb_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 加载模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# model=xgb.Booster(params)\r\n",
    "# model.load_model('model.xgb') \r\n",
    "# pred_value = model.predict(xgb_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 结果预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "test_pred = pred_value # 本来做了bagging，效果不好"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Preparing Predict dataset...\n",
      "Making submission...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customer_id</th>\n",
       "      <th>pred</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000014</td>\n",
       "      <td>0.546633</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000046</td>\n",
       "      <td>0.256358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000069</td>\n",
       "      <td>0.545569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000105</td>\n",
       "      <td>0.917949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000109</td>\n",
       "      <td>0.592108</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   customer_id      pred\n",
       "0      1000014  0.546633\n",
       "1      1000046  0.256358\n",
       "2      1000069  0.545569\n",
       "3      1000105  0.917949\n",
       "4      1000109  0.592108"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Preparing Predict dataset...\")\r\n",
    "\r\n",
    "print(\"Making submission...\")\r\n",
    "y_test = np.array(test_pred).transpose()\r\n",
    "df_preds = pd.DataFrame(\r\n",
    "    {    \r\n",
    "    \"customer_id\": X_test.customer_id, \r\n",
    "    \"pred\": y_test\r\n",
    "    }\r\n",
    ")\r\n",
    "df_preds.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "\r\n",
    "\r\n",
    "sub = pd.read_csv('data/data19383/submission.csv')\r\n",
    "\r\n",
    "# 合并预测结果\r\n",
    "submission = pd.merge(sub, df_preds, on='customer_id', how='left')\r\n",
    "submission.fillna(0,inplace=True)\r\n",
    "submission = submission[['customer_id','pred']]\r\n",
    "submission.rename(columns={'customer_id':'customer_id','pred':'result'}, inplace=True)\r\n",
    "\r\n",
    "# 将概率值转换为用户是否购买的标签\r\n",
    "def f(x):\r\n",
    "    if x <= 0.30:   # 调整阈值\r\n",
    "        return 0\r\n",
    "    else:\r\n",
    "        return 1\r\n",
    "    return x\r\n",
    "submission['result'] = submission['result'].map(f)\r\n",
    "\r\n",
    "# 保存结果\r\n",
    "submission.to_csv('submission.csv',index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "customer_id    3033146322233\n",
       "result                334857\n",
       "dtype: int64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "\t未来，我们可以继续探索更为先进的机器学习模型，并使用更多的评估指标来更准确地评估模型的性能。此外，我们还可以尝试使用更为复杂的特征工程方法来处理数据集，以提升模型的性能。\n",
    "\t总的来说，本实验提供了一种使用XGBoost模型进行商家订单数据分析的方法。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 参考资料\n",
    "[用户购买预测时间滑窗特征构建](https://aistudio.baidu.com/aistudio/projectdetail/276829)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}