Perceptron Binary Classification Learning Algorithm Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Perceptron Binary Classification Learning Algorithm Tutorial\n",
    "\n",
    "註：這個 Tutorial 主要還是介紹怎麼使用 FukuML，如果非必要並不會涉入太多演算法或數學式的細節，若大家對機器學習有興趣，還是建議觀看完整的課程。\n",
    "\n",
    "Perceptron Binary Classification Learning Algorithm（PLA）是最基礎的機器學習算法，主要用在讓機器學習分類，基礎我們會使用在二元分類，再慢慢推廣至多元分類。其核心想法也不難，追根究底就是個知錯能改的演算法，只要有錯就修正分類器，直到不會犯錯為止。PLA 也是一個最基礎的類神經網路的運算神經元，現在很紅的 Deep Learning 的最基礎概念其實就是 PLA，因此了解 PLA 對未來學習機器學習這門課程是很有幫助的。\n",
    "\n",
    "底下列出幾個 PLA 相關的數學式，方便大家日後學習時查閱：\n",
    "\n",
    "### PLA 假設\n",
    "\n",
    "$$\n",
    "h(x) = sign(w^Tx)\n",
    "$$\n",
    "\n",
    "表示 PLA 對資料每一個維度的權重假設，這個權重向量在式子中以 w 表示，所以我們利用 PLA 學習出最能夠分好類的 w 之後，將 x 丟進去這個 PLA 假設，它就會告訴你分類的結果。\n",
    "\n",
    "### PLA 犯錯 \n",
    "\n",
    "$$\n",
    "sign(w_t^Tx_{n(t)}) \\neq y_{n(t)}\n",
    "$$\n",
    "\n",
    "表示 PLA 對哪個資料點是預測錯誤的，其實就是對目前的假設 $w_t$ 對 $x_{n(t)}$ 點進行內積再取正負號，如果與 $y_{n(t)}$ 不同，那就代表 PLA 犯錯了。 \n",
    "\n",
    "### PLA 修正假設\n",
    "\n",
    "$$\n",
    "w_{t+1} = w_t + y_{n(t)}x_{n(t)}\n",
    "$$\n",
    "\n",
    "表示 PLA 犯錯之後怎麼修正，如果 PLA 猜 +1 但答案是 -1，那就往 $-1(x_{n(t)})$ 對 $w_t$ 做修正；如果 PLA 猜 -1 但答案是 +1，那就往 $+1(x_{n(t)})$ 對 $w_t$ 做修正。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用 FukuML 的 PLA 做二元分類\n",
    "\n",
    "接下來讓我們一步一步學習如何使用 FukuML 的 PLA 來做二元分類，首先讓我們將 PLA 引進來："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import FukuML.PLA as pla"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然後建構一個 PLA 二元分類物件："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pla_bc = pla.BinaryClassifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我希望 FukuML 能儘量簡單易用，因此大家只要牢記 1. 載入訓練資料 -> 2. 設定參數 -> 3. 初始化 -> 4. 訓練 -> 5. 預測 這五個步驟就可以完成機器學習了～\n",
    "\n",
    "現在第一個步驟要先載入訓練資料，但如果現在要讓大家生出一筆訓練資料應該會有困難，所以 FukuML 每個機器學習演算法都會有一個 Demo 用的內建資料，讓我們先用 Demo 用的內建資料來試試看。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[ 1.      ,  0.97681 ,  0.10723 ,  0.64385 ,  0.29556 ],\n",
       "        [ 1.      ,  0.67194 ,  0.2418  ,  0.83075 ,  0.42741 ],\n",
       "        [ 1.      ,  0.20619 ,  0.23321 ,  0.81004 ,  0.98691 ],\n",
       "        ..., \n",
       "        [ 1.      ,  0.50468 ,  0.99699 ,  0.75136 ,  0.51681 ],\n",
       "        [ 1.      ,  0.55852 ,  0.067689,  0.666   ,  0.98482 ],\n",
       "        [ 1.      ,  0.83188 ,  0.66817 ,  0.23403 ,  0.72472 ]]),\n",
       " array([ 1.,  1.,  1.,  1.,  1.,  1., -1.,  1., -1., -1.,  1.,  1.,  1.,\n",
       "        -1., -1.,  1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "        -1.,  1.,  1., -1., -1.,  1.,  1., -1.,  1.,  1., -1., -1.,  1.,\n",
       "        -1., -1.,  1., -1.,  1.,  1.,  1., -1., -1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1., -1., -1.,  1., -1.,  1., -1., -1.,  1.,\n",
       "        -1.,  1., -1., -1.,  1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1., -1.,  1.,  1.,  1., -1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1., -1.,  1., -1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,\n",
       "        -1.,  1.,  1.,  1.,  1., -1.,  1., -1.,  1.,  1., -1.,  1.,  1.,\n",
       "         1.,  1., -1.,  1., -1., -1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1., -1., -1.,  1.,  1., -1.,  1., -1.,  1.,  1.,  1., -1.,  1.,\n",
       "        -1., -1.,  1., -1., -1.,  1.,  1.,  1.,  1., -1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1., -1., -1., -1.,  1., -1.,  1., -1.,  1.,\n",
       "        -1.,  1.,  1., -1., -1.,  1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1., -1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,  1., -1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1., -1., -1., -1., -1.,  1., -1.,  1.,  1.,\n",
       "        -1.,  1., -1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,  1., -1.,  1.,\n",
       "        -1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1., -1.,\n",
       "         1., -1.,  1.,  1., -1.,  1.,  1.,  1.,  1., -1.,  1., -1.,  1.,\n",
       "         1.,  1.,  1.,  1., -1.,  1., -1.,  1.,  1.,  1., -1., -1.,  1.,\n",
       "         1.,  1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1., -1.,  1.,  1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1.,  1.,  1., -1.,  1., -1.,  1.,  1.,  1.,\n",
       "        -1., -1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1., -1.,\n",
       "         1.,  1.,  1., -1.,  1.,  1., -1., -1., -1.,  1.,  1., -1., -1.,\n",
       "         1., -1., -1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1., -1.,\n",
       "         1.,  1.,  1.,  1., -1.,  1.,  1., -1., -1.,  1., -1.,  1.,  1.,\n",
       "        -1.,  1.,  1.,  1.,  1.,  1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1., -1.,  1.,  1.,  1., -1.,  1., -1.,  1.,  1.,  1., -1.,\n",
       "         1.,  1.,  1., -1., -1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.]))"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pla_bc.load_train_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "這樣就載入了 PLA 的 Demo 訓練資料，不信的話大家可以使用 `pla_bc.train_X` 及 `pla_bc.train_Y` 印出來看看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.        0.97681   0.10723   0.64385   0.29556 ]\n",
      " [ 1.        0.67194   0.2418    0.83075   0.42741 ]\n",
      " [ 1.        0.20619   0.23321   0.81004   0.98691 ]\n",
      " ..., \n",
      " [ 1.        0.50468   0.99699   0.75136   0.51681 ]\n",
      " [ 1.        0.55852   0.067689  0.666     0.98482 ]\n",
      " [ 1.        0.83188   0.66817   0.23403   0.72472 ]]\n"
     ]
    }
   ],
   "source": [
    "print(pla_bc.train_X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "訓練資料的特徵資料就存在 `train_X` 中，矩陣的每一個列就代表一筆資料，然後每一個行就代表一個特徵值，請注意矩陣的第一行都是 1，這是我們演算法自己補上的 $x_0$，並不是原本訓練資料就會有的特徵值，以這個 Demo 資料來說，每筆資料只有 4 個特徵值（feature），像第一筆資料的 4 個特徵值就是 0.97681 0.10723 0.64385 0.29556，演算法將前面補上 $x_0 = 1$，就變成了現在看到的樣子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.  1.  1.  1.  1.  1. -1.  1. -1. -1.  1.  1.  1. -1. -1.  1.  1.  1.\n",
      " -1.  1.  1.  1.  1.  1.  1.  1. -1.  1.  1. -1. -1.  1.  1. -1.  1.  1.\n",
      " -1. -1.  1. -1. -1.  1. -1.  1.  1.  1. -1. -1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1. -1. -1.  1. -1.  1. -1. -1.  1. -1.  1. -1. -1.  1.  1.  1.\n",
      " -1.  1.  1.  1.  1.  1.  1. -1.  1.  1.  1. -1.  1.  1. -1.  1.  1.  1.\n",
      "  1.  1.  1.  1. -1.  1. -1.  1.  1. -1.  1.  1.  1.  1. -1.  1.  1.  1.\n",
      "  1. -1.  1. -1.  1.  1. -1.  1.  1.  1.  1. -1.  1. -1. -1. -1.  1.  1.\n",
      "  1.  1.  1.  1.  1. -1. -1.  1.  1. -1.  1. -1.  1.  1.  1. -1.  1. -1.\n",
      " -1.  1. -1. -1.  1.  1.  1.  1. -1.  1.  1.  1.  1.  1.  1.  1.  1. -1.\n",
      " -1. -1.  1. -1.  1. -1.  1. -1.  1.  1. -1. -1.  1. -1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1. -1.  1.  1. -1.  1.  1.  1.  1.  1. -1.  1.  1.  1.  1.\n",
      "  1.  1. -1. -1. -1. -1.  1. -1.  1.  1. -1.  1. -1. -1.  1.  1.  1.  1.\n",
      "  1.  1.  1. -1.  1. -1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  1. -1.\n",
      "  1. -1.  1.  1. -1.  1.  1.  1.  1. -1.  1. -1.  1.  1.  1.  1.  1. -1.\n",
      "  1. -1.  1.  1.  1. -1. -1.  1.  1.  1.  1.  1. -1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1. -1.  1.  1.  1.  1. -1.  1.  1.  1.  1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1. -1.  1. -1.  1.  1.  1. -1. -1. -1.  1.  1.  1.  1.\n",
      "  1.  1.  1.  1.  1. -1.  1.  1.  1. -1.  1.  1. -1. -1. -1.  1.  1. -1.\n",
      " -1.  1. -1. -1. -1.  1.  1.  1.  1.  1.  1.  1.  1. -1.  1.  1.  1.  1.\n",
      " -1.  1.  1. -1. -1.  1. -1.  1.  1. -1.  1.  1.  1.  1.  1. -1.  1.  1.\n",
      "  1.  1.  1.  1.  1.  1. -1.  1.  1.  1. -1.  1. -1.  1.  1.  1. -1.  1.\n",
      "  1.  1. -1. -1.  1.  1.  1.  1.  1.  1.  1.  1.]\n"
     ]
    }
   ],
   "source": [
    "print(pla_bc.train_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然後訓練資料的答案就存在 `train_Y` 中，也就代表每筆訓練資料的答案是什麼，正分類就是 1，負分類就是 -1。\n",
    "\n",
    "接下來讓我們進行下一步，設定參數："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('naive_cycle', 1)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pla_bc.set_param(loop_mode='naive_cycle', step_alpha=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PLA 這個演算法我只提供兩個參數可以調，一個是 `loop_mode`，用來調整 PLA 選擇訓練資料來檢查自己猜錯或猜對的選法，預設是使用 `naive_cycle` ，會照著訓練資料的順序一個一個檢測，有錯就修正 w。你也可以設成使用 `random`，這樣 PLA 檢測時就會隨便選擇一個點來檢測，有錯就修正 w。\n",
    "\n",
    "另一個參數是 `step_alpha`，用來調整 PLA 每次有錯就修正 w 要修正多少量，原則上設成 1 就可以了。\n",
    "\n",
    "接下來就可以再進行下一步，初始化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.,  0.,  0.,  0.,  0.])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pla_bc.init_W()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "初始化時，我們可以得到一個最初的權重值 w，通常就是個 0 向量了，但有時我們可以用 Linear Regression 來初始化，加速演算法，之後我們會再介紹，一樣我們將初始化的 w 印出來看看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.  0.  0.  0.  0.]\n"
     ]
    }
   ],
   "source": [
    "print(pla_bc.W)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "好！果然是 0 向量，一切準備就緒，接下來就是重頭戲「訓練」了："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-3.       ,  3.0841436, -1.583081 ,  2.391305 ,  4.5287635])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pla_bc.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "登登登！訓練完成，我們會得到一個全新的權重值 w，根據 PLA 的運算，這個 w 可以將資料完全分類正確！這就是機器學習神奇的地方！\n",
    "\n",
    "我們一樣把 PLA 計算出來的 w 印出來看看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-3.         3.0841436 -1.583081   2.391305   4.5287635]\n"
     ]
    }
   ],
   "source": [
    "print(pla_bc.W)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "果然不是個 0 向量了呢！\n",
    "\n",
    "有了這個 w，我們就可以用它來預測未來的資料，讓我拿一筆測試資料 0.97959 0.40402 0.96303 0.28133 1 來預測看看，前面 4 個值是這筆測試資料的特徵值，後面的 1 代表這筆測試資料的答案，我們來看看預測結果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test_data = '0.97959 0.40402 0.96303 0.28133 1'\n",
    "prediction = pla_bc.prediction(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "將預測結果印出來看看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'prediction': 1.0, 'input_data_x': array([ 1.     ,  0.97959,  0.40402,  0.96303,  0.28133]), 'input_data_y': 1.0}\n"
     ]
    }
   ],
   "source": [
    "print(prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "prediction 這個方法會把預測結果回傳成一個 dictionary，預測結果的 key 是 prediction，value 是 1，測試資料的答案也是 1，所以 PLA 正確預測了結果！\n",
    "\n",
    "假設我們現在要預測的是未知的資料、一些我們還沒有分好類的資料，那我們就是把資料特徵值向量丟進去 prediction 方法，並設定 ｀`mode='future_data'`，代表是做未知資料的預測，就可以進行預測了，比如丟進去 0.29634 0.4012 0.40266 0.67864 這筆特徵資料試試看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "future_data = '0.29634 0.4012 0.40266 0.67864'\n",
    "prediction = pla_bc.prediction(future_data, mode='future_data')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "將預測結果印出來看看："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'prediction': 1.0, 'input_data_x': array([ 1.     ,  0.29634,  0.4012 ,  0.40266,  0.67864]), 'input_data_y': None}\n"
     ]
    }
   ],
   "source": [
    "print(prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PLA 會忠實的觀察資料給出答案，它認為這筆資料的答案也是 1。（事實上真的是）\n",
    "\n",
    "當然，如果只是看一、兩筆資料猜對，大家可能會認為這只是運氣好，所以我們必須計算 PLA 在整個訓練資料集及整個測試資料集的預測表現如何。我們提供了很簡易的方法可以計算整體的錯誤率，如果要看 PLA 在整個訓練資料集的預測錯誤率（$E_{in}$）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n"
     ]
    }
   ],
   "source": [
    "print(pla_bc.calculate_avg_error(pla_bc.train_X, pla_bc.train_Y, pla_bc.W))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PLA 在訓練資料的預測錯誤率是完美的 0！這是當然的，因為 PLA 在線性可分的資料裡，一定會調整到沒有錯誤為止。\n",
    "\n",
    "現在我們來看看 PLA 在整個測試資料集的預測錯誤率（$E_{out}$），在此之前，我們必須先載入測試資料集，一樣 FukuML 有提供 Demo 版本的測試資料集:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[ 1.      ,  0.97959 ,  0.40402 ,  0.96303 ,  0.28133 ],\n",
       "        [ 1.      ,  0.29634 ,  0.4012  ,  0.40266 ,  0.67864 ],\n",
       "        [ 1.      ,  0.34922 ,  0.99751 ,  0.23234 ,  0.52115 ],\n",
       "        [ 1.      ,  0.65637 ,  0.7181  ,  0.72843 ,  0.93113 ],\n",
       "        [ 1.      ,  0.079695,  0.57218 ,  0.70591 ,  0.33812 ],\n",
       "        [ 1.      ,  0.71206 ,  0.51569 ,  0.18168 ,  0.5557  ],\n",
       "        [ 1.      ,  0.17528 ,  0.2625  ,  0.8306  ,  0.029669],\n",
       "        [ 1.      ,  0.93895 ,  0.93941 ,  0.72496 ,  0.95655 ],\n",
       "        [ 1.      ,  0.046136,  0.94413 ,  0.038311,  0.26812 ],\n",
       "        [ 1.      ,  0.072491,  0.2242  ,  0.62592 ,  0.67238 ]]),\n",
       " array([ 1.,  1., -1.,  1., -1.,  1., -1.,  1., -1.,  1.]))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pla_bc.load_test_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "載入測試資料之後，我們就可以計算 PLA 在測試資料集的預測錯誤率（$E_{out}$）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n"
     ]
    }
   ],
   "source": [
    "print(pla_bc.calculate_test_data_avg_error())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PLA 在測試資料的預測錯誤率也是完美的 0，當然這某種程度是因為我們的 Demo 資料有設計過，不過理論上測試資料的預測錯誤率應該不會和訓練資料的預測錯誤率差太多，只要實驗過程是一個客觀的過程、沒有經過人為的污染，機器學習的演算法的確可以做到正確的預測。\n",
    "\n",
    "以上，你大概已經學會使用 FukuML 提供的 PLA 做訓練，然後使用訓練完成的 w 來進行未知資料的預測了，真的五個步驟就可以做完了！很簡單吧！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用自己的訓練資料集和測試資料集\n",
    "\n",
    "前面的教學我們是使用 FukuML 所提供的訓練資料集和測試資料集，真實情況你當然使用自己的資料，那要怎麼做呢？FukuML 提供了很簡易的方法可以讓大家載入自己的資料：\n",
    "\n",
    "```\n",
    "your_training_data_file = '/path/to/your/training_data/file'\n",
    "pla_bc.load_train_data(your_training_data_file)\n",
    "\n",
    "your_testing_data_file = '/path/to/your/testing_data/file'\n",
    "pla_bc.load_test_data(your_testing_data_file)\n",
    "```\n",
    "\n",
    "就是這麼簡單，讓我們來實際演示一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[ 1.        , -0.49475104,  1.60851023],\n",
       "        [ 1.        ,  0.99350955,  2.53942025],\n",
       "        [ 1.        ,  0.67365802,  2.41859411],\n",
       "        [ 1.        , -1.91676615,  0.48923093],\n",
       "        [ 1.        , -0.80964166,  1.26206511],\n",
       "        [ 1.        , -0.45285374,  1.82885284],\n",
       "        [ 1.        ,  0.27463815,  2.08049683],\n",
       "        [ 1.        ,  0.89694355,  3.7834262 ],\n",
       "        [ 1.        , -1.72520564,  0.87640485],\n",
       "        [ 1.        ,  0.7349451 ,  3.39882197],\n",
       "        [ 1.        , -1.02461018,  1.44258081],\n",
       "        [ 1.        , -0.60392455,  0.98807458],\n",
       "        [ 1.        ,  0.08098387,  2.15878467],\n",
       "        [ 1.        ,  0.48213089,  2.18476304],\n",
       "        [ 1.        ,  0.74123261,  3.22706092],\n",
       "        [ 1.        , -0.57649605,  0.27757466],\n",
       "        [ 1.        , -1.60301663,  0.85311484],\n",
       "        [ 1.        , -1.90040634,  1.14021401],\n",
       "        [ 1.        ,  0.7943513 ,  2.68559323],\n",
       "        [ 1.        ,  0.15398661,  2.61447653],\n",
       "        [ 1.        ,  2.4192871 ,  4.18943591],\n",
       "        [ 1.        ,  0.13016586,  2.53128795],\n",
       "        [ 1.        , -1.00057111,  1.2998211 ],\n",
       "        [ 1.        , -2.24935866, -0.51829791],\n",
       "        [ 1.        , -0.11745011,  2.36365622],\n",
       "        [ 1.        , -0.18131864,  1.90732415],\n",
       "        [ 1.        , -1.0669876 ,  1.84490598],\n",
       "        [ 1.        , -0.41819858,  1.20384123],\n",
       "        [ 1.        , -1.27557363,  1.58879675],\n",
       "        [ 1.        , -0.48455613,  1.56688674],\n",
       "        [ 1.        ,  1.76857878,  2.70393626],\n",
       "        [ 1.        ,  0.6178306 ,  1.41965757],\n",
       "        [ 1.        ,  0.24021005,  2.07796794],\n",
       "        [ 1.        , -0.40745049,  1.12846498],\n",
       "        [ 1.        , -0.5450063 ,  1.64924578],\n",
       "        [ 1.        , -0.89149772,  1.29851015],\n",
       "        [ 1.        ,  1.12855231,  1.96797717],\n",
       "        [ 1.        , -0.62563244,  1.87988573],\n",
       "        [ 1.        , -0.91508504,  1.62532636],\n",
       "        [ 1.        , -1.21008395,  0.41751392],\n",
       "        [ 1.        ,  1.0369232 ,  3.32224131],\n",
       "        [ 1.        ,  0.82678315,  3.36840655],\n",
       "        [ 1.        ,  0.00522133,  2.57820823],\n",
       "        [ 1.        , -1.06147755,  1.06473163],\n",
       "        [ 1.        , -0.45700467,  2.00276916],\n",
       "        [ 1.        ,  0.13487671,  0.65962212],\n",
       "        [ 1.        ,  1.22494293,  2.4905672 ],\n",
       "        [ 1.        ,  0.82587401,  1.6469229 ],\n",
       "        [ 1.        , -0.46393125,  2.81795857],\n",
       "        [ 1.        , -2.36851079,  1.01187775],\n",
       "        [ 1.        ,  0.12587105,  2.55995705],\n",
       "        [ 1.        , -0.35712397,  1.88322814],\n",
       "        [ 1.        ,  0.68857731,  2.45378334],\n",
       "        [ 1.        , -1.11846239,  1.7060288 ],\n",
       "        [ 1.        , -1.73549484,  1.16778056],\n",
       "        [ 1.        ,  0.18491969,  1.6888773 ],\n",
       "        [ 1.        ,  1.10350087,  2.55392247],\n",
       "        [ 1.        , -0.44246031,  1.49684599],\n",
       "        [ 1.        , -0.22148107,  2.66175094],\n",
       "        [ 1.        ,  0.30829778,  2.25791677],\n",
       "        [ 1.        , -0.29287034,  2.04485062],\n",
       "        [ 1.        , -0.44357665,  1.58064718],\n",
       "        [ 1.        ,  0.01694366,  1.60172119],\n",
       "        [ 1.        , -0.35169509,  2.20195385],\n",
       "        [ 1.        ,  0.55527319,  1.84184212],\n",
       "        [ 1.        , -0.59067181,  1.19101348],\n",
       "        [ 1.        ,  0.21534601,  2.95975298],\n",
       "        [ 1.        ,  0.79769729,  2.79259136],\n",
       "        [ 1.        ,  0.41191044,  1.86899517],\n",
       "        [ 1.        , -1.39417234,  1.15327164],\n",
       "        [ 1.        ,  0.71641377,  2.87832566],\n",
       "        [ 1.        ,  0.44264983,  3.19840287],\n",
       "        [ 1.        , -1.11935978,  1.0214965 ],\n",
       "        [ 1.        ,  0.25788802,  3.2897688 ],\n",
       "        [ 1.        , -0.33296609,  0.88930833],\n",
       "        [ 1.        ,  2.07653897,  3.77544829],\n",
       "        [ 1.        , -0.35754265,  0.91029036],\n",
       "        [ 1.        ,  0.38998221,  2.7169355 ],\n",
       "        [ 1.        ,  0.88980695,  2.23294531],\n",
       "        [ 1.        , -0.30374769,  2.16560662],\n",
       "        [ 1.        ,  0.46858362,  2.22595082],\n",
       "        [ 1.        ,  1.5953943 ,  3.86456874],\n",
       "        [ 1.        ,  0.80516593,  1.14755445],\n",
       "        [ 1.        , -1.07078848,  1.07630871],\n",
       "        [ 1.        , -0.61184666,  1.08231727],\n",
       "        [ 1.        , -1.22082978,  1.441157  ],\n",
       "        [ 1.        ,  0.42133054,  2.00527312],\n",
       "        [ 1.        , -1.15371133,  0.39545553],\n",
       "        [ 1.        , -1.16529981,  0.55726593],\n",
       "        [ 1.        , -0.0753288 ,  2.65117295],\n",
       "        [ 1.        ,  1.6801046 , -0.08598257],\n",
       "        [ 1.        ,  1.8497788 , -0.01692729],\n",
       "        [ 1.        ,  1.85056814,  0.08177583],\n",
       "        [ 1.        ,  2.80034537,  0.81539772],\n",
       "        [ 1.        ,  0.40956132,  0.55189673],\n",
       "        [ 1.        ,  1.31484126, -0.82314199],\n",
       "        [ 1.        ,  0.74578981, -0.63417363],\n",
       "        [ 1.        ,  1.52094099,  0.07809939],\n",
       "        [ 1.        ,  2.07568652, -0.1150392 ],\n",
       "        [ 1.        ,  0.99162865, -0.53539644],\n",
       "        [ 1.        ,  0.7038055 , -0.19221186],\n",
       "        [ 1.        ,  2.06231887, -0.03769865],\n",
       "        [ 1.        ,  2.5799191 ,  0.07069567],\n",
       "        [ 1.        ,  0.87544838, -0.5637692 ],\n",
       "        [ 1.        ,  2.11936394, -0.12933603],\n",
       "        [ 1.        ,  1.72361776, -0.83946543],\n",
       "        [ 1.        ,  1.00174566, -0.38122767],\n",
       "        [ 1.        ,  2.67256573,  1.39825995],\n",
       "        [ 1.        ,  1.95670692, -0.76033255],\n",
       "        [ 1.        , -0.12377975, -1.66555773],\n",
       "        [ 1.        ,  3.49754388,  0.86099572],\n",
       "        [ 1.        ,  2.30620253,  1.41859895],\n",
       "        [ 1.        ,  2.55608362,  0.42400762],\n",
       "        [ 1.        ,  2.48991027,  0.76553132],\n",
       "        [ 1.        ,  2.55940671, -0.57984346],\n",
       "        [ 1.        ,  2.32904612, -0.51945896],\n",
       "        [ 1.        ,  1.7353993 , -0.75519976],\n",
       "        [ 1.        ,  2.51829003,  0.37786517],\n",
       "        [ 1.        ,  1.8706277 , -0.93869733],\n",
       "        [ 1.        ,  0.22236542, -2.44483319],\n",
       "        [ 1.        , -0.09038213, -1.79941358],\n",
       "        [ 1.        ,  2.70225343,  0.94731516],\n",
       "        [ 1.        ,  1.88566698,  0.2798723 ],\n",
       "        [ 1.        ,  1.58910203, -0.70947294],\n",
       "        [ 1.        ,  3.0973127 ,  0.92156856],\n",
       "        [ 1.        ,  2.69809437,  0.38175539],\n",
       "        [ 1.        ,  3.3289721 ,  0.41484516],\n",
       "        [ 1.        ,  1.87232143, -0.61040703],\n",
       "        [ 1.        ,  2.33296818,  0.02254511],\n",
       "        [ 1.        ,  1.81407758, -0.17053957],\n",
       "        [ 1.        ,  2.96737955,  1.44181063],\n",
       "        [ 1.        ,  2.9380551 ,  0.47943516],\n",
       "        [ 1.        ,  0.73973305, -1.28050721],\n",
       "        [ 1.        ,  2.08422916, -1.39634791],\n",
       "        [ 1.        ,  1.66061566, -0.98458495],\n",
       "        [ 1.        ,  1.98635728, -0.28509211],\n",
       "        [ 1.        ,  2.56435931, -0.47953988],\n",
       "        [ 1.        ,  2.20241294,  0.39511807],\n",
       "        [ 1.        ,  3.87224268,  0.76225007],\n",
       "        [ 1.        ,  1.64152869,  0.42732398],\n",
       "        [ 1.        ,  2.01228488,  0.83947942],\n",
       "        [ 1.        ,  1.12248906, -0.86437473],\n",
       "        [ 1.        ,  0.92242964,  0.68317263],\n",
       "        [ 1.        ,  1.60673796,  0.18415559],\n",
       "        [ 1.        ,  1.50243849,  0.01270292],\n",
       "        [ 1.        ,  2.85000032,  0.26811154],\n",
       "        [ 1.        ,  1.11113213, -1.08291552],\n",
       "        [ 1.        ,  1.40913806, -0.73020544],\n",
       "        [ 1.        ,  1.73676161,  0.22610285],\n",
       "        [ 1.        ,  2.55634878,  0.73043033],\n",
       "        [ 1.        ,  1.68015105, -0.51196788],\n",
       "        [ 1.        ,  2.07339426, -0.69056912],\n",
       "        [ 1.        ,  2.51559063,  0.28962156],\n",
       "        [ 1.        ,  1.91898721, -0.38112982],\n",
       "        [ 1.        ,  2.84276804,  1.00034359],\n",
       "        [ 1.        ,  2.48044735,  0.97198591],\n",
       "        [ 1.        ,  1.00833589, -1.14853349],\n",
       "        [ 1.        ,  1.33359442, -0.28894411],\n",
       "        [ 1.        ,  1.77539826, -1.30014904],\n",
       "        [ 1.        ,  0.67101677, -1.91851934],\n",
       "        [ 1.        ,  3.17326187,  1.70334529],\n",
       "        [ 1.        ,  2.08009842, -0.74647001],\n",
       "        [ 1.        ,  2.29719679, -0.16319915],\n",
       "        [ 1.        ,  1.83769012, -0.19705278],\n",
       "        [ 1.        ,  1.80368554, -0.12629935],\n",
       "        [ 1.        ,  1.56859964, -0.2302815 ],\n",
       "        [ 1.        ,  1.47931541, -0.54655241],\n",
       "        [ 1.        ,  1.9768354 , -0.26445185],\n",
       "        [ 1.        ,  2.50410495,  1.52574828],\n",
       "        [ 1.        ,  2.35432514, -0.47573114],\n",
       "        [ 1.        ,  1.07167578,  0.15158119],\n",
       "        [ 1.        ,  1.11479351, -1.433885  ],\n",
       "        [ 1.        ,  1.12664849,  0.11512314],\n",
       "        [ 1.        ,  2.08937432, -0.48560459],\n",
       "        [ 1.        ,  0.9236267 , -0.89739827],\n",
       "        [ 1.        ,  2.22914995,  0.08347266],\n",
       "        [ 1.        ,  1.87811234, -0.22916618],\n",
       "        [ 1.        ,  3.32948946,  0.77274808],\n",
       "        [ 1.        ,  2.60176922,  0.08437287],\n",
       "        [ 1.        ,  1.98635292,  0.81549389]]),\n",
       " array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,\n",
       "         1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1., -1., -1., -1., -1.]))"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pla_bc = pla.BinaryClassifier()\n",
    "pla_bc.load_train_data('/Users/fukuball/Projects/fuku-ml/FukuML/dataset/linear_separable_train.dat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[  1.00000000e+00,   3.12179041e-01,   3.26300582e+00],\n",
       "        [  1.00000000e+00,  -7.88545922e-01,   1.84177454e+00],\n",
       "        [  1.00000000e+00,   3.45018856e-01,   2.02971487e+00],\n",
       "        [  1.00000000e+00,  -5.90936663e-02,   2.06095580e+00],\n",
       "        [  1.00000000e+00,  -6.04229133e-01,   1.89545186e+00],\n",
       "        [  1.00000000e+00,   2.92639463e-01,   2.21847534e+00],\n",
       "        [  1.00000000e+00,   1.37291076e+00,   3.10397301e+00],\n",
       "        [  1.00000000e+00,  -8.55850926e-01,   7.43968659e-01],\n",
       "        [  1.00000000e+00,  -2.33116362e-04,   1.45262917e+00],\n",
       "        [  1.00000000e+00,   5.63747692e-01,   2.65759454e+00],\n",
       "        [  1.00000000e+00,   3.27474170e+00,   5.16394190e-01],\n",
       "        [  1.00000000e+00,   1.00982446e+00,  -9.92472127e-01],\n",
       "        [  1.00000000e+00,   1.63602318e+00,  -7.66844250e-01],\n",
       "        [  1.00000000e+00,   2.81507689e+00,   2.63441093e-01],\n",
       "        [  1.00000000e+00,   1.83736479e+00,  -8.03493918e-01],\n",
       "        [  1.00000000e+00,   2.26025418e+00,   4.02606276e-01],\n",
       "        [  1.00000000e+00,   2.18689341e+00,   7.86296427e-01],\n",
       "        [  1.00000000e+00,   1.34179804e+00,   1.26613719e-04],\n",
       "        [  1.00000000e+00,   2.75190511e+00,  -3.67967235e-01],\n",
       "        [  1.00000000e+00,   2.92122264e+00,   1.35934066e-01]]),\n",
       " array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1., -1., -1., -1.,\n",
       "        -1., -1., -1., -1., -1., -1., -1.]))"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pla_bc.load_test_data('/Users/fukuball/Projects/fuku-ml/FukuML/dataset/linear_separable_test.dat')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看吧，都順利載入資料了，接下來的問題只剩下資料集的格式是怎麼樣，這個可以直接看 FukuML 提供的資料集一窺究竟：\n",
    "\n",
    "https://github.com/fukuball/fuku-ml/blob/master/FukuML/dataset/pla_binary_train.dat\n",
    "\n",
    "其實格式真的很簡單，就是將每筆資料的特徵值用空格隔開，然後放成一橫行，然後將這筆資料的答案用空格隔開放在最後，答案是正分類就是 1，負分類就是 -1，這樣就完成了。\n",
    "\n",
    "所以比如你想做銀行核卡預測，然後審核的特徵是年薪、年齡、性別，那假設小明年薪 100W、年齡 30、性別男性且通過核卡了，那這筆資料就是：\n",
    "\n",
    "100 30 1 1\n",
    "\n",
    "假設小華年薪 20W、年齡 25、性別男性，沒有通過核卡，這筆資料就是：\n",
    "\n",
    "20 25 1 -1\n",
    "\n",
    "假設小美年薪 30W、年齡 24、性別女性，有過核卡，這筆資料就是：\n",
    "\n",
    "30 24 0 1\n",
    "\n",
    "以此類推，簡簡單單、輕輕鬆鬆，大家就可以使用自己的資料來玩玩看機器學習囉～"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用二維資料來幫助理解\n",
    "\n",
    "其實 PLA 分類演算法計算出來的 w 就是去找出一條可以將資料點完美分開的線，書本上的範例可能會使用二維的資料集並畫成圖示呈現給大家看，但在真實世界中，我們的資料通常不會只是二維的，這樣找出來的 w 就會是一個在高維空間將資料完美分類的超平面，我們很難在平面上呈現這樣的結果，因此還是請大家多去從抽象化的高維空間去思考機器學習的過程，不要遷就於圖示。不過如果你剛接觸機器學習，使用二維資料來慢慢理解機器學習演算法也是一個不錯的學習方法，我這邊稍微展示一下如何印出二維資料點及機器學習訓練出來的 w。\n",
    "\n",
    "載入資料點時，我們就可以在平面上印出所有的資料點，正分類印成紅色的，負分類印成藍色的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD7CAYAAAChScXIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnX9wHGeZ579vrMQa1nbiEsnJscNmM3tbmKQAqzBl6rY0\ns9my2od2C6iroOwGkixx4cReSbcchIt+RAO2d2+dOmolXX7cXbJswMuSYqFyh2ZPGhVhZkxdCLlb\nBeLEuYWBPbBjxcHn2A5IDjLv/TEzUs9M90z/ePvX9PdT1WVNT0/327L97We+7/M8r5BSghBCSDS4\nIugBEEIIsQ5FmxBCIgRFmxBCIgRFmxBCIgRFmxBCIgRFmxBCIkSH1xcQQjCnkBBCHCClFPX7fIm0\npZTKtomJCaXnC8PWjvfE+4rWpvqeRvv6IIGGbUzTIn1ffm5mKBFtIcQ6IcSCEOKbKs5HCIk2fUND\nGE0ma/aNJJPYPTgY0IjaB1X2yDCAlwFsVHQ+QkiE6e3vBwCMT09j3fIyLnd2Ys/g4Op+4hzXoi2E\n2AbggwAOA/iU6xG1IJ1Oe30J32nHewJ4X1HCi3vq7e8PXKTb8e9KNPNOLJ1AiK8B+HMAmwB8Wkr5\nh3XvS7fXIISQuCGEgDSYiHQVaQsh/gDAGSnlghAibXZcJpNZ/TmdTrfl048QQtyQz+eRz+dbHucq\n0hZC/DmAjwNYAdCJcrT9dSnlnbpjGGkTQohNzCJt1/aI7gIp0B4hxDOK2SxyU1PouHQJK+vXo29o\nKHDPmHiHJ/aIAVRnQjygmM1ibngYh0ul1X2jlZ8p3PFCWaRtegFG2iRihDGiHdM0HMrlGvaPaxoO\nzs4GMCLiNX5F2oREmrBGtB2XLhnuX7e8rPQ6YXxgkVoo2oToyE1N1Qg2ABwulTA+PR2oeK2sX2+4\n/3Jnp7JrhPWBRWphlz9CdPgV0drFj7JwswfW/PS0smsQ9zDSJkSHHxGtE/woCw/rA4vUQtEmREff\n0BBGS6WaiHMkmcSeEDQ68rosPKwPLFILRZsQHXFudBTmBxZZgyl/hJBVitks5nUPrN0xeWCFEc8r\nIptcmKJNCCE2MRNtZo8QQkiEoGgTQkiEoGgTQkiEoGgTQkiEYMofiRXsrUGiDkWbxAaz3hrHn38e\nrz77LIWcRAKKNokNRr01tFIJXzlyBI8tLa3uY5MkEmboaZPYYNRbIwfUCDbQuklSMZvFmKYhk05j\nTNNQzGZVD5UQUxhpk9Dgtd9s1FvD7D+AWZMkti8lQUPRJqHADzE06q1xIpEA6iJtwLxJUlj7bZP4\nQHuEhAI/ejn39vdDm5zE3h07cPvmzbhr82ZgyxZ8qru75rhmfarZvpQEDSNtEgrsiKFbG+VfXLiA\nx8+dK784dw73dHfjQE8Prt24sWVXP7YvJUFD0SahwKoYurVRjCL6JxYXMf6e9yBjYYFcti8lQUPR\nJqHAqhi69ZTd2htx7rdNwgFFm4QCq2LoVnRV2BteryBDSDMo2iQ0WBFDt6JLe4NEHVeiLYToBFAA\nsB7AVQD+m5TyARUDI8QIt6JLe4NEHdcr1wgh3ial/KUQogPAdwB8Wkr5Hd37XLmGKIVLYpE44Ply\nY0KIt6Ecdd8lpXxZt5+iTQghNjETbdeethDiCgD/CCAJ4FG9YBNiBbZLJcQ6rkVbSvlrAO8VQlwN\nYE4IkZZS5l2PjMQC9vIgxB7KskeklOeFEFkA7wOQ17+XyWRWf06n00in06ouSyIOe3kQUiafzyOf\nz7c8zpWnLYR4O4AVKeUbQogEgDkAn5NSfkt3DD1tYkomnUamUGjcn0ohY+EfsB1ow5Ao4ZWnvQXA\nkxVf+woAX9YLNiGt8KuXB20Y0i646vInpXxRStkjpXyvlPLdUsqHVA2MeEPYGvj3DQ1hNJms2des\ny55T/OgiSIgfsCIyRoQx2vSr2MWs/P3iqVMY0zRaJiQyULRjRFgn/fzo5WFmw5wulfDU8eOrr4N+\niBHSCi6CECOi3MDfra1jZMPsSyRwwOb6kIQEDSPtGBHVBv4qbB0jGwYnT6L3pZcajo3CQ8wMZsi0\nPxTtGOG02VIrIfBaKFTZOvU2zP6eHsPjXr940dlAAyaMcxZEPRTtGOFk0q+VEPghFF7ZOm8BGAVw\nWLdvBMCliNYVhHXOgqiFoh0z7E76tRICJ0JhNzL3ytbZtmkTbgUwDmAdgMsA9gB4ZtMmV+cNiijP\nWRDrULRJU1oJgdn7P33uOWTS6QZRfiSTQeHIEWxfWsIKgD4Acy0ic68WLlhZvx69AHrr9s+H3OM3\nI6pzFsQeFG3SlFZCYPb+O954Y7U8fVQntgt/8Rd46q23Vl9/CsCHKxkbZqLtVS63nYeB2beDME38\ncVWeeEDRJk1pJQSG76NsM1Sp2iVnz5zBf9UJNgB8AcABANe2+ArvRS631YeBmW9//Pnncero0dBM\n/HFVnnigbBEE0wuwYVTkabVSjP79//ODH+C+c+caLIdMKoWffP/7ePKNNxrOfxeAd2gaDs7Oensj\nDhnTNBzK5Rr2D3R14amzZxv2j1fuJUxROIkeni2CQNqfVlGu/v0xTUOvgcBd7uzEJdHw7w8A8IYQ\nuMfjr/BuBNTMt0+srBjuX7e8zPQ74hmsiCRKadYAasONN2K07vgRAInf+i1PheyRTAZfue02HMrl\nkCkUcCiXw9zwsOWqSjPffqnDOOa53NnJBlXEMxhpE6W08lWf3LsX44uLqyl2i93d2D815dl4itks\nCkeO4CmDcnWr+ctmvn7qYx/DaJ2nXfX7n3nIuOEl0++IWyjaxDFmloOZndLb3w88/ng52lxeBjo7\ncbfHE2W5qSlsrxPsKlYFtNmDqLhzp+H+nMmDiOl3xC0UbeIIp56tHx399HRcugRj59megDZ7EBnt\nZ/od8QqKNnFEVEqmV9avRx8ay9X3JRK4w0MBZfod8QqKNnFEVEqm+4aGMFcqQSuVVsvVTyQSSN1/\nv+cC6ve3ChIPKNrEEV6XTKvKca5+Rp9nfsBCkyzmV5OwQtEmAOwLlZeerRW/3M547US8zK8moUdK\n6elWvgQJM4WZGTmSTEoJrG4jyaQszMy0/NyYpsmJVEqOaVrL460y2tdXM5bqNqZppuPdl0jIhycm\nPL82IX5R0c4GTWWkTRomFYsARKmEJz7+ceR27jSNYq1GsHaj+FZ+udEk6GNLSxg4cgS37NzpKiKO\nildP4gtFm9QIVRHAHCqZFufOAbmcK3vAid3Qyi83E9btS0tNuwVage1NSdhhGTupEaocalPjAHfl\n107KuZuVwtePV89luI+IW12bkKBhpE1qJhXN/kE4FUOrdkPVQnnz1Vdx+vRp/Po3fgMDXV3YsmUL\nNm7dWpPj3Dc0hHuPHcNjukrHajtYtwsYML+ahB1Xoi2EuAHAlwBcB0AC+C9SSu8aSRBP0AvVD7/3\nvbItUodTe8CK3WBooZw9Cw3A3DXXNLSC7e3vx/H778dAZQWc6jJhs4qyV5hfTcKMW3vkVwD+TEp5\nM4BdAA4IIba7HxbximI2izFNQyadxpimrXa66+3vx8HZWfQODeHeuu51+zo6sGXXLkfXs2I3GFoo\nAObRaKVUx38mn0fXO9+Jkzt2YF0qhXlNw57JSYotaXtcRdpSykUAi5Wf3xRCnABwPYATCsZGFGNl\nUvDVZ5/FH6+s1Cx2e8fKCua/+11H17RiN5haKNU/K1aK4fiTSdx68GDgYs2CHOIXyjxtIcSNAHYA\neE7VOYlarPQL6bh0yXCx22dcTPC1shuaTSwCa1ZKWPudsCCH+IkS0RZCbADw9wCGpZRv1r+fyWRW\nf06n00in0youS2xiZVKwmQftVTTZbJ1JfZVlWHOow/owIdEin88jn8+3PM61aAshrgTwdQBHpZRP\nGx2jF23iHa1E1Yogv37qFO5NJGozM5JJbNu1y7NoUm+hXDx1CounT+Pq7m7Mb9tWY6WENYc6rA8T\nEi3qA9rPfe5zhse5zR4RAJ4A8LKU8q/cnCsueBWtWvmKbtYvpF6QiwAGEglsSSZX0+28jiatZGxc\n/4EPNKb6haBHdVgfJqRNMaptt7oB+F0AvwbwAoCFyran7hgfqvSjgdMeH1aw2jOjMDMj9/f0yDuv\nuUbevnmzvGfHDnnfjh0tPzuRShkeM5FKuR67Faq/uwIgxwA5AciP1vUbKczMyNG+PjmRSsnRvj5l\nvVCsjk3/e3lA0d8riS/woveIlPI7YFWlZbyMVu18Rb/m/Hk8/MYb5RfnzuFOk4jQqtftB/rf3eok\n6dISxitZLUFOBrIgh/gJKyJ9xEvv06qoGj043mFyff1ng14+y0kTKT8nA1mQQ/yCou0jXkarVkXV\nSPz6ANzX2YlHdeJd/9mgo0mnTaQ4GUjaDYq2D+j7agwkEjiwtLT6FV9VtGpVVI3ErxfAl7Zvx/h1\n1zX9bJDRZKuHUtD2DYke2WwRU1M5XLrUgfXrVzA01If+/voKhRBiZHSr3BDziUizhv2fvPlmpQsH\nuBlPUJNmdicOmy26EKb7IuFnZqYgk8mRmnn1ZHJEzswUgh7aKjCZiBTl97xDCCG9vkaYGdM0HMrl\nGvaPaxoOzs56fn2jFEOgds3E+oZMfmBWkq656B9SzGYd3RdL0OOHpo0hlztksH8cs7MHAxhRI0II\nSClF/X7aIx5j12tVKSBGwnhfsYhfbd+OOwPu1+HFxKET+4Yl6PHk0iVj6VteXme4P0xQtBXQTGjt\neK2qBcRIGB9dXsbehQU8fNtt+EYyiQ3XX+9JZNnq4ROWicOgs05IMKxfv2K4v7PzsuH+MEHRdkkr\nobWTKqdaQIyEsVj586mlJeD4ceD4ceWRpZWHT1gmDsPy8CD+MjTUh1JpFKXS2jpNyeQIBgf3BDgq\na1C0XdJKaO2kyqkWECNhzAF4vG6f6sjSysMn6LzvKmF5eMSBMGVrVK87PT2O5eV16Oy8jMHBPZHI\nHqFou8SK0Fr1WlULiJEw/tTkWJWRpdXfCRB8FWFYHh7tTjZbxPDwXE1kWyqNAkCgwh0Fka6Hou0S\nlUKrWkCqAnjgwQfx84UF/E6TLJ7XL150dA0jrP5OwlBFGJaHR7szNZWrEWwAKJUOY3p6PJLCGSQU\nbZfohbaIsv3w085ObDhzBsVs1vQ/f7OJOpUCUhXG/T09OLiwgL0ARlG74voIgEsK0zKjFr2G4eHR\n7viVrREmC8YrKNouqf5n3zs+jitPnCiXgi8vAwsLGB0erjmmSquJOi8E5PaDBzE6PIxtpRJuBWqW\nE9sD4JlNm5RdKwzRK3Ov/cOKUPqRrRFGC8YTjCpuVG6ISUWk1daodo9VSWFmRn60qyuQa/uJly1w\nSS1WKwuNj3vA8Li+vlGZSk3Ivr5RWxWKfX2jRv+0paaNKblXv4EXrVnJGnYyP4JKM+vt7weefBKj\ndVF+mK0LJzD32j+setVWsjXcRspRLpixA0VbEXYmJINMMwuDdeE1zL32DztC2Spbw+1kZZQLZuxA\n0VaEncm36rFaqYQcyn8JJxIJpHbtAuC9H9vuE2/MvfYPlULpNlKOcsGMHSjairATwfb29+P488/j\nK0eOrK13uLSE0aNH8QiAU0ePNkxSHn/+ebz67LOcWLNA1LJXooxKoXT7AIhywYwd2OUvIMy6/w10\ndeGps2cb9ycS5dLzCm474rU7Tjv+Eftks0VMT8/rhHK3I6E08rSTyRFMTraf8FrBrMsfRTsgMuk0\nMoVCw/67r74af3P+fOPxlU2PWXtXq/YK0+JI2FD1AGgHYtuaNazCZOa7LnUY/5UYfUE0mliz2imQ\nLUlJGIlqabmvGOUBqtzgYZ52q5VPVObr2l1lxcr5jFZaeXhiomH/JxMJWVCcAx5UrnjYUP33Sogq\n0G552lYiRVX5ul5Epc0mLos7d9bsf8+uXZg7ehS9FibWrKa7dVy6tFp23wFgBeUFfuOUFsdvGySK\nRFa0rQiyqnxdlcUaVuwao5S8eiE3y0w5eeECxlArxL1oTHc7eeEC5lDbg2QUwGsXLti6nyjDIhwS\nRSIr2lYEWVW+rirxL2azeHrvXnxhcXF136d+8APg8cdbioSV3OpiNotNp09Dv/LdKIC/6e7G3XVR\n+VWoFWxUXh8QDfMebQuLcEgUucLtCYQQfy2EeE0I8aKKAVnFiiD3DQ1hNJmseX8kmcRum/m6bsW/\nmM1iTNMwdfvtNYINAF9YXMRTDz5oazxm5KamGs5/GEDi+usbBP86kwZR127cqGQsUYBFOCSKuBZt\nAF9EuVGcKWOahmI2q+BSa1gR5N7+fmiTkxjXNGRSKYxrGvY4yG12I/5V3/RQLodb3nzT8Jg3f/IT\nW+MxwyxyvPKttxr2UbDUPdSJv2SzRWjaGNLpDDRtDNlssfWH2gjX9oiU8pgQ4sZmxxzK5ZRP8Fit\nQFRRsu2mX4feNzWu9wIaJdUZZkJ8ulRq6O3NqsF49GEJC6r6XMem/WozjFJK7G4AbgTwosl7sU0n\nk1LKiVRq9f4LgBypS7F7AJD37Nih5FqFmRm5L5FoOH/B5HdfmJmRY5omJ1IpOaZpTHcjnmC1fWv9\nZ4xatLZb+9VmIMiUv0zlz2OvvIJ8Po90Ou3HZUOBPvqtxgHjKK/V+A4Ai93duPvgQSXX6u3vx9/e\ndBPGX3qpZoGDXgDPGEyu6b+FVLNannnooVAVIZHoY7d7X7No2k1TqbCvapPP55HP51sfaKTkdjeE\nONIOunjCqIjmk4mE/OTNN3sS3TopmjEa475EQj48MaF0bCSepFIThtFxKjVheHyzaNpppO0k2g8a\nBF1cE4Rf2qp4wo8SdyPf9A4PfVMnXrVRvvJjS0sYOHIEt+zcyYi7zfE6ArXbva9ZNP2Zz9zqqKug\nlwsL+x3BuxZtIcTfAUgB6BJC/AzAg1LKL+qPGde0QCZ4mhVPAPCtGs7P/tVOJtfMsk62Ly1hnoUm\nbU0m8wiOHClgaWk7quVYpdIcAHUTe3bbt5qJ/MWLr2NqKofOzp+jq2sAW7ZswdatGy21X/VqVZtA\nJkaNwm+VGwJcI1I/CajfqlYJe2+UMf1dVH5XpD2ZmSnIRGJf3V/7iAQKyif2ZmbK50ylJqSmjbWc\nhKy3Mrq7/63s7v6EY3vDqwlMLydGEbQ9EgTNcpE7TKre4lgN1zc0hHuPHVtbkAHACMqTmPMxytuO\nG1NTOSwtPVa39zCAceXrKlajzqmpHJaXOzA1lavZb3SsfjGDM2cuYmHhiZrj7NgbThdraGV9eBHB\nV69pRluLdjN/Nzc1ZfiZOBWXVOnt78fx++/HwJEj2L60tJp1MhuzvO24YSY4wDrl6yo22ghFHDv2\nMJLJb+D66zc0iGF9i9Z0OmN4Xqvi6GRVGyvWh+p1KWuvWd9oooJR+K1yQ4D2iJTmuchmrVHjnKvM\nvO14YfbVPpH4qPKsitprFSo2jHWrI4j8bCvXNM5KecDx76/2mjG0RwDzSUBWwzXix4RpWBelqCfs\nOb0qMLIMEol9uP/+lPJ7rY3qc6iPIltZHUEs2mvF+lC9LqX5t5812l60m9Huq5KHjaj0r45LqbSx\n4NzhyT3W2gj2feAgFu21an2oXG3H7Jo1GIXfKjcEbI+Q8BCVjJ04lUr7Ra2NEI3fr2rrw/41Y2qP\nkPAQlf7VXuX0xhl9pHzy5Ov48Y/vrclc8drqcEIQ0b3+mnNzxsdQtD0iKt6tn7htB+uXz6w6I4CU\n0dsI5VXX/RNDpwSx0HD1mkIcMj7AKPxWuSGG9ojKBYXbCTcZO372jgjiazEh9cDEHhHl97xDCCG9\nuEaYI9kxTcOhXGNy/Lim4eDsbAAjCg/FbBbzuoyd3RYzdjRtDLlcY+ShaeOYnVXTJVFPORKc10WC\nu0MZCZI12i3jRwgBKWXD+n+RtEfCnoUQFe82CJxm7PjtMwfxtZg4Jy4ZP4Ca5cZ8x6wR1HylEVTQ\ncCkv9dBnXqMdlttSfQ/mXfzmXZ03jEQy0g5zJFvMZrH4+uu4r7MTj+rGsy+RwHt27QpwZNEmiOKK\nMBJkRBnmJcO87OIXNsslkqId1ki2ats8XirhEQADALajvILMHUtLmDt6FEX2p3ZEEOlXYcTLvtDN\nUCW02WwRd931MM6e3Q5gDEAfgN6Ge7Arll58Ewut5WI0O6lygwfZI2HtG6IvHhk1qh4IYSEJiRZ2\nV4FRhYqCI6OsnGorWP09OF1TUnXGT9BFVmin4pqw9g3R2zam/dNCYOGQ6BKUt6/CfjD6llBtBQv0\nrt6Dk28TXnwTC2uRVSRFGwhn3xC9bWPWQcAvCyfMKZHEOV71hW6FiodFWQSLKDeM6kB1pRxgXc09\nOBVL1Rk/YZ38jqxohxF9/+4+AKOo7WXm1zqZYU+JJM7xqi90K1RMBF+4cBLAHGr/V4xiw4YXMDn5\nWVs9qv2YIAzt5LeRZ6JyQ8wqIvU9qe/ZsUPu7+nxvT91VBozEXXMzBRkX9+oTKUmZF/faI2Xq8qb\ntbNkmBE7dtxnOI6env0N12nmT/tdHevmnt2AdvK0w0wYbJswp0QS55hFl60iaVXerFv7YdOm6wz3\nb9x4bcN1APNvE35m0ISxyIqi3YaENSWynfA7f7eZMLcSsbB4s3bG0UwswzpB6BcU7Tak2dqYxD1B\n5O82E+ZWIhYWb1bVOMLyEAoKinYbEtaUyHahUUCLKJUEPv7xJ7BzZ86TqLuZMLcSsbAUJqkaR1ge\nQkER2S5/hARFOp1BoZCpvCqiPiMimRzF5KTmWhT1Fszx4ydw9uxTDcdo2jgGB3c3RP7J5AgmJ8si\nptrGUVnOPjWVw6lTr2Nx8Q1s2bLFcGV2s8+2exdGsy5/KrJD9gB4BcAPAXzW4H0/JloJ8Y3abAxv\nquYaMyQKsqNjX9OMivosBy+yLFSdc+089ldmjwswyR5xK9jrAPwIwI0ArgTwAoDtdcf4d5eE+ECt\ncE14Ula+Y8c9lQfCROXPggQKsqtrwHL6mRdl2Gbn7OnZb5py2Pw80VgvMgjMRNutp/1+AD+SUv5z\nJZz/KoAPATjh8ryEhBa9N/u97/0Q5841HuO2UdGJE1cC0C/6MApAwy23vBP5fMbSebzIsjA758sv\nX8Ty8sOrr1tNzK6dJ96ZIE5w2097K4Cf6V6frOwjpK3p7+/F7OxBfPnL+5FMjta8V54U2+343FNT\nOSwvP1q39zCAeVsPgwsXzhjuv3jxddPPtOpzbTbpubz8mzWvW/WyXjtPvDNBnOA20uYMI4k1fjYq\n6uz8vxgc3Lv6uvWE4FswaKYAKY2Lr6ykMhplbnR23ovl5T9uOF+zaHntPFrDGOOUCeIEt6J9CsAN\nutc3oBxt15DJZFZ/TqfTSKfTLi9LSHjwq1HRu961sabfdCuB3bRpG4BbUe6itw7lzu57sGnTM4bn\nt1JpaPSQOnNmBQsLjfffLFpeO888Tp78ORYXb8eWLd3YunVjLPukA0A+n0c+n299oJHRbXVDWfRL\nKE9EXgVORBLiGiu9oVtNMs7MFGRX10frJjKbT/Ldcstw3fkKEhiVb3vbHU0nF7l6vTfAi4lIKeWK\nEOJPUU5UXQfgCSklJyEJcYEVy6XZJGM2W8TevU/X5XV/CgCQTM6aWg+nT5/WvVrLP//lL4Fcznhy\nsWrRJBK/QFfXALq7r8G2bdfFNlr2BSMlV7mBkTYJCc064UWNZpG2WTe9des+KCcmHjY95803f1KX\nM906Fc/PbntxBCaRdiRXYyfELlUPOJc7hEIhg1zuEIaH5yKxkrlRRsfQUJ9p1spPfvILg7MUcfny\nRvzlX34XPT37De9769ZrAWgoe+ANU1MAaicX47QCephg7xESC4JaENctZhOOk5MaJic1QwtFiEfq\nzlK1Or6K5WVgYQEYHh7F888fx7PPvrqaffKBD1yPUql6rTHD8egnF+PebS8oKNokFoRJYOz07mj2\nsJmdPWj4uRtv3IBz5/RpdDnUpv2Vz3HkyACWltZ872PH7sWHP3w1fvu3x3Hy5Ov48Y/vxdLSY6vv\n16fixb3bXlBQtEksCIvA2G3r6uRhc/Dgndi790ksLg4A2I7a+rc1lpa2171+DE8/PYCvfe3A6uIK\nzSZD495tLygo2iQWhEVg7No0Th42/f29ePxxYHz8Szhx4jUsL99gcmTjOZaWtmN6en4197yZdRSW\nlq9xg6JNYoEKgVHRktRu5Oz0YVMV3Gy2WBHv+2pK44X4BKS82+CTl21ZRmFcjqvdoWiT2OBGYFSt\nVmM3cnb7sNGL94MPHqg0dvpNSPk+lCcn9ecZAbAHnZ3M/gg1RnmAKjcwT5u0ASpXNA+qerDxHgoS\nGJPAnZU/C6xkDBHgauyEOEfliuZAMD5w4z30AujF5s134d3vXofOznl60hGAok2IBVRmnwTlA5vd\nw/vf/w7MzmZWXzv17u1+zu8V7dsF30W7mM0iNzWFjkuXsLJ+PfqGhrjgLAk9Yck+0WNX9Kzcg1Pv\nPpN5BEeO/KAmr7vZ54JY0b5tMPJMVG7QedqFmRk5kkzWmIIjyaQszMx46g0R4paZmYLcseMeuXnz\n7fKaa+6UPT37PfF+rfZHcdr3w2gtST1OvPuZmYJMJD5q63NeLIXWbiAMnnZuagqHS6WafYdLJYxP\nTzPaJqHAKHoFUIkKH1897vz5UbNTuLq21ejTaVl+K2vGiXc/NZVrKNRp9blXX33T9nVIGV9Fu+OS\n8YoZ65aX/RwGIYaYieamTa/VCHZ5v/q+JXaE2I642rFRnHj35bFY/1w2W0SpdNrgaJbAW8FX0V5Z\nv95w/+XOTj+HQWKOmYiZiebmzX9keB7VUaEdIbYqrna9YyfefXksfahfNiyR2IfBwTsaji9H5gcs\nH09q8VW0+4aGMFoq1VgkI8kk9gwO+jkMEmOaiZiZaEp5leF+1VGhnSjXqrjatVGcpCSWxzJXWe+x\nvLRZInEC99+fatJPpbp/bSm0m27iJKQVfBXtqm89Pj2NdcvLuNzZiT2Dg/SziW80E7H1643Xqb7p\npg04f977zJFmQmz07cCsNaseux51/XUGB1un4enXe1wbywELFkwv9BWZ27aNN70OKeN7yl9vfz9F\nmgRGMxGf14+sAAAIz0lEQVT7zGduNRTNz39+AIB3BTF6ody06TX09BzAxo3Xrl4HgGlP7dnZg03P\nbSd6d5OGZyf3PIzpk1GCxTUkVjQTsVbWgBdf3Y2EMpkcxec/f+vq9TRtrKXFYebTmwnkrl3boGlj\nNcevfQspotyDuwOlksD4+JeU3ju7A7qDok1iRasoz+9qRSuecyuLw0qErBfIXbu24ejRUw3HJxK/\ngH5B3yonTtyHbLaoXLgp0s6gaJNYEXSUVx8RW8lXbmVxtBL+eoE0i9y7ugZgtMrN8vKjoV+WLU5Q\ntEnsCCrKM4qIE4kBw2P1nnOrbwd2C1XMIvfu7mvwi1/8FEZlEyx6CQ8UbUJcYrV4xSgiXlo6gESi\n+VqMzb4dOClUMYvct227DldddRYLC9bPRfyHok2ICVbE2E7GhXGE24ubbvpbbNvW3K4x+3bgpFCl\nVeQ+PMzMjjBD0SbEAKtibKd4pVmE2yp1zwwrhSpOcryZ2RFijLpIWdkA3AbgJZRXB+1pcpxfTbEI\nUYbVLnSp1IThcanURMM5vVi1ptU4nXYDJMEDD7r8vQjgIwD+s8vnBiGhw2oloZ3iFZWZK9Xo+dVX\n30QiMVCxSMrn0dsZTrsBkvDiWLSllK8AgBBC3WgICQlWxdhudZ+KzBXjLJR7K974dTUPAlXLpJHw\nQE+bEAOsinEQed/GWSiPYdu28QZvXOUyaSQcNBVtIcQ8gG6Dt0aklN+0epFMJrP6czqdRjqdtvpR\nQgLBrhiXLci1P73ETvTMPh/RIZ/PI5/PtzxOuP1HJoT4NoB/J6X8R5P3pR//kAkJArPeIZOTmmfR\ntqaNIZc7ZLC/MdKujrG2A99u+tkRQAgBKWWD/6xKtD8tpfzfJu9TtElbYJQ6NzWVsyWgqsbR+KAY\nweQkU/PaCTPRduxpCyE+AmAKwNsBZIUQC1LKf+1ijISEFrO87c7Onxse7+VEX9D9U0iwuI60W16A\nkTZpA8wsia6uAZw9+5TB8d5F2iQemEXaVwQxGEKihtnk35YtW5BM1q7MXp7o2+3HsEgMYcofIRYw\nS53bunUjBgd306ogvkF7hMQOq1356j8Thck/J/dGwonyiUhCoojTdRCjMPnnZo1HEh0YaZNYYTfH\nOUq0873FEU5EEoL27sXRzvdG1qA9QmJFO/fiUHVv9MXDDUWbxAovenGEReRU3Bt98fBD0SaxQvWE\nYphETsW9sf92+OFEJCEuaLfJv3Q6g0Ih07A/lcogn2/cT7yDE5GEeEC7Tf61s+ffLlC0CXFBu4nc\n0FAfy/JDDj1tQlzQbosMRKGIKO7Q0ybEJVxkgHiBZ4sgWLgwRZsQQmzC3iNEKWHJTSYkblC0iW3C\nlJtMSNxg9gixjXkBxnxAIyIkPlC0iW3aLTeZkChB0Sa2qc1NLgIYA5DB8eMnkM0WAxoVIfGAnjax\nzVpusgZgDkDZKjl7FhgeprdNiJcw5Y84Ipst4q67HuZK5IR4BHuPEKX09/fillu2G75Hb5sQ76A9\nQhwTlb4bzCkn7YRj0RZCPATgDwC8BaAE4E+klOdVDYyEnyj03WBOOWk3HHvaQojdAL4lpfy1EOI/\nAICU8t8bHEdPu40Je9+Ndut3TeKD8jJ2KaW+kuI5AP/G6blIdOnv7w2VSNfDnHLSbqiaiPwEgH9Q\ndC5ClBEV350QqzQVbSHEvBDiRYPtD3XHjAJ4S0r5Fc9HS4hN2NSftBtN7REpZdN/2UKIuwF8EMDv\nNzsuk8ms/pxOp5FOp62OjxBXsKk/iQr5fB75fL7lcW4mIvcA+I8AUlLKnzc5jhORhBBiE+WLIAgh\nfgjgKgD/r7LrWSnlfoPjKNqEEGITrlxDCCERgmXshBDSBlC0CSEkQlC0CSEkQlC0CSEkQlC0CSEk\nQlC0CSEkQlC0CSEkQkROtK2UeUaNdrwngPcVJdrxnoD2vC+Kdghox3sCrN9XNluEpo0hnc5A08ZC\nv6J7O/59teM9Ae15X1xujAQKV5YhxB6Ri7RJezE1lasRbAAolQ5jenre5BOExBtfeo94egFCCGlT\nAmkYRQghRB20RwghJEJQtAkhJEJETrSFEA8JIU4IIb4vhPiGEOLqoMekAiHEbUKIl4QQl4UQPUGP\nxw1CiD1CiFeEED8UQnw26PGoQAjx10KI14QQLwY9FpUIIW4QQny78m/vuBBiKOgxqUAI0SmEeE4I\n8YIQ4mUhxF8EPSZVRE60AeQA3CylfA+AfwLwQMDjUcWLAD4CINxJyi0QQqwD8J8A7AHwLgB/JITY\nHuyolPBFlO+p3fgVgD+TUt4MYBeAA+3w9yWlXAbwe1LK9wJ4N4DfE0L8bsDDUkLkRFtKOS+l/HXl\n5XMAtgU5HlVIKV+RUv5T0ONQwPsB/EhK+c9Syl8B+CqADwU8JtdIKY8BOBf0OFQjpVyUUr5Q+flN\nACcAXB/sqNQgpfxl5cerAKzD2tKIkSZyol3HJwD8Q9CDIDVsBfAz3euTlX0k5AghbgSwA+VgKPII\nIa4QQrwA4DUA35ZSvhz0mFQQyopIIcQ8gG6Dt0aklN+sHDMK4C0p5Vd8HZwLrNxXG8Ac0ggihNgA\n4O8BDFci7shT+Ub+3sq815wQIi2lzAc8LNeEUrSllLubvS+EuBvABwH8vi8DUkSr+2oTTgG4Qff6\nBpSjbRJShBBXAvg6gKNSyqeDHo9qpJTnhRBZAO8DkA94OK6JnD0ihNgD4DMAPlSZbGhHGqqgIsT/\nAvAvhRA3CiGuAjAA4L8HPCZighBCAHgCwMtSyr8KejyqEEK8XQhxTeXnBIDdABaCHZUaIifaAKYB\nbAAwL4RYEEI8EvSAVCCE+IgQ4mcoz+BnhRD/I+gxOUFKuQLgTwHMAXgZwFNSyhPBjso9Qoi/A/A/\nAfyOEOJnQog/CXpMivhXAD6GcnbFQmVrhyyZLQCeqXjazwH4ppTyWwGPSQksYyeEkAgRxUibEEJi\nC0WbEEIiBEWbEEIiBEWbEEIiBEWbEEIiBEWbEEIiBEWbEEIiBEWbEEIixP8HfsYG+iK597kAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d485710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import FukuML.PLA as pla\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "pla_bc = pla.BinaryClassifier()\n",
    "pla_bc.load_train_data('/Users/fukuball/Projects/fuku-ml/FukuML/dataset/linear_separable_train.dat')\n",
    "\n",
    "for idx, val in enumerate(pla_bc.train_Y):\n",
    "    if val==1:\n",
    "        plt.plot(pla_bc.train_X[idx,1], pla_bc.train_X[idx,2], \"ro\")\n",
    "    else:\n",
    "        plt.plot(pla_bc.train_X[idx,1], pla_bc.train_X[idx,2], \"bo\")\n",
    "        \n",
    "plt.axis(\"tight\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "機器訓練完之後，我們可以得到 w，這時只要使用 $w_2*x_2+w_1*x_1+w_0*x_0=0$ 的線性方程式找出斜率，就可以在平面上畫出 w："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD7CAYAAABzGc+QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXt8FOXVx3+P3BKFCEbocqtgtIJgKSoKaslqX7LU1Fra\nKl4Q8A5q4KW1WpPsZkO4XywkxAsFFQsIWMVLtm9IKibBC4pyDQQIqVpuEYwCARMgyXn/2Gyym8zs\nzu7M7OxszvfzmY/J7Ow8ZyP727PnORdBRGAYhmHMxwVGG8AwDMOEBgs4wzCMSWEBZxiGMSks4AzD\nMCaFBZxhGMaksIAzDMOYlPZ6LyCE4DxFhmGYECAi4e/xsHjgRBTxR0ZGhuE2RIudZrCR7YwsO9OS\nkkBAqyPdZosoO8N5KEETARdCtBNCbBNCvK/F/RiGaVskTZmCtIQEn3OpCQkYlZJikEXmQKsQylQA\newB00eh+DMO0IUYmJwMA7Dk5aFdbi/qYGIxOSWk6z0ijWsCFEH0A3A5gJoA/qbbIIKxWq9EmKMIM\ndprBRoDt1Bq1do5MTg6LYJvl76kEoTTWInsDId4EMAtAHICnieiOFo+T2jUYhmHaGkIIUIBNTFUe\nuBDiNwCOEdE2IYRV7jqn09n0s9VqjapPQIZhGC0oKipCUVFRUM9R5YELIWYBeABAHYAYuL3wt4ho\nvNc17IEzDMMEiRIPXHUIxWuxRHAIhWFMRYnLhYLsbLQ/exZ1nTohacoU3jiMEHQPoUjASs0wJqHE\n5cKGqVMxs6Ki6Vxa488s4uZAMw9cdgH2wBlGEeH2htNtNswoKGh13m6zISs/X7d1GWUY4YEzDBMC\nRnjD7c+elTzfrrY25HtySCa8sIAzTARQkJ3tI94AMLOiAvacHN0EsK5TJ8nz9TExId2PQzLhh7sR\nMkwEoIc3HAity9flPoQKc3JCtpHxD3vgDBMBaO0NK0Hr8nUjPoTaOizgDBMBJE2ZgrSKCh8PNjUh\nAaN1buakZfm6ER9CbR0WcIaJAKKhmZNRH0JtGU4jZBhGM0pcLhR6fQiNMtmHUCQR1kpMP0awgDMM\nwwSJEgHnLBSGYRiTwgLOMAxjUljAGYZhTAoLOMMwjEnhNEKGCSPcK4TREs5CYZgwIdUr5GGLBZ16\n9kSPuDgWdMYH7kbIMBFEy14hJQAslZWYWVnZdI6bPzHBwDFwhgkTLXuFFACY2eIaqeZPJS4X0m02\nOK1WpNtsKHG59DWUMQ3sgTOMH7SMWbfsFSL35vNu/sQtWhl/sIAzjAxai2fLXiF1Mtd5N38yok84\nYx44hMIwMmjd33pkcjJ6jxuHsfHxmHjxxdjauTOe6tbN55qW/bi5RSvjD/bAGUYGJeIZTIilxOXC\n4ZUrsbaqquncwxYLnrz2WnTv0kWyAyG3aGX8wQLOMDIEEs9gQyxSHv3yykrYhwyBU2aIMLdoZfzB\nAs4wMgQSz2Dj06GEQ6KhTzijHyzgDCNDIPEMVpBDDYdoOTWHiS5YwBnGD/7EM1hB5nAIozWqBFwI\nEQOgGEAnAB0BvEtEz2lhGMNEOsEKModDGK1R3QtFCHEhEf0ohGgP4CMATxPRR16Pcy8UJmrhEWKM\nXoR1pJoQ4kK4vfEJRLTH6zwLOMMwTJCEpZmVEOICAFsBJAB40Vu8GcascNtXxgyoFnAiagDwCyHE\nxQA2CCGsRFSk2jKGMQjuP8KYBc2yUIjopBDCBeB6AEXejzmdzqafrVYrrFarVssyjOZw/xHGCIqK\nilBUVBTUc1TFwIUQlwKoI6ITQohYABsAZBLRB17XcAycMRVOqxXO4uLW5xMT4QzyDeYNh2WYYAhH\nDLwngBWNcfALAPzDW7wZxozo0X+EwzKMHqjqRkhEu4joWiL6BRH9nIjma2UYw3gTzqEGSVOmIC0h\nwedcyy6BwaJ1Z0Mmetm9ezfuvvtuRddyJSYT8YTbe9Wj4Eau7P6/n30Gp9XKIRUG+/fvR2ZmJgoL\nC/H000/jzTffDPwkItL1cC/BMKGTlpREBLQ60m02o01TjOxr8Po5NSGBivPyjDaVCTNfffUVPfjg\ngxQfH09ZWVl06tQpIiJq1E6/+soDHZiIJxKHGgQb0pEMywAY5fU7h1TaFocOHcLkyZNx3XXXoXfv\n3igvL0d6ejq6dOmi+B4cQmEinkgbahBKSKdlWGbfzp2Y/MMPGNniukiatMNZM/rw7bffYvbs2Xj9\n9dfxyCOPYN++fbj00ktDu1kgF13tAQ6hMCopzsuj1IQEn9DDczLhhuK8PEpLSqKMxERKS0pqukbu\nfChoEdKZPHSo5D2euPbakO3SEqm/OYd41PHdd9/RM888Q926daOUlBQ6cuSI3+uhIITCHjgT8Sjd\nVJTzjEu3bMHhlSs12wTVIqRzDkAagJle51IBnI2QmgkuZtKOEydO4Pnnn0dubi7uuusu7NixA337\n9tXk3izgjClQMtRATnTGLlniM4fSc15KjJSEDbQI6fSJi8NtAOwA2gGoBzAawMa4OMX30JNI3Hcw\nG9XV1cjOzsaiRYuQnJyMLVu24PLLL9d0DRZwJmqQE53YujrJ8y1T+Eq3bEHxvHkYWFODOgBJADZI\neOpaDGao69QJI4FWMfDCCBlWHGn7Dmbixx9/xIsvvoh58+bhtttuw0cffYSrrrpKl7VYwJmoQU50\natpL/zP/6YkTTSXzD+/cCfruO6z1Evs/AfhdY2aIt4BrkSeu5EOg5beBXiNG4Minn4ZlU5GnBwXP\n2bNnsXTpUsyePRvDhw/Hv//9b1xzzTX6LhooSK72AG9iMmFCbrMzNyOj9XmAir1+T5PYUCSAngAo\nIzFRN3vTbTbKSEykdJvNZ4Ow5WspBujx9u3Duqnozz6mmXPnztHSpUupb9++dPvtt9MXX3yhyX3B\nm5hMW8KfZ1wybJjfFD65N8JpAJfoFDbwF9dvGc8vAPBSi1CQJ47vuV5rz5yHKfunvr4eq1atQmZm\nJvr374+1a9dixIgRYbWBBZyJKuREx/t8us2GkQUFPo9LR8mBE0LgYQ3CBsHmVLeM58u9UasPH+Ym\nWWGmoaEBb775JjIyMtC9e3csX77cuBbZgVx0tQc4hMJEGFKhlj927EjTWoRPngNo7OWXq14vNyOD\nHo+NDSr80TLXXC7Ec3d8vOqcdEYZDQ0NtH79errmmmto2LBhtGHDBmpoaNBtPXAIhWFaIxVquXX4\ncHz58suwV1Y2pfVVWix4Ijtb1VolLheK583D2poan/OBcqpbbiImAZjUvr1PGCU1IQFdY2KAFimS\nAKf7aQkRIT8/H3a7HXV1dZg5cyZ+85vfQAi/rbrDAgs4E9XIhS6kQi0lw4a5e5HU1gIxMZiowYT5\nguxsDGwh3h78iazUh8zPhw+HffNmn/h+QXY2sHt3q+dzup82bNy4Eenp6Thx4gQyMzPxhz/8ARdc\nEDktpFjAmagl2J4lemzatT97Vja+HkhkldrD6X7a8/HHH8Nut+PgwYPIyMjAvffei3bt2hltVitY\nwJmoJRLKwes6dUISWpfNPx4bi/s1EFk9epe3ZbZs2QKHw4GysjI4HA6MHz8e7WXqCCKByLWMYVQS\nCeXgSVOmYENFBWwVFU1l82WxsUh85hnNRJbT/dSzY8cOOBwOfPHFF0hLS8M777yDTjKFYZEECzgT\ntWhZDh5qa1XPNYVeHvKTMh4yt28NP2VlZXA6nSguLsazzz6LNWvWIDY21mizlBMoTUXtAU4jZHQi\nUIvYYNrQBlrHX2tVLVrVcvvW8HLgwAF64IEHqHv37jRnzhw6ffq00Sa1AgrSCFnAGVOiVPC0KAf3\n1/+7OC+PplksPuenWSxBrxMNY+PMwNdff02PPPIIxcfHk9PppBMnThhtkixKBJxDKIwp8d6gLIG7\n1LxDRQVyJ0wAVqxoCj0oiQ8HCl34i6WvsdvxQmWlz/nnKyvxpMMRVPgjEuL10cyRI0cwa9YsrF69\nGpMmTcL+/ftxySWXGG2WaljAGVPiEbwSABvgleFRVYW0qVMBKCslV5Jq6C+WfmbHDsnHTn/1VeAX\n4QW3b9WHY8eOYe7cuXj11Vfx4IMPYu/evejRo4fRZmlG5GSkM0wQeASvAL7peUBww4HlUg29ny85\nkDghAaNSUnBWphrvnKLVm/G3BhM833//PVJTUzFw4ECcPXsWpaWlWLhwYVSJN8AeOGNSPKXmHVqI\nrweloQd/oQtPaOX0kSPY/+23GHPRRYjr0AGdL78cY6dPx8jkZLzerx/Sfvih1Wi0i/r1C+r1cD63\nNpw8eRKLFi1CdnY2fv/732Pr1q247LLLjDZLN1QJuBCiL4DXAfQAQACWEpG65hEMowCPsOVOmCDZ\nC0Rp6EEudHHo1KnWoRUANgAbTp5sOjc+KwsrHnmkVQ+ViVlZSl9KE5zPHTpnzpxBTk4OFi5ciF//\n+tf47LPPcMUVVxhtlu6o9cDPA5hGRNuFEJ0BfCmEKCSiMg1sY5gm5DYaS596CpNmzvRp8vR4+/YY\nMny4ovvKTZ7pCLQOrcA9w7JlH+4ul16K/efPo6vFgh59+mjSQ4VRRk1NDV566SXMnTsXiYmJKCkp\nwcCBA402K2yoEnAiqgRQ2fjzaSFEGYBeAFjAGc3wt9F45NNPcV9dnc9w4Pvr6lC4ebOie8uFLjbO\nny95vacbxrFDh1rb1LUrRoVBvLngxz2+bPny5Zg1axauv/56bNiwAUOGDDHarPATKM9Q6QGgH4Bv\nAHRucV7nbEkm2vGXI52RmCj5mNoxaLJrGtyHu60X/Jw7d46WLVtGl112GY0ePZo+//xzo03SDYQr\nD7wxfPJPAFOJ6HTLx51OZ9PPVqvVuOkVjCnxt9EoF8M+Xl2NdJstZC9VMrQCYDTcIZaesbGG9OGO\nhAZdRlBfX4833ngDmZmZ6Nu3L1atWoWbb77ZaLM0paioCEVFRUE9R7WACyE6AHgLwEoiekfqGm8B\nZxg55EIDUiJdAqCstBRdLRZMio3FS149t6dZLKg9cgQztm5tOhfsmDHv0Er14cOoPHoUF1ssKOzT\np7kPd2lpq+fpnbfd1gp+Ghoa8Pbbb8PhcKBbt254+eWXcdtttxltli60dG4zMzMDPkdtFooAsBzA\nHiJapOZejHnRIibrL87d0hsuAbC6fXusraoCqqpQAmBsbCx6JiSgS+/eqD52DMu3bfO5fyheaqCs\nkD/t3Innvaowp1ksGKNz3nZbKfghIrz//vtwOBxo3749Fi5ciNGjR0fEFJxIQq0HfjOAcQB2CiE8\n75jniChf5X0ZkxDs0AQ5/IUGsvLzUbplC8YuWYLYujpUnTmD972yTkYCGFlTA3vv3sjKz4dTJkSn\ntZd6EvDZPD3VeF7PTUa5rJloGeBARCgoKIDdbkdtbS2mT5+OO++8k4VbBrVZKB+BqznbNFrFZAMV\n1BxeudLtcQNwytzDI9Dh8FILsrOxvEUPFDT2QOl68qRuU+KjueCnuLgY6enpOH78ODIzM3HXXXdF\n1PiySIQrMRlVaBWT9Se6LT8kAo0oC4eXKve6T//nP8g9ccLnnNabjNFW8PPpp5/Cbrfjq6++QkZG\nBu67776InoITSfBfiVGFVt6uP9FtmZMtNaLMW6DD4aXKve5zMl/1o3WTUQ1ffvklHA4Hdu3aBbvd\njokTJ6JDhw5Gm2UqWMCZkPDuEzI2NhZP1tRgZONjoXi7/kS3INu3O4NnnXvi4zFg8GBJgdbbS5X7\nwLkoLg744YdW10fbJqMadu3ahYyMDGzevBmpqal4++23DR9f5nKVIDu7AGfPtkenTnWYMiUJyckj\nAz/RaAIliqs9wIU8UYdUMcnjsbH02KBBIQ9NCHa9UCbrBLOekgk7UsMiwm2rmdi7dy/dc8891KNH\nD1qwYAGdOXPGaJOIiCgvr5gSElJ9arISElIpL6/YULugoJBHuK/TDyEE6b0GE17SbTbMKChodd5u\nsyErX5sEpJaZHL1GjMDRzZubvHO9StYls2oSEmBbvFjxeiUul88MTH+2toWy+P/85z+YPn06XC4X\npk2bhpSUFHTp0sVos5qw2dJRUDBD4rwd+fnBNyXTCiEEiMhv+g2HUJigUbpxGao4SYnopE2bQJdf\nju69eyNJx6wLLbJqlIZvtErBjFQOHjyIGTNm4J///CeeeuoplJeXo2vXrkab1YqzZ6VlsLa2neT5\nSIIFnPGLlAgr2bhUI05SIvpSTQ0e2b0b2L0by0pKsGbgQNyTlRWy0Ml9uISz0jFay+KPHj2K2bNn\nY+XKlXjsscewf/9+xMfHG22WLJ06Sec1xcTUh9mS4GEBZ2SRE+He48YFTNNTI05SIlrS+N8ZAFBb\nC2zbFtToNCWvCwhvpWO0lcV/9913mDdvHpYtW4YJEyagrKwMP/nJT4w2KyBTpiShoiINFRXNeU0J\nCalISRltoFXKYAFnZJEV4c2bYVu82G+anhpxkhLRAgDLWpwL1Vv19+ESzkrHaCmLP3HiBBYuXIgX\nXngBY8eOxc6dO9GnTx9V9wxnVojnvjk5dtTWtkNMTD1SUkabIguFBZyRxZ8IB4rzqhEnKRH9r8y1\noXirgV4XEJ5KR7OXxVdXV2Px4sVYtGgRfvvb3+KLL75A//79Vd/X5SrB1KkbfDziioo0ANBVxM0g\n2C1hAWdk0VqElYqTt4h+tWkT+v/4o+y1x6urA96vJYFeV7gqHc1aFv/jjz8iNzcX8+fPx6hRo/DJ\nJ5/gZz/7mWb3z84u8BFvAKiomImcHLspRVZPWMAZWbxFuATuMMZ/Y2LQ+dgxlLhcPkIjtSkYKMzi\nD4+IpttsyCoowCOQqL4EcDaEFNVI8nzNVBZfW1uLpUuXYvbs2bjlllvw4YcfYtCgQZqvo3VWiGmL\ndBTAAs7I4hGWR+x2dCgrw4u1tZIbiHKbgrbFi1XnhXvEtk9FBW6Db/e/0QA2xsWF/Lr09nyjJcf7\n3LlzePXVVzFjxgz84he/wL/+9S8MHTpU1T39iaqWWSFGhGPCSqBKH7UHuBLT9PgbaabkcbUU5+UZ\nNsIsVKJh9Nn58+fp1Vdfpf79+9OoUaNo8+bNmtw3UOWj9OPP+TyelJRGiYkZlJSU5rdiMikpTeqf\nDdls6Zq8Fj1BuEaqMdFNoIwSvdPhRiYnAytWIK2Flx/JG35mzvFuaGjA2rVr4XQ6YbFYsGLFCvzy\nl7/U7P6BYtz+skKC9ajNXKSjBBZwJiCBNv3CkQ5ntg0/M+Z4ExHWr18Ph8OBzp07Izc3F7/61a80\nH6agRFTlskKC3eA0c5GOEljAmYAE2vTzPG6rqEAB3P+oymJjkTh8uKZxYDNt+Jkpx5uI8K9//QsO\nhwNEhLlz5+L222/XbQqOGlEN1qM2c5GOIgLFWNQe4Bh4VCDVec+b3IwMejw21ifQ+JDFQtMsFsmu\nhf66/EUDZuhK2NDQQIWFhTR8+HAaNGgQvfXWW1RfX6/7uoFi3P4IJaadl1dMNls6JSZmkM2WbniX\nQaVAQQycBZzRBKmNzDSpdxpA6Sbd1AuWQB96RlJSUkKJiYl05ZVX0urVq6muri6s64cqqmrE32wo\nEXBuJ8togtNqhbO42PccpOdXep+XakHrL+wSLal5RvH555/Dbrdj//79yMjIwLhx40w3vszlKkFO\nTqHXBueo6EgJbAG3k22jGCFyUjFf2dmVXj9LtaCVazQFIKrbr+rJ9u3b4XA4sG3bNqSlpeGhhx5C\nx44djTYrJMxa9q4HLOAmRU6k1bRxVSP8UhudRywW/AnA817T21PhLsDx0HJTz1/6HRGZNjVPinB8\n0O7ZswcZGRn46KOP8Ne//hXr1q1DTARupDKhwQJuQvyJdKj5x2qHC0il+U1szFKx5+Sg+vBhHK2o\nCDg7M1D6naekvz3cHn4SIjs1Tw69hzmUl5cjMzMTBQUF+Mtf/oLXXnsNF110ker7MpEFC7gJ8SfS\noeYfqxH+QF6kd/y6MCcHG/3kcR86dQrp8BXokXB76kePHcMG+PZDSQPw7alTfl9bJKJXoc/XX3+N\nrKwsvPvuu5g6dSpeeOEFxIXQboAxByzgJsSfSIeafxyK8MuNPit95hk84XS2uj5QHneJy4W4o0fh\nPZ0wDcBrFgsmpqRgjd2OmS2eMxPAkzrlK+uJ1oU+hw8fxsyZM7F27VpMnjwZ5eXl6NatmxoTGRNw\ngdobCCFeEUJ8K4TYpYVBTGD8iXTSlClIS0jwOZ+akIBRAUrOgxX+EpcLuRMmSI4+K543DyUul9/1\npCjIzvaJlwNugY7t1Qsjk5PRQ8aT7B5BA3KVolWhz7fffotp06bhmmuuQefOnbFv3z7MmDGDxbuN\noIUH/iqAHACva3CvqEXLDSt/lZGhlpwH02LV43kPrKqSvNfAmhoUhhAKkPNKPQJtpurGQKhtaVtV\nVYX58+fj73//O+6//37s3r0bPXv21Mtc0xDNrWOlUC3gRLRJCNFPvSnRi9YbVoFEOpSS82CE3xO/\nTZe5Vz1CCwUck4lle4Y2RFIfb7WE+kF78uRJPP/881iyZAn++Mc/Yvv27ejbt284TA4boYpw1LeO\nlSJQpY+SA0A/ALtkHtO1WskM6N1uNdxkJCYSAVQMUGqL1/Rc4/lQXtvDQ4dK3u/hoUObronk6kY9\nqa6uppkzZ9Kll15KEyZMoIqKCqNN0oVArWa9r2vZUtbMrWOlQKS0k3V6bWhZrVZYrdZwLBsxmLEz\nnT88oQyPT+MZsrAXwBMA8kP0ivvExQUc2uD5duEJSW2cPx8F2dlRW5FZU1ODF154AfPnz8ett96K\njz76CFdddZXRZumGkm6Dcp52bOwZyXvKNbqKtHBLUVERioqKgntSIIVXcoA9cL+E2wMvzsujtKQk\nykhM1KVplFSjpscam1Sp8YqV/p2k1n88NpZyMzI0eHWRQW1tLeXk5FCvXr1ozJgxtHPnTqNNCguJ\niRmSXnRiYkbTNXKednz83Yo9cKWevpEgUjzwtk44Y7dy8fbSLVtw5NNPNWvrCvjGb+/XoDe30r+T\nVA71SzU1GDtvHgYPG2ZqT/z8+fNYsWIFsrKyMHjwYLz33nu47rrrjDYrIFp5s0pazcq1lLVYuqJr\nV2WtY7UanGy0F69awIUQbwBIBBAvhDgIwEFEr6q2LIoI5zACKXGzVVRg9bx5eKmmpumc2qo/PXpz\nK/07yYWkQs1+iQTq6+uxatUqTJ8+Hf369cOaNWswYsQIo80KiMtVArt9DcrKTqO29qcAbgMwMuTN\nQyX9u+VEvrLyBCwWID5+LHr27Inevbs0TfJpiRaTeiJi0zSQi672AIdQwopng1FRW1eTbqLKhloA\n9+s3EfX19bRmzRoaMGAA3XLLLfThhx8abZJipMIQQCoBxao2DwO1mpVat337x5rWVRIO0WLDU+9N\nU3AIpe0hlSst9z/ZrJuoSVOmYNKmTT7fKDxNsgpNkhNORHjvvffgcDjQqVMnLFq0CElJSbpNwdED\nqTCEu/TKDmBkyHMnPd5rdnYBamvbIzu7wOd8y5mZpaVlqKp6Es3b6oHDIcFM6pELk6j14rUIv7CA\nRxlSceSy2FjAS+w8mLEABnCHWkqfeQZj583DwJqapmyVULNfwgkRYcOGDbDb7Th//jyysrJwxx13\nmEq4PcgJmDuHKPS5k76hCXf7spKSZRg4cA2ysu5paifrETur1Yni4tbC509I/Q1OlrfFjSdMomY0\nnGbhl0AuutoDHEIJOy1zpXMzMiJ+vFcomC0nfOPGjXTTTTfRwIEDad26dWEZX6YnciEEIF3VlJzm\n+xY3hmT8Z4roGcrwd2+9R8OBQyhtE6kNxpJhw0wz0V0pWm+k6tWf++OPP4bdbsd///tfOJ1O3Hvv\nvWjXzhy5yf6QCkPExEzCwIF1yMoaH7LdzZ59AYDAmSJ6Di72FyZR6sUHe99gYAFvI5hporsR6NGf\n+4svvoDdbseePXvgcDgwfvx4dOjQQfb6iMhqCAJpAbtPta3NoQllIqdGSJXb4osnTBLqdCA14Rcf\nArnoag9wCIUxAVoWW+3YsYN+97vfUe/evSk3N5dqa2sVPS/aSsFDpTk0YfzfQ68hykruCw6hMIwy\ntGh3UFZWBqfTieLiYjz77LNYvXo1YmNjFT9fq6/VZsfj0drtr6OsbDJqa19sekyr0Eiwtmjt3Wt1\nXxZwJmiicTJ8sK1qvWPV9fXH0aHDAezatRV//vOfsXz5cnTu3DloGzT7Wh0FeEIT7gn02odGQrEl\nIu8byEVXe4BDKFGFVB+S1CjJaFGaqdP89fcbAh4lIJ66dfslrV3rUmWDXl/XGXMCBSEU4b5OP4QQ\npPcabQ0jPeB0mw0zCgpanbfbbMjKzw+LDXrhmdnpydQZJZOpY7VORXFxA4DVAB4H8DSAS2Cz2ZGf\nn6XKBrfHWejlcY6KyA1Ms2CmrJ6WCCFARH4LBDiEYjL0nmYeiGhrjetNoEyd48ePY+7cufj446Vw\nN84tA9Cj6XEtYtV6fV1vi5gtqycUVM/EZMKL3DTzwpycsKwfTWPNlPL9998jLS0NAwYMQG1tLW66\n6TEAC+Et3oC5Y9UuVwlstnRYrU7YbOlwuUqMNgmAOrvkOw4Wam2mYbAHbjKM9IBLXC5UHj+ORzt0\nwN/Pn286P81iwZgIL2EPhVOnTmHRokXIzs7GmDFjsHXrVlx22WVwuUpw+LA+hSNGoLenatSINK2y\neiI5DMMCbjKM8oA9oZvxFRVYAd+pOdKTLM3LmTNnsGTJEixcuBCjR4/G5s2bccUVVzQ9rmfhiBFo\n1RtbCjUibLe/jooKCwAngDoASU12eez2J6paZPVEfBgm0C6n2gOchaIpwWRLaImn0CXaWtN6U1NT\nQ3/729/IYrHQ3XffTXv27DHapLCgZApOqIRanJSXV0wxMZMkW9UOGvSY4rmZarN6jCyuAhfyRB/h\nHA7hjSd0E22taQHg3LlzWL58OWbOnInrr78e+fn5GDJkiNFmhQ09889DDWO4W8m+2OKsu1VtZeUJ\nVFW97POI1DcGLb4pRXpxFQu4CTGir4kndCP9Vtc2hBOuNMm6ujq8/vrryMrKwoABA7B+/XoMGzZM\n83UiHS0Pxr1FAAAa8ElEQVR6Y8sR6oeDWzjdrWTdMuUOocTEfIOePXuiqqr1c6REVW1WT6QXV7GA\nM4rw9Bm3VVQgDb494rSc7xmONMn6+nqsWbMGmZmZ6N27N1auXImbb75Zk3ubES16Y2sxOMGbU6cO\nAdgA339paejV6xx69eqM0tLWz/GIqpabjnp2OtSEQDEWtQc4Bh41ePpvPzZoEI2Nj6epgwdr3odb\ny6ZSLamvr6c333yTrr76ahoxYgR98MEHGlgcneTlFVNSUholJmZQUlJa0++hxrP9jUiTYujQyZJr\nXXvtE35j23pMmw/Ffi0Ax8AZLQlH6EaPNEkiQl5eHhwOB9q1a4cFCxZg9OjRppyCowctPdYRI3ph\n5crDrTztmJjvJJ8fKB4cShgjLq6H5PkuXbr7/cZgs6VrnlETycVVLOBMRKFlmiQRobCwEHa7HTU1\nNZg+fTruvPNOUwq3XrnIUmGRTZvGoqZmrc91FRUzER8/VvIeesSDQ+3DHembjlrDAs5EFFIzPUOJ\nsZeUlCA9PR3Hjh1DZmYm7rrrLlxwgTkLj/XMRZbKAa+pGSh5bc+ePdG1a3jiwaHGniN901FrWMCZ\niEJtmuTmzZtht9tRUVEBp9OJ++67D+3bm/ufua/IujMzKio6YMKEXKxYoU7EpT1WaRHs3bsLUlJG\nhaWAKdQUwIjfdNQY7kbIRAVbt26Fw+HAzp07YbfbMXHiRL/jy8yEe+q6E27x9s3MSEhIw+LFtqBE\n1DscU1pahqqqtS2uKEFs7GrU1LzktU4qxo3rg08/PRJyGEdNSb3d/jq+/vo0iDqhf/+LmqbTy10f\nDR0dlXQj1CLLZDSAvQDKATwr8bjOe7VMW2bXrl30+9//nnr27Ek5OTmKx5eZiebsD/VVga2zNIqp\nffvHW2V0ZGTk+mReZGTkqsruCDU7JC+vmCyWh6jldHqLZVrU90mHgiwUteLdDsABAP0AdACwHcDA\nFteE6eUybYl9+/bRvffeSz169KAFCxbQmTNnjDZJN5rFL0N1yfvQoQ83fhBkNP63mIBiio8f6zdN\nTm1Judzz4+Pv9klVlH6e8bMxjUCJgKsNDt4A4AARfd3o8q8BcCfcjZIZRnO++uorTJ8+HXl5eZg2\nbRpefvlldOnSxWizdMXz9X/ChFzJCkSlG3QuVwnKyjoAmOF1Ng2ADYMHD0BRkVP2uWqzO+SeX1U1\nsDE8JL0xK/e8YNaOZtRuy/cGcNDr90ON5xhGUw4dOoRJkybh+uuvx09/+lOUl5cjNTU16sXbQ3Ly\nSKxY8SQSEtJ8zrs36EYpuod8f5HCgB8Cp04dkzxfXX3c53e5/t1y2SHufpZupHp1u5/XtjJLgkGt\nB867k4yuVFZWYvbs2Vi5ciUeffRR7N+/H/Hx8UabZQhqmzPJebMxMd8gJeURAP42Gs8BrZsogKi5\n8MpfuqNUdgiQCvcWWjMtveopU5Kwc+cKVFb6rm2xTENKyhhFrzuaUSvghwH09fq9L9xeuA9Op7Pp\nZ6vVCqvVqnJZJtr57rvvMH/+fCxbtgwPPPAAdu/eDYvFYrRZhqOmKlDOC7766i5NE+DlBDgurg+A\n2+DbCX404uI2Nl3rr6+4Z1ao58PHnf3yJADf19LSq05OHolly9CYhXIvgI7o378zpk8fa8rMEn8U\nFRWhqKgouCcFCpL7O+D+AKiAexOzI3gTk1HJDz/8QOnp6XTJJZfQpEmT6ODBg0abFDUE6o8tt9E4\ndOjDFB9/d4uNz9YbiYMHT/XJbvFslnbu/AddenVHO9B7E5OI6oQQT8GdnNoOwHIi4g1MJmiqq6ux\nePFiLF68GHfccQe++OIL9O/f32izoopAIRjpEEsJSksFzp/3zhX/EwAgISHfp0Dm6NGjTc/xzlc/\nfRqYOrV5g9ITpomNPYP4+LGwWLqiT58epp5qZBiBFF7tAfbAGT+cOXOG5s+fTz169KD77ruP9u3b\nZ7RJuiHV4S+SkPbApbsCxsYmt7J/0KDHGvO15dP+9OgWGK2AuxEykUptbS2WLl2KOXPm4KabbsLG\njRsxaNAgo83SjUicrSjVhbDlRqMQx0E+qQruUv6amgtht68B0Gx/797dsXt3EoDlkuvV1rbTdf5m\nW4QFnAkr58+fx6uvvooZM2ZgyJAhcLlcGDp0qNFm6U6kCZfcB8q4cb2xeXNziOXjjwmnT3uu8A2N\nbNsGPPLIn9Cz5+uIi+uDU6e+h8WyApWVfVsuB8C9QVlb27a6BeoNCzgTFurq6rBq1SpkZmbiiiuu\nwJtvvokbb7zRaLPCRjjbnCrpOSL3gbJ5c3PGCABce+0j2LZtMoAX4R5v5vucysrnUVlph3tyPNCt\n21O4/PIyHDky2Sfn3NNQKju7QNJmzukODRZwRlcaGhqwbt06ZGRk4Cc/+Qlee+01jBzZ9r4qh6vN\nqdJQjdIPlKys8fj97+fg3Dk7JDKEG2l+zg8/LEH//k8iO3us7GZpW+oWqDuBguRqD/AmZpukoaGB\n3n77bRo8eDDdeOONVFBQQA0NDUabZRjhSptT2rMkmN4mGRm5FBv7uOzmJJDu83u3buMD/i2MGFFm\nNsCbmEy4ISL83//9H+x2O4gIc+bMwe23327KKThaEkoVZSjtV5V61sH0zXY6n8CwYe6WrmVlk1uU\n47eupnRXbcoTySPKzAYLOKMJRISNGzciPT0d1dXVmD59On73u9+ZdgqOHgQjXKFmrSgN1QT7geKx\n3eUqgcPxJPbsqUZtbRcAtfCtpkxFv34XBX6BjDYEctHVHuAQStSzadMmslqtdOWVV9KqVauorq7O\naJNMj5oJ8HqHanxtK24MoWQQMJYslgc5JKIR4BAKoyeff/45HA4H9u3bB4fDgQceeMD048sihVCz\nVtQ2vAretpHweODduk3AsmUTOTwSRvjdxgTN9u3b4XA4sG3bNqSlpeGhhx5Cx44djTYrqlCTtaJ3\njFnOthtu+KlPqbyS2L2Sa0MdxdYmCOSiqz3AIRRZivPyKC0piTISEyktKYmK8/KMNskvu3fvprvu\nuossFgstWrSIampqjDYpajGq2ZOScn9/tgVTKt+c3SJ/bVsuvYfeI9WUHCzg0hTn5VFqQoJPgDM1\nISEiRby8vJzGjRtH3bt3p7lz59Lp06eNNqlNkJGRS/Hxd9PFF0+g+Pi7KSMjV9X9AolzMGIplwqo\nNHafl1dMsbF3B7xW7Sg3M6NEwDmEYhAF2dmYWVHhc25mRQXsOTkYmZxskFW+fPPNN8jKysI777yD\nqVOnIjc3F3FxcUabFZVI9SVZufKwz8T4lSvTMGxYSUjhAyVZLcGU+8uFaZTG7rOzC1BTMzDgtUeO\nnA54TVuGBdwg2p89K3m+XW1tmC1pzeHDhzFr1iysWbMGkydPRnl5Obp162a0WVGLlLhu2jQJNTX3\n+VynpneKEnFWIr6B4tFKY/futfxf63KVoKLiqN9r2jos4AZR16mT5Pn6mJgwW9LMsWPHMGfOHLz2\n2mt4+OGHsXfvXnTv3t0we6KVliJ4/Pj3qKh4weeampqX4J5+4yvWoXqeSsQ5kPgq8eKVFgi510pC\nyzFtsbGPIyXlfgAeL/1Jv9e0dVjADSJpyhSkVVT4hFFSExIwOiUl7LZ8//33mD9/PpYuXYr7778f\nu3fvRs+ePcNuR1tASgRjYsbLXN1arEP1PJV4xoHEV4kXrzSN0b3WBlRU2OAZ0xYbW4Znnkls8Y3A\n87zmUW6XX25cC95IgwXcIDxxbntODtrV1qI+JgajU1LCGv8+efIk/va3v2HJkiX4wx/+gO3bt6Nv\nX+lWoIw2SIlgbe1PJa+NjS1DTU3z72qaPsmJ8/DhfWCzpTd9G2jZTjbwxB7/IZaUFOmUv2ahL/Ra\n60mZcExzrjkA9OljD+lvEI2wgBvIyORkQzYsT58+jZycHDz//PNITk7G559/jssvvzzsdrRFpEUw\nCTExrduvjhuXKCumSvEW1Li4b3HttU+iS5fuiImpx/DhfbBy5eFWIZHFi22S62gRYvEmUL56MP1a\n2ios4G2ImpoavPjii5g3bx5uvfVWbNq0CQMGDDDarDaFtAiOxMCBr6NHD22rJ6UENSEhDdOn34bk\n5JGw2dJlQyIAWm1WSglqbOzjOHQIsNnSG2P59wBIh1ta6lBRYUNOTmFIryUcVaWmJ1CeodoDnAdu\nOLW1tbRkyRLq1asXjRkzhnbu3Gm0SW2WcBboBMqhTkzMkHx88OCpsvngnvzvwYOnNuZxN0+o79Dh\njsaZmN73S6VBgx7T/LW1BcB54G2b8+fPY8WKFcjKysLgwYPx3nvv4brrrjParDaN3l6ld8hkx46D\nktd4YtZyIZGjR4/65J8DzZ55fn5Wk/deWup7zfnzsWg5sQeYicrKe0J6LUxgWMCjkPr6eqxevRqZ\nmZno168f1qxZgxEjRhhtFtOIXr1KWodM0iWv88Ss5WLMdXXtUVXV+nnem5XSsXzpzKWePS2K7GeC\nhwU8imhoaMA///lPOJ1OxMfHY9myZbBarUabxWiIv0Ka1hkurfOsvTcBpb4NDB/eB/Pm+VYIe/BO\nOZT23jtLPq937y6KXx8TJIFiLGoPcAxcdxoaGujdd9+ln//85zRs2DDKz89v0+PLzIy/fiWBepVI\nx7SLqVu3exSPL3PHzYtbxbJjYx8LaIvF8iBZLNPC3oArWgHHwKMbIsKGDRvgcDhw7tw5ZGVl4Y47\n7mjz48vMSqA0vECFNHIZLjfcUIj8fKciGwIVz/imJf6AoUMfQVxcn8ZY/kQAnDUSVgIpvNwB4C4A\nuwHUA7jWz3Xh+LBqc3z44Yd0880304ABA2jdunVUX19vtEmMSkLNGklMzCAibTJc/NnQllu7GgF0\n9sB3ARgD4GWVnyFMEHzyySew2+345ptv4HQ6ce+996JdO+7MFg0EqnQMVEijNsPF5SrB8ePfIyZm\nfGN1aBKAkU1x82C6FTLhIWQBJ6K9APjrepj48ssvYbfbsXv3bjgcDowfPx4dOnQw2ixGQwIJtJLK\nxFAzXJrDN81NtWJiJmPgwNeRlTUeyckjMX/+RsnncmtX4+AYeISza9cuOBwOfP7550hLS8P69evR\nSaaTIWNuAgm0njnk0j1aXkSPHs3etZoxb4w++BVwIUQhAKkkzlQiel/pIk6ns+lnq9XKqW0K2Lt3\nL5xOJ4qKivDss89i9erViI2NNdosRkeUCrQ7PNr8Xy1Q0qiKe5PoS1FREYqKioJ7UqAgeaADwIfg\nTUzNqKiooPHjx1P37t1p9uzZVF1dbbRJTISg5yZiMKPQpEapMdqDMKYRciBcJQcPHsSMGTPw1ltv\nISUlBeXl5bj44ouNNosxGO+0vdLSMlRVPenzuFabiEq9a70n3jPBEbKACyHGAMgGcCkAlxBiGxH9\nWjPL2ghHjx7FrFmzsHr1ajz++OPYv38/LrnkEqPNYiIAqbxwd2Ul4N0fW4tNRO78Z07UZKGsB7Be\nQ1vaFMePH8e8efPwyiuvYOLEiSgrK0OPHj2MNouJIKQ2Ft1l8b6j1rTaRGTv2nxcYLQBbY0ffvgB\n6enpGDBgAGpqarBr1y4sXLiQxZtphdzGoveoNXeYY1R4DGIiDk4jDBOnTp3C4sWLsXjxYowZMwZb\nt27FZZddZrRZTAQjl7YXH78Xgwc7OczBsIDrzZkzZ5Cbm4sFCxbAZrNh8+bNuOKKK4w2izEIf90E\nWyK3sbh48RNhE+1g7GXCDwu4TtTW1uLll1/GnDlz8Mtf/hJFRUW4+uqrjTaLMZBQZkYCxm0sBmsv\nYwCB8gzVHmhjeeBnz56lF198kfr06UO//e1vafv27UabxEQISnOtIwWz2RttgNvJho+6ujr84x//\nwPTp03HVVVfhrbfewg033GC0WUwEoaTaMZIwm71tERZwldTX12Pt2rVwOp3o3bs3/vGPf+CWW24x\n2iwmAjFbL5FQ7eW4efhgAQ+RhoYGrF+/Hg6HAxdffDFeeukl3HbbbUabxUQwWvQSCac4hmIvx83D\nCwt4kBARXC4XHA4HhBBYsGABRo8ezW11mYBo0a87nOIYir3cMzzMBAqSqz0QJZuYDQ0NVFBQQDfe\neCMNHjyY1q9fz3MnmbBihk3FQFODGOWANzG1oaSkBHa7Hd9++y2cTifuvvtuXHABF7Ey4cUMm4pm\ni/ObHVYhP3z22WdISkrCxIkT8fDDD6O0tBT33HMPizdjCGYQxylTkpCQkOZzjsv99YM9cAm2bdsG\nh8OBHTt2wG63Y+LEiTy+jDEcMwxUMLr4qK0hSMOpHpILCEF6r6EVu3fvRkZGBj755BM899xzePTR\nRxETE2O0WQzThMtVgpycQi9xHMXiGKUIIUBEfrMjWMAB7N+/H5mZmfj3v/+Nv/zlL3jiiSdw4YUX\nGm0WwzBtGCUC3qaDuV9//TUeeugh3Hzzzbj66qtx4MABPP300yzeUYTLVQKbLR1WqxM2WzpcrhKj\nTWIYzWiTMfBDhw5h5syZePPNN/HEE0+gvLwcXbt2NdosRmO4qISJdtqUB15ZWYn//d//xZAhQxAX\nF4e9e/di+vTpLN5RinxRSaFBFjGMtrQJAa+qqsKzzz6LQYMGAXBvVs6dOxeXXnqpwZYxemKGvGmG\nUUNUC/iJEyfgcDhw1VVX4dSpU9ixYwcWLVoEi8VitGlMGPDNmy4BkA7AidLSMo6FM1FBVMbAq6ur\nkZ2djUWLFuGOO+7Ali1b0L9/f6PNYsJMc960DcAGuAcCA1VVwNSpHAtnzE9UeeA//vgjFixYgCuv\nvBJ79uzBxx9/jFdeeYXFu42SnDwSixfbEB+fC494e+BYOBMNRIWAnz17Fjk5ObjiiiuwefNmfPDB\nB1i1ahV+9rOfGW0aYzDJySMxePBAycc4Fs6YHVOHUM6fP4/XXnsNWVlZGDJkCFwuF4YOHWq0WUyE\nYXQPER5wwOhFyAIuhJgP4DcAzgGoAPAgEZ3UyjB/1NfXY9WqVcjMzERCQgLWrVuH4cOHh2NpxoQY\n2UOEc9EZPQm5lF4IMQrAB0TUIISYAwBE9FeJ6zQrpW9oaMC6devgdDrRo0cPzJgxAyNH8puACYxR\nPURstnQUFMyQOG9Hfn6W7usz5kVJKX3IHjgRee8AfQbgD6HeS8FaePfdd+FwOHDhhRciJycH//M/\n/8NTcBjFJCePNMTj5Vx0Rk+0ioE/BOANje7VBBEhPz8fdrsd9fX1mDVrFpKTk1m4GdNgdPydiW78\nCrgQohCAVNVLKhG933hNGoBzRLRaS8M2btyI9PR0nDp1CpmZmRgzZgwPUmBMhxl6eDPmxa+AE5Hf\nMRpCiIkAbgfwK3/XOZ3Opp+tViusVqvstR999BHsdjsOHz4Mp9OJsWPHol07/rrJmBMecMAopaio\nCEVFRUE9R80m5mgACwEkEtF3fq5TtIm5ZcsW2O127Nu3Dw6HAw888ADatzd1liPDMEzI6DrQQQhR\nDqAjgO8bT31KRE9IXOdXwHfs2AGHw4Evv/wS6enpeOihh9CxY8eQbGIYhokW9M5CuTLU5wJAWVkZ\nnE4nSkpK8Ne//hVr167l8WUMwzBBEPZdwQMHDmD8+PFITEzEddddhwMHDmDq1Kks3gzDMEESNgH/\n5ptv8Oijj2L48OG48sorceDAATzzzDO46KKLwmUCwzBMVBGWXcKnnnoKb7zxBiZPnozy8nJ069Yt\nHMsyDMNENWER8NjYWOzduxfdu3cPx3IMwzBtgpCzUBQvoGEvFIZhmLaCkiwULm1kGIYxKSzgDMMw\nJoUFvJFgS1iNwgx2msFGgO3UGrYz/LCAN2KW/6lmsNMMNgKB7XS5SmCzpcNqdcJmSzdskn20/D0j\nBbPYqQRuNsIwEvAkHcYMsAfOMBJkZxf4iDfAk+yZyCMsaYS6LsAwDBOl6NaNkGEYhjEWDqEwDMOY\nFBZwhmEYkxI2ARdC/FkI0SCEuCRcawaDECJLCLFDCLFdCPGBEKKv0TZJIYSYL4Qoa7T1bSHExUbb\nJIUQ4i4hxG4hRL0Q4lqj7WmJEGK0EGKvEKJcCPGs0fZIIYR4RQjxrRBil9G2+EMI0VcI8WHj/+9S\nIcQUo21qiRAiRgjxWeP7e48QYrbRNvlDCNFOCLFNCPG+v+vCIuCNYjgKwDfhWC9E5hHRECL6BYB3\nAGQYbZAMBQAGEdEQAPsBPGewPXLsAjAGgDHJ034QQrQDsATAaABXA7hXCDHQWKskeRVuGyOd8wCm\nEdEgAMMBPBlpf08iqgVwa+P7++cAbhVC3GKwWf6YCmAPAL+blOHywJ8H8EyY1goJIqr2+rUzANk5\nn0ZCRIVE1ND462cA+hhpjxxEtJeI9htthww3ADhARF8T0XkAawDcabBNrSCiTQB+MNqOQBBRJRFt\nb/z5NIAyAL2Mtao1RPRj448dAbRD8zjIiEII0QfuYfHLABjbzEoIcSeAQ0S0U++11CKEmCmE+C+A\nCQDmGG2PAh4C8C+jjTAhvQEc9Pr9UOM5RiVCiH4AhsLtXEQUQogLhBDbAXwL4EMi2mO0TTL8DcBf\nADQEulCTSkwhRCEAi8RDaXB/xU/yvlyLNUPBj52pRPQ+EaUBSBNC/BXuP+KDYTWwkUB2Nl6TBuAc\nEa0Oq3FeKLEzQuHcWR0QQnQG8E8AUxs98Yii8ZvrLxr3jTYIIaxEVGSwWT4IIX4D4BgRbRNCWANd\nr4mAE9EoGWMGA+gPYIcQAnB/3f9SCHEDER3TYu1gkLNTgtUw0LMNZKcQYiLcX7F+FRaDZAji7xlp\nHAbgvUndF24vnAkRIUQHAG8BWElE7xhtjz+I6KQQwgXgegBFBpvTkpsA/FYIcTuAGABxQojXiWi8\n1MW6hlCIqJSIfkJE/YmoP9xvkmuNEO9ACCGu9Pr1TgDbjLLFH0KI0XB/vbqzcWPGDBj2rUuGLwBc\nKYToJ4ToCGAsgPcMtsm0CLd3thzAHiJaZLQ9UgghLhVCdG38ORbupIqIe48TUSoR9W3Uy3sAbJQT\nbyD8eeCR/NV1thBiV2OMzArgzwbbI0cO3JushY1pRi8YbZAUQogxQoiDcGcluIQQ/2e0TR6IqA7A\nUwA2wL3Tv5aIyoy1qjVCiDcAfALgZ0KIg0IIQ0J6CrgZwDi4Mzu2NR6Rlj3TE8DGxvf3ZwDeJ6IP\nDLZJCX41k0vpGYZhTApXYjIMw5gUFnCGYRiTwgLOMAxjUljAGYZhTAoLOMMwjElhAWcYhjEpLOAM\nwzAmhQWcYRjGpPw/4ahgcIjBncYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e380bd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pla_bc.set_param(loop_mode='naive_cycle', step_alpha=1)\n",
    "pla_bc.init_W()\n",
    "pla_bc.train()\n",
    "\n",
    "for idx, val in enumerate(pla_bc.train_Y):\n",
    "    if val==1:\n",
    "        plt.plot(pla_bc.train_X[idx,1], pla_bc.train_X[idx,2], \"ro\")\n",
    "    else:\n",
    "        plt.plot(pla_bc.train_X[idx,1], pla_bc.train_X[idx,2], \"bo\")\n",
    "\n",
    "a0 = -4;\n",
    "a1 = (-pla_bc.W[0]-pla_bc.W[1]*a0)/pla_bc.W[2]\n",
    "b0 = 4;\n",
    "b1 = (-pla_bc.W[0]-pla_bc.W[1]*b0)/pla_bc.W[2]\n",
    "\n",
    "plt.plot([a0, b0], [a1, b1], \"k\")\n",
    "\n",
    "plt.axis(\"tight\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "這樣就可以畫成圖示了，但記得圖示只是用來幫助理解，在使用或學習機器學習這門課時，千萬不能被圖示牽著走喔！"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}