LogisticRegression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression\n",
    "\n",
    "Logistic regression is a classification method. Its main goal is learning a function that __returns a yes or no answer__\n",
    "when presented as input a so-called __feature__ vector. \n",
    "\n",
    "As an example, suppose we are given a dataset, such as the one below:\n",
    "\n",
    "| Class| Feature1 | Feature2  |\n",
    "|---| |---|\n",
    "| 0 |5.7| 3.1|\n",
    "| 1|-0.3|2 |\n",
    "|---| |---|\n",
    "|  $y_i$| $x_{i,1}$ | $x_{i,2}$ |\n",
    "|---| |---|\n",
    "| 1|0.4|5 |\n",
    "\n",
    "The goal is learning to predict the labels of a future dataset, where we are given only the features but not the labels:\n",
    "\n",
    "| Class| Feature1 | Feature2  |\n",
    "|---| |---|\n",
    "| ? |4.8| 3.2|\n",
    "| ? |-0.7|2.4 |\n",
    "|---| |---|\n",
    "\n",
    "More formally, the dataset consists of $N$ feature vectors $x_i$ and the associated labels $y_i$ for each example $i=1\\dots N$. The entries of $y$ are referred typically as class labels -- but in reality $y$ could model any answer to a true-false question, such as 'is object $i$ a flower?' or 'will customer $i$ buy product $j$ during the next month?'. We can arrange the features in a matrix $X$ and the labels in a vector $y$:\n",
    "\n",
    "\\begin{eqnarray}\n",
    "X & = &  \\begin{pmatrix}\n",
    "  x_{1,1} & x_{1,2} & \\dots & x_{1,D} \\\\\n",
    "  x_{2,1} & x_{2,2} & \\dots & x_{2,D} \\\\\n",
    "  \\vdots & \\vdots & \\vdots & \\vdots  \\\\\n",
    "  x_{i,1} & x_{i,2} & \\dots & x_{i,D}  \\\\\n",
    "  \\vdots & \\vdots & \\vdots & \\vdots  \\\\\n",
    "  x_{N,1} & x_{N,2} & \\dots & x_{N,D} \\\\\n",
    "\\end{pmatrix} = \n",
    "\\begin{pmatrix}\n",
    "x_1^\\top \\\\\n",
    "x_2^\\top \\\\\n",
    "\\dots \\\\\n",
    "x_i^\\top \\\\\n",
    "\\dots \\\\\n",
    "x_N^\\top\n",
    "\\end{pmatrix} \n",
    "\\\\\n",
    "{y} & = & \\begin{pmatrix}\n",
    "y_1 \\\\\n",
    "y_2 \\\\\n",
    "\\vdots \\\\\n",
    "y_i \\\\\n",
    "\\vdots \\\\\n",
    "y_N\n",
    "\\end{pmatrix}\n",
    "\\end{eqnarray}\n",
    "where $x_{i,j}$ denotes the $j$'th feature of the $i$'th data point. \n",
    "\n",
    "It is common, to set a column of $X$ entirely to $1$'s, for example we take $x_{i,D}=1$ for all $i$. This 'feature' is artificially added to the dataset to allow a slightly more flexible model -- even if we don't measure any feature, the relative numbers of ones and zeros in a dataset can provide a crude estimate of the probability of a true or false answer. \n",
    "\n",
    "Logistic Regression is a method that can be used to solve binary classification problems, like the one above. \n",
    "We will encode the two classes as $y_i \\in \\{0,1\\}$.  The key idea is learning a mapping from a feature vector $x$ to a probability, a number between $0$ and $1$. \n",
    "\n",
    "The generative model is \n",
    "$$\n",
    "\\Pr\\{y_i = 1\\} = \\pi_i = \\sigma(x_i^\\top w)\n",
    "$$\n",
    "Here,\n",
    "$\\sigma(x)$ is the sigmoid function defined as\n",
    "\\begin{eqnarray}\n",
    "\\sigma(x) & = & \\frac{1}{1+e^{-x}}\n",
    "\\end{eqnarray}\n",
    "\n",
    "To understand logistic regression as a generative model, consider the following metaphor: assume that for each data instance $x_i$, we select a biased coin with probability $p(y_i = 1| w, x^\\top_i) = \\pi_i = \\sigma(x_i^\\top w)$, throw the coin and label the data item with class $y_i$ accordingly. \n",
    "\n",
    "Mathematically, we assume that each label $y_i$, or more precisely the answer to our yes-no question rearding the object $i$ with feature vector $w$ is drawn from a Bernoulli distribution. That is: \n",
    "\\begin{eqnarray}\n",
    "\\pi_i & = & \\sigma(x_i^\\top w) \\\\\n",
    "y_i & \\sim &\\mathcal{BE}(\\pi)\n",
    "\\end{eqnarray}\n",
    "\n",
    "Here, we think of a biased coin with two sides denoted as $H$ (head) and $T$ (tail) with probability of side $H$ as $\\pi$, and consequently the probability of side $T$ with $1-\\pi$. \n",
    "\n",
    "We denote the outcome of the coin toss with the random variable $y \\in \\{0, 1\\}$. \n",
    "\n",
    "For each throw $i$, $y_i$ is the answer to the question 'Is the outcome heads?'. We write the probability as $p(y = 1) = \\pi$ and probability of tails is $p(y = 0) = 1-\\pi$. More compactly, the probability of the outcome of a toss, provided we know $\\pi$, is written as\n",
    "\\begin{eqnarray}\n",
    "p(y|\\pi) = \\pi^y(1-\\pi)^{1-y}\n",
    "\\end{eqnarray}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Maximum Likelihood\n",
    "\n",
    "Maximum likelihood (ML) is a method for choosing the unknown parameters of a probability distribution, given some data that is assumed to be drawn from this distribution. The distribution itself is referred as the probability model, or often just the model.\n",
    "\n",
    "#### Example\n",
    "Suppose we are given only $5$ outcomes when a coin is thrown:\n",
    "$$\n",
    "H, T, H, T, T\n",
    "$$\n",
    "\n",
    "What is the probabilty that the outcome is, say heads $H$ if we know that the coin is biased ?.\n",
    "\n",
    "One reasonable answer may be the frequency of heads, $2/5$.\n",
    "\n",
    "The ML solution coincides with this answer. For a derivation, \n",
    "we define $y_i$ for $i = 1,2,\\dots, 5$ as\n",
    "\n",
    "$$\n",
    "y_i  = \\left\\{ \\begin{array}{cc} 1 & \\text{coin $i$ is H} \\\\ 0 & \\text{coin $i$ is T}  \\end{array} \\right. \n",
    "$$\n",
    "hence \n",
    "$$\n",
    "y = [1,0,1,0,0]^\\top\n",
    "$$\n",
    "\n",
    "If we assume that the outcomes were independent, the probability of observing the above sequence as a function of the parameter $\\pi$ is the product of each individual probability\n",
    "$$\n",
    "\\Pr\\{y = [1,0,1,0,0]^\\top\\} = \\pi \\cdot (1-\\pi) \\cdot \\pi \\cdot (1-\\pi) \\cdot(1-\\pi) \n",
    "$$\n",
    "\n",
    "We could try finding the $\\pi$ value that maximizes this function. We will call the corresponding value as the maximum likelhood solution, and denote it as $\\pi^*$. \n",
    "\n",
    "It is often more convenient to work with the logarithm of this function, known as the loglikelihood function.\n",
    "\n",
    "$$\n",
    "\\mathcal{L}(\\pi) = 2 \\log \\pi + 3 \\log (1-\\pi)\n",
    "$$\n",
    "For finding the maximum, we take the derivative with respect to $\\pi$ and set to zero.\n",
    "$$\n",
    "\\frac{d \\mathcal{L}(\\pi)}{d \\pi} = \\frac{2}{\\pi^*} -  \\frac{3}{1-\\pi^*} = 0 \n",
    "$$\n",
    "When we solve we obtain $$ \\pi^* = \\frac{2}{5} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More generally, when we observe $y_i$ for $i=1 \\dots N$, the loglikelihood is\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\mathcal{L}(\\pi)& = & \\log \\left(\\prod_{i : y_i=1} \\pi \\right) \\left(\\prod_{i : y_i=0}(1- \\pi) \\right) \\\\\n",
    "& = & \\log \\prod_{i = 1}^N \\pi^{y_i} (1- \\pi)^{1-y_i} \\\\\n",
    "& = & \\log  \\pi^{ \\sum_i y_i} (1- \\pi)^{\\sum_i (1-y_i) } \\\\\n",
    "& = & \\left(\\sum_i y_i\\right) \\log \\pi + \\left(\\sum_i (1-y_i) \\right) \\log (1- \\pi) \n",
    "\\end{eqnarray}\n",
    "\n",
    "If we define the number of observed $0$'s and $1$'s by $c_0$ and $c_1$ respectively, we have \n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\mathcal{L}(\\pi)& = & c_1 \\log \\pi + c_0 \\log (1- \\pi) \n",
    "\\end{eqnarray}\n",
    "\n",
    "Taking the derivative and setting to $0$ results in\n",
    "\n",
    "$$\n",
    "\\pi^* = \\frac{c_1}{c_0+c_1} = \\frac{c_1}{N} \n",
    "$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pylab as plt\n",
    "\n",
    "from ipywidgets import interact, interactive, fixed\n",
    "import ipywidgets as widgets\n",
    "from IPython.display import clear_output, display, HTML\n",
    "from matplotlib import rc\n",
    "\n",
    "import scipy as sc\n",
    "import scipy.optimize as opt\n",
    "\n",
    "mpl.rc('font',**{'size': 20, 'family':'sans-serif','sans-serif':['Helvetica']})\n",
    "mpl.rc('text', usetex=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "def sigmoid(x):\n",
    "    return 1/(1+np.exp(-x))\n",
    "\n",
    "def dsigmoid(x):\n",
    "    s = sigmoid(x)\n",
    "    return s*(1-s)\n",
    "\n",
    "def inv_sigmoid(p=0.5):\n",
    "    xs = opt.bisect(lambda x: sigmoid(x)-p, a=-100, b=100)\n",
    "    return xs\n",
    "\n",
    "def inv_sigmoid1D(w, b, p=0.5):\n",
    "    xs = opt.bisect(lambda x: sigmoid(w*x+b)-p, a=-100, b=100)\n",
    "    return xs\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting the Sigmoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "513043a8e74249ae806631bfe93f689c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>interactive</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "interactive(children=(FloatSlider(value=1.0, description='w', max=5.0, min=-5.0), FloatSlider(value=0.0, description='b', max=10.0, min=-10.0), Output()), _dom_classes=('widget-interact',))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "  \n",
    "fig = plt.figure(figsize=(10,6))\n",
    "ax = fig.gca()\n",
    "ax.set_ylim([-0.1,1.1])\n",
    "x = np.linspace(-10,10,100)\n",
    "ax.set_xlim([-10,10])\n",
    "\n",
    "ln = plt.Line2D(x, sigmoid(x))\n",
    "ln2 = plt.axvline([0], ls= ':', color='k')\n",
    "ln_left = plt.axvline([0], ls= ':', color='b')\n",
    "ln_right = plt.axvline([0], ls= ':', color='r')\n",
    "\n",
    "ax.add_line(ln)\n",
    "plt.close(fig)\n",
    "ax.set_xlabel('$x$')\n",
    "ax.set_ylabel('$\\sigma(wx + b)$')\n",
    "\n",
    "def plot_fun(w=1, b=0):\n",
    "    ln.set_ydata(sigmoid(w*x+b))\n",
    "    if np.abs(w)>0.00001:\n",
    "        ln2.set_xdata(inv_sigmoid1D(w,b,0.5))\n",
    "        ln_left.set_xdata(inv_sigmoid1D(w,b,0.25))\n",
    "        ln_right.set_xdata(inv_sigmoid1D(w,b,0.75))\n",
    "    display(fig)\n",
    "    \n",
    "res = interact(plot_fun, w=(-5, 5, 0.1), b=(-10.0,10.0,0.1)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'D' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-5-bd68f9235c4a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0mN\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m20\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[0;31m# Some random features\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 11\u001b[0;31m \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mN\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mD\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     12\u001b[0m \u001b[0mX\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'D' is not defined"
     ]
    }
   ],
   "source": [
    "\n",
    "\n",
    "def LR_loglikelhood(X, y, w):\n",
    "    tmp = X.dot(w)\n",
    "    return y.T.dot(tmp) - np.sum(np.log(np.exp(tmp)+1))\n",
    "\n",
    "w = np.array([0.5, 2, 3])\n",
    "\n",
    "N = 20\n",
    "# Some random features\n",
    "X = 2*np.random.randn(N,D)\n",
    "X[:,0] = 1\n",
    "\n",
    "# Generate class labels\n",
    "pi = sigmoid(np.dot(X, w))\n",
    "y = np.array([1 if u else 0 for u in np.random.rand(N) < pi]).reshape((N))\n",
    "\n",
    "\n",
    "xl = -5.\n",
    "xr = 5.\n",
    "yl = -5.\n",
    "yr = 5.\n",
    "\n",
    "\n",
    "fig = plt.figure(figsize=(5,5))\n",
    "\n",
    "plt.plot(X[y==1,1],X[y==1,2],'xr')\n",
    "plt.plot(X[y==0,1],X[y==0,2],'ob')\n",
    "\n",
    "ax = fig.gca()\n",
    "ax.set_ylim([yl, yr])\n",
    "ax.set_xlim([xl, xr])\n",
    "\n",
    "ln = plt.Line2D([],[],color='k')\n",
    "ln_left  = plt.Line2D([],[],ls= ':', color='b')\n",
    "ln_right = plt.Line2D([],[],ls= ':', color='r')\n",
    "\n",
    "ax.add_line(ln)\n",
    "ax.add_line(ln_left)\n",
    "ax.add_line(ln_right)\n",
    "plt.close(fig)\n",
    "ax.set_xlabel('$x_1$')\n",
    "#ax.grid(xdata=np.linspace(xl,xr,0.1))\n",
    "#ax.grid(ydata=np.linspace(yl,yr,0.1))\n",
    "ax.set_ylabel('$x_2$')\n",
    "ax.set_xticks(np.arange(xl,xr))\n",
    "ax.set_yticks(np.arange(yl,yr))\n",
    "ax.grid(True)\n",
    "def plot_boundry(w0,w1,w2):\n",
    "    if w1 != 0:\n",
    "        xa = -(w0+w2*yl)/w1\n",
    "        xb = -(w0+w2*yr)/w1\n",
    "        ln.set_xdata([xa, xb])\n",
    "        ln.set_ydata([yl, yr])\n",
    "\n",
    "        xa = -(-inv_sigmoid(0.25) + w0+w2*yl)/w1\n",
    "        xb = -(-inv_sigmoid(0.25) + w0+w2*yr)/w1\n",
    "        ln_left.set_xdata([xa, xb])\n",
    "        ln_left.set_ydata([yl, yr])\n",
    "\n",
    "        xa = -(-inv_sigmoid(0.75) + w0+w2*yl)/w1\n",
    "        xb = -(-inv_sigmoid(0.75) + w0+w2*yr)/w1\n",
    "        ln_right.set_xdata([xa, xb])\n",
    "        ln_right.set_ydata([yl, yr])\n",
    "        \n",
    "        \n",
    "    elif w2!=0:\n",
    "        ya = -(w0+w1*xl)/w2\n",
    "        yb = -(w0+w1*xr)/w2\n",
    "        ln.set_xdata([xl, xr])\n",
    "        ln.set_ydata([ya, yb])\n",
    "\n",
    "        ya = -(-inv_sigmoid(0.25) + w0+w1*xl)/w2\n",
    "        yb = -(-inv_sigmoid(0.25) + w0+w1*xr)/w2\n",
    "        ln_left.set_xdata([xl, xr])\n",
    "        ln_left.set_ydata([ya, yb])\n",
    "\n",
    "        ya = -(-inv_sigmoid(0.75) + w0+w1*xl)/w2\n",
    "        yb = -(-inv_sigmoid(0.75) + w0+w1*xr)/w2\n",
    "        ln_right.set_xdata([xl, xr])\n",
    "        ln_right.set_ydata([ya, yb])      \n",
    "        \n",
    "    else:\n",
    "        ln.set_xdata([])\n",
    "        ln.set_ydata([])\n",
    "        \n",
    "    ax.set_title('$\\mathcal{L}(w) = '+str(LR_loglikelhood(X, y, np.array([w0, w1, w2])))+'$')\n",
    "    display(fig)\n",
    "\n",
    "res = interact(plot_boundry, w0=(-3.5, 3, 0.1), w1=(-3.,4,0.1), w2=(-3.,4,0.1)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic Regression: Learning the parameters\n",
    "\n",
    "The logistic regression model is very similar to the coin model. The main difference is that for each example $i$, we use a specific coin with a probability $\\sigma(x_i^\\top w)$ that depends on the specific feature vector $x_i$ and the parameter vector $w$ that is shared by all examples. The likelihood of the observations, that is the probability of observing the class sequence is\n",
    "\n",
    "$\\begin{eqnarray}\n",
    "p(y_1, y_2, \\dots, y_N|w, X ) &=& \\left(\\prod_{i : y_i=1} \\sigma(x_i^\\top w) \\right) \\left(\\prod_{i : y_i=0}(1- \\sigma(x_i^\\top w)) \\right)\n",
    "\\end{eqnarray}\n",
    "$\n",
    "\n",
    "Here, the left product is the expression for examples from class $1$ and the right product is for examples from class $0$.\n",
    "We will look for the particular setting of the weight vector, the maximum likelihood solution, denoted by $w^*$.\n",
    "\n",
    "$\n",
    "\\begin{eqnarray}\n",
    "w^* & = & \\arg\\max_{w} {\\cal L}(w)\n",
    "\\end{eqnarray}\n",
    "$\n",
    "\n",
    "where the loglikelihood function\n",
    "\n",
    "$\n",
    "\\begin{eqnarray}\n",
    "{\\cal L}(w) & = & \\log p(y_1, y_2, \\dots, y_N|w, x_1, x_2, \\dots, x_N ) \\\\\n",
    "& = & \\sum_{i : y_i=1} \\log \\sigma(x_i^\\top w) + \\sum_{i : y_i=0} \\log (1- \\sigma(x_i^\\top w)) \\\\\n",
    "& = & \\sum_{i : y_i=1} x_i^\\top w - \\sum_{i : y_i=1} \\log(1+e^{x_i^\\top w}) - \\sum_{i : y_i=0}\\log({1+e^{x_i^\\top w}}) \\\\\n",
    "& = & \\sum_i y_i x_i^\\top w - \\sum_{i} \\log(1+e^{x_i^\\top w}) \\\\\n",
    "& = & y^\\top X w - \\mathbf{1}^\\top \\text{logsumexp}(0, X w)\n",
    "\\end{eqnarray}\n",
    "$\n",
    "\n",
    "$\\mathbf{1}$ is a vector of ones; note that when we premultiply a vector $v$ by $\\mathbf{1}^T$ we get the sum of the entries of $v$, i.e. $\\mathbf{1}^T v = \\sum_i v_i$.\n",
    "\n",
    "We define the function $\\text{logsumexp}(a, b)$ as follows: When $a$ and $b$ are scalars, \n",
    "$$\n",
    "f = \\text{logsumexp}(a, b) \\equiv \\log(e^a + e^b)\n",
    "$$\n",
    "\n",
    "When $a$ and $b$ are vectors of the same size, $f$ is the same size as $a$ and $b$ where each entry of $f$ is\n",
    "$$\n",
    "f_i = \\text{logsumexp}(a_i, b_i) \\equiv \\log(e^{a_i} + e^{b_i})\n",
    "$$\n",
    "\n",
    "Unlike the least-squares problem, an expression for direct evaluation of $w^*$ is not known so we need to resort to numerical optimization. \n",
    "\n",
    "Before we proceed, it is informative to look at the shape of $f(x) = \\text{logsumexp}(0, x)$.\n",
    "When $x$ is negative and far smaller than zero, $f = 0$ and for large values of $x$, $f(x) = x$. Hence it looks like a so-called hinge function $h$\n",
    "$$\n",
    "h(x) = \\left\\{ \\begin{array}{cc} 0 & x < 0 \\\\x & x \\geq 0  \\end{array} \\right.\n",
    "$$\n",
    "\n",
    "We define\n",
    "$$\n",
    "f_\\alpha(x) = \\frac{1}{\\alpha}\\text{logsumexp}(0, \\alpha x)\n",
    "$$\n",
    "When $\\alpha = 1$, we have the original logsumexp function. For larger $\\alpha$, it becomes closer to the hinge loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsMAAAJOCAYAAABSogpEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlcVXX+x/HXUVFBTdz3DaufZosC\nNjPZYgXVVNM2qNO0NylTWWObZKWljQtUWmkLWFnNWJnY1LQHltoylUBapuWCG64IXlGU/fv74wC5\nsHvvPZd738/H4z7g3nPuOW8jvnzu93zP92sZYxARERERCURNnA4gIiIiIuIUFcMiIiIiErBUDIuI\niIhIwFIxLCIiIiIBS8WwiIiIiAQsFcMiIiIiErBUDIuIiIhIwFIxLCIiIiIBS8WwiIiIiASsZt48\nWceOHU3fvn29eUoREbdIT0/fY4zp5HSO2liWFVX+bbQxJq6K7TGACwg3xiTUdCy12SLSmNW13fZq\nMdy3b1/S0tK8eUoREbewLGuz0xlqU14IjzDGxFqWFWdZVrgxJuOw7eEAxphUy7LCjt5+NLXZItKY\n1bXd1jAJERE/YYxJNcbElj8Nq6LQHYXdKwyQCUQhIhLgVAyLiPgZy7LGA7FVbAoFcg973qGK946x\nLCvNsqy07OxsT0UUEfEZtRbDlmVFlT/iq9keU759vPvjiYhIfZWPBY61LCu0Ae9NMsZEGmMiO3Xy\n+SHSIiLHrcYxw+4ef1aV4uJisrKyKCgoaNA/QDynZcuW9OzZk6CgIKejiEgdHNYmZ2APgxgDHH6T\nnAtoX/59KJBT33OozfZvavclENVYDBtjUoHU8qfVjT9LKf++YvxZvYrhrKws2rRpQ9++fbEsqz5v\nFQ8yxpCTk0NWVhb9+vVzOo6I1M3hbXAosBzAsqxQY4wLWABElm8P47f2vc7UZvsvtfsSqOo0ZtiT\n488KCgro0KGDGlUfY1kWHTp0UO+PSOOSBIRZljUGwBiTXP764vLnGVB51c9V3yt5oDbbn6ndl0BV\np6nVjDEJlmUttCwrrbx3oc6MMUnYDTSRkZGmqn3UqPom/VxEGpfy9jmpitcjDvv+mO31pbbBf+ln\nK4Goxp5hy7LCK8ag8dv4s8Md9/gzERGvMlV+JhcREV/khTa7tmESURxZ7GaCPf6s/LUF2OPOoIHj\nz5yWnJxMXNwxizQ1OhX/hri4OKKjo4mIiCAzM7NO701KSiIjo95XS0Uan19/hfBwWLHC6STSQIHY\nZsfGxhIdHU3//v1JTk6ucp/DVdWmV5yvvscScVJZQQH7IyNhwQKPnqe2Ytjj48/k+CUkJDBq1Cgy\nMjLIyMggJSWFuXPnEhtb1TDvY40ZM4bp06d7OKWID5g+3S6Iu3d3OokEsPq02ampdh9TSkoK6enp\njB49utbjH92mZ2RkMHTo0AYdS8RJTZKTycrI8Piwg9pmk/DK+LNK48a5v8dm8GB4+uk67TpixAhc\nLhe5ubnEx8cTFRV1xOthYWGkpaWRnp5OZmZmZcMVGhrK3LlzSU1NJTMzk/Hj7SmXIyIiSE9PJzU1\nlfj4eEJDQ8nIyCA2NpaUlBRyc3OZO3cu4eHhxMbGVi57eskll7Bnzx4SExMZMWIEsbGxZGRksGHD\nBjIzM8nNzWXChAnExMQAdsM2fvx4EhISiI6OBiA8PLzKZVSTk5NZUP4JKzU1lblz5xITE0P79u3J\nzMwkLCzsmPeI+IXMTPj3v+Huu6FzZ6fT+Ae12YBn2+ywsLDKXt3Q0FDat29/xPa6tOkLFiwgPj6e\nzMzMGo8l4lOMgSefZOApp8DIkR49VZ1uoAsECQkJhIWFER8fj8vlol+/fuzdu5e4uDiGDh3K+PHj\nSU1NrfxknZycTHh4OPHx8aSmppKbm1vLGWDhwoUkJyeTmJhISkpKZSNW0QCmp6fjcrmIiIggPDyc\nuLg42rdvT1RUFBkZGWRmZpKSYs9k165dO2JiYnC5XISG2qNWcnJyGDp0aI0ZYmJiiImJITU1FZfL\nVdk4R0REkJGRoWJY/NeMGdCsGdx/v9NJxA0Cpc2uaJMzMzMZMWLEMUNE6tOm13YsEV+S9vzzvLdy\nJffPmUNbD9/Y6VvFcB17Azxhw4YNjBgxAqCyoQK70Rg1ahRAZa8D/HYZKjo6urJBrkl4eHjlsSsa\npNDQUFwuF+np6aSlpR1x/vj4ePr378/evXsrj1HRgwAQGRlJZmYmLper8tN9hw4djhhz5nJVPfFH\nRkYGcXFxpKenV74WFhZGSkpKZUMq4le2bIFXX8V1440Ed+hAC6fz+Au12ZWve7LNTkhIYMGCBZW9\n0kerqU0PCws7IkdtxxLxFUuefZbnLYvx5b/PnlSneYYDQf/+/Ss/wR/eOIWFhVX2LFR8BXj77bcZ\nNWoUKSkp9O/fn6Qke7RITo49sqU+NyZEREQQFRXFwoULWbhwIaNGjSI2Npb4+PgjxnRV5ANIS0sj\nLCyMsLCwyh6OqKioyn0yMjKO+ENQweVyMXr0aBYvXnzE65mZmbX2UIg0Wgn2Imw7//pXxo0bR2Fh\nocOB5HgFSpudmppaOca3quK1tjZ9wYIFlcet7VgiPuPHH7l/7Vo2TJxIm44dPX463+oZdtD48eMZ\nMWIE0dHR5ObmsnDhQgAmTJjAiBEjSElJOaLxiIyMZMSIEYSFhVX2CoSFhZGYmEh0dHS9GpoxY8ZU\nnhugc+fOREdHM378eGJjY49opCvyzZ07F/itpwLsnozw8PDK4yQmJgJ2oxgREcHevXsZPXp0ZeMJ\nMGrUKGJiYkhPT9clM/FPO3bASy/BTTcx4IILeOGCC5xOJG4QKG12SkoKaWlp9O/fv/KYGzZsqPy+\ntjZ9+fLllfvWdiwRX1GQkEDLVq0IHTfOOyc0xnjtERERYY62evXqY17zJSkpKSYlJcUYY0x6erqJ\niopyJEd8fLxJTEysdlt6enqN76/uvRViYmKqfN3Xfz4itbr3XmOaNjVL33jD7Nmzp8GHAdKMF9tL\nX3iozW44T7fZtamuTa8LX/8Zi3/L+fFH0wHMq9HRx32surbbGiZRi8jIyMo7hOPi4io/ufuS8ePH\nV95NXJ2aboxLSkpiwoQJ7o4l4rzdu+HFFykaNYqYf/xDU0kFgEBos2ujNl0as6IXXuDPlsWQe+7x\n2jkt48XVmCIjI83RU8esWbOGgQMHei2D1I9+PtKoPfigPV549Wp+Li0FYNCgQQ06lGVZ6caYSHfG\n83VqswOTfsbimLw86NUL/vhHeOut4z5cXdttjRkWEf+UkwPPPQejRsGAATSsBBYREW/55uGH6ZiX\nx8lengJTwyRExD898wwcOMC7v/89Y8eOZf/+/U4nEhGR6hQXc3diIqNat8ZERNS+vxupZ1hE/I/L\nZRfD11zDmoMHWbJkCSEhIU6nEhGR6rz9Nh8WF7P9ySexPLzIxtHUMywi/mf2bHvs2cSJTJgwgR9+\n+IGmTZs6nUpERKpSvvRyl4EDGTJ2rNdPr2JYRPzL/v0waxb86U/s7NoVgKCgIIdDiYhIdX5ITOSa\nFSvYcsst0MT7pWnAF8OpqalVLjaRnJxMQvmqVe5S3aIWSUlJZGRkuPVcIgHruedg715+HDWKnj17\n8s477zidSNxIbbaI/9nw4otkNGnCCTfe6Mj5fa4YHj58OK+++ioAxcXFDB8+nH//+98AHDx4kOHD\nh1fOz7hv3z6GDx9e+cduz549DB8+nPfffx+AnTt3NjhHTEwM48ePP45/yZESEhIYVc362mPGjGH6\n9OluO5dIwDpwAJ56Ci65hC5RUYwfP57zzz/f6VR+TW22iByXH38kZuVKNjz2GKFdujgSweeKYSdk\nZGQQHR1NRERE5Xr1ycnJxMXFkZqaWrnsZkRExBHLbFa8HhsbS8Rhdz5WPI+IiKjsPahYGjQ5OZkR\nI0YwYsQI2rVrV3m89u3bk5mZ6cV/tYgfeuEF2LMHJk2iS5cuTJs2jXbt2jmdStxMbbaI/8iaMgVa\ntaLpnXc6lsHnZpNYsmRJ5fdBQUFHPA8JCTniedu2bY943rFjxyOedy0fL1ib3Nxc0tPTcblcRERE\nMGbMmCO2Z2ZmHrE9JiaGuLg4hg4dyvjx40lNTSU1NRWgsmE+fP/09HRCQ0MBu/ciJiaG1NRUXC4X\nMTExAJWN8PGsOiQS0A4ehCeegOhoXl+3jlOCgoiMDKg1MhyhNltttkhD7Vu9mlMWLeKeM89kcvv2\njuVQzzAQFRUFUNn41WV7ZmZm5esVX8FuUNPS0hgxYgSjR48mNDSUzMxM2h/2Q87IyCAuLo6UlJTK\n18LCwli+fLn7/lEigSYxEbKzKX7oIR5++GGeeeYZpxOJh6jNFvEPzefOZZplcc2jjzqaw+d6hhuL\nsLAwUlNTCQ8Pr+xhALu3IDQ0lPj4eMAedxYWFkZubi4ALpeL0aNHs3jx4iOOl5mZydChQ733DxDx\nJ4cO2csun38+QcOH8/PPP5Ofn+90KvEharNFfExeHsGvvMLYkSPh0ksdjaKe4QaaMGECKSkpREdH\nH9FbMGbMGDIzM4mOjiY6OpqwsDBCQ0NxuVwAjB49urJxHTFiROX4s/T0dMLDwx35t4g0ei+9BDt3\nYiZNAuCEE06gW7duDocSX6I2W8S3pDzwAO/m5VF2771ORwFjjNceERER5mirV68+5rXGICUlxaSk\npBhjjElPTzdRUVE17h8fH2/S09Or3R4TE+PWfO7SWH8+EkAOHTKme3djzjnHJCYmmvPOO8/s3bvX\n7acB0owX20tfeKjNbnxttjs01p+xNCJFReaKli3NwJAQU1pa6rHT1LXd1jCJBoqMjGT06NEkJibi\ncrlITEyscf/x48cTFxdXZU9CUlISEyZM8FRUEf/28suwfTu8/johO3bQrl072rZt63Qq8TFqs0V8\nyIIFLCooIOvll2niwCIbR7Pswtk7IiMjTVpa2hGvrVmzhoEDB3otg9SPfj7i0woLoX9/6NcPli0D\nD65nb1lWujEmoKanUJsdmPQzFo8yBjN4MFZxMaxa5dEV5+rabjtfjmMP1RDfo5+L+LyXX4Zt2zCT\nJrF02TLKysqcThQQ1Db4L/1sxdNWzZ3LwB9/JGPECEeWXq6K4ylatmxJTk6OfgF9jDGGnJwcWrZs\n6XQUkaoVFsL06XDWWXzdsiXDhw9n/vz5Tqfye2qz/ZfaffGG/JdfpktQEH1iY52OUsnxMcM9e/Yk\nKyuL7Oxsp6PIUVq2bEnPnj2djiFStXnzICsLXn6Z3/3+98yfP5+rr77a6VR+T222f1O7Lx61ciW/\n+/57lk6bBt27O52mkuNjhkVE6q2oCE46yW5Mv/nGo2OFK2jMsIjI8Vl+6aWcunQpwVlZ0K6dx8/X\nqMYMi4jUy6uvwpYt8OijTJ4yhTfeeMPpRD7Dsqwx5Y/4arbHV+zn3WQiEsjy167loo8/5o4+fbxS\nCNeHimERaVyKimDqVPj97ym58EI+/PBDvv32W6dT+QTLsqKAVGNMEhBW/vxoYyzL2gBkejediASy\nkKQk/tOkCffNmuV0lGM4PmZYRKReKnqFk5JoFhTEd999R0FBgdOpfEVY+SMJu9gNq2Kf0caYZK+m\nEpHAlpeHNXcuw0eOhIsvdjrNMWrtGdYlNxHxGYf1Ch865xwKCwuxLIvg4GCnk/kEY0xSea8wQDhQ\n1YDfMMuyoizLGl/VMcrb+zTLstJ0k5yIuMMH48bxeF4eBXfd5XSUKtVYDOuSm4j4lIpe4cce49nZ\nswkLCyM3N9fpVD7HsqxwIMMYk3H0NmNMgjEmFehQVZteXlBHGmMiO3Xq5I24IuLPiopYtnAh/woO\nJuh3v3M6TZVqGyahS24i4hsO6xXmoov4Q3Aw+/bto3379k4n80VRxpi4o18sv4KXW95m51B1my4i\n4j5vvUXCgQNMWrSIpk2bOp2mSjUWw4ddbgP7ktuCKnar6DEON8YkHL2xvPEdA9C7d+/jiCoiAe2w\nscJYFueeey7nnnuu06l8jmVZYyraYsuyoowxqZZlhRpjXNjDJiqu4vUHEp3KKSIBwBjyExJodeqp\ntPbheeDrNJuELrmJiKMKC+1e4d/9jrKoKJ5//nlcLpfTqXxOeRscb1nWBsuy9h62aTFAeRs+0rKs\nGGBDVW26iIi7/Pryy3T9+Wc+jIryynzwDVXX2SR0yU1EnPPKK5W9wt/873/ceeedtGvXjmuvvdbp\nZD6lvGPimAk8jTERh32fdPR2ERFPaDFvHqNCQoi8916no9So1mJYl9xExFGFhTBtGpx1Flx0EWdb\nFitWrGDQoEFOJxMRkepkZND3m294acYM6NXL6TQ1qstsErrkJiLOeeklyMqCyZMrL7OdccYZNGum\nadJFRHzVx/fdR2ZICMTGOh2lVrXdQKdLbiLinIICu1f47LPhwgu54YYbOPnkk5k4caLTyUREpBql\nmZnctmQJkWFhvBca6nScWqlrRUR819y5sH07/OtflJaVOZ1GRETqoOns2XzfpAkHX3nF6Sh1omJY\nRHzToUN2r/C558L559PUsvjXv/7ldCoREanJ3r0wdy49rr0WzjvP6TR1Uqep1UREvC4xEXbuhMmT\n2ZOTw5YtW5xOJCIitfhg3DhG5eeTM3q001HqTMWwiPie/HyYPh0uuACGD+fJJ5/k5JNPZs+ePU4n\nExGR6hQWsuPdd1nbpg1thw1zOk2daZiEiPie55+H3bvtGSSAO+64g4EDB9KxY0eHg4mISLXmz2d0\nXh63ffopViOa8Uc9wyLiW/bvh/h4uOgiexYJ7KXcb7rpJoeDiYhItcrKyJw2DQYPxoqOdjpNvagY\nFhHfMmcO5OTAlCkUFRVx55138ssvvzidSkREapD5yiuctGEDLwwe7NNLL1dFxbCI+I68PHjiCbjs\nMvjd71i5ciWvv/46mzdvdjqZiIjUoNO8eTwRGsqVjz3mdJR6azwDOkTE/z39tD0tT/lY4aFDh7J1\n61batm3rcDAREanW99/T5ptvuHfmTOjTx+k09aaeYRHxDXv3wsyZcOWVEBHBoUOHAAgNDcVqZJfc\nREQCyVt3382HISFw221OR2kQFcMi4hueegr27YMpUwC45JJL+Nvf/uZwKBERqYlZt45Z333H7C5d\noE0bp+M0iIZJiIjzsrPtIRKjRsHpp1NWVsall15K586dnU4mIiI1sGbN4qugIPa8847TURpMxbCI\nOC8hwV5+ufzGiyZNmhAXF+dsJhERqZHZvRvzyisE3Xgj3QYPdjpOg2mYhIg4a8cOezq166+HAQPY\nuHEjn332GcYYp5OJiEgNPrnnHk4rLGTDiBFORzkuKoZFxFnTpkFJCUyaBMCcOXO44ooryM7OdjiY\niIhU6+BBgt5/nz6dO9P7ggucTnNcVAyLiHO2bIGkJLjlFujfH4Bp06axePFijRcWEfFl8+YRtX8/\nHy1aRFBQkNNpjouKYRFxzuOP218feaTypRYtWjBs2DCHAomISK1KSlg6dSqFZ54JftBeqxgWEWes\nWwfz5sHf/w69e7N//36GDx/O0qVLnU4mIiI12PbSS1y4YweP9+7d6JZeropmkxARZzz2GLRoAQ89\nBMCWLVvYvXs3LVu2dDaXiIhUzxi6z53Lx927M/DJJ51O4xYqhkXE+1atgjffhLg46NIFgEGDBvHz\nzz9rtTkREV+2ZAlWRgbRiYmNcunlqmiYhIh438SJ9kpFDzwAwKZNmyguLlYhLCLi416++27iW7Wi\n7PrrnY7iNiqGRcS7li+Hd9+F+++H9u0xxnDFFVdwxRVXOJ1MRERq8tNPfLlqFZ927kyTkBCn07iN\nhkmIiHc98gh07AjjxlW+NH36dJo1U3MkIuLTnnySV1u14tCXXzqdxK3010dEvGfJEvjsM3jySXuY\nBGBZFpdddpmzuUREpEZmyxb2z5/PCWPHEtyjh9Nx3ErDJETEO4yBCROgRw+44w4AfvjhB2bOnMnB\ngwcdDiciIjX5/L776FFaynfDhzsdxe1UDIuId3zwAXz7LTz6KAQHA/Df//6XyZMnU1JS4nA4ERGp\nlstFj48+4roTT+SMSy5xOo3bqRgWEc8rK4OHH4aTToKbb658+dFHH2XNmjWccMIJzmXzM5ZljSl/\nxFezPcayrCjLssZ7O5uINFIvvsiAgwd5ceFCv5wLXsWwiHjem2/CTz/BlClQvoZ9RW9w9+7dnUzm\nVyzLigJSjTFJQFj588O3hwMYY1IBV8VzEZFqFRayYMYMtpx9Ngwe7HQaj1AxLCKeVVwMkybZjejI\nkQDs3r2b3r1788477zgczu+EARUFcGb588ONAlyHbY9CRKQGeXPncuu+fSS0b+90FI+pdTYJy7LG\nlH/b3xgTV8X2GOzGNdwYk+DmfCLS2L30EmRm2mOGm9ifvw8dOsTZZ5/NKaec4nA4/1LeI1whHFhw\n1C6hQO5hzzt4PJSINF5lZZwwZw6rBw2i+QsvOJ3GY2oshg+75JZpWdZCy7Kiyi+vVWyvvORmWVaY\nZVnhxpgMD2cWkcYiP98eGnHOOXDppZUv9+nTh7ffftvBYP6tvG3OaEh7XN4BMgagd+/e7o4mIo3J\nf/8Lv/5KnzffBD8e0lbbMAldchORhnvmGdi5E2bMgPKllr/++mu2b9/ucDC/F1XVlTzs9rriWmco\nkHP0DsaYJGNMpDEmslOnTp7MKCK+zBheuOcerm/VioLLL3c6jUfVWAyXN4oVl93CgbSjdtElNxGp\nWk4OxMfDFVfAWWcBYIzhpptu4rrrrnM4nP+yLGtMxZC1ihvoLMsKLd+8gN86NcKA1GOPICICfP01\n+zdtIqd3b1q2bu10Go+q0w10x3vJzbKsNMuy0rKzs+sdUEQaqRkzYP9+mDq18iXLsvjss8+YNWuW\ng8H8V3nxG29Z1gbLsvYetmkxQEUbXr6fS8PaRKRaCQmM79CBj5YvdzqJx9V1OebjuuQGJAFERkaa\nhoQUkUYmKwtmz4YbboBTTz1iU1jY0aOtxF3K7+loV8XrEYd9n3T0dhGRw5mff+bX999nwGOPYbVq\n5XQcj6u1Z1iX3ESk3h57zF5+efLkypc+//xzbr31VnJyjvnMLCIiPuSrBx5gIPBuv35OR/GKGoth\nXXITkXpbvRrmzYPbb4e+fStfXrt2LcuWLaO1n489ExFp1LZt49SUFJ446ywuiolxOo1XWMZ4b+RC\nZGSkSUs7+h48EfErV10Fn39uzy3cseMRm0pKSmjWrK6js3yLZVnpxphIp3N4k9pskQB0//3w9NOw\nbh008p7hurbbWoFORNzn66/hvfcgLu6IQrhiKrXGWgiLiAQEl4s5s2ez+NxzG30hXB8qhkXEPYyB\n8eOhWzcYN67y5Y0bN9KnTx9eeuklB8OJiEhtiufMYWZREW/78dLLVVE3jYi4x3//C998A4mJcNjd\nx6GhoUycOJFLLrnEwXAiIlKjggKC5sxhTXQ0+UmBNemMimEROX4lJfDgg3DyyXDrrUdsateuHZMm\nTXIomIiI1EXpq6/SZNcuWjz4IC0CrGdYwyRE5PjNmwe//ALTp8Nh44IXLFjA0qVLHQwmIiK1Ki1l\n7qRJDAkOJuf0051O43XqGRaR45OfD5Mm2UsuX3115cvGGB5//HH69u3Leeed52BAERGp0X/+Q9fs\nbE475xzad+jgdBqvUzEsIsfnqadg505YtAgsq/Jly7JYvnw5ubm5DoYTEZEaGQPx8Vx14olc9cUX\nR7TjgULDJESk4XbtgoQEuOYau2e4nDEGYwzBwcH06NHDwYAiIlKjL77g47Q0iu+9F5o2dTqNI1QM\ni0jDPfYYFBbaY4UPk5yczJlnnlk5v7CIiPim9Icf5lLg5ZISp6M4RsMkRKRhfvkF5s6Fv//dnkXi\nMM2bN6dz58506dLFoXAiIlKrH35gyLff8v5NN3HezTc7ncYxKoZFpGEefBBCQuDRR4/ZdOWVV3Ll\nlVc6EEpEROosPp4mJ5zA5c88A23aOJ3GMRomISL1t3Spvezygw9Cp05HbFqyZAklAXy5TUSkUdiw\ngWlvv82cM8+Etm2dTuMoFcMiUj9lZXDvvdCrF9xzzxGbVq1axfnnn8+cOXMcCiciInVhnnySr4G0\nAFtgoyoaJiEi9TN/PmRkwL/+BcHBR2waOHAgixYt4vzzz3conIiI1GrXLqxXX+XD226jSJ0XKoZF\npB4OHoSHHoKICPjrX4/Z3LRpU6655hoHgomISF0VzZpFYUEBbR54gObNmzsdx3EaJiEidTdrFmRl\n2QttNDmy+UhISCAxMdGhYCIiUid5efz72WfpFRREZoDOK3w0FcMiUje7dsGMGXDllXDU8srGGBYv\nXsxXX33lUDgREamTF19kyKFDjP7LX+jXr5/TaXyChkmISN1MnAgFBfaKc0exLItPP/2UQ4cOORBM\nRETqpKAAZs1iSHQ0Q15/3ek0PkM9wyJSu5Ur4eWXYezYYxbYKCwsJD8/H4Dgo26oExERH/L66yTt\n3Mn2225zOolPUTEsIjUzxp5KLTQUJk06ZvNrr71G79692bx5swPhRESkTkpL2TR1KrcD8zdtcjqN\nT9EwCRGp2fvvw+efw+zZ0K7dMZvDw8O55ZZb6N27twPhRESkTpKT6btlC+uee46O11/vdBqfYhlj\nvHayyMhIk5aW5rXzichxKiqCQYOgWTP48UcICnI6kWMsy0o3xkQ6ncOb1GaL+AljYMgQOHQIVq+G\nAJlFoq7ttoZJiEj1nnsO1q+HmTOPKYSNMTz//PPs3r3boXAiIlInn3zCQytXMqZrV0wTlX5H038R\nEaladjZMngwXXwx//OMxm9esWcPYsWNJTk52IJyIiNTZ9OmUtWmDOfFELMtyOo3P0ZhhEanaxImQ\nn28vtFGFU045hTVr1tCrVy8vBxMRkTr7+mv48ktmzJoF48Y5ncYnqWdYRI61YgUkJdlTqQ0ceMzm\ninsN/u///o+QkBBvpxMRkToqnjqV1W3bwujRTkfxWSqGReRIxsDdd0OHDvDoo1Xucscdd3Dfffd5\nOZjUlWVZ4TVsiy//OsZ7iUTEET/+yIKPP2bQvn18t2qV02l8lophETnSwoXw5Zcwdao9t/BRjDEE\nBQXRNEDuRm5sLMuKAhbWsMvJKbadAAAgAElEQVQYy7I2AJleiiQiTpkxg0tateKZadMYOnSo02l8\nVp2mVrMsK9wYk1HNtnhjTJxlWWOMMUk1HUfT9Ij4uIMHYcAAu1c4LS1gpt+pi8Y0tZplWSnGmOhq\ntsUYY+p016PabJFGLDMTTjoJ7rkHnnzS6TSOcNvUauplEAkg8fGwdSs880yVhfC+ffv49ddfHQgm\nbhRmWVaUZVnjnQ4iIh4UH89ky+Lrc85xOonPq7UYNsakUnOhO9oY0798PxFprDIz7WL4L3+Bc8+t\ncpcXX3yRgQMHsnHjRi+HE3cxxiSUt9cdyjs7jmBZ1hjLstIsy0rLzs52IKGIHLdt29g7bx7PN2/O\nEo0VrpU7plYLK29Qw40xCW44nog44d577ZXmnnii2l1uvvlmOnXqRL9+/bwYTNyl/Ka53PJhEjlA\n2NH7lA93SwJ7mIR3E4qIW8ycSbuyMjalpVHau7fTaXzecd9AV1svg4g0Ap98Au+9Z88t3LNntbt1\n6dKFW2+91YvBxB0sy6q4EzINqLiK17/8uYj4kz17KHrhBcxf/kLwKafQunVrpxP5vOMqhssvp8WU\nP62yl0GX3ER8XGGhPZXaySdXOyF7WVkZY8eO5YcffvByOKmv8jY58rC2GWAxQPmN0CPLt22o7sZo\nEWnEnn2WRw8d4qxVqygqKnI6TaPQoGESlmWFGmNc2L0KFeOJ+wOJR++rS24iPu7pp2HdOvj4Y2jR\nospd1q9fzxtvvMHZZ5/NkCFDvBxQ6qN8CETyUa9FHPZ9jbP+iEgjlpcHs2dzSkQExcOH07x5c6cT\nNQq1FsOH9zIcNh3PYiDCGJNR3vObi3oZRBqfrVthyhS48kq45JJqdzv55JPZunUrLaoplkVExAe8\n8AK4XNzwwgvcoHmF66zWYli9DCJ+7J577BXnnn662l0OHjxISEgIrVq18mIwERGpl0OHKHnqKd4/\n4wz+NGSIW2ZICBRagU4kUH36KSxaBA8/DH37VrvbtddeyxVXXOG9XCIiUn8vvcQH2dlcs3Iln3zy\nidNpGhV9cBAJRAUFMHasfdPc/fdXu5sxhosuuoi6rFQpIiIOKSyEhAT+NGwYH0yYwCU1DHuTY6kY\nFglETzwB69dDSkq1N80BWJbFnXfe6cVgIiJSb6+/DllZNH3pJS67+GKn0zQ6GiYhEmgyM2HaNBg5\nEqKqnxp8x44dvPvuu5SVlXkxnIiI1EtJCcyYwV2dO/Pajh1Op2mUVAyLBBJj4M477ZXmZs6scddX\nXnmFP//5z2zatMk72UREpP7eeovCzEx+aN+ezI0bnU7TKGmYhEggWbjQXm3u6aehR48ad42Li2P4\n8OGEhR2zlo6IiPiCsjKYNo0Wp57KVytXUqIreQ2iYlgkUOzbB//4B0RE2DfP1aJZs2YMGzbMC8FE\nRKRB3nmH/WvW0HTePEKaNKFZE13wbwj9VxMJFA89BLt3Q2IiNG1a7W5FRUVccMEFfPTRR14MJyIi\n9VJWBo8/zlPt29PngQdwuVxOJ2q01DMsEgi+/95emeiuu+ye4Rps376dvLw8mqiHQUTEd73/Pvz4\nI5dMnEiL4GBCQ0OdTtRoWd6cPzQyMtKkpaV57XwiAhQXw9ChkJ0Na9bACSfU+paKdsGyLE+nazQs\ny0o3xkQ6ncOb1GaL+Chj7HZ971749Vf7pmg5Rl3bbXX9iPi7mTNh5UqYM6fWQnjz5s0cOnQIy7JU\nCIuI+KqPP6Y0PZ0nIyLYnZvrdJpGT8WwiD9bvx4eewyuuQauvrrW3W+88UbOPfdcz+cSEZGGMQYe\nf5zvu3blgYULWbJkidOJGj31q4v4K2NgzBh7hbnZs+v0lsmTJ7Nv3z4PBxMRkQZbvBi+/ZY/vPAC\nv15wgaa/dAMVwyL+at48+OILe/aI7t3r9Jbhw4d7NpOIiDScMTBlCqZ7d6xbbuHkFi2cTuQXNExC\nxB/t2gX33w/nnAO33Vbr7uvWrePxxx9Xr7CIiC/74gv48ktu6tmTuEmTnE7jN1QMi/ijO++Egwch\nKQnqMEXaJ598wrRp0ygoKPBCOBERqTdjYPJkTLdutB48mJCQEKcT+Q0NkxDxN8nJsGgRTJ8OAwbU\n6S133XUXI0aMoEuXLh4OJyIiDbJkCSxbhvXsszx/111Op/Er6hkW8Sd79ti9whER9jCJOiguLgag\na9eunkwmIiLHY/JkXF26sFb3dridimERfzJunD0J+7x5dZqE/cCBA4SFhTFv3jwvhBMRkQZZsgSW\nLuX5yEgGDh7Mli1bnE7kVzRMQsRfvP8+zJ9vzyt82ml1ekt+fj5RUVEMGjTIs9lERKThJk+Grl25\ndfZsui9dSu/evZ1O5FdUDIv4g9xciI21i+AJE+r8ti5duqhXWETEly1davcMz5pF1379uLlfP6cT\n+R0NkxDxB3ffDdnZ8Npr0Lx5nd7y/fffk5mZ6eFgIiLSYMbApEmUde3K3WvXkpGR4XQiv6RiWKSx\ne+cde3jExIkwZEid33bnnXdy9dVXY4zxYDgREWmwzz+HZcvIHDOG+QsWsHr1aqcT+SXLm38IIyMj\nTVpamtfOJ+L3du+GU0+F3r3hf/+DoKA6v3Xbtm1s376doUOHejCg/7AsK90YE+l0Dm9Smy3iIGNg\n2DDYuhXWrSO/tJTmzZsTVI92PtDVtd3WmGGRxsoYuP122LfPHh5RzwayR48e9OjRw0PhRETkuHz6\nKfzvfxQ8+ywtW7akldN5/JiGSYg0VvPn20MkHn8c6jEbRFpaGiNHjmTbtm0eDCciIg1WPlaY3r35\n2zff8Mc//lFD2jxIPcMijdHmzfbiGsOGwX331eut69at47vvvqNNmzYeCiciIsflww9h+XKYO5ez\ni4s5cOAAlmU5ncpvacywSGNTWgoXXAA//AArV0IDptkpLS2ladOmHgjnvzRmWES8oqzMXkV03z74\n9dd6D4GT39S13dYwCZHG5sknYdkymDOn3oVwVlYWgAphP2dZVngN22Isy4qyLGu8NzOJSB0tWgQr\nVrAvLo73PvqIsrIypxP5vToVw2pYRXzEDz/YU6jFxMANN9TrrdnZ2Zx00kkkJCR4KJz4AsuyooCF\n1WwLBzDGpAKumtp2EXFAaak9VviUU/hXYSFXXXUVP/30k9Op/F6txbAaVhEfcfAgXHcddOoEL74I\n9Rw/FhISwtSpU7nyyis9FFB8QXl7XN1qKqMAV/n3mUCUV0KJSN38+9/wyy8wZQp/v+MOUlJSOOOM\nM5xO5fdqvYHOGJNqWVZNDWtK+fcVDauWRxHxhHvusRvJzz6DDh3q/fZWrVpx7733eiCYNCKhQO5h\nz+v/P5KIeEZRETz2mD1e+JpraGZZREXp86o3HO+YYTWsIt6QnAxJSTB+PDSgcXz33Xf56KOPNDWP\n1MqyrDGWZaVZlpWWnZ3tdByRwPHyy7BpE+bxxxkxciTvvPOO04kChm6gE/F1mzfD6NFw5pn2nMIN\nMHPmTKZOnaqpecQFtC//PhTIOXoHY0ySMSbSGBPZqVMnr4YTCViHDsE//wlnn01OZCQbN25k7969\nTqcKGMc7z3CtDatlWWOAMQC9e/c+ztOJBJiSEnuccGkpvPFGg6fYWbx4Mbt27XJzOGksLMsKNca4\ngAVAxTRDYUCqc6lEpNLs2bB9O7z1Fh07dWL58uW6kudFDeoZtiwrtPzbBdgNKlTTsKqXQeQ4PPYY\nfP01vPAC9O9f77cbYygrKyMoKIiePXu6P5/4HMuyYoDI8q8VFgMYYzLK94kCXBXPRcRBe/fC9Olw\n2WVkDxhAQUEBlmXRpIku3ntLXWaTUMMq4oRPP4Vp0+DWW+3e4QZYvHgxp512GuvWrXNzOPFVxphk\nY0w7Y0zyYa9FHPZ9kjEm1RiT5ExCETnCjBn2AhvTpzN+/HhOOeUUSkpKnE4VUOoym0QykHzUa0c0\nrB7IJRLYsrLg+uth0CD78lkDWZZFz549NURJRMQXZWXBs8/a7f1pp3HTTTfxhz/8gWbNjncUq9SH\n/muL+JqSErj2WvuGioULISSkwYe68MILufDCC90YTkRE3GbyZHv55SlTABg+fDjDhw93NlMA0oAU\nEV/zyCPw1Vf2VGoDBjT4MEuWLKGwsNCNwURExG1++QVeeQVuv50DHTsyffp0cnNza3+fuJ2KYRFf\n8u67EB8PsbHw1782+DBbt24lKiqKqVOnujGciIi4zUMP2Vf+Hn6YlJQUHnroIdauXet0qoCkYRIi\nvuLXX+HGG2HoUHjmmeM6VM+ePfnoo4849dRT3RRORETc5ssv4T//secW7tSJq6++mvXr19O/AbMG\nyfFTMSziC/bvh6uvhhYtYNEi++txsCyLiy66yE3hRETEbYyB+++HHj3gnnswxmBZlgphB2mYhIjT\njLGnT/v1V1iwAHr1Oq7DPffccyQkJGjCdhERX/T22/D99/DPf2KCg7nggguIj493OlVAUzEs4rSE\nBEhOtiddv+CC4z7c119/zZdffqmll0VEfE1hITz4IJxxBtxwAwUFBfTt25eOHTs6nSygaZiEiJM+\n/BAmTICRI+GBB9xyyDfeeIOCggK3HEtERNzouedg0yZISYGmTQkODmbevHlOpwp46hkWccqaNfZ8\nwoMHw7x5cJw9uaWlpbhcLgBatmzpjoQiIuIuubnw+ONwySUQFUVWVhaZmZlOpxJUDIs4Y+9euOIK\nCA62p1M7joU1Krzzzjv06tWLVatWuSGgiIi41aOPQl4ePPEEAP/85z85/fTTOXDggMPBRMMkRLyt\nuBhGjYLNm+GLL8BNSyWfcsop3HbbbQwcONAtxxMRETf5+Wd44QX4+9+hfMrLiRMnctFFF9G6dWuH\nw4mKYRFvMgbGjrXHi738Mgwb5rZDDxo0iFmzZrnteCIi4gbGwD33wAknVC67DNCjRw+uueYaB4NJ\nBQ2TEPGmp56yl1meMMGeTs1NEhMT2bp1q9uOJyIibvL++3YHyGOPQYcOHDx4kFtvvZXVq1c7nUzK\nqRgW8Zb//AfGj4cRI+xVh9xk27ZtjB07ltdff91txxQRETcoLIT77oOBA+H22wH48ccfWbRoEXv2\n7HE4nFTQMAkRb/j+e7juOvjd7+C116CJ+z6H9ujRg/Xr1xMaGuq2Y4qIiBs8+yysXw+ffAJBQQD8\n/ve/Z9u2bbRq1crhcFJBxbCIp61dC5ddBt26wXvv2TNIuEnFMp59+vRx2zFFRMQNsrJg8mS4/HK4\n+GIA8vPzadWqlW6a8zEaJiHiSTt32o2gZcGnn0Lnzm49fFxcHLfccouWXhYR8TX33w+lpfDMM5Uv\nXXzxxdx0000OhpKqqGdYxFPy8uCPf4TsbFiyBE480e2naNmyJcHBwVp6WUTElyxeDAsW2D3DYWEA\nlJWVcc0112jpZR9kebNHKTIy0qSlpXntfCKOKSiASy+FL7+EDz6ovEQmjZdlWenGmEinc3iT2myR\nBigqgtNPh5ISWLUKtCKoY+rabmuYhIi7FRfDyJH2ghrz5nmkEC4oKGDlypVuP66IiBynWbPg119h\n9uzKQnjz5s189NFHlJWVORxOqqJiWMSdSkvh5pvteSWffx6uv94jp3n99dcZPHgwGRkZHjm+iIg0\nwObN9sIaV11lD5Mr9+KLL3LVVVexa9cuB8NJdTRmWMRdjIE77oA33oD4+Mo5JT1hxIgRNGnShCFD\nhnjsHCIiUg/G2O2+ZR1x0xzAlClT+NOf/kS3bt0cCic1UTEs4g4Vy20mJcFDD9mLa3hQu3btuO22\n2zx6DhERqYe33oKPP4ann4bevY/YFBQUxFlnneVQMKmNhkmIHC9j4N577Z6Ae+5x6+pyx57KMG7c\nOL766iuPnUNEROopJwf+8Q8480wYO7by5cLCQi688EJSUlIcDCe1UTEscjyMsZfafPppGDcOnnrK\nvkTmITt37mTBggX89NNPHjuHiIjU0/33w969MHcuNG1a+fLWrVvZs2cPTdy46qi4n4ZJiDRURSE8\na5bdIzBzpkcLYYBu3bqxadMmj55DRETqYfFiePVVmDDBnlLtMCeeeCIrVqxwJpfUmT6qiDREWZl9\no8SsWXD33fZXDxfC+fn5GGNo0aIFLVq08Oi5RESkDvbvh9tusxdVmjjxiE1ZWVkUFhZiWZYWRvJx\nKoZF6qukBG66CRIT7Z6Ap5/2eCEMMHbsWM455xzNUyki4ivuv9+eTu3VVyE4+IhNN954I+eee64z\nuaReNExCpD4KC+Haa+E//4GpU+2ZI7zkggsuYNCgQRp7JiLiCz791J5B6P77YdiwYzbHxcVx4MAB\nB4JJfakYFqmrvDy4+mr4/HN75oi77/bq6W+44Qavnk8aJ8uyYgAXEG6MSahie7wxJs6yrDHGmCTv\nJxTxAy4X/O1vMHAgPP54lbtc7IHVR8Uzau1isiwrxrKsKMuyqpw41bKs+PKvY9wdTsRn7NgB554L\ny5bB6697tRB2uVzMnz+f4uJir51TGifLssIBjDGpgKvi+VHGWJa1Acj0ajgRf/KPf8DOnfbfg/Il\nlyts2bKF+Ph48vLyHAon9VVjMayGVQR7jfmzzoL16+GDD8DLPbRvvvkm119/PT///LNXzyuN0ijs\nXmGw2+SoKvYZbYzpX96ui0h9LVxoF8ETJkBk5DGbP/jgAx555BH27dvnQDhpiNp6htWwSmBbutQu\nhPPzYckScOCyV2xsLF9//TWDBw/2+rml0QkFcg973qGKfcJqutonIjXYvBlGj7YX15g0qcpd7rjj\nDjZs2ECvXr28HE4aqrZiWA2rBK5XX4XoaOjUCf73vyp7ALyhSZMmWsZT3MYYk1DeedHBsqxjOjgs\nyxpjWVaaZVlp2dnZDiQU8VElJfDXv9pTa775JgQFHbNLxWw/vY9ajll823Hflq6GVfxOWZl9+euW\nW+xxwv/7H/Tv70CMMi666CLeeustr59bGi0X0L78+1Ag5/CN5e1xTPnTHCDs6AMYY5KMMZHGmMhO\nnTp5NKxIozJlCnzzjT2tZtgxvzoUFRVx6qmnkpSk+1Ibm9qKYTWsElj27YMrr4QZMyA2Fj7+GNq1\ncyRKTk4OhYWFjpxbGq0F/NYOhwGpAJZlhZa/llbxGtC//LmI1GbJEvjnP+Hmm+3pNauQl5dHeHi4\neoUbodqmVlsAVFwbPqJhNca4sBvSihvn+gOJnggp4hWrV8NVV8HGjTB7Ntx5p1cW06hOp06dWLp0\nKcYYxzJI42KMybAsK7L8Kp3LGJNRvmkxEFG+fYxlWbnAhsO2i0h1tm+Hv/wFTjrJ/ttQjY4dO/Lv\nf//bi8HEXWoshtWwSsBYtMj+xB8SYs8jfM45jsbZunUroaGhtGnTRst4Sr1UNXewMSaipu0iUo2i\nIoiJgQMHYPFiaN26yt1++eUXWrVqpZvmGqlaF91Qwyp+rbAQHnjA/rT/u9/ZRXGPHk6n4q677mLV\nqlWsXbtWK86JiDjl3nvt+0YWLIBBg2rY7V5WrVrFxo0badq0qRcDijtoBToJXBs2wKhRkJ4O48ZB\nfDw0b+50KgAefvhhNm3apEJYRMQpr70Gzz1nL7c8cmSNu77wwgusW7dOhXAjpWJYAo8x9rQ4t98O\nTZvCe+/BFVc4neoIQ4cOZejQoU7HEBEJTN99B3//O5x/PkyfXuvuffr0oU+fPl4IJp6gbicJLLm5\n9p3A110Hp50GP/zgU4Xwjh07mDBhApqGUETEIRs32n8XunWzh0c0q77fcPv27dx0001s3LjRiwHF\n3VQMS+D47DO7AF60CKZNs1eX87FP8p9//jlPPfWU1rQXEXHC3r1w2WVQXAwffWQvulSDjIwM3nvv\nvcrFNqRxUjEs/s/lgr/9zV5KuW1b+/LXhAn2EAkfc91115GVlUV/Bxb5EBEJaEVF8Oc/w/r18M47\nMGBArW+5/PLL2bFjh9rsRk7FsPi3d9+FU06xb4R48EH7ZrnwcKdTVamoqAiAzp07O5xERCTAlJXB\nrbfCF1/ASy/B8OG1vmXfvn0ABAcHeziceJqKYfFPmzfD1Vfbj86d7d7g6dPBRxutoqIiBgwYwKxZ\ns5yOIiISWIyxF1maP99eZe7GG2t9S0lJCYMHD+aBBx7wQkDxNM0mIf6lqAieegoef9xePW76dLjv\nPggKcjpZjQ4ePMill17Kaaed5nQUEZHAYYw91/yLL9pXDx96qE5vKy4uJjY2lnAfvdIo9aNiWPyD\nMfDBB/Z8kGvX2j3CTz8NjWSN+NDQUObMmeN0DBGRwDJlit2BMnasfWN1HVf8DA4O5sEHH/RwOPEW\nDZOQxm/FCoiKsqfCsSz7DuB33mk0hfAPP/zA6tWrnY4hIhI4jIFHHoHHHoObb4ZnnqlzIbxixQpS\nU1Mxxng0oniPimFpvDIz7bFd4eGwcqW9pPJPP8Ef/+h0snp58MEHufTSSyktLXU6ioiI/ysrg3/8\nA6ZOhdtus2+Yq8dqnzNnzuTaa6+loKDAgyHFmzRMQhqfbdvsmxxeesmeDP3+++2p0tq1czpZg8yf\nP5/169drGU8REU8rKbEL4Ndes+8neeKJOvcIV5g7dy6rV6/WLBJ+RMWwNB6bN0N8PLzyCpSWwpgx\n8PDD0L2708mOS8eOHenYsaPTMURE/Nv+/fbqo++/b48VfuSRehfCAC1atGDIkCEeCChO0TAJ8X1r\n1tjzP554ot0bfOON9k1yzz3XqAvhtWvXcvnll7N+/Xqno4iI+LdNm2DYMPuekuefh4kT610I7969\nmz/84Q98++23nskojlHPsPgmY+zJz596ym68WraEO+6wh0T06uV0OrdYv349P/74I23atHE6ioiI\n//r6a3uGoeJi+PhjiI5u0GG2bt3K/v37addIh+RJ9VQMi285cADeeMP+5L5ypb1gxuTJcPvtta4R\n39hceumlbNy4UWOFRUQ8wRj7xuoHHoA+fezhEf/3fw0+XEREBD/99BNWA4ZWiG/TMAnxDStX2vM8\ndu8OsbH25auXXrLHCU+a5HeF8JYtWzDGqBAWEfGEnBy48kp71oiLL4Zvvz2uQnj9+vUUFxerEPZT\nKobFObm5MGcORETA4MEwdy5cdRV88w1kZMDf/mYPj/AzBw4c4IwzzmDChAlORxER8T9ffAFnnAGf\nfmrPH/zee9C+fYMPV1payqWXXsrIkSPdGFJ8iYZJiHfl59uXqt580x67VVwMQ4bAs8/CX/8KHTo4\nndDjgoKCmDFjBkOHDnU6ioiI/9i7F8aPt68qnnQS/O9/9jz0x8myLGbOnEmrVq3cEFJ8kYph8bz9\n+39bFe7DD+2CuEcPuPtuuP56u1c4gLRo0YLY2FinY4iI+AdjIDkZ7roL9uyxC+JHH4WQELccvkmT\nJlx++eVuOZb4JhXD4hlZWXbh+8EHkJIChYX2zXDXXWf3AJ9zTr1W/PEXn3zyCXl5ecTExNAkAP/9\nIiJutXy5PcvQsmV2L/DHH9tXG90kPT2dZcuWERsbS4ibimvxPSqGxT0KCuCrr+zC99NP7RviAPr1\ng7//Hf78ZzjrLAjwG8YSExNZt24dMTExTkcREWm8NmywF8146y27o+X552H0aHtVUjd67733mD17\nNqNHj3brccW3qBiWhikqsj+RL1kCS5fCl1/aBXFQkF30JiTA5ZfDgAENWuHHXyUnJ7N9+3b1CouI\nNMRPP8GMGXYR3KKFXRCPHw8emq99ypQp3H777bRu3dojxxffoGJY6iYnx74Z4Ztv7Mf338OhQ/a2\n006zp0OLjobzzgM1GlUqKyujadOm9PKTRUNERLyirAw++8xedfSDD6BVK7j3XrjnHo+uQlpSUkKz\nZs3o1q2bx84hvkHFsBwrNxdWrLCnN1u+3H5s3Ghva9bMHo81Zoxd+J5zDnTs6GzeRiAtLY1Ro0aR\nnJysNe1FROpixw549VV72s2NG+355idPtuekP46p0uoiJyeH008/nWeeeUbD2gKAiuFAVlgIa9fC\nqlW/PVasgC1bftunTx8YOtQe9/v730NkpNvu0A0kpaWlnHjiifTv39/pKCIivis7GxYtsodBLFtm\nzxRx/vkwfbo9D32LFl6JkZ+fz7BhwxgwYIBXzifOsowxXjtZZGSkSUtL89r5BPvy0rZtsG4drF9v\nf/3lF/uRmWlvB/vGtpNPticqHzLEnu5s8GD7xgQRwbKsdGNMpNM5vElttnhcWZl9w/VHH9mPb7+1\nXxswAEaNsmcfOvlkp1NKI1XXdls9w41dURFs325PZbZli/3YvNm+pLRxI2zaZO9ToUULu2EZMgSu\nvdZucE47zX7NS5+4A83SpUuJjIzUhO0iIkVF8OOP9k3Xy5bZsxDt2WNvi4yEhx+2Zx86/XTHbr5e\nvnw5PXr0oLsHxyOLb1Ex7IvKysDlsi8X7d4Nu3b99ti+3R5HtWOH3eO7e/ex72/f3p7S7Iwz7MtK\nYWH2ajwnnWQvdqGZDLxm7969XHLJJdx2223Mnj3b6TgiIt6Tmws//2w/VqyA9HS7EK7ooAkLs2cd\nOv98uPhi6NLF2byAMYZbb72VkJAQvvvuO6fjiJfUWgxblhUDuIBwY0xCfbcHrNJSOHAA8vLsx759\ndoFb8XXvXvuRm2s/cnJ+e+zZY7//aJZlNxbdutmPyEjo2dMucHv2tMf39upl32krPqFdu3akpqbq\nbmTxGrXZ4jXFxXanzJYtsHWrfTVyw4bfhuTt2vXbvm3bQkQE/OMf9t+uYcPsv10+xrIs3nvvPfZU\n9FZLQKixGLYsKxzAGJNqWVaYZVnhxpiMum73OWVl9ifSoiL75rGKR0HBb18rHocO/fb14MHfvubn\n//b18MeBA/Zj/377kZ9fe57gYGjXzu7J7dDBHrLQoYN9x2ynTvYsDZ072wVwly72czdPKC6eN2zY\nMKcjSIDwuzZbvMMY++/a/v1Hdt5UdNrk5NhXKvfssa9G7tgBO3faz4++76h7dzjxRLjsMhg4EAYN\nsh+9ejWaOefDwsIIC5SqCakAACAASURBVAtzOoZ4UW2V1Sggpfz7TCAKyKjH9uNWcN11bM/Lo2uz\nZoQAhwoK2JGfT7emTQkuK+NgQQE7Dx2iu2XRsrSU/MJCdhUU0KOsjBYlJRwoLGR3YSE9i4tpXlrK\nfiAb6AUEAXnAHqA39n+Mo5/vA3KAPkBTwNWyJbnBwfRt04YmrVqxt2VL9rZoQb/u3bFOOIHcZs1w\nBQUR1rMntG1LTpMm5DVrRr+TToLQUHLKyshr2pR+Z5wBLVqwZ88eDhw4QN++fQHIzs7m4MGD9OnT\nB4Ddu3dTUFBA765dAdi1axdFRUWVc9Xu3LmTkpISevbsCcCOHTsoKyujR/kn7qOfb9++HaByLNS2\nbdto0qRJZc9lVlYWzZo1o2v5+bZu3Urz5s3pUn75asuWLbRs2ZLO5TfWbd68mZCQEDp16lTl802b\nNtG6dWs6lk+/tnHjRk444QQ6dOgAQGZmJqGhobRv3x5jDBs3bqRdu3a0a9eOsrIyNm3aRPv27QkN\nDT3meWlpKZs3b6ZDhw60bduWkpIStmzZQseOHTnhhBMoLi5m69atdOrUiTZt2lBUVERWVhadO3em\ndevWFBYWsm3bNrp06UKrVq0oKChg+/btdO3alZCQkGOeHzp0iB07dtCtWzeCg4M5ePAgO3fupHv3\n7rRs2ZL8/Hx27dpFjx49aNGiBXPnzmXr1q1MmjSJZvoQI97heJvN9u1krl1Lu7Ztade2rf17vXVr\n5fOysjI2ZWXRPjSU0DZt7N/jbdvo0K4dbdu0sX+Pt2+nY7t2nNCmjf17vGMHndq3p03r1vbv8Y4d\ndO7QgdatWtnPd+6sfF5YWMi2nTvp0rEjrSp+j3fvpmvHjoS0bGn/DcnOplunTgS3aMHBQ4fYuWcP\n3Tt1omWLFuQfPMiunBx6dO5Mi6AgDhw8yO49e+jZtSvNmzWzn+fk0LNzZ5oHBbH/wAGyc3Pp1bkz\nQUFB5O3fz569e+nduTPNmjZl3/795Lhc9OnShaZNmuDat4/cffvo27kzTSyLvfv2sTcvj36dO2MZ\nw968PPbu20dYp05QVkbuvn249u8nrH17KC0lx+Ui78AB+oWGQkkJe1wuDuTn07dVKyguJnvfPvtv\nSMuWUFjIbpeLgoMH6d2kCRw6xK68PIoOHqRXSQnk5//2vLyo3QmUAD3Lf5w7gDKgR+vW0LEj29u2\nhW7d6H7WWdC1K9tat6ZJz550GzwYevcmKze36r8h5YVwff+m1OdvSMXzir8htf1NOfpvyPLly3n+\n+eeZPn16ZX4JEMaYah9AIvalNLAbzfj6bD/6ERERYerry169DGBSevc25vTTTeqJJxrALDv1VGPO\nO898NHiwAcz/s3fncVFX+x/HX18RxR1xyzQXSC01U0BT86YV2N4tL2ilNy0DyxZLDbJstxS19NoK\nZt2xskyy5baD1a+6pQZkZmqpaC6puaG5I5zfHwxz1ZBBBM7AvJ+PBw/nzHfmO2+cmc8czpzv9yzs\n08eYAQPMu336GMBkXX21MbfeauZeeqkBzM9xccY88oh5NTbWAGbVQw8Zk5JiXrr5ZgOY9S+9ZMwn\nn5jn777bAGbL558b8+uvZvojjxjA7Ny0yZj8fJOUlGQAs2/fPmOMMY8//rgBTG5urjHGmPHjx5tq\n1ap58ickJJigoCBP++677zb169f3tG+77TbTpEkTT3v48OGmRYsWnvaQIUNMaGiopz1w4EBz1lln\nedrXXHON6dKli6d92WWXme7du3vaF198sTn//PM97T59+piLLrrI0+7Ro4e59NJLPe2uXbuaq6++\n2tM+++yzTWxsrKcdFhZmBg8e7Gm3bNnS3HzzzZ5206ZNza233uppBwcHm7vuusvTrlWrlrn33ns9\n7YCAAPPAAw8YY4w5cuSIAcxjjz1mjDFm//79BjCTJk0yxhiTk5NjAPP0008bY4zZunWrAcxzzz1n\njDFmw4YNBjAzZ840xhizevVqA5jZs2cbY4xZvny5Acybb75pjDFmyZIlBjDz5883xhizaNEiA5gP\nP/zQGGPM119/XfDaS0szxhiTnp5e8Nr76itjjDEfffRRwWtv4UJjjDHvvvtuwWsvK8sYY8zMmTNN\n7969jVQNQIYppr75wo8v1GzTt68BzMMFY4bmEBjAPOFu73G3p7rb293tGe727+72i+72Wnf7FXf7\nF3f7dXd7qbud6m5nuNvvu9vfutufuNtfuNtfuNufutv/dbffd7cz3O1Ud3upuz3H3V7pbr/ibq91\nt5Pd7U3u9gx32z2Gaqa623vc7Sfd7YPu9sPutnH/jAMTeFR7NJi6R7XvABNS2A4IMPHVq5vTHMeY\nZs2MOeMMM7RePdM6MNCYiAhj/vY3c/1pp5l2deoYc/31xsTFmQFnnmk6N2lizKRJxjz/vLmyWzcT\nHhZmzNdfG/PTT6Z/376mZ48eRz29fU3fvn097V69epno6GhPOzw83Fx55ZWedufOnc2AAQM87fbt\n25vrrrvO027durUZOnSop928eXMTFxfnaYeEhJg77rjD065bt64ZPXq0p12jRg1z3333edqAefjh\nh40xxhw6dKjgtffEE8YYY/bs2VPw2ps61RhjzPbt2wteezNmGGOMWblypalXr57Jyckp2WtdfF5J\n63a5D1c5jhMPxAO0atXqpO/f7vvvcX36KZ2io6F5czpu3owrLY327sn2XTZtwrVgAaGXXQZNmtBt\n/XpcX35J6yuvhJAQeqxbh+urrzj96qshOJje2dm4rrySptdcA/Xrc0G/frj69iXkH/+AOnW4qE0b\nXN26Ub9nT6hVi/4DB+Jq25bajRqB43D55Zdz2mmnUaNGDQCuvvpqWrVq5Vled8CAAbQ/6jQwAwcO\npEuXLp72DTfcQI8ePTztG2+8kX79+nnaN998M5deeqmnPWLECK699lpPe+TIkeTk5Hjad911F3v3\n7vW0x4wZw8GDBz3thIQE8gtPnwY88MADxywF/PDDD3t+FyhYevLoZScnTpxIcHCwpz1lyhTPX+gA\n06ZNO+aI22eeeeaY5/mFF1445uummTNnHnPexldeeYXOnTsDBXO1XC4XXbt2BSAwMBCXy0V4eDgA\ntWrVwuVyERlZcJaUevXq4XK5OO+884CC+bkul4tevXoB0LRpU1wuF7179wagefPmx9z+jDPOwOVy\nERERARR8NeZyuTzPV7t27XC5XHTq1AmAjh074nK5PM9vly5dcLlcnt+vW7duuFwuz6h+VFQUl1xy\nCSKVyanWbMaNw9W+Pee2agWtWhGQn4/r22/p5j6mIejIEVwLFxLRpg20bEndI0dwLVpE97ZtoUUL\nGhw6hOv77+kZFgbNm9P44EFcGRn0btcOmjXjtAMHcGVm0rNDB2jalJb79uH64QcizzoLGjemzZ9/\n4vrxR7p27AiNGnHmnj24li6lc6dOEBLCWbt341q6lLO6dIHgYDrv2oVr2TLOPPdcCA6m6/btuJYv\np023blC/PpHudsvISKhbl55//IFrxQpO69ED6talz+bNuH75hca9ekHt2vTbtAnXqlUEn38+1KpF\n1IYNuFatom7fvlCzJpdu2ECT1asJuvhiqFGDK9eupcWqVVS/9FKoUYNrV68mdPVq+PvfISCAmJUr\n6bh2LQwYANWrc/2yZUT89lvBaceqV+efS5Zw/oYNMHgwOA7DvvuOqI0bITYWgLj//pertmwpOEMD\ncNvXXxfMh3V/rtz55Zfs2bMHrr4agHs6dGD//v3Qpw8A944fz+Gjzkg0bty4Y57uBx988Jhvvh59\n9FFqH3Uu+ieeeIL69et72klJSZ5RXICnnnrqmFHYf/3rX55vOgGee+65Yz5DkpOTj/mMnTVrFh07\ndvS0XS4X5557LgABAQG4XC7PQkdBQUHH1Py6devicrno3r07UPCZ8MUXX9CgQYMiX9pSdRV7nmHH\ncZKANFMwvywGCDVHHXDhbfvxdM5KEamsKsN5hlWzRUT+p6R129s5tuYChX+ShQLp7p0HF7ddRESs\nUM0WETlJxXaGjfsoY8dxooAc87+jjhd42S4iIhVMNVtE5OR5nTNsjEkp4rqI4raLiIgdqtkiIidH\nS5GJiIiIiN9SZ1hERERE/JY6wyIiIiLit9QZFhERERG/pc6wiIiIiPgtdYZFRERExG8VuwJdmT+Y\n42wDfivFXRsD28s4TmkpS9GUpWjKUrTKmKW1MaZJeYfxJarZZU5ZiqYsRVOWop1MlhLV7QrtDJeW\n4zgZvrIMqrIUTVmKpixFU5aqzZf+T5WlaMpSNGUpWlXPomkSIiIiIuK31BkWEREREb9VWTrDvrR8\nqLIUTVmKpixFU5aqzZf+T5WlaMpSNGUpWpXOUinmDIuIiIiIlIfKMjIsIqfAcZxw2xlERKTkVLcr\njjrDp8hxnATbGeTEbBUTx3FiHMeJ8oXXh+M4UcA82zkAHMeJd/8k+UCWKPeP9SxSsXzhfSknprqt\nul1MlnKp25WqM1z4yzuOE287C3herNG+kMNXPtR97U2DhWJSWMiNMelAju2/7t05sm1mAM/zkW6M\nSQFC3W2bWWLd/zfhtp+jqkx1+8Q5VLeLzKK6jep2MVnKpW5Xqs4wEO84zhp84AXiK3zpQ92X3jRg\ntZgMAnLcl7MBq/8PPiSU//1fZLvbVhhj0o0xI9zNUGNMlq0sfkB1+ziq2yemuu1z/KJuV7bOcJwx\nJsz9ZrHKcZxwX8jhYx/qPvOmsSwY2HlUu5GtIL7EGJPi/sAFCAcybOYBz9flI7zeUE6F6vZxVLd9\nkup2Efylble2znCor8znAUJsBziaL3yo++KbRnyPexQsyxdGY40xk4ERjuME285Shalun4DqtlQW\nVb1uVy+rHVUE938AjuNEO44TVZ5/4Z9gflu2MSbdV0YXjmaMmew4zjz3MoU53u9RfirqTVPcc1Se\nj1sCOfzvQzcY2GExiy+KMsYk2gxw1PzALApGw+KByTYzVVWq2yemuu2huu37qnTd9qnOsJdCFg/s\nNMakUvAiLdevco76S7kooY7jhFLwxglxF9lyKyDeCjxU3Id6CQtZhbxpvDxHNs0FCtdNDwWsFnnH\ncWKASMdxYtzvH5tZ4o/qHJVrx8iLKKDwPRsMfG8pR6Wnul001e2iqW6XjOp2kcqtbvtUZ9jLmySD\n/02qDwOSyz9R0QpfmO4CU+5fr3r5f6nQD3VvhcyH3jTWiokxJstxnEj3gSg5tr9Wcv/uVospeA7U\nSXIcJ5GCDkmsxTgpwMDCToLtD5vKTHX7hI+nul0KqtuePKrbf1VudbtSrUBXOMpAwQEH+koTcM+Z\nGehuRhx1UIaNLIWnxNmJ+03jA199iYhFqtt/pbot4lsqVWdYRERERKQsVbazSYiIiIiIlBl1hkVE\nRETEb6kzLCIiIiJ+S51hEREREfFb6gyLiIiIiN9SZ1hERERE/JY6w1LpOY4T417SdJ7jOLvcJ20X\nEREfpJotvkbnGZYqw33y+ERjTLTtLCIiUjzVbPEV6gxLleA4Tjgw0xgTYTuLiIgUTzVbfIk6w1Lp\nuZc2XQBcbIzJsZ1HREROTDVbfI06w1LpOY4zDwgHstxXzTXGpFqMJCIiJ6CaLb5GnWERERER8Vs6\nm4SIiIiI+C11hkVERETEb6kzLCIiIiJ+S51hEREREfFb6gyLiIiIiN9SZ1hERERE/JY6wyIiIiLi\nt9QZFhERERG/pc6wiIiIiPgtdYZFRERExG+pMywiIiIifkudYRERERHxW+oMi4iIiIjfUmdYRERE\nRPyWOsMiIiIi4rfUGRYRERERv6XOsIiIiIj4LXWGRURERMRvqTMsIiIiIn5LnWERERER8VvVvd3A\ncZwo98VoY0xiEdtjgBwg3Bgzubh9NW7c2LRp06Y0OUVErMrMzNxujGliO4c3qtkiIgVKWreL7Qy7\ni2qsMWaE4ziJjuOEG2OyjtoeDmCMSXccJ/T47cdr06YNGRkZJ/FriIj4BsdxfrOdwRvVbBGR/ylp\n3S52moQxJt0YM8LdDC2iaA6iYIQBIBuIQkRErFDNFhE5eSWaM+w4TgIwoohNwcDOo9qNyiKUiIiU\nnmq2iEjJlagz7J5XNsJxnOCTfQDHceIdx8lwHCdj27ZtJx1QREROjmq2iEjJeZszXDi/LIuCr9Ti\ngaMPuMgBQtyXg4Edx+/DGJMCpABERkaa47fn5uayceNGDh48WJr84sOCgoJo2bIlgYGBtqOI+AXV\nbKlIqvFSVXg7m0QUUDjnLBj4HsBxnGBjTA4wF4h0bw8F0k82wMaNG6lXrx5t2rTBcZyTvbv4KGMM\nO3bsYOPGjbRt29Z2HBF/oZotFUI1XqoSb9MkUoBQx3HiAYwxqe7rF7jbWeA5gjmnuKOST+TgwYM0\natRIRbWKcRyHRo0aafRIpGKpZkuFUI2XqqTYkWH3SEJKEddHHHX5L9tPlopq1aTnVaRiqWZLRdLr\nQKoKrUAnIiIiIn5LnWER8R+HD8MHH0B+vu0kIiJSEh99BPv3l+tD+H1nODU1lcTEv6xYWukU/g6J\niYlER0cTERFBdnZ2kbdt2LAhERERREREMGJEUaciPVZKSgpZWcdOLSx8vJPdl4hNW5KTSb3qKg5/\n9pntKFJK/lizoaAOT5587OrZJb1v4f1PVMfLYv8i5SF3xQpSr7iCzQ89VK6P4+1sEhXr7rthyZKy\n3WfXrjB9etnu08dMnjyZQYMGkZWVRVZWFmlpaWRlZTFixAjS0tKOuW12djZRUVHMmzevxPuPj48n\nNjbWc5+srCy6d+9eqn2JWGMMp730EgM6dcLp3992mqpBNbtUTqZmA0RHR5Oenk5SUpLnupLet9CJ\n6nhZ7V+kPAQ++ywxNWqQd8895fo4fj8yfLTY2FjPX8Hp6el/uX7EiBFERBQch5KdnU10dDTR0dHE\nxsaSk5NDamrqMX9ZF942PT3dc7uwsDAmT57seZzCv9QL9x0REcEDDzzgGWWNjY0lPT2dyZMnM2LE\nCM/9UlNTPY+TlpZGeHi453EAwsPDycjI+MvvmJ2dTXZ2tud3On6kIDU1ldjYWGJjY2nYsKHncUJC\nQjwjA3PnziUmJsbrvkR8yZa33uLI0qVUGzMGp5pKX1XgDzW78PbJycnHXFfcfU+mjpdm/yIVYc/a\ntex++WUYMoSAFi3K9bF8a2TY4mjA5MmTCQ0NJSkpiZycHNq2bcuuXbtITEyke/fuJCQkkJ6e7im4\nqamphIeHk5SURHp6Ojt37vTyCDBv3jxSU1NJTk4mLS2N1NRU5s6d6ykymZmZ5OTkEBERQXh4OImJ\niYSEhBAVFUVWVhbZ2dmev8wbNmxITEwMOTk5BAcXLDK1Y8cOz1/6JxISEsK4ceOIiYkhKyuL2NhY\n1qxZ49keExNDTEwM6enp5OTkeIpl4YdAaGhoifcl4kuG3n47ewID+e6GG2xHqTpUs8u9Zp9Icfc9\nmTpemv2LVIR/3XILUw8eZM1NN9G4nB/LtzrDFq1Zs4bY2FgAT6GCgtGEQYMGARAVFeW5Pj4+nokT\nJxIdHe0pyMUJDw/37LuwEAUHB5OTk0NmZiYZGRnHPH5SUhJhYWHs2rXLs4/Cv9IBIiMjyc7OJicn\nh5CQggWlGjVqdMy8rpycnCJzFGYJDw9n586dxxRnKPh6LDExkczMTM91oaGhpKWlERoaesxogbd9\nifiEpUu5Y8cO9g4aBDVr2k4jZcBfavaJeLtvSet4afcvUq4OHuTKH36gZocONO7Tp9wfTt8VuoWF\nhXn+gj+6AISGhnpGFo7+Gu6tt95i0KBBpKWlERYWRkpKwak7d+woWN306K/EvImIiPDMvZ03bx6D\nBg1ixIgRJCUlERcX57nd0fO1MjIyCA0NJTQ01DPCERUV5blNVlbWMR8EhSZPnuz5WjA7O5uQkJBj\nPkhycnKIi4tjwYIFx9wvOzub7t27M3fuXM9+ve1LxGc8/TRX1anD9c8/bzuJlBF/qdknUtx9T6aO\nl2b/IuXutdfotmsXCRVUszUy7JaQkOCZZ7Zz507PQQbjxo0jNjbWM8erUGRkJLGxsYSGhnpGBUJD\nQ0lOTiY6OvqY23pTeGBD4V/qTZs2JTo6moSEBEaMGHFMkS7MN3PmTOB/IxXwv5Hawv0UzgHLzs4m\nIiKCXbt2eX7Pwrlxxx/8FhcX5ymkAIMGDSImJobMzEwSExP5/vvv//J/dqJ9ifiCTRkZpL76KsPj\n4qjrHpGTys9favaJnOi+cHJ1vDT7FylP+UeOkHT//fyzUydaXnhhxTyoMabCfiIiIszxli9f/pfr\nfElaWppJS0szxhiTmZlpoqKirORISkoyycnJJ9yWmZlZ7P1PdN+SiomJKdX9fP35larvuehoUw3M\nmi++OKX9ABmmAuulL/yoZpee7ZpdlNLW8eL4+utBKp/MadNMAJjXR4485X2VtG5rZNiLyMhI4uLi\nSE5OJicnxyf/Ok5ISCAxMbHYkY2SHDBxIikpKYwbN67U9xex5s8/Gbl4Mf0vvZTQfv1sp5EKoJpd\nNNVxqSzC332X1aedRospUyrsMZ2CjnPFiIyMNMefnmXFihWcffbZFZZBKpaeX7HJTJuGM3o0LFwI\n5513SvtyHCfTGBNZRtEqBdVs8UavBylL5vvvcXr0gKlTYcyYU95fSeu2DqATkSrpyMGDRNx3H7Pa\ntTvljrCIiJS/oQMHcmdgIBx1IGpFUGdYRKqknFdfJfTwYZq6T38lIiK+y6xdS9PffqNRZCTUr1+h\nj605wyJS9RhD4+RkUtu1g8cft51GRES8cGbMYGpAAMydW+GPrZFhEalyVs+Zw5bMzII5Z1p6WUTE\np+3+7Td+Sk6G666DM86o8Mf3+0+J9PR0EhMT/3L98WvWl4WiHgcKjvItXO9eRE7d6DFj6F6tGnmD\nB9uOImVMNVuk6kkZOZIuBw6w2tK0Np/rDPfr149///vfAOTm5tKvXz9ee+01APbv30+/fv2Y6x5C\n3717N/369WP+/PkAbN++nX79+vGf//wHgC1btpQ6R0xMDAkJCafwmxxr8uTJniVCj1e4TKiIlIGV\nK3l661aSr7uOgLp1baep8lSzReSUHD7M8KwsXu3YkTOvvtpKBJ/rDNuQlZVFdHQ0ERERniU6U1NT\nSUxMJD093bPSUERExDErCxVeP2LECM8qbICnHRER4Rk9KFwNKTU1ldjYWGJjY2nYsKFnfyEhIccs\nKSoipfT005wZFMTl06fbTiLlRDVbpAp5801CtmxhyNSp9jKUZGWOsvrxxdWM0tLSTHh4uDHGmF27\ndpnQ0FBjjDHz5s0zCQkJJ9yekJBgkpKSPPsovD45OdnEx8cfc/tdu3b9ZeWftLS0Y1ZGSk5ONvPm\nzSvH39QO28+v+JetP/1k4qtVM+tuuKHM941WoDPG2H9Pq2b7FtuvB6nc8vPyzO2NGplvQ0ONyc8v\n8/2XtG5rZBiIiooCCtaML+n27Oxsz/WF/wJkZmaSkZFBbGwscXFxBAcHk52dTUhIiOc2WVlZJCYm\nkpaW5rkuNDS0ROvFi8iJLX7iCV7Lz+fgkCG2o0g5Us0WqRrWz5lD6o4drOzbFxzHWg6dWq2UQkND\nSU9PJzw8nPT0dM/1ERERBAcHk5SUBBTMOwsNDWXnzp0A5OTkEBcXx4IFC47ZX3Z2Nt27d6+4X0Ck\nqtm/nyvT0thy+eXUu+wy22nEx6hmi/ie1q++yrpmzaj2r39ZzaGR4VIaN24caWlpREdHHzNaEB8f\nT3Z2NtHR0URHRxMaGkpwcDA5OTkAxMXFeYprbGysZ/5ZZmZmsevUi0jx9qWkwI4d1LvvPttRxAep\nZov4lv2LFmE++4ygUaOoUa+e3TAlmUtRVj++OP+stNLS0kxaWpoxxpjMzMxj5pIVJSkpyWRmZp5w\n+/Hz06qKyvr8SuVy5NAh0z4w0CQ2b14u886M0ZzhQpX1Pa2aXT4q6+tB7BsaFmb+Vq2ayd++vdwe\no6R1W9MkSikyMpK4uDiSk5PJyckhOTm52NsnJCSQmJhY5EhCSkoK48aNK6+oIlXe4Xfe4YbcXCL+\n+U+r887Ed6lmi/iQTZvot24dnXr1wmnUyHYanIKOc8WIjIw0GRkZx1y3YsUKzj777ArLIBVLz69U\niD59YNMmWLUKqpfP3/iO42QaYyLLZec+SjVbvNHrQUrlvvtgypSCmh0aWm4PU9K67RNzhiuyQy4V\nR8+rVISf58zh6//+FzNqVLl1hOVYem8L6HUgpfPn77/z5owZ5F5zTbl2hE+G9c5wUFAQO3bs0Juq\nijHGsGPHDoKCgmxHkSpu8v33c7XjcOCGG2xH8Quq2QKq8VJ6c0eP5voDB/jhiitsR/GwPozSsmVL\nNm7cyLZt22xHkTIWFBREy5YtbceQqmzNGl5Yv56R//wntZs2tZ3GL6hmSyHVeDlpR45w83ff0e6c\nc+hx882203hY7wwHBgbStm1b2zFEpDKaPp3a1atz3sSJtpP4DdVsESm1t9+m2vr19H33XdtJjuF1\nmoTjOPHun6QTbE8qvF1ZhxMROZEdq1Zx4fPPs7h/fzj9dNtxfIZqtoj4IpOfT8ytt/J6s2Zw1VW2\n4xyj2M6w4zhRQLoxJgUIdbePF+84zhoguzwCiogUZe20aWzMz6f28OG2o/gM1WwR8VW7P/qIP3Jy\n2HvxxVDN+iFrx/A2TSLU/ZNCQeEs6rC/OGNMalkHExE5oUOHiJw/n18uuYRq115rO40vUc0WEZ8U\nnJLCV40bk+/lHN82FNs1N8akuEcYAMKBjCJuFuo4TpTjOAlF7cP9dV2G4zgZOuBCRMrC+hkzyNu6\nlWpjx9qO4lNUs0XEF2379lv2/uc/MHIk1erWtR3nL0o0Tu04TjiQZYzJOn6bMWayMSYdaFTUV3Lu\n4hxpjIls0qTJqScWEb+Wf+QIl44fT0z9+nDxxbbj+CTVbBHxJQ8MH04H4HBcnO0oRSrp2SSijDGJ\nx1/pPgBjp/srS1SizQAAIABJREFUtx0U/ZWciEjZ+eQTJhw+TJ1Ro7T08ompZouIb/jjD4avWUP3\nCy6gho+eiq9EZ5Mwxkx2X45y/xvs3pwBpLsvh1H0V3IiImWm2tNPM6BFCy554gnbUXySaraI+JTn\nnuO83FziUlK839aSkpxNIslxnDWO4+w6atMCAPdXcAMdx4kB1hT1lZyISFlZOncuz3/xBQduvx0C\nA23H8Tmq2SLiS/Zv386jU6awtX9/6NDBdpwTKnaahHteWcMiro846rLvdvVFpEpJnTCB6cDgwYOp\nZTuMD1LNFhFf8n+PPMKjBw5w0VVX0cx2mGL41oneREROZP16Hl2+nJ9uuYUGrVrZTiMiIsXJy+Oy\nzz5j7bnn0mfkSNtpimV9OWYRkZLInz6dao5D6wcftB1FRES8yH/vPaqtWkXrt97yuUU2jufb6URE\ngN2//Ua76dOZ36sXaFRYRMTn9b/5Zh5q0AAqwcJI6gyLiM/LSU6mszG0ufVW21FERMSLw19/Tfvd\nu2lxySVQ3fcnIfh+QhHxb4cP03r2bN678EIYPNh2GhER8aLGjBk8HxwMs2bZjlIiGhkWEZ/245Qp\nbNu0CbT0soiIz9v03/+y4u234bbbwAeXXi6KOsMi4ruM4aYnnuCyoCC49FLbaURExIvJI0fSzRh2\nDRliO0qJaZqEiPiuzz/n9QMH2JGY6PNHI4uI+L0dOxi/ahUXXnghDTt2tJ2mxNQZFhHfNXUqZzdr\nBo88YjuJiIh48+KLNDlwgGtmzLCd5KRoqEVEfNKyd97htk8+YetNN0FQkO04IiJSjEN79nDjhAks\n6dULOne2HeekqDMsIj4p4+mnmQtUv/lm21FERMSL5U89xUcHD7L973+3HeWkaZqEiPiezZsZtmgR\nA+Pjqd2une00IiJSnPx8us2bx4ZzziHo3nttpzlp6gyLiM/ZM3Uq9Y8coXZCgu0oIiLixZ7586m3\nYgW1Xn21Uh7sXPkSi0iVtnfLFkKnTWNq584QFmY7joiIeHFNXBwxQUEwaJDtKKWikWER8Sl5Lhcj\njOGCUaNsRxERES9MZiaxOTnUGjgQAgNtxykVdYZFxHccOUKD5GSeOP98GD7cdhoREfHCefppbqtX\nD1JSbEcpNU2TEBGf8e2kSSxauxbGjLEdRUREvNi4eDFvv/kmecOHQ4MGtuOUmjrDIuIbjOGRKVMY\nEhhI/pVX2k4jIiJevHLPPQzKz2fTwIG2o5wSTZMQEd/wzTfM37OH7Pvvp1olnXcmIuI3du/m/qVL\n6R8dTatevWynOSUaGRYR3/DUU9Rt1IguDzxgO4mIiHgzcyYBe/dy3sSJtpOcMnWGRcS6Xz7+mL7v\nvcfygQOhdm3bcUREpBi5+/dz4fjxvH/OORARYTvOKVNnWESs2/jcc2x1HBqNHGk7ioiIeLF55kz2\nHzpE9X/8w3aUMqE5wyJi17ZtXLxgASuGD8fp3Nl2GhERKY4xtPr3v1l41lkwfrztNGVCI8MiYtWq\nxx8n/+BBHJ1OTUTE52148032L1mCM3YsTkCA7ThlQp1hEbHmwM6d9H72WUa2bg1nnWU7joiIeBF/\n112cV7065oYbbEcpM5omISLWBM6dyzPGEDZunO0oIiLizbJl3L99O1sHDcKpVct2mjKjzrCI2JGf\nT/Xp07kuMhLi422nERERb55+mr/Vrg3PPWc7SZnSNAkRseKLJ5/kpV9/Jffuu8FxbMcREZFibF6y\nhEddLnZefz00amQ7TplSZ1hErHjjueeYEBCAU0VOzSMiUpV9Nn48j+fns6sKzRUupM6wiFS8xYtJ\n3rKFbx98kOpBQbbTiIhIcfbuZeh//8v6yy8n7KKLbKcpc17nDDuOUziZL8wYk1jE9hggBwg3xkwu\n43wiUgXlTZlCQIMGnD56tO0oVY5qtoiUtbxZswjIyeH0Bx+0HaVcFDsy7DhOFJBujEkBQt3to7eH\nAxhj0oGcwraIyIms/eorWqem8vlll0G9erbjVCmq2SJS1vIOHeLce+9letu20LOn7Tjlwts0iVCg\nsJhmu9tHG0TBCEPh9ihERIpxICWFro5DBy2yUR5Us0WkTO1980165eYSOmiQ7SjlpthpEu7RhULh\nwNzjbhIM7Dyq/ZfDC91f2cUDtGrVqnQpRaRq2LmTju++ywdDhkBkpO00VY5qtoiUKWNo8PzzzDzz\nTJgwwXaaclOiA+jcX6VlGWOyTvYBjDEpxphIY0xkkyZNTjqgiFQd3z3wADv37YOxY21HqdJUs0Wk\nLPzyxhusXrwYRo+GKrL0clFKejaJqKIOxKDg67YQ9+VgYEeZpBKRKufIvn3EpKQwrEkT6NLFdpyq\nTjVbRE7Z/WPH0ttxyK2Cp1M7mtfOsOM48YVHHBcejOE4TrB781z+NyctFEgvj5AiUvlVnzePj/Pz\nefyxx2xHqdJUs0WkTPz6K89u3sycG24gsEED22nKVUnOJpHkOM4ax3F2HbVpAUDhV3Du2+WU5is5\nEfEDxsDUqXTp0oVzR4ywnabKUs0WkTIzbRrNa9Yk6qmnbCcpd94OoEsHGhZxfcRRl1OO3y4icrQF\nkybx/s8/89gLL9BASy+XG9VsESkL21as4M6UFB4dMIAOzZrZjlPutAKdiJS7JS+/zPsBAdQaMsR2\nFBER8WLppEmk5edjhg2zHaVCqDMsIuVryRLGrF7Nysceo0bdurbTiIhIcQ4c4OKPP+b3yy7jrCuu\nsJ2mQqgzLCLlateTT0LdutQcOdJ2FBER8WLXiy/Ctm3UTEiwHaXCqDMsIuVmw6JFnD5vHrN794bg\nYO93EBERa/KPHKFnYiK3NW4MffvajlNhij2ATkTkVNR0ubjDcbjgwQdtRxERES+OvPceI3NzOfOm\nm8CPDnZWZ1hEyseePTR9/XWmDBwIffrYTiMiIl7U+Ne/GNWqFTz5pO0oFUrTJESkXHw0ZgxZe/Zo\n6WURkUrgpzlzeP/rr8kfNQqq+9dYqX/9tiJSIczhwyS6XDRq0IAvIyNtxxERES9eeOgh5gAbrruO\nerbDVDB1hkWkzDlvv803ublsS0qyHUVERLxZu5YZ2dncOmwY9U4/3XaaCqfOsIiULffSyw06dKBB\nXJztNCIi4s306VQPCKDLhAm2k1ihOcMiUqa+mDaNi7OyWD9sGFRTiRER8WW71q6lx7PP8uWFF0KL\nFrbjWKFPKhEpUzmvv05O9eo0vfVW21FERMSL32fMIC8/n5D4eNtRrNE0CREpO8uXc21WFtc8+iiO\nFtkQEfFthw7Rae5cMqKjcWJibKexRiPDIlJmfn7wQUxQEI6WXhYR8Xmrpk3j4ObNOH5+Ckx1hkWk\nTGxZupTw+fN5rHNnaNzYdhwRESmGyc9nwCOPcFXduhAdbTuOVZomISJlIuSNN5gJnD9pku0oIiLi\nzWefMf3QIcyoUX619HJR1BkWkVO3bx81kpO58dpr4eKLbacREREvnKee4uLTT4fHH7cdxTpNkxCR\nUzbvrrtw7dpF/j332I4iIiJe/JSaypPp6fw5YgTUqGE7jnXqDIvIqcnL47U33yS5bl2q/e1vttOI\niIgXn0ycyCQgb+hQ21F8gqZJiMipefdd3t2/n+0vv2w7iYiIeLNxI/cuXcrQ+HiCW7e2ncYnaGRY\nRErN5OeTm5SEExZGkxtvtB1HRES8yJ02DYyh6bhxtqP4DHWGRaTUvk1Ops333/NDTAwEBNiOIyIi\nxdizcSNtpk1jdkQEtGljO47PUGdYREqt5ty59AgMpP3o0bajiIiIF/tnzSLKGDrefrvtKD5Fc4ZF\npHRWrSLyq69454EHoGlT22lERKQ4ubmc9vLLuPr2BU1rO4ZGhkWkVNJGj2Z39epwxx22o4iIiBcZ\nkyezbv16GDPGdhSfo86wiJy03WvW8PcPPiAxLAyaNbMdR0REimMMdzz5JFfWqIG5/HLbaXyOpkmI\nyElr8PrrfAMET59uO4qIiHjzf/9H6v79bLzvPhwd7PwX6gyLyMk5cACefZbwK66ASy6xnUZERLyZ\nOpWWTZvS8uGHbSfxSSWaJuE4Tngx25Lc/8aXVSgR8V2v33kn927bxqG77rIdRU5ANVtECi1//31u\n+PBDNgwZAkFBtuP4JK+dYcdxooB5xdwk3nGcNUB2maUSEd+Un8/P77zDV7VrUyMqynYaKYJqtogc\n7ed//YvPgVrx+vv3RLxOkzDGpDuOU1zRjDPGpJZhJhHxVR9+yJM7d5I7ezZONR1/64tUs0XEY8sW\nYr/5hmvi4wns0MF2Gp9VFp9moY7jRDmOk1AG+xIRH7Zt4kRo1YrA666zHUVKTzVbxE9smzwZcnMJ\nHDvWdhSfdsqdYWPMZGNMOtDI/fWciFRBGbNnc/p33/FR//4QGGg7jpSSaraIf9j3xx+cNX06D3fo\nAO3a2Y7j006pM+w4TrzjODHu5g4g9AS3yXAcJ2Pbtm2n8nAiYtFpqamMqlGDPjoaudJSzRbxI6+9\nxjhjuGL0aNtJfF6pOsOO4wS7L2YA6e7LYe72MYwxKcaYSGNMZJMmTUqXUkTsWreOlh9+yNRRo6jf\nsqXtNHKSVLNF/ExeHnWee46xPXvS45ZbbKfxeSU5m0QMEHnUaALAAgBjTBYw0L1tjbstIlXMG7fd\nxlLHAZ1OzeepZovI1xMn8nF2NmbMGHAc23F8nmOMqbAHi4yMNBkZfxmIEBEflvvHH7Q67TSi2rTh\n1Wz/PRuX4ziZxphI2zkqkmq2SOV0TUgIP/35J7/u20dAjRq241hT0rqtFehEpFiBL7/Mz8Zw4MUX\nbUcRERFvvv2Wt3btYu348X7dET4Z6gyLyAmZQ4dwZswgJDoa+ve3HUdERLwwU6dSo2FDOtx3n+0o\nlYbOmi8iJ/TqnXdy6ebN7NTKRSIiPm9Vejrd3nmHrGuvhTp1bMepNNQZFpGiGUP+Rx+RV7cuDQcM\nsJ1GRES82P7CCwQ6DqfrYOeTomkSIlK0zz5j2KZNDHO5QEsvi4j4th076PXxx3x/001w7rm201Qq\n+oQTkSJlPfQQpnlz0NLLIiI+b9kjj3D4wAHQIhsnTZ1hEfmLpW+9RcTixczs1Qt0NLKIiE87mJPD\nxc89xy2nnw6dOtmOU+lomoSI/EX7d99lZs2aDJg61XYUERHxoua8ebiMocmDD9qOUimpMywix9q4\nkaB587jljjugbVvbaUREpDj5+ThPP82l3brBiBG201RKmiYhIsd48eabmZuXB6NG2Y4iIiJefD5p\nElNWruTgXXdp6eVSUmdYRDzM7t24Pv+ct1u2hDZtbMcREREvPn7xRZ4JCCBg4EDbUSotTZMQEQ9n\n1iz+m5fHHpfLdhQREfEmM5MpGzYw7rHHCKxd23aaSksjwyICQN7BgxyZNo1q/foRfOGFtuOIiIgX\nh5KSoH59QjSt7ZSoMywiAKTeey9nbtzI2iFDbEcREREvfvvvfzl93jw+uPhiqF/fdpxKTZ1hEQFj\nOO2TT/hb/fq0HjrUdhoREfEib9YsLnEczk1MtB2l0tOcYRGBL76g7+rV9H3pJaiusiAi4tNycgid\nN485gwfDeefZTlPpaWRYRHj33nv5s0kTGDzYdhQREfHiy8RENu7dC2PG2I5SJagzLOLn1n36Kddm\nZTG9SxcICrIdR0REipF/8CBDZ81ieEgIdO1qO06VoO9DRfxcm7lzWVizJmHPP287ioiIeFHtrbf4\nv7w89j7xhO0oVYY6wyL+bPNmeO01zouLg/btbacREZHiGANPPUWbTp209HIZ0jQJET82dcgQxufm\nYu65x3YUERHxYsGUKQxdupTt8fFaerkMaWRYxF/9+Se/fvMN208/HefMM22nERERL9a4XHwXEEC9\nYcNsR6lS1BkW8Vcvv0zK4cPkzZtnO4mIiHizdCnxy5czfMIEArTIRpnSNAkRP5R36BBbp06FPn0I\n6N3bdhwREfFi84QJULs2AbfdZjtKlaPOsIgfev/++2m1cSMZ11xjO4qIiHixKSODNvPm8Xz37hAS\nYjtOlaPOsIi/MYaun33GmIYN6XrnnbbTiIiIF3Vff52HHYdLH3nEdpQqSXOGRfzN//0fbZct48kX\nX4QaNWynERGR4vz5Jw1eeYX7//EP6NfPdpoqSSPDIn7mxTvvZEXDhnDjjbajiIiIF++PHs3nu3dr\n6eVypM6wiB/Z9d13jF22jFc7d4ZatWzHERGR4hw5wpOzZ/NI/frQs6ftNFWWpkmI+JGGL73E2qAg\nAmbNsh1FRES8efttvjx8mK3PPms7SZVWopFhx3HCi9kW4zhOlOM4CWUXS0TKmvn9d3jtNZrcfDMh\n7drZjiPlSDVbpPIz+fmYKVMIateO1sOH245TpXntDDuOEwUUeVb+woJrjEkHcoorwCJi16TBg7n6\n8GEO33677ShSjlSzRaqGr559lsjMTFYPGQLVNKu1PHn933UXzewTbB4E5LgvZwNRZZRLRMrS3r3U\nXrSIBq1aUaNjR9tppBypZotUDYfmzKF29eq00ABGuTvVOcPBwM6j2o1OcX8iUh5efplRBw7A3Lm2\nk4hdqtkilcHKlfRftIj+Dz8MjfQ2LW8adxep4vIPH2bhxImY3r11NLKISCXw/bhxHKlRA0aOtB3F\nL5xqZzgHKFwXMBjYcfwNHMeJdxwnw3GcjG3btp3iw4nIyfrkoYfotWULH+hk7aKaLeLz/li2jL+9\n+y73n302NG1qO45fKFVn2HGcYPfFuUCo+3IokH78bY0xKcaYSGNMZJMmTUqXUkRKxxj6ffYZM5s1\n49KHHrKdRixRzRapPBq/9RbzgBFTptiO4jdKcjaJGCDS/W+hBQDGmCz3baKAnMK2iPiIL7+k9g8/\ncMtjjxFYs6btNFIBVLNFKrH9+6n2/PNcdfXVhEVH207jN7weQGeMSQVSj7su4qjLKeWQS0TKwOMj\nRtC1QQOu0tLLfkM1W6TyevPOO9m6Ywd33HMPAbbD+BEdQCdSRR3KyuL1Vav4slMnCAqyHUdERIqT\nl8fH8+Yxp04dql1wge00fkXLMYtUUTWfeYafa9XiwJw5tqOIiIg377+P688/2f3vf+NokY0Kpf9t\nkSro0Nq15L32GgHDh1O3dWvbcURExIsDkydD27Y0GDzYdhS/o86wSBX0zM03c9aRI+RoPXsREZ+3\n8KWXaLFwId/+/e9QXV/aVzR1hkWqmj176LRoEVe0a0dw166204iIiBf13nqLKwID6ZKYaDuKX9Kf\nHyJVTUoKlx04wGWaKywi4vvWrKFTejqv3ncfnHaa7TR+SSPDIlWIOXSIOU8+yf4LLoDISNtxRETE\niw9GjWJLQADceaftKH5LnWGRKmThhAkM3rWLueHhtqOIiIgX+9av5/oPP+SBtm2heXPbcfyWpkmI\nVBXG0HP+fL5q25YeEyfaTiMiIl7UefVVsoDAZ56xHcWvqTMsUlV8/DHO8uX8zeXSIhsiIr7u0CF4\n5hnaXXopXHKJ7TR+TdMkRKqIsSNGkFS/Plx3ne0oIiLixWt33MEtW7eyb+RI21H8njrDIlWAWbSI\ndRs3sqlLF6hRw3YcEREpTn4+G997j+W1a1P7iitsp/F7miYhUgU4U6aQ2qAB+R98YDuKiIh488kn\n3LdtGwkul5Ze9gF6BkQquX0//si2t9+GkSOp1qCB7TgiIuLF+gkToEULql1/ve0ogjrDIpXerJEj\naQX8NmCA7SgiIuLFD3Pm0Oa770i96CIIDLQdR9A0CZHKbcsW+n//Pft69KC1FtkQEfF5Z7z9No/W\nqEH0hAm2o4ibOsMildmMGZx15AjjXnvNdhIREfFm/Xoav/ceD44aBa1a2U4jbpomIVJJmT17SJo2\njTWXXALt2tmOIyIiXrji4/nGGBg1ynYUOYo6wyKV1JqJE3no4EE+69LFdhQREfEib+dOHv3sM144\n4wyNCvsYTZMQqYwOHeLM2bP57fzzqf/ww7bTiIiIFwGzZrHMGPakpNiOIsdRZ1ikEjKzZ+P8/jun\nuVxQu7btOCIiUgxz6BBMn07tiy6idv/+tuPIcTRNQqSyycvjjnvvZXBICOaii2ynERERL+bcfTe9\nfv+drcOH244iRVBnWKSyeecdmu3eTYs+fbRykYiIrzOGoI8/pkndujQZNMh2GimCpkmIVCbGwMSJ\nPNS+PcyfbzuNiIh4s2AB//jtN/7x8ssQEGA7jRRBw0oilciBDz9kUVYWJCSoqIqIVAJfP/AAec2a\nwQ032I4iJ6DOsEgl8urYsfQEMjt2tB1FRES8WP7OO1yweDHPREZCzZq248gJaJqESGXx3Xdc/8sv\n1Lr+esJ79rSdRkREvOjwzjuk1qhBv+nTbUeRYqgzLFJZPPEE9Ro14p8zZ4Lj2E4jIiLF2bSJgDff\n5B+33QZnnmk7jRRD0yREKoMlS7j7ww9Jv+oqqFPHdhoREfHimaFDefHIEbj7bttRxAt1hkUqgR0P\nP8w7jsOy9u1tRxEREW/27OGjL78kvUULaNvWdhrxQtMkRHzdihU0+s9/WJOYSN7o0bbTiIiIN7Nm\n8VFeHvtff912EikBryPDjuPEOI4T5ThOwgm2J7n/jS/rcCIC+x5/nPygIKqPHk1NHY0sXqhmi9iV\nd/AgB556CqdvX+pccIHtOFICxXaGHccJBzDGpAM5he3jxDuOswbILod8Iv5t7Vruf/NNutapQ25w\nsO004uNUs0Xsm5+YSKtNm/jluutsR5ES8jYyPAjIcV/OBqKKuE2cMSbMXXxFpCxNnMgFAQHEDhtG\nYGCg7TTi+1SzRWwyhraffsq1DRpw5vDhttNICXnrDAcDO49qNyriNqFevpKLdxwnw3GcjG3btpU2\np4j/WbcOXnmFf9x6Kw9OmWI7jVQOqtkiNn3+OZG//ELK1KkEaACj0jjls0kYYya7RxgaOY7zl1EI\nY0yKMSbSGBPZpEmTU304Eb9x6PHHcRnDQZ2WR8qQarZI+Xlj9Gi2N2kCQ4bYjiInwVtnOAcIcV8O\nBnYcvdE9ghDjbu4AQss2noifWreOD/79b4bl5fHVmjW200jloZotYsnv6ekMWbqUGZ07Q1CQ7Thy\nErx1hufyv2IZCqQDOI5TeCRPRuF1QJi7LSKn6sknGRAQwNfz5xMdHW07jVQeqtkilpw+ezbLatXi\nrpQU21HkJBXbGTbGZAG4v0rLKWwDC47aPtA90rDmqO0iUlruucLOiBH0ufZaHC29LCWkmi1iyYYN\n8MYbnB0fT2MtvVzpeF10wxjzlz9xjDERxW0XkVPwxBMMyc+nT4sW3Go7i1Q6qtkiFW/y4MH8lpfH\nM6NGaWnfSkjPmYgvWbWKgy+/zI6WLdlTTW9PERGfl5PDjoUL2dqyJdW09HKlpOWYRXzJgw8SVKsW\nHy9ejGna1HYaERHxJjmZpNxczLvv2k4ipaShJxFfkZXFjrlz2XXrrdCsmeYKi4j4uPz9+8meOhUu\nvhgnvKgFH6UyUGdYxFfcfz+TgoJoO3Mme/bssZ1GRES8+ODeezlz+3a+uvxy21HkFGiahIgv+OIL\n+PRTbhwzhtCwMOrXr287kYiIFCcvjx4ff8yE00+n95132k4jp0CdYRHbjIFx46BlS855/HHOqVXL\ndiIREfHmnXc4be1a7n/rLdDSy5WapkmI2DZ/PrmLFjH+3HNZv22b7TQiIuKNMUwfPZrFLVvCgAG2\n08gpUmdYxKYDB2DsWDJCQ0n69FOWLl1qO5GIiHix98MPeWLDBuZ36gQBAbbjyCnSNAkRm556Ctat\no9eCBazr0IHmzZvbTiQiIl7UnTGDtU2acOTf/7YdRcqAOsMitmzcCBMnkjdgAAEXXUQL23lERMSr\nvEWLCEhLo+7EiXDaabbjSBnQNAkRWxISID+fAXv2cNddd9lOIyIiJTD5ppu4oHp1Dtx0k+0oUkY0\nMixiwzffwBtvkD9+PB0OHeI0jS6IiPi+JUtovmIF7bp1o1azZrbTSBlRZ1ikouXmwp13whlnUG3c\nOCbXrm07kYiIlMSECQyrX59hn39uO4mUIU2TEKlokybBkiX88dhjZK1caTuNiIiUgPnpJ9Lefpv8\nO++E4GDbcaQMqTMsUpGWLoXHH4frr+e5tWuJjIxk48aNtlOJiIgXC+66i/5Aatu2tqNIGdM0CZGK\nkpsLw4ZBw4bwzDOMrl6drl270rJlS9vJRESkOCtX0vfLL5n7979zzT//aTuNlDF1hkUqyqRJ8MMP\nMH8+NGpEA+Daa6+1nUpERLx57DECa9dm4MyZUKOG7TRSxjRNQqQi/PCDZ3pE3tVXM3ToUL799lvb\nqURExJvFi3nojTd4LSoKmjSxnUbKgTrDIuVt+/aCteubNoUZM1i7di0LFizg999/t51MRESKYwxH\nxozh0+rVydSUtipL0yREytORIzBoEGzeDF9/DY0bc2bjxmRnZxOg9exFRHzbe+9R/ZtvWPjCCxwa\nNsx2GiknGhkWKU/33guffw4pKdC9O7t378YYQ40aNdQZFhHxZYcPc2DsWA516IBzyy0EBQXZTiTl\nRJ1hkfIyezZMnw6jRsGNNwJw0003ceGFF2KMsRxORESKlZzMC2vW0HrrVrbt2mU7jZQjTZMQKQ/z\n5sHw4dCvH0yZ4rl6wIAB7Nu3D8dx7GUTEZHibd8Ojz5Kj4gIhl18MU104FyVps6wSFl77TUYOhR6\n9YL33oPAQM+mIUOGWAwmIiJeGQM33QR//kmfV16hzznn2E4k5UzTJETK0ssvF0yJ6NsXPvkE6tcH\nYPv27cyePZvDhw9bDigiIsWaMQPzwQekXHstu3QGCb+gzrBIWcjNhQcfLJga0b8/fPgh1K3r2fzm\nm28ydOhQVq9ebTGkiIgU64cfICGBHy+4gBFz55Kammo7kVQATZMQOVWrV8PgwbB4ccFyyy+8AMcd\ndXz77bfTs2dPOnbsaCejiIgUb+9euO46aNyYrm+/zdLNmznzzDNtp5IKoM6wSGnl5hZMixg7FqpX\nh7lzYeBJ1cqNAAAPMUlEQVTAIm/qOA6RkZEVHFBEREpk796Cc8KvWgULFkDjxpzTuLHtVFJBNE1C\n5GTl5sKsWXDWWXDrrRAZCUuXFtkRzs/Pp3///syZM8dCUBER8Wr9ejj/fPj0U3jhBe55/33GjRtn\nO5VUIHWGRUpq+XJ49FFo1w5uuQVCQuD99wsW1TjjjCLvsnPnTvLy8io4qIiIlMjChdCjB6xbBx9+\niImPZ9++fezfv992MqlAXqdJOI4TA+QA4caYySe7XaTS+v33gnnACxcWdHpXrADHgQsugOefh8su\nK2gXo3HjxixYsECLbEiFUc0W8cIY+OabgjqemlowmPH559CxIw6QkpKimu1niu0MO44TDmCMSXcc\nJ9RxnHBjTFZJt4v4JGNg/37YvRu2bSv4+eOPgq/KsrMLflauhE2bCm4fGAi9e8Ptt8OAAdC8eYke\nZtOmTdSrV4/69etrkQ2pEKrZIkXIzS2YC7xsGfz0E7z7bsHlBg0K6vr48dC4Mfv372fbtm20bt1a\nNdvPeBsZHgSkuS9nA1FA1klsP2W/xsQw8ccfGdupE50aNmR5Tg5Tli1j3Dnn0L5BA5bu3Mm05csZ\n36ULYfXrk7VjB8+sWMEjXbvSum5dFm/bxgu//MKEbt1oUacO3/7xBzN//ZVJERE0q1WLr7Zs4ZXV\nq5kaGUmjoCA+37yZV9esYXqPHjSoUYO0339nTnY2z553HnUCA/lo40bmrVvHi716UTMggPfXr+ed\n9et5qXdvAqpVY/5vv/HBhg283KcPAG+tXUv65s2k9O4NwBvZ2Xy9dSvP9+oFwOzVq8nYsYMZ550H\nwCurVvHTrl083aMHACm//MLqP/9ksvvgqxdWrmTDvn08GREBwDMrVrDt4EEe69YNgKd//pm9ubk8\n1LUrAFOWLSM3P5/7u3QB4MmlS6nuOCS4TyL++I8/Uqd6dUZ36gTAwz/8QKOaNbnLfdaDB7KyaFG7\nNiPPOguAxIwMwurVI75DBwDGfP89nRo04OZ27QAYtXgxEY0acWNYGAC3L1xI76ZNGRwaCsCIb7/l\n4ubNGdimDQDDv/2WK1q0YEDr1uTn5zP8u++4tmVLrm7ZksN5eYxYvJjYM87g8tNPZ39uLrdnZHDd\nGWdwSbNm7Dl8mFFLlvDPli25qGFDdh44wJiVKxnWtCl969blj/37SVy3jrj69eldrRq/79vHAzt3\ncuuRI5xnDOuBh4E7gAgKXsCPBwVxT7t2dLnwQn5t3ZqJS5Yw9rHH6BQezvLly5ly//2MGzeO9u3b\ns3TpUqZNm8b48eMJCwsjKyuLZ555hkceeYTWrVvz4osv4nK5yM7Opnp1HasqFcJ6zWbSJG6aNYu/\nn3EG17RuzZH8fOK+/ZZ/tG7NlWecwYEjRxi5cCGD2rTh0pYt2Zuby52LFjE4NJSo008n59Ah7vn+\ne4aGhdGveXO2HzzIvRkZDG/Xjj7NmrFl/37GZWUxon17ejZtysZ9+3jwhx+4/ayziGzcmHV//smj\nP/7IqLPPpmujRqzes4cnli5ldKdOnNOwIStzckhatozEzp05KziYZbt28dTPP3N/ly60q1+fJTt2\n8K8VK3j43HNpU68eGdu389zKlTzerRst69Rh4R9/kPzrrzwZHk7z2rX5ZutWZq1axZTISBoHBfHl\n5s241qzh6e7daVizJum//87r2dk8c9551A0M5JONG5m7bh3P9+xJrerV+WDDBt7+7TdSevcmsFo1\n3v3tN97bsIFX3J8hqevW8cmmTbx0/vkAvJmdzf9t3coL7s+Q19asYeG2bTzbsycA/161iiU7dzLd\n/Zny0q+/snL3bqZ27w7AiytXsm7vXia5P1OeXbGCLQcOMCE8HIDpP/9MzuHDPOL+THlq2TIO5OUx\n/txzC57epUsBuM/9mfLEjz9SMyCAsZ07A/DIDz8QXKMGd7s/U8ZnZXFarVrccfbZAIzLzKR1nTrc\n6v5MGfv995zVoAG3tG8PwD2LFnFuSAjD3GdxuGPhQno2acKQ0FAwhtsWLqRv06Zc1+b/27u/37bq\nM47j76dpmpaClJTBxCZgSwfbVaU11RDaRbUuhTF1YtoKRVSTNlVN1WnrxSpR/gW44gqJjjuuUPkL\nUDkUKL/WEBBMrCtrq5JfTuKfcRzbcZLvLvy1G8pJnJQ45zj+vKRI+cZN88T2+TxPzjn2+REsLnL8\n4495/N57OXzffVCp8JehIZ7s6eH3O3Ywn05zfGSEP1YqHHKOIvBXM4489BC/efVVZg4d4u/PP8/R\nzz6jv7+fDz74gMcee4zLly/zsK9H2kOjDt0NpJes717j7ZjZADAA8MADD6y5wNyHHxJMTHBsehq2\nbydTLhMkk5zM56Gri1SpRJBKcWpmBrZtI1ksEqTTnPbrSb+eLRSgs5OJ2VmCTIbi7Cxs3cpYoUCQ\nzVL261G/nisWoaOD4ZkZglyO+VIJtmzh63yeYHqaRb++4deuXAYzrk1P83Y+D/7iCldzOS4UClAq\nAfBVLse7s7NQLAJwJZvlYrEIhQIA/8lkeL9crr6yFfgyk+GTuTmYngbg3+k0lyuV6l5N4ItUiq8X\nFsBfN/3zVIr04iKkqw/Lp8kkZeeql5YEhqam6DSr7g0FBqem6N6yBSYmALg0OckPOjogkQDgo4kJ\nftrZWT1lAPhwYoJSZ2d9r+n7iQQdXV319cXxce4cHYXhYQDeHRvjnkSiutcVuDA6yoNTU/X12yMj\n/CyVghs3cM4RjIzw80wGhodZdI5geJhf5PMwOsr84iLByAi/LJUgmWRucZFgfJxfOQelEuWFBYJk\nksfvugt6eiju3ElQLPK7hx+G3bsplEoEb77JH554Avr6mCkUCF5+mSPPPQeHDjGdThM8+yx/eukl\nOHCA3KVLBK+9xjF/7lgmkyEIAk6ePAlAKpUiCAJOnToFVC+sEQQBp0+fBuCRRx6hp6dHg7BspMgz\nm08/Jbh+nT3JJFy9Wt2Ox8boS6Xgq69Y8Nvto+k0XLlCxa/3p9Nw553MLSwQJBL0ZzKwcyfF+XmC\niQl+m83CHXcw69dP+vVMpUIwOcnTuRzs2EF+bo5gaoqjvmfk/PrPt/SQAd9D0r6H/K3WQ/z6H349\n5XvImVt6SK2njC/TU2o9ZMT3kEqth/j1wi09ZNH3kOvT0wRLesi16WkuzMxAuQzA/3I53ikUvtFD\n3isWq0e7gP9ms1wslb7RUz4qlyGfB+DLdJrPK5V6T/kineb6/Dxks0C1h0ws6SmfJZPMOAepVPXh\n9b2k1lMGp6bYaVY9uka1h3y/o6PeU/41OcmPt26F8XGg2lNmlvaURALX1QUjIwBcTCToGh2tr98b\nG2NXIlFfvzM6yg+TyWqPMuPC8DA/yeerPXPbNoJkkj3bt8P997O4ezfB5CR9Bw7A0aMs9PYSPPMM\nj54+DceOUfGZvn//fgD27NnDkSNHeMjv3JH2YSudF2NmrwCvOOeGzKwfOOicO7Pa22+1b98+Nzg4\nuI7li4hsDDP7xDkX6/fHU2aLiNy02txu9G4SWWCX/7wbSK3xdhER2TjKbBGRNWo0DL8O9PrPe4Hz\nAGbWvdLtIiISCWW2iMgarTgM115l7A+nZZe86vitBreLiMgGU2aLiKxdw1f2OOfOhnytb6XbRUQk\nGspsEZG10RXoRERERKRtaRgWERERkbalYVhERERE2paGYRERERFpWxqGRURERKRtrXgFunX/YWZT\nwI3b+NbvAcl1Lud2qZZwqiWcagnXirU86Jy7p9nFxIkye92plnCqJZxqCbeWWlaV2xs6DN8uMxuM\ny2VQVUs41RJOtYRTLZtbnO5T1RJOtYRTLeE2ey06TUJERERE2paGYRERERFpW60yDMfpikmqJZxq\nCadawqmWzS1O96lqCadawqmWcJu6lpY4Z1hEREREpBlaZc+wiHwHZrY36hpERGT1lNsbR8Pwd2Rm\nz0VdgywvqjAxs8Nm1h+H54eZ9QPnoq4DwMwG/McLMail339EXotsrDhsl7I85bZye4VampLbLTUM\n1355MxuIuhaoP1kPxqGOuDT1uG00RBAmtSB3zp0HslH/de/ruBZlDVB/PM47584CvX4dZS1P+ftm\nb9SP0Wam3F6+DuV2aC3KbZTbK9TSlNxuqWEYGDCzq8TgCRIXcWrqcdpoINIwOQJk/efXgEjvhxjp\n5eZ9cc2vI+GcO++cO+GXvc65oahqaQPK7Vsot5en3I6dtsjtVhuGjzvndvuNJVJmtjcOdcSsqcdm\no4lYN5Besr47qkLixDl31jdcgL3AYJT1QP1w+YmG/1C+C+X2LZTbsaTcDtEuud1qw3BvXM7nAXZF\nXcBScWjqcdxoJH78XrChOOyNdc69CJwws+6oa9nElNvLUG5Lq9jsub11vf6jjeDvAMzsoJn1N/Mv\n/GXOb7vmnDsfl70LSznnXjSzc/4yhdnG39E8G7XRrPQYNfPnrkKWm023G0hFWEsc9TvnzkRZwJLz\nA4eo7g0bAF6MsqbNSrm9POV2nXI7/jZ1bsdqGG4QZANA2jn3BtUnaVMP5Sz5SzlMr5n1Ut1wdvmQ\nbVqANAp42Limvsog25CNpsFjFKXXgdp103uBSEPezA4D+8zssN9+oqxlYMlw1NTBqIF+oLbNdgOX\nIqqj5Sm3wym3wym3V0e5HappuR2rYbjBRjLIzZPqdwOvNL+icLUnpg+Yph9ebXC/bGhTbxRkMdpo\nIgsT59yQme3zL0TJRn1Yyf/ukYYp1F+o84KZnaE6kDwVYTlngadrQ0LUzaaVKbeX/XnK7dug3K7X\no9z+tqbldktdga62l4HqCw50SBPw58w87Zd9S16UEUUttbfESeM3mhgc+hKRCCm3v025LRIvLTUM\ni4iIiIisp1Z7NwkRERERkXWjYVhERERE2paGYRERERFpWxqGRURERKRtaRgWERERkbalYVhERERE\n2paGYRERERFpWxqGpeWZ2WEzO+c/Mv4KRiIiEkPKbIkbXXRDNg1/JaUzzrmDUdciIiIrU2ZLXGgY\nlk3BzPYC/3TO9UVdi4iIrEyZLXGiYVhanpl1A28Bv3bOZaOuR0RElqfMlrjRMCwtz8zOAXuBIf+l\n151zb0RYkoiILEOZLXGjYVhERERE2pbeTUJERERE2paGYRERERFpWxqGRURERKRtaRgWERERkbal\nYVhERERE2paGYRERERFpWxqGRURERKRt/R/0O9m8U5VDpgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1512b6e2e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pylab as plt\n",
    "\n",
    "def logsumexp(a,b):\n",
    "    m = np.max([a,b])\n",
    "    return m + np.log(np.exp(a-m) + np.exp(b-m))\n",
    "\n",
    "def hinge(x):\n",
    "    return x if x>0 else 0\n",
    "\n",
    "xx = np.arange(-5,3,0.1)\n",
    "\n",
    "plt.figure(figsize=(12,10))\n",
    "for i,alpha in enumerate([1,2,5,10]):\n",
    "    f = [logsumexp(0, alpha*z)/alpha for z in xx]\n",
    "    h = [hinge(z) for z in xx]\n",
    "    \n",
    "    plt.subplot(2,2,i+1)\n",
    "    plt.plot(xx, f, 'r')\n",
    "    plt.plot(xx, h, 'k:')\n",
    "    plt.xlabel('z')\n",
    "    #plt.title('a = '+ str(alpha))\n",
    "    if alpha==1:\n",
    "        plt.legend([ 'logsumexp(0,z)','hinge(z)' ], loc=2 )    \n",
    "    else:\n",
    "        plt.legend([ 'logsumexp(0,{a} z)/{a}'.format(a=alpha),'hinge(z)' ], loc=2 )\n",
    "\n",
    "    \n",
    "    \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The resemblance of the logsumexp function to an hinge function provides a nice interpretation of the log likelihood. Consider the negative log likelihood written in terms of the contributions of each single item:\n",
    "\n",
    "$$\n",
    "- \\mathcal{L}(\\pi) = - \\sum_i l_i(w) \n",
    "$$\n",
    "\n",
    "We denote the inner product of the features of item $i$ and the parameters as $z_i = x_i^\\top w$.\n",
    "\n",
    "Then define the 'error' made on a single item as the minus likelihood\n",
    "$$\n",
    "E_i(w) \\equiv -l_i(w) = - y_i x_i^\\top w + \\text{logsumexp}(0, x_i^\\top w) = - y_i z_i + \\text{logsumexp}(0, z_i)\n",
    "$$\n",
    "\n",
    "Suppose, the target class $y_i = 1$. When $z_i \\gg 0$, the item $i$ will be classified correctly and won't contribute to the total error as $-l_i(w) \\approx 0$. However, when $z_i \\ll 0$, the $\\text{logsumexp}$ term will be zero and this will incur an error of $-z_i$. If instead the true target would have been $y_i = 0$ the error reduces to\n",
    "$E_i(w) \\approx \\text{logsumexp}(0, z_i)$, incurring no error when $z_i \\ll 0$ and incuring an error of approximately $z_i$ when $z_i \\gg 0$. \n",
    "\n",
    "Below, we show the error for a range of outputs $z_i = x_i^\\top w$ when the target is $1$ or $0$. When the target is $y=1$, we penalize each negative output, if the target is $y =0$ positive outputs are penalized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAFKCAYAAADmEmJ8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XuclGX9//H3xUFBFBcQJU2DhdQ0\nSRc8/NCkcs20r5m2qJVmhS5qKXI+iICAwi6IJ1KB0lLTVErLzAo8lJYaC/r1kKdY0y9JiSxriiLC\nXr8/PvfIsOzuLLsz93XPzOv5eNyPnd2ZnfvN7HDtZ6/7OjjvvQAAAAA0r0PoAAAAAEDSUTQDAAAA\nGVA0AwAAABlQNAMAAAAZUDQDAAAAGVA0FyDnXKVzbpVzzkcfK7PwnE0us+Kcq2ruvnzknCtzzq2K\n8Vwr0j730cfSuDI0kWmFc640xLmBYkR73XbF3l4jfp1CB0B2OefGSxoRHTWSBku6xzlX571fkoNT\njpfUIwfPWwxqJU0IHUKSnHPlkoZJKgudBSgWtNd5JTHtdTrnXIWkekklkmq99ysDRypo9DQXEOdc\niaQqScd775d57+u998tk/9HPSHtcRVrPxj3R96XuW+icWx8d46OvLY0+rm90vqXRzddaet7oL/Gl\nUS/HCjXinCuPvm996vui51qa9phVUWGX6plZH53n457R6Dwr0v4NS1M9EdFjx6flXBida5vnaE22\nll6rtPtWpOUdn967k/ZvKY1+Xk2+xtHrtT56fK6LWYplIEa017TX7RW9Dqn3zxJJk+I4b1Hz3nMU\nyCGpXNKKDI8pleSjx5ZIukfSwui+Ckmroq+XRY8rie7zzTyfb8Xzpu5bKKms0feXSFqf9n0LJd0T\n3bciyjQ+9bXUOaN8qfNUNTpPRXTfqui5S6Ln92n/Ti+pMu2cK6L7yiStakW2Zl+r6P6qtFxLo/tL\nG71mZek/ryZey/Fpz7U0pvfQ+lRODg6O3B2017TXWXgPVaZyR5/f0/hnxpHdg+EZhaVUUl2Gx1RI\nWuStR0POuQmyxm5E+vN471c653p47+tbee6Mz+u9H9HE950uaVmj73stum9Y9Bx1kgalfc/HuZxz\ndbKGMKXeR5c1nXPLos/rJS1zzimt52Gl935RdHtE1IOR/jyZsqU091otVdQrIbvkukhSuXOuVlJr\nLp/Ve++ro9t3yX4RfCzKOljWyzAh1WvVzGsMIHlor2mv29tel0hal/Z5nex9xRCNHKFoLiy1kno2\n/mL0H/T0qNHpJfuLW5Lkva9NNT7e+yXOuZ6yMXU9Jc2WVN34+ZrR7POmZWtKf0kVjS4lpvLURo1W\nbaMGblJ0uSz1Cyf9udN/CdVr2wYlXeM8tbLGprXZWnytvPfLokuNpdFzL5Vdcl0uaVkzmdJl+mWa\net5U5nLZL6xtpH72LTzPMu99cz8bALlDe017vQ3a6+SjaC4sNZLKnHOljf5jnS4bJ7dI1ij1T90R\n/SdN9QKUyv5TLopuL3XOrUz95Z5Bs8+bwSpJS7z3w9K+NzXmLfXXennq3+Rs0kO5pOO89/XOZpoP\n2u5ZM2vc4DZu1DJla81rtVL2ui+LjipZI97uySRRb0mlrFdDsgb+vCYeVy/7uQNIFtrr1qO9blpq\nAmBKTzX/Bw+ygKK5gESN0gRZg5CajV0u+8+f+o+/RNIq59w90f2LJd0d3Vch6Qzn3LC0p218Caw5\nLT1vS+6WVBU1uDVR1lJJx8vGpA2TXdZaGH2tp6S66N9aIruc2JZGoixqxO6Ozrkyes7WZmvNa7VM\nNr5vWPTcPSX19Nmb3Xy8tja8JU1dmqXnAkgm2usdQnsdZWzUXqdej5SSLOZFU0IPqubI/iH7j79K\nNjlhlaTKRvdXpN1/j7adEJGaBLFe208w8E2cy2d6XlnDtaqFvOVp37dU1piNVzQxJXrMKkkVaRnX\nyy5vlWvr5I9tziNrTManZ42euyJ6jnuir63Q1kkfZY2eY7tsmV6rRt+bPjnnHqVNENH2E0tSeRr/\nO7Z5XNrXK6NzbPNatfO9w0RADo4YD9rrj7+H9rpt75/y9CP0+7nQDxe96EDRiC4ZnuHTLuPlm9QS\nSN4uNy6U/RKgxxhAQaG9RpIwPAPIT7XSx79Q7qEBBoDEor0uEBTNQB6KGl0aXgBIONrrwsHwDAAA\nACADttEGAAAAMqBoBgAAADKgaAYAAAAySOREwD322MP37ds3dAwAaJMVK1a87b3vHTpHnGi3AeSr\n1rbZiSya+/btq5qamtAxAKBNnHOvh84QN9ptAPmqtW02wzMAAACADCiaAQAAgAwomgEAkiTnXFmj\nzyucc+XOufGhMgFAUlA0AwDknCuXdE/a52WS5L1fJqm+cUENAMWGohkAkCqO07f6PUNSfXS7VlJ5\n7KEAIEFyVjRzmQ8A8lqJpLq0z3uFCgIASZCTopnLfAAAACgkOSmaucwHAHmvXlLP6HaJpHWNH+Cc\nq3TO1TjnatauXRtrOACIW1xjmuO5zLd5c06eFgCK0F2SSqPbpZKWNX6A936R936w935w795FtQEi\ngKTZvFnyPqenKJyJgPPnS4cfTuEMAG3gnKuQNDj6KO/9yujr5ZLqU58DQCJVVUlf+pL0wQc5O0Vc\nRXPuL/P16yc984y0eHG7ggJAMfLeL/He9/DeL0n72iLv/TLv/aKQ2QCgRf/+tzRnjtSjh9S1a85O\nE1fRnPvLfF//ujR0qDR1qvTOO+0KCwAAgDwxfbq0caP1NudQrlbPiP8yn3M2RGPdOunKK7P+9AAA\nAEiYF16wUQYXXih9+tM5PVWuVs8Ic5mvrEz6zneka66RamszPx4AAAD5a9w4qXt3G2mQY4UzETDl\niiukTp2kiRNDJwEAAECuLF0qPfigNGWK1Cv3+y8VXtG8zz7S+PHSPfdIf/lL6DQAAADIti1bpLFj\nbSGIH/4wllMWXtEs2Yu4997SqFFSQ0PoNAAAAMimn/1MevZZWzVj551jOWVhFs3dutlkwOXLpTvv\nDJ0GAAAA2bJhgw3JOOooadiw2E5bmEWzJJ19tk0MnDhRev/90GkAAACQDfPmSWvWSFddZaunxaRw\ni+YOHWwJutWr7SMAAADy25tvStXV1sM8ZEispy7colmyzU5OPdXGu6xZEzoNAAAA2mPqVOmjj6TZ\ns2M/dWEXzZL9NbJpk3TZZaGTAAAAoK2efVa6+Wbpoouk/v1jP33hF80DBtiLe/PN0v/+b+g0AAAA\naItx46SSEpsEGEDhF82Svbg9ekijR0veh04DAACAHfH730t//KMNz+jRI0iE4iiae/SQpk+XHn5Y\n+u1vQ6cBAABAa23ebHtwDBggXXhhsBjFUTRL0vnnSwccYC/6Rx+FTgMAAIDWuOUW6YUXpKoqaaed\ngsUonqK5c2db1++VV6QbbwydBgAAAJm8+64t5nDMMbYiWkDFUzRL0le/Kh13nA3VqKsLnQYAAAAt\nqa6W/vMf6/iMcSOTphRX0eyc7R5TXy/NmhU6DQAAAJqzerXVbWeeKR15ZOg0RVY0S9LnPicNHy4t\nWCC9+mroNAAAAGjKlClSQ0OQjUyaUnxFsyTNnGkDycePD50EAAAAjT39tHTrrdLIkVLfvqHTSCrW\norlPH2nSJOm++6RHHw2dBgAAACneS2PGSD17Wr2WEMVZNEu20cm++9rHhobQaQAAACBJDzwgPfKI\nLdxQUhI6zceKt2ju2lWaM2dr9z8AAADC2rzZtsvef39pxIjQabZRvEWzZLMxjzhCmjxZ2rAhdBoA\nAIDitnix9NJLttRc586h02yjuIvmDh2kq6+W1qyR5s4NnQYAAKB4/fe/0rRp0tCh0te+FjrNdoq7\naJakIUOk00+3v2hWrw6dBgAAoDjNmSOtXWtrMwfeyKQpFM2S/ZC2bJEuvTR0EgAAgOLzxht29f+s\ns6RBg0KnaRJFsyT16yeNGmUTAmtqQqcBAAAoLpMn28crrgibowUUzSmTJkm9e9sSdN6HTgMAAFAc\namqkn//cOjD32y90mmZRNKfsvrs0Y4b02GPSvfeGTgMAAFD4UhuZ7LmnNHFi6DQtomhOd+650sEH\n2/baH34YOg0AAEBh+/WvpT//Wbr8cql799BpWkTRnK5TJ5uxuWqV9KMfhU4DAABQuD76yDoqP/MZ\n67hMOIrmxk44QfrKV2yoxttvh04DAABQmG66SXr1Vdsro1On0Gkyomhuyrx50nvv2aUCAAAAZFd9\nvdVZxx0nnXRS6DStQtHclIMPliorpRtvtK0cAQAAkD1XXinV1VlHZQI3MmkKRXNzpk+XunWTxo4N\nnQQAAKBwvPaadO210jnnSIceGjpNq1E0N2fPPW2HwAcekJYuDZ0GAACgMEyaZGOYZ80KnWSHUDS3\n5OKLbbfAMWNsm20AAAC03ZNPSnfdZVfy99kndJodQtHcki5dpKoq6bnnpJtvDp0GAAAgf3lvOy/3\n6SONGxc6zQ6jaM6kokIaMkSaMkV6993QaQAAAPLTL38pPfGENHOmtOuuodPsMIrmTJyTrr5aeust\nac6c0GkAAADyz4cfShMmSIccIn3ve6HTtAlFc2sccYT07W/bboGvvx46DQAAQH750Y+k2lpbYq5j\nx9Bp2oSiubWuvNJ6nSdNCp0EAGLhnKtwzpU75ypDZwGQx9atsyEZX/mK9OUvh07TZhTNrbXffraK\nxp132sxPAChgzrkySbXe+2WSaqPPAWDHzZwp/fe/tl12HoutaC6IHouJE23G5+jRNgMUAApbVfSx\n1Hu/MmgSAPnp1VdtaMbw4dJnPxs6TbvEUjQXTI/FrrvaQtxPPCHdfXfoNACQM1GRXOucWy+pLnQe\nAHlq4kRbwnfGjNBJ2i3O4RmF0WPx3e9Kn/uczQDduDF0GgDICedciaR6SbMlLXbOlTbxmErnXI1z\nrmbt2rWxZwSQcI89Jv3qV1Yz9ekTOk27xVI0F1SPRceOW1fRuPba0GkAIFcqJc323ldLOk9SReMH\neO8Xee8He+8H9+7dO/aAABKsocHmgu2zjw1rLQBxDc8orB6L446TTj5ZuuIK6T//CZ0GAHLKe79E\n1oYDQOv84hfS8uW2+tguu4ROkxVxDc8ovB6LuXOlDz6Qpk0LnQQAsi5qryujSdyV3vtFoTMByBMf\nfGBL9B52mHTWWaHTZE3sS84VTI/FAQdIF1wgLV4sPf986DQAkHXe+2rv/RIKZgA75LrrpDfesOGs\nHQpndeO4xjQXZo/FtGlS9+7S2LGhkwAAAIS3dq0NyTj5ZOmLXwydJqtiK/8LsseiVy9p6lTpD3+Q\nHnwwdBoAAICwpk+XNmyQqqtDJ8m6wukzD+UHP5AGDLAZops3h04DAAAQxosvSgsXSuefLx14YOg0\nWUfR3F477WSTAl980cY3AwAAFKPx46Vu3Qp2kQSK5mw45RRp6FAbqlGf/3McAQAAdsjDD0u//a00\nebKUD6ugtQFFczY4J82fL61bZ4PfAQAAikVDgy2K8KlPSSNHhk6TMxTN2VJWJp1zju0SWFsbOg0A\nAEA8brtNevppafZsqUuX0GlyhqI5m2bNkjp1sj3WAQAACt3770uXXiodcYR05pmh0+QURXM27bOP\nDYJfskR6/PHQaQAAAHLrqqukf/3LPjoXOk1OUTRn29ixVjyPHm1jfAAAAArRmjVSVZV02mnSMceE\nTpNzFM3Z1q2bTQZcvly6887QaQAAAHJj2jRp0yYrnIsARXMunHWWTQycONHG+gAAABSS55+XfvKT\nrZu8FQGK5lzo0EG6+mpp9Wpbig4AAKCQjB0rde8uXXZZ6CSxoWjOlWOPtTE+c+bYmB8AAIBC8Ic/\n2HHZZVLPnqHTxIaiOZeqqmysz5QpoZMAAAC035Yt1stcWmpDM4oIRXMuDRggXXSRdMst0jPPhE4D\nAADQPrfcYuOZq6qknXcOnSZWFM25lrp0MWaM5H3oNAAAAG3z3ntW1xx9tPSNb4ROEzuK5lwrKZGm\nT5cefli6//7QaQAAANqmulr697+LYiOTplA0x2HECOnAA20M0KZNodMAAADsmNWrpXnzpDPOkI48\nMnSaICia49C5s73RXn1Vuumm0GkAAAB2zJQpNglw9uzQSYKhaI7LSSdJ5eU2VKOuLnQaAACA1nn6\naenWW6WRI6V+/UKnCYaiOS7O2Rig+npp5szQaQAAADLz3oaX9uwpTZ4cOk1QFM1xGjhQGj5cWrBA\neuWV0GkAAABa9sADtpjB9Om2uEERo2iO28yZUpcu0oQJoZMAAAA076OPpHHjpP33t0UNihxFc9z6\n9JEmTZLuu0965JHQaQAAAJq2eLH00ku21FznzqHTBEfRHMKoUdJ++0mjR9tMVAAAgCR55x1p2jRp\n6FDpa18LnSYRKJpD6NpVmjPHtta+7bbQaQAAALY1Z4709ttFu5FJUyiaQznzTFscfPJk25YSAAAg\nCV5/Xbr6aunss6VBg0KnSQyK5lCck+bPl9askebODZ0GAADATJ5sdcoVV4ROkigUzSENGWLbUc6d\na9tTAgAAhPS3v0l33GHzrvbdN3SaRKFoDm3OHKmhQbr00tBJAABAMfNeGjNG2nNPaeLE0GkSh6I5\ntL59pUsuse0pa2pCpwEAAMXqvvukxx+XZsyQdtstdJrEoWhOgsmTpd697VKI96HTAACAYrNpkzR+\nvHTQQbZ7MbZD0ZwE3bvbToGPPSbde2/oNAAAoNjceKP0j39I8+ZJnTqFTpNIFM1JMXy4dPDBtl3l\nhx+GTgMAAIrF+vU2JKO8XPrKV0KnSSyK5qTo1MkWEK+tlRYsCJ0GAAAUi1mzrHCeN4+NTFpA0Zwk\nJ5wgnXiiDdV4++3QaQAAQKGrrZWuv1763vekz30udJpEo2hOmnnzbIfA6dNDJwEAAIVu4kSpc2fr\nsEOLKJqT5qCDpMpK6aabpBdfDJ0GAAAUqr/+VbrnHls1Y++9Q6dJPIrmJLr8cqlbN5sUCAAAkG2p\njUw+8Qlp7NjQafICRXMS9e4tTZkiPfCAtHRp6DQAAKDQ3H239OSTNgmwW7fQafICRXNSXXSR1K+f\n/RW4ZUvoNACKkHOuzDlX4ZyrCJ0FQBZt3GhjmQcOlM45J3SavEHRnFRdukhVVdJzz0k33xw6DYDi\nNMl7v0RSqXOuLHQYAFmyYIH0z3/aUrcdO4ZOkzdiK5rpsWiDigrp6KNtqMa774ZOA6CIRG31ckny\n3ld771cGjgQgG95+24ZknHSSbWaCVouzp5keix3lnDR/vvTWW9Ls2aHTACguh0vqFXV4jA8dBkCW\nzJhhHXHV1aGT5J1YimZ6LNrhiCOkb3/biufXXw+dBkBxWZdqr5u6Suicq3TO1TjnatauXRt/OgA7\n5pVXpBtvlM47Tzr44NBp8k5cPc30WLTH7NnW6zxpUugkAIrHOkm10e16WTu+De/9Iu/9YO/94N69\ne8caDkAbjB9vc6Yuvzx0krwU5/AMeizaat99bQ3FO++05WEAIPeWSCqNbpcouloIIE/96U/Sr39t\nHXB77RU6TV6Kq2imx6K9JkyQ+vSRRo2yBckBIIe897WS6qNOjl7RnBQA+aihwZaw/eQnrY5Am8RV\nNNNj0V677ipdcYX1NN99d+g0AIpA1JmxxHs/IXQWAO1wxx3SihXSlVdKXbuGTpO3Yima6bHIknPO\nkT73Oet13rgxdBoAAJB0H3wgTZ4sDRpkCwugzWIb00yPRRZ07Lh1FY1rrgmdBgAAJN3VV0v/93+2\nkUkH9rRrD169fPOlL0lf+5pdYvnPf0KnAQAASfWf/9gKXKecIg0dGjpN3qNozkfV1Xa5Zdq00EkA\nAEBSTZ9uwznZyCQrKJrz0QEHSBdeKC1eLD33XOg0AAAgaf7+d2nRIun886X99w+dpiBQNOerqVOl\n3Xe3JWRYgg4AAKQbN07abTeuSmcRRXO+6tXLCuelS6Xf/z50GgAAkBTLlkm/+5106aXSHnuETlMw\nKJrz2YUXSgMGWG/z5s2h0wAAgNC2bLG6oG9f6aKLQqcpKBTN+WynnaS5c6UXX7RxSwAAoLjdeqv0\n7LPSnDlSly6h0xQUiuZ8d8op0he+YGOW6utDpwEAAKFs2GBDMo48Ujr99NBpCg5Fc75zzhYsX7fO\n1m4GAADFad48ac0aqwucC52m4FA0F4KyMtti+9prpdra0GkAAEDc/vUvW4/59NOlo48OnaYgUTQX\niiuukDp1kiawSzkAAEVnyhRbFGDOnNBJChZFc6HYe28rmJcskR5/PHQaAAAQl5UrpZ/9TLrkEqlf\nv9BpChZFcyEZM0baZx9p1CipoSF0GgAAkGve2+//Xr2kyZNDpylorSqanXPdG33eNxdh0E7dutlk\nwJoa6Y47QqcBEBDtNlAkfvMb6dFHpcsvt52CkTOt7WnuKUnOuTnOudmSKnIXCe1y1lnSoEHSpEnS\n+++HTgMgHNptoNBt2mTbZX/mM1JlZeg0Ba9Tax7kvf9ndPMu7/3TzjkGzCRVhw7S/PnS0KH2ccqU\n0IkABEC7DRSBG2+UXn1VeuABWwwAOdVkT7Nzbqxz7ktN3OUlyXv/Wk5ToX2OPVY67TSbQfvmm6HT\nAIgB7TZQZOrqbEjG8cdLJ54YOk1RaG54xh6S+jvnbnLO3eWcu9E5d64kpT4i4aqr7bLNZZeFTgIg\nHrTbQDGZOVN65x02MolRk3353vuJ0c3FkuSc213SYEnHS1oVTzS0S//+0sUX2xCNiy6SDj00dCIA\nOUS7DRSRV16RFiyQhg+XDjkkdJqi4bz3oTNsZ/Dgwb6mpiZ0jPxXXy8NGCANHCg99BB/iQIxcc6t\n8N4PDp0jTrTbQIxOPVVatszGM/fpEzpN3mttm93qdZqbGSuHJCspsfFOjzwi3X9/6DQAYka7DRSg\nRx+V7rvPVsmiYI7VjmxuUpqzFMidykrpwAOlsWNtjDOAYkK7DRSShgZp9Ghp331tIzPEakeKZq7t\n56POnaV58+wSzo03hk4DIF6020Ahue026emnbXWsrl1Dpyk6O1I0J2/wM1rnpJOk8nIbqlFXFzoN\ngPjQbgOFYsMG2yb7iCOkM88MnaYo7UjRjHzlnK2i8c47tkQNAADIL/Pm2d4L8+fbRmaIHcMzisUh\nh9jSNAsW2FI1AIoB7TZQCN580/ZfGDZMOvro0GmKVquLZu/94lwGQQxmzJC6dLF96gEUPNptoEBM\nmSJt3mxjmREM/fvFpE8f6dJLpd/8xtZtBgAAyfb009JPfyqNHCmVsiBOSBTNxeaSS6R+/ezj5s2h\n0wAAgOZ4L40ZI/XqZZMAERRFc7Hp0kWaO1d6/nnpxz8OnQYAADTn/vttg7Lp023DMgRF0VyMTjtN\nOvZY6bLLbKttAACQLJs22cZkBx5oG5UhOIrmYuScdM010rp1LEEHAEAS3XSTbUw2b55tVIbgKJqL\n1WGHSd//vnTddSxBBwBAktTV2ZCM8nLboAyJQNFczGbNsm04WYIOAIDkmDXLhk9edZVdHUYiUDQX\ns/Ql6JYtC50GAAC8+qptRDZ8uDRwYOg0SEPRXOxGjrQl6EaNYgk6AABCmzBB2mkn5hwlEEVzsevS\nxSYZPP+8tJjNwwAACOZPf5LuvVeaNMmuBiNRKJohnXqqNHQoS9ABABBKQ4M0erS07772EYlD0Qyb\nZHD11TZbd8aM0GkAJIxzbnzoDEDBu/12aeVKafZsm6SPxIm9aKbxTajDDrNJB9dfzxJ0AD7mnCuX\ndHzoHEBBe/992yb78MOlb34zdBo0I9aimcY34VJL0I0dGzoJAADFY+5c6V//kubPlzowCCCp+Mlg\nq732kqZMsb3uly4NnQZAYM65Mu8961ECubR6tVRVJZ1+unTMMaHToAWxFc00vnli5EiptJQl6ABI\nUs/QAYCCN3GiTQKsqgqdBBnE2dNM45sPdt7ZlqB74QVp0aLQaQAE0pqODudcpXOuxjlXs3bt2rii\nAYXjySeln/9cGjNG6ts3dBpkEEvRTOObZ77+dekLX5CmTpXWrw+dBkAYpc65CudcpaSezrmyxg/w\n3i/y3g/23g/u3bt3gIhAHvNeuuQSW4954sTQadAKcfU00/jmE5agA4qe936J935J9GlJ0DBAIbrz\nTumpp2yJud12C50GrRBL0Uzjm4cOPVQ691xpwQLp5ZdDpwEQSNSh0d97vzJ0FqBgbNhg22WXlUnf\n+U7oNGilWFfPoPHNMzNn2hJ0Y8aETgIAQOGYN89WzbjmGpaYyyP8pNC8vfayrbUfeED6wx9CpwEA\nIP+llpgbNkz6/OdDp8EOoGhGyy6+WOrfXxo9miXoAABor0mTbIm56urQSbCDKJrRstQSdH//u7Rw\nYeg0AADkryeflG6/nSXm8hRFMzI75RTpi19kCToAANqKJebyHkUzMnPOJivU10uXXx46DQAA+Se1\nxNyVV7LEXJ6iaEbrDBy4dQm6F18MnQYAgPzx/vtbl5g755zQadBGFM1ovVmzpF13tctL3odOAwBA\nfmCJuYLATw6t17u37RD4xz9Kv/lN6DQAACQfS8wVDIpm7JgLLpAOOkgaNUrauDF0GgAAkm3SJGnL\nFiuckdcomrFjOneWrr1Weu016aqrQqcBACC5nnrKlpgbPVrq1y90GrQTRTN2XHm5dNppNgN49erQ\naQAASJ70JeYmTQqdBllA0Yy2ueoq29Fo/PjQSQAASJ4777TNTFhirmBQNKNt+vaVxo2zRuGxx0Kn\nAQAgOd57z35HDhokfec7odMgSyia0XYTJ0r77itdfLFNcgAAANa7/Oab0nXXSR07hk6DLKFoRtvt\nsoutPfnMM9KPfxw6DQAA4a1aZUMYzz5bGjIkdBpkEUUz2mfYMGnoUOnSS6X160OnAQAgrNGjpZ12\nkubMCZ0EWUbRjPZxzi4/rV8vTZsWOg0AAOH8/ve2+ddll0l77x06DbKMohntN3CgdP750g03SM89\nFzoNAADx27TJlpgbMEAaOTJ0GuQARTOyY8YMaffdraHwPnQaAADidf310ssvS9dcI+28c+g0yAGK\nZmRHr17SrFnSI49Iv/xl6DQAAMTn3/+WLr9cOukk6atfDZ0GOULRjOyprLShGmPGSO+/HzoNAADx\nmDxZ2rhRuvrq0EmQQxTNyJ64RhA/AAAUDElEQVSOHW1S4BtvSNXVodMAAJB7f/ubdMst0qhR0v77\nh06DHKJoRnYNHSqdeaZUVSXV1oZOAwBA7jQ0SBddJPXpI02ZEjoNcoyiGdk3b57UqROzhwEAhe3W\nW62nuapK2m230GmQYxTNyL599pGmT5d++1vp/vtDpwEAIPv++19p4kTpqKOks84KnQYxoGhGblx8\nsXTwwfaRSYEAgEIzY4b01ls2l6cD5VQx4KeM3OjcWfrRj6R//pOtRAEAheWFF6Rrr5W+/33p8MND\np0FMKJqRO0OHSt/+to31+sc/QqcBAKD9vJcuvFDq3p1OoSJD0YzcmjtX6tLFhmmwUyAAIN/9/OfS\nn/8szZ4t7bFH6DSIEUUzcusTn7BxXw8+KN13X+g0AAC0XX29NHasdMQR0rnnhk6DmFE0I/d+8APp\nkEOkSy6RNmwInQYAgLaZOtUm/91wA5P/ihA/ceRep07WwLzxhnTllaHTAACw455+2ia4X3CBNGhQ\n6DQIgKIZ8TjmGOmcc2yM88svh04DAEDrNTTY5L899pBmzQqdBoFQNCM+VVXSLrvYcA0mBQIA8sUt\nt0hPPmkdPz16hE6DQCiaEZ+99rLleR56yGYfAwCQdOvWSRMmSJ//vHT22aHTICCKZsSrstK2HB01\nyhoiAACSbPJkWzXjRz+SnAudBgFRNCNeHTpIixZZAzRhQug0AAA0729/kxYvlkaOtFWgUNQomhG/\nQw6RxoyRfvITWyAeAICk2bxZGjHC9huYPj10GiQARTPCmDpV6tvXGqQPPwydBgCAbV1zjfTMM9L1\n10u77RY6DRKAohlh7LKLrd380ktSdXXoNAAAbPXaa9a5c8op0qmnhk6DhKBoRjgnniidcYZ0xRXS\nK6+ETgMAgC2JesEFUseO0oIFTP7DxyiaEdY110hdulgDxdrNQKI45yqjoyp0FiA2d94p/eEP0uzZ\n0ic/GToNEiS2opnGF03q08c2PXn4YVs8HkAiOOfKJS3z3i+SVBp9DhS2deukSy6RjjzSOnOANLEU\nzTS+aNF550nHHiuNHi29+WboNABMqaRUW10bfQ4UtnHjpPXrbWnUjh1Dp0HCxNXTTOOL5nXoYMvP\nbdoknX8+wzSABPDeL4o6OiSpTFJNyDxAzqWueI4bJw0cGDoNEiiWopnGFxkNGCDNmiXdf7/0i1+E\nTgMg4pwrk7TSe7+yifsqnXM1zrmatWvXBkgHZMnGjdZp07+/dNllodMgoWKdCEjjixaNHGnjyC66\nSHrrrdBpAJhy732T23dGHSKDvfeDe/fuHXcuIHumTZNefVVauFDq2jV0GiRU3Ktn0PiieR072jCN\nd9+1whlAUM65Su99dXSbuSgoTE89Jc2bZ/NrjjsudBokWKyrZ9D4IqODD7YF5e++W/rVr0KnAYpW\n1E5XOedWOefWh84D5MTGjdL3vifts48VzkAL4lw9g8YXrTN+vHToodKFF0p1daHTAEXJe7/Me9/D\ne98/+rgsdCYg6y6/XHrxRWnxYql799BpkHBxTQSk8UXrde5sM5jXrbNxzgAAZNvy5VJ1tTR8uHTC\nCaHTIA+wIyCS6dBDpcmTpdtvl5YsCZ0GAFBINm6Uvvtdae+9pauuCp0GeYKiGck1ZYo0eLA0YoS0\nZk3oNACAQjF1qvT3v9smJrvvHjoN8gRFM5Krc2fpttuk99+3y2dsegIAaK9HH7VJf5WV0oknhk6D\nPELRjGQ78EBp7lzpwQdt/UwAANqqvl76zndsQ63580OnQZ6haEbyXXih9OUvS2PGSK+8EjoNACBf\n/eAH0ptv2nyZbt1Cp0GeoWhG8nXoIN18s7TzztLZZ0sffRQ6EQAg39x5p3THHTae+YgjQqdBHqJo\nRn7YZx+bsPG3v0mXXho6DQAgn7zxhnTBBdJRR9nKTEAbUDQjf1RUSOefb2Ocf/e70GkAAPlg82bp\nrLPs4+23S506hU6EPEXRjPwyf740cKBN5PjXv0KnAQAk3dSp0mOP2WTy/v1Dp0Eeo2hGfunaVbrr\nLluY/lvfsp4DAACa8uCD0uzZ0rnnSt/+dug0yHMUzcg/Bx4o3Xij9Oc/SzNnhk4DAEii1att8vjA\ngdJ114VOgwJA0Yz8dPbZtgXqzJnWkwAAQMrmzdKZZ0offijdfbddpQTaiaIZ+WvBAutB+Na3pFdf\nDZ0GAJAUkydLf/mLrbp0wAGh06BAUDQjf3XrJt13n9Sxo/T1r0vvvhs6EQAgtDvusFWWLrhA+uY3\nQ6dBAaFoRn7r29cmBr78snTOOVJDQ+hEAIBQli+Xhg+Xjj1Wuuaa0GlQYCiakf+OO856Fe69V7ry\nytBpAAAhrFljVx332ktaskTaaafQiVBgWOEbheGSS6SVK6XLLpMOPlg69dTQiQAAcdm40dr9d96R\n/vpXqXfv0IlQgOhpRmFwziZ8HHWUTQx88snQiQAAcfBeqqyUnnpKuvVWmyAO5ABFMwpH167Sr38t\n7b23dPLJ0qpVoRMBAHJtyhTpttukGTOk004LnQYFjKIZhWXPPW3d5oYG6cQTpbfeCp0IAJArN9xg\nc1nOO8+KZyCHKJpRePbfX/rNb2w3qBNOkOrrQycCAGTbL38p/fCHdmXxhhtsmB6QQxTNKExHHy39\n6lfSCy9I//M/0oYNoRMBALLld7+zNZiPOkr6xS+kTqxrgNyjaEbh+spXbJH7J56wZYg++CB0IgBA\ney1damOXDznEiudddgmdCEWCohmFraJCuvlm6aGHpK9+lR5nAMhnjzwinXKKDcP74x+lkpLQiVBE\nKJpR+M45x5Yh+tOfbHIg220DQP757W+tDe/XT1q2TOrVK3QiFBmKZhSHs86yoRp//av0pS+xqgYA\n5JO77rLNSz77WesA2XPP0IlQhCiaUTzOOMO22n7hBWnIEOkf/widCADQEu+l666zSX9DhkgPPyzt\nsUfoVChSFM0oLiefbI1ufb01wE88EToRAKApmzfbknIjR9o45gcflLp3D50KRYyiGcXnqKNsmEb3\n7tLQobb9NgAgOerqbLnQG26Qxo2zNZlZJQOBUTSjOO2/v7R8uXTccdKIEXZs3Bg6FQBg+XKprMyu\nCi5aJFVXSx0oVxAe70IUrx49bDb2pEnWMB9xhPT886FTAUBxamiQrr9eOuYYG8v8+OO2PTaQEBTN\nKG4dO0pXXmkL5L/1ljR4sHTttdZ4AwDi8cYb0gknSBdfLJWXSytXWkcGkCAUzYBka38++6w11pdc\nYttwP/dc6FQAUNi2bJEWLrTd/Z54QrrpJrsCyBrMSCCKZiBlzz2l+++XbrvNlqMrK5PGj7eVNgAA\n2fXUU9KRR0rnny8NGmQdFSNGSM6FTgY0iaIZSOecbYTy0kvS2WdL8+ZJAwbYkI1Nm0KnA4D898or\n0re+ZSsZrVljG0899JDt9AckGEUz0JRevaSbb5ZWrJAOO8yGbPTvb4vsv/9+6HQAkH9eekkaPlw6\n6CDp17+WJk+2r33zm/QuIy9QNAMtOeww6Y9/tKNfP1tk/1Ofki69VHr99dDpACDZGhps6biTT5Y+\n8xnp5z+XLrpIeu016YorpN12C50QaDWKZiAT56Tjj5f+/Gc7/t//k+bMkUpLbfH9O+6Q3n03dEoA\nSI7XX5dmzZI+/WlbD//JJ6Vp02yVjKuvtjkkQJ7pFDoAkFc+/3k7Xn/d1nb+6U+lBx6QunSRvvpV\nadgw6ctftjWgAaBYeG/r3N93n3TvvdLTT9vXv/hF6fLLpW98Q+raNWxGoJ0omoG2+NSn7NLizJm2\nJfddd0lLlthWrx062Ezw446z48gjuQQJoLB4L9XWSn/6k12Be/RR60xwzib4VVdboVxaGjopkDXO\nex/PiZyrkFQvqcx7X93SYwcPHuxrampiyQVkzZYtts7osmU2E/zJJ6XNm+2XyGc+YxunDBokDRwo\nHXigtNdeTH4pUM65Fd77waFztBftNiRJH31kY5CffVZ65hk7Vq60lS8kaY89pGOPtatsX/ua9IlP\nhM0L7KDWttmx9DQ758okyXu/zDlX6pwr896vjOPcQGw6drTtX485Rpo+3cY5P/64tHy5VFNjkwlv\nvXXr47t3t+K5f39p3323PT75SVvBo2PHYP8cFDfa7SLy3nvSv/9tx5o11mO8atXW4/XXrVNAsjbp\noIPsKtqQIdLQodYpQAcAikBcwzPOkLQ0ul0rqVwSjS8K22672U6DJ5649Wtvvim98IL08su21NKL\nL1qP9JIl1puTzjmppMSK55497WPq9q672tGt2/bHrrvaGOvOnaWddrKj8e3OnW0YCdA82u2kaGiw\nq1ZbttjH1LFxo/TBB1uPxp9/8IEtkfnOO3bU19uRur1+vRXKGzZsf84ePewP+sMPl84809arHzhQ\nOvhga1+AIhRX0VwiqS7tc/bHRHHae287jj9+2683NEhr10r/9392rF4tvf22tG6dVFdnH996y4rs\nujrrGWpoaF+WTp2seO7UyXqPOnRo3+Hctr1NSbi9I49ryhe+YDP+i1Nu2+0nnrB1etOHCKZuN/cx\nW49J0jlTBXDjgjj9a9kYRrnLLvZHeOro3Vvaf3+pT5/tj09+ksnMQBMSMxHQOVcpqVKS9ttvv8Bp\ngJh16GBjnPfay8Y+Z+K99OGH1kP03nv2Mf348EPbwXDTJuvBbu72hx/aL+aGhuwc6flC3870uNYU\nIjHN+chX7Wq3vd/6nkn9AZP+h1dzH1u6b0cek+3na+s5O3Xa/ujYMfPXOna0Ht+uXe1Iv53+tV12\nkXbf3f5ABtAucRXN9ZJ6RrdLJK1r/ADv/SJJiySbUBJTLiA/OWe/ELt0sSEbQPbltt0eMsRWXgCA\nPBHXoMa7JKXWnSmVtCym8wIA2oZ2GwDSxFI0p2ZcO+fKJdUzAxsAko12GwC2FduY5ugyHgAgT9Bu\nA8BWrDkFAAAAZEDRDAAAAGRA0QwAAABkQNEMAAAAZEDRDAAAAGRA0QwAAABkQNEMAAAAZOC8T96O\n1c65tZJeb8O37iHp7SzHaSuybC8pOSSyNIcs22tLjk9573vnIkxStbHdTsrPWCJLc5KSJSk5JLI0\nJ5+ztKrNTmTR3FbOuRrv/eDQOSSyJDmHRJbmkCW5OQpRkl5bsjQtKVmSkkMiS3OKIQvDMwAAAIAM\nKJoBAACADAqtaF4UOkAasmwvKTkksjSHLNtLSo5ClKTXlixNS0qWpOSQyNKcgs9SUGOaAQAAgFwo\ntJ5mANhhzrmyRp9XOOfKnXPjm3l8i/cDAHIrRLud90Uzv+yQSZLeE865yuioaub+qtTjcpyjxfPE\n9Zo458qcc945tyo6Fu5o1ixkKJd0T3omSfLeL5NU30Qb0+L9yIx2Gy1J0vshKW12a84Vx+uShDY7\neu4g7XZeF81J/GWXlDdUa85TDA1PEt4Taecql7TMe79IUmn0eWOVzrlVkmpzlSPTeWIuCnt67533\nvr+kYZKaen/k9DWJ/p3pz32GpProdq2kxj+nTPejBUlrt5PUZrfmXDH+QUubnaw2u8Vzxfi6BG+z\npXDtdl4XzQn9ZZeIN1Sm8xRRw5OE90RKadrz10afN3ae975/9HPJpZbOE9tr0uj8g733Tf3843pN\nUkok1aV93msH70cLEthuJ6nNbvFccbXbtNkfS1KbnelcsbwuCW2zpZja7bwumpsQ/Jddwt5Qwf+D\nRUI2PMHfEyne+0XRLyFJKpNU08TDSmPq/W/pPLEXhdEv5bubuTuu1wRhBP0/mrA2O9O54mq3abOV\nuDY707libbeLtc0utKI5MRLyhkrEf7AENjxBRb1DK733Kxvf572vjn4J9Wqmdycr4jrPDjjee1/f\n1B0BstZL6hndLpG0bgfvRx5KSJud6VyxtNu02dtKQpsd97laIUltthRTu92pTdFi0sxYqdoW/rJN\n0i+745vL6b2vliTn3PHOufJc9V7EdZ7WytTwRI/JdtYkvSdSyr33Exp/MXq/13nvl0Q5murdabdW\nnCfEa9LkZea4XpNG7pKU2n61VNKyKEtJ9Euiyfth8rjdDt5mx32uTGizPxa0zW7lueJ+XZLUZksx\ntduJLprT/tJtrVhetFb+Usj5G6qlHHH/B2vlaxKi4UlUAeScq0z7ZVMe/axSWWq0dXxgf0nbTUjK\nkibPE/A12e7nHedr4pyrkDTYOVfhvV/ivV/pnBsc9ZDUpxUMD0ka1ML9UDLb7aS02ZmyxNlu02a3\nTkLabDV3rhCvS+g2OzpfmHbbe5+3h6QKSeslVaR9rVI2Fqsy7WsrWro/B7lKJS1t9LWS6GNZ2u2F\nkspylKHJ8zTKURndHp+rHOmve9rt8jhfkyS8J1L/7uj9uir6WN5MlgpJ43P982h8nhCvSXSuUkkL\nG30t9teEI54jie12Etrsls4Vot2mzU5Wm93cuQK9LkXbZrMjYA5Ef4VN8N6PSPvaCu/9oOh2pWxc\nWqmP/oLNUY7tztNEjtro/pxtf+m2LjFVJ+slGebtr/XYXxMAaCwpbXZz54q73abNBppG0QwAAABk\nwOoZAAAAQAYUzQAAAEAGFM0AAABABhTNAAAAQAYUzQAAAEAGid7cBMimaImk42XLNS33tjA/ACCB\naLORNCw5h6ISrcda5b0flvZ5nbedjAAACUKbjSShaEbRcM6VyBbsH0aDCwDJRpuNpGFMM4rJYkkj\nJGuMnXNlzrmqwJkAAE2jzUaiUDSjKDjnyiTNljXAVd77eu/9ysCxAABNoM1GEjEREEUhrbH9uNGN\nxsatc86Veu9rwyQDADRGm40kYkwzAAAAkAHDMwAAAIAMKJoBAACADCiaAQAAgAwomgEAAIAMKJoB\nAACADCiaAQAAgAwomgEAAIAMKJoBAACADP4/q2TEX4yWj34AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1510cf77f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "xx = np.arange(-10,10,0.1)\n",
    "y = 1\n",
    "f = [-y*z + logsumexp(0, z) for z in xx]\n",
    "f0 = [logsumexp(0, z) for z in xx]\n",
    "\n",
    "plt.figure(figsize=(12,5))\n",
    "plt.subplot(1,2,1)\n",
    "plt.plot(xx, f, 'r')\n",
    "plt.xlabel('$z_i$')\n",
    "plt.ylabel('$-l_i$')\n",
    "plt.title('Cost for examples with $y = $'+str(y))\n",
    "plt.subplot(1,2,2)\n",
    "plt.plot(xx, f0, 'r')\n",
    "plt.xlabel('$z_i$')\n",
    "plt.ylabel('$-l_i$')\n",
    "plt.title('Cost for examples with $y = 0$')\n",
    "\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Properties of the logsumexp function\n",
    "\n",
    "If \n",
    "$$\n",
    "f(z) = \\text{logsumexp}(0, z) = \\log(1 + \\exp(z))\n",
    "$$\n",
    "The derivative is\n",
    "$$\n",
    "\\frac{df(z)}{dz} = \\frac{\\exp(z)}{1 + \\exp(z)} = \\sigma(z)\n",
    "$$\n",
    "\n",
    "When $z$ is a vector, $f(z)$ is a vector. The derivative of\n",
    "$$\n",
    "\\sum_i f(z_i) = \\mathbf{1}^\\top f(z)\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{d \\mathbf{1}^\\top f(z)}{dz} = \\left(\\begin{array}{c} \\sigma(z_1) \\\\ \\vdots \\\\ \\sigma(z_N) \\end{array} \\right) \\equiv \\sigma(z)\n",
    "$$\n",
    "where the sigmoid function $\\sigma$ is applied elementwise to $z$.\n",
    "\n",
    "### Properties of the sigmoid function\n",
    "Note that\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\sigma(x) & = & \\frac{e^x}{(1+e^{-x})e^x} = \\frac{e^x}{1+e^{x}} \\\\\n",
    "1 - \\sigma(x) & = & 1 - \\frac{e^x}{1+e^{x}} = \\frac{1+e^{x} - e^x}{1+e^{x}} = \\frac{1}{1+e^{x}}\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\sigma'(x) & = & \\frac{e^x(1+e^{x}) - e^{x} e^x}{(1+e^{x})^2} = \\frac{e^x}{1+e^{x}}\\frac{1}{1+e^{x}} = \\sigma(x) (1-\\sigma(x))\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\log \\sigma(x) & = & -\\log(1+e^{-x}) = x - \\log(1+e^{x}) \\\\\n",
    "\\log(1 - \\sigma(x)) & = &  -\\log({1+e^{x}})\n",
    "\\end{eqnarray}\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "Exercise: Plot the sigmoid function and its derivative."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Solve \n",
    "\n",
    "$$\n",
    "\\text{maximize}\\; \\mathcal{L}(w)\n",
    "$$\n",
    "\n",
    "### Optimization via gradient ascent\n",
    "\n",
    "One way for\n",
    "optimization is gradient ascent\n",
    "\\begin{eqnarray}\n",
    "w^{(\\tau)} & \\leftarrow & w^{(\\tau-1)} + \\eta \\nabla_w {\\cal L}\n",
    "\\end{eqnarray}\n",
    "where\n",
    "\\begin{eqnarray}\n",
    "\\nabla_w {\\cal L} & = &\n",
    "\\begin{pmatrix}\n",
    "{\\partial {\\cal L}}/{\\partial w_1} \\\\\n",
    "{\\partial {\\cal L}}/{\\partial w_2} \\\\\n",
    "\\vdots \\\\\n",
    "{\\partial {\\cal L}}/{\\partial w_{D}}\n",
    "\\end{pmatrix}\n",
    "\\end{eqnarray}\n",
    "is the gradient vector and $\\eta$ is a learning rate.\n",
    "\n",
    "#### Evaluating the gradient (Short Derivation)\n",
    "\n",
    "$$\n",
    "\\mathcal{L}(w) = y^\\top X w - \\mathbf{1}^\\top \\text{logsumexp}(0, X w)\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{d\\mathcal{L}(w)}{dw} = X^\\top y - X^\\top \\sigma(X w) = X^\\top (y -\\sigma(X w))\n",
    "$$\n",
    "\n",
    "#### Evaluating the gradient (Long Derivation)\n",
    "The partial derivative of the loglikelihood with respect to the $k$'th entry of the weight vector is given by the chain rule as\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial{\\cal L}}{\\partial w_k} & = & \\frac{\\partial{\\cal L}}{\\partial \\sigma(u)} \\frac{\\partial \\sigma(u)}{\\partial u} \\frac{\\partial u}{\\partial w_k}\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\begin{eqnarray}\n",
    "{\\cal L}(w) & = & \\sum_{i : y_i=1} \\log \\sigma(w^\\top x_i) + \\sum_{i : y_i=0} \\log (1- \\sigma(w^\\top x_i))\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial{\\cal L}(\\sigma)}{\\partial \\sigma} & = &  \\sum_{i : y_i=1} \\frac{1}{\\sigma(w^\\top x_i)} - \\sum_{i : y_i=0} \\frac{1}{1- \\sigma(w^\\top x_i)}\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial \\sigma(u)}{\\partial u} & = & \\sigma(w^\\top x_i) (1-\\sigma(w^\\top x_i))\n",
    "\\end{eqnarray}\n",
    "\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial w^\\top x_i }{\\partial w_k} & = & x_{i,k}\n",
    "\\end{eqnarray}\n",
    "\n",
    "\n",
    "So the gradient is\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial{\\cal L}}{\\partial w_k} & = & \\sum_{i : y_i=1} \\frac{\\sigma(w^\\top x_i) (1-\\sigma(w^\\top x_i))}{\\sigma(w^\\top x_i)} x_{i,k} - \\sum_{i : y_i=0} \\frac{\\sigma(w^\\top x_i) (1-\\sigma(w^\\top x_i))}{1- \\sigma(w^\\top x_i)} x_{i,k} \\\\\n",
    "& = & \\sum_{i : y_i=1} {(1-\\sigma(w^\\top x_i))} x_{i,k} - \\sum_{i : y_i=0} {\\sigma(w^\\top x_i)} x_{i,k}\n",
    "\\end{eqnarray}\n",
    "\n",
    "We can write this expression more compactly by noting\n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial{\\cal L}}{\\partial w_k} & = & \\sum_{i : y_i=1} {(\\underbrace{1}_{y_i}-\\sigma(w^\\top x_i))} x_{i,k} + \\sum_{i : y_i=0} {(\\underbrace{0}_{y_i} - \\sigma(w^\\top x_i))} x_{i,k} \\\\\n",
    "& = & \\sum_i (y_i - \\sigma(w^\\top x_i)) x_{i,k}\n",
    "\\end{eqnarray}\n",
    "\n",
    "$\\newcommand{\\diag}{\\text{diag}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test on a synthetic problem\n",
    "\n",
    "We generate a random dataset and than try to learn to classify this dataset\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pylab as plt\n",
    "\n",
    "# Generate a random logistic regression problem\n",
    "\n",
    "def sigmoid(t):\n",
    "    return np.exp(t)/(1+np.exp(t))\n",
    "\n",
    "\n",
    "def generate_toy_dataset(number_of_features=3, number_of_datapoints=20, styles = ['ob', 'xr']):\n",
    "\n",
    "    D = number_of_features\n",
    "    N = number_of_datapoints\n",
    "    \n",
    "\n",
    "    # Some random features\n",
    "    X = 2*np.random.rand(N,D)-1\n",
    "    X[:,0] = 1\n",
    "\n",
    "    # Generate a random paramater vector\n",
    "    w_true = np.random.randn(D,1)\n",
    "\n",
    "    # Generate class labels\n",
    "    pi = sigmoid(np.dot(X, w_true))\n",
    "    y = np.array([1 if u else 0 for u in np.random.rand(N,1) < pi]).reshape((N))\n",
    "\n",
    "    return X, y, w_true, D, N"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUcAAAExCAYAAADvMmC3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEBFJREFUeJzt3b1uG1m2huFvaToZd0JYR6ltyMF0\nbFMXIIAnnIkEO2DcBHwDSgw7sI+D1gVMw8wFjNW6BBlMBxDtaFILlsPW2GAwENAYQOsEVSVS9BbJ\nYrH+WO8DNOqPci2wqU+7anPvMncXAOCmjbILAIAqIhwBIIBwBIAAwhEAAlYSjmb2aMaxX+JlbxXn\nAoAiZA5HM+tI+m3GS3pm9knSWdZzAUBRfsj6D7j7iZnNCr6f3f0463kAoEhF3HPcNrOOme0XcC4A\nWIncw9HdD9z9RNJmfAkOAJWX+bJ6lrgT5lt8Wf1V0vYtr+lJ0o8//vj4p59+yrMkAA304cOHf7v7\nVpqfySUczazl7iNJQ407Yh5Kejv9WnfvS+pLUrvd9uFwmEdJABrMzM7T/swqeqv3JLXjZeK9JLn7\nR0lP4mOf4m0AqLxV9FYfSzqe2vd4Yr2f9RwAUDRGyABAAOEIAAGEIwAEEI4AEEA4AkAA4QgAAYQj\nAAQQjgAQQDgCQADhCAABhCMABBCOABBAOAJAAOEIAAGEIwAEEI4AEEA4AkAA4QgAAYQjAAQQjgAQ\nQDgCQADhCAABhCMABBCOABBAOAJAAOEIAAGEIwAEEI4AEEA4AkAA4QgAAYQjAAQQjgAQQDgCQADh\nCAABhCMABBCOABBAOAJAwErC0cwezTi2Z2YdM9tfxbkAoAiZw9HMOpJ+u+XYI0ly9xNJo1khCgBV\nkjkc4+A7u+XwU0mjeP1MUifr+QCgCHnfc2xJ+jaxvZnz+QBgJUrvkDGznpkNzWx4cXFRdjkAICn/\ncBxJuhuvtyR9nX6Bu/fdve3u7a2trZzLAYDF5BKOZtaKV99J2o7XtyWd5HE+AFi1VfRW70lqx8vE\ne0ly94/xazqSRsk2AFTdD1n/AXc/lnQ8te/xxHo/6zkAoGild8gAQBURjgAQQDgCQADhCAABhCMA\nBBCOABBAOK6Rw0PpwQNpYyNaHh6WXRFQX5m/54hqODyUej3p8jLaPj+PtiWp2y2vLqCuaDmuiefP\nx8GYuLyM9gNIj3BcE1++pNsPYDbCcU3cu5duP4DZCMc18eaNdOfOzX137kT7AaRHOK6Jblfq96X7\n9yWzaNnv0xkDLIve6jXS7RKGwKrQcgSAAMIRAAIIxzwcHEiDwc19g0G0H0AtEI552NmRnjwZB+Rg\nEG3v7JRbVxoEPBqOcMzD7q50dBQF4suX0fLoKNpfF+sQ8EAGhGNednelZ8+k16+jZZ2CUVqPgAcy\nIBzzMhhIv/4qvXgRLacvUeug7gEPZEA45iG5BD06kl69GrfA6haQ6xDwwJIIxzycnt68BE0uUU9P\ny60rjXUJeGBJhGMe9ve/vwTd3Y3218U6BHyMSYCxDHP3smu41m63fTgcll0G1sj0JMBSNCEH486b\nxcw+uHs7zc/QcsRaYxJgLItwxFpjEmAsi3DEWmMSYCyLcMRaYxJgLItwxFpjEmAsi8lusfaYBBjL\noOUIAAGEIwAEEI4AEEA4VhjD3oDy0CFTUdPD3s7Po22JzgWgCLQcK4phb0C5Moejme2ZWcfMglPO\nmNkv8bKX9VxNwrA3oFyZwtHMHkmSu59IGiXbU3pm9knSWZZzNQ3D3oByZW05PpU0itfPJHUCr/nZ\n3R/GAYoFMewNKFfWcGxJ+jaxvRl4zfacy+6emQ3NbHhxcZGxnPXBsDegXLl3yLj7Qdxq3DSz71qW\n7t5397a7t7e2tvIup1a6XenzZ+nqKloSjEBxsobjSNLdeL0l6evkwbhVuBdvfpW0nfF8AFCIrOH4\nTuPA25Z0Iklm1or3DZN9kh7G20Dk4OD7B3YNBtF+oGSZwtHdP0pSfLk8SrYlvZ84/iRuPX6aOA5I\nOzs3n2iYPPFwZ6fcugDxgC2ULQnEZ8+iZ2NPPvEQWBEesIX62d2NgvH162hJMKIiCEeUazCIWowv\nXkTL6XuQQEkIR5QnuaQ+OpJevYqWk/cggRIRjijP6enNe4y7u9H26Wm5dQGiQwZAA9AhAwArQjgC\nQADhCAABhCMABBCOABBAOAJAAOEIAAGEIzADzw5vLp5bDdyCZ4c3Gy1H4BY8O7zZCEfgFjw7vNkI\nR+AWPDu82QhH4BY8O7zZCEfgFjw7vNnorQZm6HYJw6ai5QgAAYQjAAQQjgAQQDgCQADhCAABhCMA\nBBCOABBAOCK9gwNpMLi5bzCI9gNrgnBEejs70pMn44AcDKLtnZ1y62ow5p1cPUbIIL3dXenoKArE\nZ8+kX3+Ntnd3y66skZh3Mh+0HLGc3d0oGF+/jpYEY2mYdzIfhCOWMxhELcYXL6Ll9D1IFIZ5J/NB\nODbY0vepknuMR0fSq1fjS2wCshTMO5kPwrGhkvtU5+eS+/g+1UIBeXp68x5jcg/y9DTXmhHGvJP5\nMHcvu4Zr7Xbbh8Nh2WU0woMHUSBOu39f+vy56GqQ1eFhdI/xy5eoxfjmDZ0xk8zsg7u3U/0M4dhM\nGxtRi3GamXR1VXw9QJ6WCcfMl9VmtmdmHTPbX+Y4ysF9KmC2TOFoZo8kyd1PJI2S7UWPQ6WNNuE+\nFTBb1pbjU0mjeP1MUiflcZQ02oTnowCzZR0h05L0bWJ7M+VxlDjahOejALcr/as8ZtYzs6GZDS8u\nLsoupxyMNkHBGIs9X9ZwHEm6G6+3JH1NeVzu3nf3tru3t7a2MpZTU4w2KVbDZxXK9B3XBskaju8k\nbcfr25JOJMnMWrOOYwKjTYrX8FmFGIu9mEzh6O4fJcnMOpJGybak93OOI8Fok+JN3ud9+XL8x6kh\ntzMYi70YvgSO5nr5MrrP++JF1GpviCaOjirlS+BALTX4Pi/fcV0M4Yjmafh9Xr7juhjCEc3DfV51\nu9El9NVVtLwOxob35E8iHNE8+/vfd77s7kb7m67hPfmTeIYMgDGeD3SNliOAmxixJYlwBDCtwT35\nkwhHAGODgQ7/9g892Piijf97pQcbX3T4t380MiAJRwDXDt/+R73//l3nv/85Gnf9+5/V++/fdfj2\nP2WXVjjCEcC15//8qy7/+NONfZd//EnP//nXkioqD+EI4BrjrscIRwDXeLbQGOEI4BrjrscIRwDX\nGHc9xggZADfwbKEILUcACCAcASCAcASAAMIRAAIIRwAIIByRCQ+Hx7riqzxYWvJw+OQZyMnD4SW+\nCoL6o+WIpfFweKwzwhFLY5ICrDPCEUtjkgKsM8IRS2OSAqwzwhFLY5ICrDN6q5EJkxRgXdFyBIAA\nwhEAAghHAAggHAEggHAEgADCEQACCEcACCAcASCAcASAgMzhaGZ7ZtYxs/1bjv8SL3tZzwUARckU\njmb2SJLc/UTSKNme0jOzT5LOspwLAIqUteX4VNIoXj+T1Am85md3fxgHKADUQtZwbEn6NrG9GXjN\n9pzL7p6ZDc1seHFxkbEcAFiN3Dtk3P0gbjVumtl3LUt377t7293bW1tbeZcDAAuZO2XZLR0pZ8l9\nRkl3430tSV8DP/vN3Y/jY9vZygWAYswNR3fvzzj8TlI7Xt+WdCJJZtZy95GkocYdMQ8lvV2+VAAo\nTqbLanf/KEnx5fIo2Zb0fuL4EzPbk/Rp4jgAVFrmmcBDLUt3fzzrOABUHSNkACCAcASAAMIRAAII\nRwAIIBwBIIBwRGqHh9KDB9LGRrQ8PCy7ImD1Mn+VB81yeCj1etLlZbR9fh5tS1K3W15dwKrRckQq\nz5+PgzFxeRntR/5otReHliNS+fIl3X6sDq32YtFyRCr37qXbj9Wh1V4swhGpvHkj3blzc9+dO9F+\n5ItWe7EIR6TS7Ur9vnT/vmQWLft9LuuKQKu9WIQjUut2pc+fpauraEkwFoNWe7EIR6AmaLUXi95q\noEa6XcKwKLQcASCAcASAAMIRAAIIRwAIIBwBIIBwBIAAwhEAAghHAAggHAEggHAEGoqJc2dj+CDQ\nQEycOx8tR6CBmDh3PsIRaCAmzp2PcAQaiIlz5yMcgQZi4tz5CEeggZg4dz56q4GGYuLc2Wg5AkAA\n4QgAAYQjAAQQjgAQsJJwNLNHM47tmVnHzPZXcS4gwdhg5ClzOJpZR9Jvtxx7JEnufiJpNCtEgTSS\nscHn55L7eGwwAYlVyRyOcfCd3XL4qaRRvH4mqZP1fIDE2GDkL+97ji1J3ya2N6dfYGY9Mxua2fDi\n4iLncrAuGBuMvJXeIePufXdvu3t7a2ur7HJQE4wNRt7mjpAxs15g91l8OT3PSNLdeL0l6WuK2oBb\nvXlzcz5CibHBWK254eju/bT/qJm13H0k6Z2kdrx7W9IigQrMlQx7e/48upS+dy8KRobDYVVW0Vu9\nJ6kdLxPvJcndP8av6UgaJdvAKnS70ufP0tVVtCQYsUqZJ55w92NJx1P7Hk+sp255AkDZSu+QAYAq\nIhwBIIBwBIAAwhEAAghHAAggHAEggHAEgADCEQACCEcACCAcASCAcASAAMIRAAIIRwAIIBwBIIBw\nBIAAwhEAAghHAAggHAEggHAEgADCEQACCEcACCAcASCAcASAAMIRAAIIRwAIIBwBIIBwBIAAwhEA\nAghHAAggHAEggHAEgADCEQACCEcACCAcASCAcASAAMIRAAJWEo5m9mjGsV/iZW8V5wKAImQORzPr\nSPptxkt6ZvZJ0lnWcwFAUX7I+g+4+4mZzQq+n939OOt5AKBIRdxz3DazjpntF3AuAFiJzC3Hedz9\nQJLM7H/NrOPuJ5PH43uRyf3IP8zsX3nXlIP/kfTvsotYUl1rr2vdUn1rr2vdkvSXtD8wNxxv6Ug5\nmw65GT/7Lb6s/ippe/o17t6X1I9fP3T39tyqK6audUv1rb2udUv1rb2udUtR7Wl/Zm44xuGVtpCW\nu48kDTXuiHko6W3afwsAyrCK3uo9Se14mXgvSe7+UdKT+NineBsAKm8VvdXHko6n9j2eWE/T8kzd\nSq2IutYt1bf2utYt1bf2utYtLVG7uXsehQBArVVm+GCdR9nMqX2PrzKhjur6uV5VllQiHOs8ymZW\n7cn/pLhnfzTrf1rR5n24q/QHaYFaK/mLWqf3eFqNP9cry5JKhGP8Js8bZfNwka8PFW1O7U8ljeL1\nM0mdQoqaY8EPdyX+IM2rtaq/qHV6j0Pq+LmWVpsllQjHBdR1lE1L0reJ7c2yCpmyyIe7Kn+Q5tVa\n1V/UOr3HaVX1c72IhbOkFuHo7gfxB2gzbjYjm0U+3FX5gzSv1qr+otbpPW6MNFmS+/BBKf9RNnnK\nUruilsPdeL2lqP5amDfsE9nV+D2u5ec6bZYUEo51HmWTsfZ3kpLhVtuSCvvwzwn1mR/usv8gTZn3\ni1jVX9Q6vccLqcLnehnLZkklLqvrPMpmgdqTHrRRkbW7ez/wX/IhfqfxL+P1h9vMWvG+ocYf+Ifx\ndlnm1Ro8XgF1eo+/U9XP9TyrzBK+BN5QccvlTNJ20jo2sw/J6KakZRMfPyiv0oVrvXG8Cur0HuN7\nhCMABFTishoAqoZwBIAAwhEAAghHAAggHAEggHAEgADCEQAC/h+nVH09zAKUKgAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x151388f828>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "styles = ['ob', 'xr']\n",
    "X, y, w_true, D, N = generate_toy_dataset(number_of_features=3, number_of_datapoints=20, styles=styles)\n",
    "\n",
    "xl = -1.5; xr = 1.5; yl = -1.5; yr = 1.5\n",
    "\n",
    "fig = plt.figure(figsize=(5,5))\n",
    "plt.plot(X[y==1,1],X[y==1,2],styles[1])\n",
    "plt.plot(X[y==0,1],X[y==0,2],styles[0])\n",
    "\n",
    "ax = fig.gca()\n",
    "ax.set_ylim([yl, yr])\n",
    "ax.set_xlim([xl, xr])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Implement Gradient Descent\n",
    "w = np.random.randn(D)\n",
    "\n",
    "# Learnig rate\n",
    "eta = 0.05\n",
    "\n",
    "W = []\n",
    "MAX_ITER = 200\n",
    "for epoch in range(MAX_ITER):\n",
    "    W.append(w)\n",
    "    dL = np.dot(X.T, y-sigmoid(np.dot(X,w)))\n",
    "    w = w + eta*dL\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5a83ae73a7e7477db3946b8342a9f6b4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/html": [
       "<p>Failed to display Jupyter Widget of type <code>interactive</code>.</p>\n",
       "<p>\n",
       "  If you're reading this message in Jupyter Notebook or JupyterLab, it may mean\n",
       "  that the widgets JavaScript is still loading. If this message persists, it\n",
       "  likely means that the widgets JavaScript library is either not installed or\n",
       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
       "  Widgets Documentation</a> for setup instructions.\n",
       "</p>\n",
       "<p>\n",
       "  If you're reading this message in another notebook frontend (for example, a static\n",
       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
       "  it may mean that your frontend doesn't currently support widgets.\n",
       "</p>\n"
      ],
      "text/plain": [
       "interactive(children=(IntSlider(value=0, description='iteration', max=199), Output()), _dom_classes=('widget-interact',))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<function __main__.plot_boundry_of_weight>"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xl = -1.5\n",
    "xr = 1.5\n",
    "yl = -1.5\n",
    "yr = 1.5\n",
    "\n",
    "fig = plt.figure(figsize=(5,5))\n",
    "ax = fig.gca()\n",
    "ax.set_ylim([yl, yr])\n",
    "ax.set_xlim([xl, xr])\n",
    "\n",
    "plt.plot(X[y==1,1],X[y==1,2],styles[1])\n",
    "plt.plot(X[y==0,1],X[y==0,2],styles[0])\n",
    "\n",
    "\n",
    "ln = plt.Line2D([],[],color='k')\n",
    "ln_left  = plt.Line2D([],[],ls= ':', color=styles[0][1])\n",
    "ln_right = plt.Line2D([],[],ls= ':', color=styles[1][1])\n",
    "\n",
    "ax.add_line(ln)\n",
    "ax.add_line(ln_left)\n",
    "ax.add_line(ln_right)\n",
    "plt.close(fig)\n",
    "ax.set_xlabel('$x_1$')\n",
    "ax.set_ylabel('$x_2$')\n",
    "ax.set_xticks(np.arange(xl,xr))\n",
    "ax.set_yticks(np.arange(yl,yr))\n",
    "ax.grid(True)\n",
    "def plot_boundry(w0,w1,w2):\n",
    "    if w1 != 0:\n",
    "        xa = -(w0+w2*yl)/w1\n",
    "        xb = -(w0+w2*yr)/w1\n",
    "        ln.set_xdata([xa, xb])\n",
    "        ln.set_ydata([yl, yr])\n",
    "\n",
    "        xa = -(-inv_sigmoid(0.25) + w0+w2*yl)/w1\n",
    "        xb = -(-inv_sigmoid(0.25) + w0+w2*yr)/w1\n",
    "        ln_left.set_xdata([xa, xb])\n",
    "        ln_left.set_ydata([yl, yr])\n",
    "\n",
    "        xa = -(-inv_sigmoid(0.75) + w0+w2*yl)/w1\n",
    "        xb = -(-inv_sigmoid(0.75) + w0+w2*yr)/w1\n",
    "        ln_right.set_xdata([xa, xb])\n",
    "        ln_right.set_ydata([yl, yr])\n",
    "        \n",
    "        \n",
    "    elif w2!=0:\n",
    "        ya = -(w0+w1*xl)/w2\n",
    "        yb = -(w0+w1*xr)/w2\n",
    "        ln.set_xdata([xl, xr])\n",
    "        ln.set_ydata([ya, yb])\n",
    "\n",
    "        ya = -(-inv_sigmoid(0.25) + w0+w1*xl)/w2\n",
    "        yb = -(-inv_sigmoid(0.25) + w0+w1*xr)/w2\n",
    "        ln_left.set_xdata([xl, xr])\n",
    "        ln_left.set_ydata([ya, yb])\n",
    "\n",
    "        ya = -(-inv_sigmoid(0.75) + w0+w1*xl)/w2\n",
    "        yb = -(-inv_sigmoid(0.75) + w0+w1*xr)/w2\n",
    "        ln_right.set_xdata([xl, xr])\n",
    "        ln_right.set_ydata([ya, yb])      \n",
    "        \n",
    "    else:\n",
    "        ln.set_xdata([])\n",
    "        ln.set_ydata([])\n",
    "        \n",
    "    display(fig)\n",
    "\n",
    "def plot_boundry_of_weight(iteration=0):\n",
    "    i = iteration\n",
    "    w = W[i]\n",
    "    plot_boundry(w[0],w[1],w[2])\n",
    "    \n",
    "interact(plot_boundry_of_weight, iteration=(0,len(W)-1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Second order optimization\n",
    "\n",
    "Newton's method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Evaluating the Hessian\n",
    "\n",
    "The Hessian is \n",
    "\\begin{eqnarray}\n",
    "\\frac{\\partial^2{\\cal L}}{\\partial w_k \\partial w_r} & = & - \\sum_i (1-\\sigma(w^\\top x_i)) \\sigma(w^\\top x_i) x_{i,k} x_{i,r} \\\\\n",
    "\\pi & \\equiv & \\sigma(X w) \\\\\n",
    "\\nabla \\nabla^\\top \\mathcal{L}& = & -X^\\top \\diag(\\pi(1 - \\pi)) X \n",
    "\\end{eqnarray}\n",
    "\n",
    "\n",
    "The update rule is\n",
    "\\begin{eqnarray}\n",
    "w^{(\\tau)} = w^{(\\tau-1)} + \\eta X^\\top (y-\\sigma(X w))\n",
    "\\end{eqnarray}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 339,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1]\n",
      "[2 3 4]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfAAAAD7CAYAAACVHMaRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEW1JREFUeJzt3T9u20gfxvHnF6RKpddetUkg30Ch\nD2BAewOvU2wdvcgFssUiW2ywhX2BIHprF/bmCAp0gCi+gQVkW732qyrt7y04tGmFEulQfzzS9wMY\n5MxwrAlt88kMadncXQAAIC6P1j0AAABwfwQ4AAARIsABAIgQAQ4AQIQIcAAAIkSAAwAQIQIcAIAI\nEeAAAESIAAcAIEKP1z2AMj/99JM/f/583cMAAGAlvnz58l93b5Yd9+AD/Pnz5xoOh+seBgAAK2Fm\nX6scxxI6AAARIsABAIgQAQ4AQIQWEuBm1p7TdmhmHTN7M68OAABUVzvAzawj6e8ZbW1Jcve+pImZ\ntYvq6o4BAIBtUzvAQxCPZjS/lDQJ+yNJnRl1APBwnZxIg8HdusEgrQfWZNn3wBuSrnPl3Rl1APBw\n7e9LR0e3IT4YpOX9/fWOC1vtwf8eOACs3cGBdH6ehvbr19L792n54GDdI8MWW/YMfCJpJ+w3JF3N\nqLvDzLpmNjSz4Xg8XvIQAaCCg4M0vN+9S7eEN9ZsKQFuZo2weyapFfZbkvoz6u5w9567J+6eNJul\n7yYHAMs3GKQz77dv0+30PXFgxRbxFPqhpCRsM58kyd0vwjEdSRN3vyiqqzsGAFiq7J73+bn055+3\ny+mEONbI3H3dY5grSRLnvdABrNXJSfrAWn7ZfDCQPn+W3vB2FlgsM/vi7knpcQQ4AAAPR9UA561U\nAQCIEAEOAECECHAAACJEgAMAECECHACACBHgAABEiAAHACBCBDgAABEiwAEAiBABDgBAhAhwAAAi\nRIADABAhAhwAgAgR4AAARIgABwAgQgQ4AAARqh3gZnZoZh0ze1PQ1jYzN7PL8PEh1B+Hbbfu6wMA\nsI1qBbiZtSXJ3fuSJlk5Z8fdzd33JP0i6TjUd83sUtKozusDALCt6s7AX0qahP2RpE6+MQR7JnH3\nLLBfufveVDsAAKioboA3JF3nyrtFB5lZR9J5rqo1a9kdAACUW9VDbD+7ezZTl7ufhNn3bgj3O8ys\na2ZDMxuOx+MVDREAgHjUDfCJpJ2w35B0NeO4m3vjIZwPQ/FKUmv6YHfvuXvi7kmz2aw5RAAANk/d\nAD/TbQC3JPUlycwa2QFmNh3Qw+w4SXuhDAAA7qFWgLv7hXRzj3uSlSV9mjp0NNXnKMzCL3N9AABA\nRY/rfgJ37xXUvcjtjyT9u6wPAACojndiAwAgQgQ4AAARIsABAIgQAQ4AQIQIcAAAIkSAAwAQIQIc\nAIAIEeAAAESIAAcAIEIEOAAAESLAAQCIEAEOAECECHAAACJEgAMAECECHACACBHgAABEqHaAm9mh\nmXXM7M2M9uOw7Vbtg8U6PZWeP5cePUq3p6frHhGA6J2cSIPB3brBIK3HStQKcDNrS5K79yVNsvKU\nrpldShrdow8W5PRU6nalr18l93Tb7RLiAGra35eOjm5DfDBIy/v76x3XFqk7A38paRL2R5I6Bce8\ncve9ENhV+2BBfv9d+vbtbt23b2k9APywgwPp/DwN7T/+SLfn52k9VqJugDckXefKuwXHtKaWy0v7\nmFnXzIZmNhyPxzWHuN3++ed+9QBQ2cGB9Pq19O5duiW8V2rpD7G5+0mYfe+aWaXZtrv33D1x96TZ\nbC55hJvt6dP71QNAZYOB9P699PZtup2+J46lqhvgE0k7Yb8h6SrfGGbSh6F4JalV1geL9ddf0pMn\nd+uePEnrAeCHZfe8z8+lP/+8XU4nxFemboCfKQ1lhW1fksysEeqGWZ2kvVAu7IPl+PVXqdeTnj2T\nzNJtr5fWA8AP+/z57j3v7J7458/rHdcWMXev9wnSXw8bSWq5ey/UfXH3F7n269B+MqvPLEmS+HA4\nrDVGAABiETI0KTvucd0XKgrgLLzntM8NbQAAMB/vxAYAQIQIcAAAIkSAAwAQIQIcAIAIEeAAAESI\nAAcAIEIEOAAAESLAAQCIEAEOAECECHAAACJEgAMAECECHACACBHgAABEiAAHACBCBDgAABGqHeBm\ndmhmHTN7M6O9Gz6Oc3XHWVvd1wcAYBvVCnAza0uSu/clTbJyrr0jqe/uPUmtUJakrpldShrVeX0A\nALZV3Rn4S0mTsD+S1Jlqb+XqRqEsSa/cfS8EPwAAuKfHNfs3JF3nyrv5xjDzzrQlnYX9bDbedveT\nmmMAAGDrrOQhtrC0fuHuF5Lk7idh9r2bW1bPH981s6GZDcfj8SqGCABAVOoG+ETSTthvSLqacVzH\n3X+TbsL5MNRf6XZZ/Ya799w9cfek2WzWHCIAAJunboCf6TaAW5L6kmRmjewAM+tmy+Rhtj3MjpO0\nF8oAAOAeagV4tiQegnmSlSV9ytUfm9mlmf0v1+cozMIvc30AAEBFdR9im35QLat7EbZ9Sf+q0gcA\nAFTHO7EBABAhAhwAgAgR4AAARIgABwAgQgQ4AAARIsABAIgQAQ4AQIQIcAAAIkSAAwAQIQIcAIAI\nEeAAAESIAAcAIEIEOAAAESLAAQCIEAEOAECECHAAACL0uO4nMLNDSRNJbXc/qdJe1gcAAMxXawZu\nZm1Jcve+pElWntde1gcAAJSru4T+UulMWpJGkjoV2sv6AACAEnUDvCHpOlferdBe1gcAAJR4kA+x\nmVnXzIZmNhyPx+seDgAAD07dAJ9I2gn7DUlXFdrL+sjde+6euHvSbDZrDhEAgM1T9yn0M0lJ2G9J\n6kuSmTXcfTKrfUYdAACoqNYM3N0vJMnMOpImWVnSp1ntc/oAAICKav8euLv3CupelLR/VwcAAKp7\nkA+xAQCA+QhwAAAiRIADABAhAhwAgAgR4AAARIgABwAgQgQ4AAARIsABAIgQAQ4AQIQIcAAAIkSA\nAwAQIQIcAIAIEeAAAESIAAcAIEIEOAAAESLAAQCIEAEOAECEage4mR2aWcfM3sxo74aP41zdcdZW\n9/UBANhGtQLczNqS5O59SZOsnGvvSOq7e09SK5QlqWtml5JGdV4fAIBtVXcG/lLSJOyPJHWm2lu5\nulEoS9Ird98LwQ8AAO7pcc3+DUnXufJuvjHMvDNtSWdhP5uNt939ZPqThqX1riQ9ffq05hABANg8\nK3mILSytX7j7hSS5+0mYfe/mltVvuHvP3RN3T5rN5iqGCABAVEpn4DMeNBtl970l7YS6hqSrGZ+m\n4+6/5T7ftbt/DMe3ZvQBAAAzlAb41DL4tDNJSdhvSepLkpk13H0S9rvZMnmYbQ91+/DanqQPPzZ0\nAAC2V60l9GxJPATzJCtL+pSrPzazSzP7X67PkZkdSrrM9QEAABXVfYitcIbu7i/Cti/pX1X6AACA\n6ngnNgAAIkSAAwAQIQIcAIAIEeAAAESIAAcAIEIEOAAAESLAAQCIEAEOAECECHAAACJEgAMAECEC\nHACACBHgAABEiAAHACBCBDgAABEiwAEAiFDtADezQzPrmNmbGe3HYdut2gcAAMxXK8DNrC1J7t6X\nNMnKU7pmdilpdI8+AABgjroz8JeSJmF/JKlTcMwrd98LgV21DwAAmKNugDckXefKuwXHtKaWy0v7\nmFnXzIZmNhyPxzWHCADA5ln6Q2zufhJm37tmVmm27e49d0/cPWk2m0seIQAA8XlcdkD+4bOcUXYP\nW9JOqGtIuiroe+3uH0Nbq6wPAAAoVxrg7t6b03wmKQn7LUl9STKzhrtPJA0VHl6TtCfpQ6j7rg8A\nAKiu1hK6u19IUlgan2RlSZ9y7Udmdijp0t0v5vQBAAAVlc7AyxTN0N39RUn7vFk9AAAowTuxAQAQ\nIQIcAIAIEeAAAESIAAcAIEIEOAAAESLAAQCIEAEOAECECHAAACJEgAMAECECHACACBHgAABEiAAH\nACBCBDgAABEiwAEAiBABDgBAhAhwAAAiVDvAzezQzDpm9qagrW1mbmaX4eNDqD8O227d1wcAYBvV\nCnAza0uSu/clTbJyzo67m7vvSfpF0nGo75rZpaRRndcHAGBb1Z2Bv5Q0CfsjSZ18Ywj2TOLuWWC/\ncve9qXYAAFBR3QBvSLrOlXeLDjKzjqTzXFVr1rJ7OL5rZkMzG47H45pDBABg86zqIbaf3T2bqcvd\nT8LsezeE+x3u3nP3xN2TZrO5oiECABCPx2UHzHjQbJTd95a0E+oakq5mfJqbe+Ph8127+8dwfOte\nIwYAAOUB7u69Oc1nkpKw35LUlyQza2QzbjObDuihbh9e25P04T4DBgAANZfQ3f1CurnHPcnKkj5N\nHTqa6nNkZoeSLnN9AABARaUz8DJFM3R3f5HbH0n6d1kfAABQHe/EBgBAhAhwAAAiRIADABAhAhwA\ngAgR4AAARIgABwAgQgQ4AAARIsABAIgQAQ4AQIQIcAAAIkSAAwAQIQIcAIAIEeAAAESIAAcAIEIE\nOAAAEVpIgJtZe07boZl1zOzNvDoAAFBd7QA3s46kv2e0tSXJ3fuSJmbWLqqrO4YbJyfSYHC3bjBI\n6wEAWIHTU+n5c+nRo3R7erqc16kd4CGIRzOaX0qahP2RpM6MusXY35eOjm5DfDBIy/v7C3sJAABm\nOT2Vul3p61fJPd12u8sJ8WXfA29Ius6Vd2fULcbBgXR+nob2H3+k2/PztB4AgCX7/Xfp27e7dd++\npfWL9iAfYjOzrpkNzWw4Ho/v1/ngQHr9Wnr3Lt0S3gCAFfnnn/vV1/G47AAz6xZUj8LSeZmJpJ2w\n35B0FfaL6m64e09ST5KSJPEKr3NrMJDev5fevk23BweEOABgJZ4+TZfNi+oXrTTAQ5jei5k13H0i\n6UxSEqpbkrLQL6qrL7vnnS2bHxywjA4AWJm//krveeeX0Z88SesXbRFPoR9KSsI280mS3P0iHNOR\nNHH3i6K6umO48fnz3bDO7ol//rywlwAAYJZff5V6PenZM8ks3fZ6af2imfv9VqhXLUkSHw6H6x4G\nAAArYWZf3D0pO+5BPsQGAADmI8ABAIgQAQ4AQIQIcAAAIkSAAwAQoQf/FLqZjSUV/Fp8qZ8k/XfB\nw4kd56QY56UY56UY5+V7nJNiP3penrl7s+ygBx/gP8rMhlUew98mnJNinJdinJdinJfvcU6KLfu8\nsIQOAECECHAAACK0yQF+7/dw3wKck2Kcl2Kcl2Kcl+9xToot9bxs7D1wAKjLzNqz/l5D+PsPE0lt\ndz9Z7cjWq+S8HLv7b2bW/ZE/hoXqNnIGbmbtOW2HZtYxszerHBMehrKv/zZ+f1Q4J8dhW/SnhTdW\n+INLf89oa0tS+LPKk3nXnE0z77wEXTO7lDRa0ZAeBDPrho/jGe0Lv7ZsXIDzQ1eMi3T5138bvz8q\n/pu38oIczsmsf/NLpbNvhWM6KxnUA1ByXiTplbvvheO2QsidflhxaIVyvn0p15aNC3B+6L7HRfpG\n2dd/G78/qvybt+6CXEFD0nWuvLuugTxArW1bxZLU0u3PziiU85Zybdm4AC+xrT90XKRTZV//bfz+\nqPJv3sYLMn6Qu5+E68ju9Ex0U7l7L3e/vy1p+m9gL+Xasm0Bvq24SOOHbeMFuYKJpJ2w35B0tcax\nPBjhHvBhKF7p+5noRgurmxezHvBbtMereJFFmnGPdlRx5sgP3QzZU7Rm9rOZdTZ0Jl729d/G74+5\n/+bw83bt7h+1hRfkaWbWcPeJpDNJ2TtstSRt4s9LZbnzMtTtbbg9SR/WN6q16Lj7bwX1S7m2RBfg\nP/JrCdvwQ1fyHxsu0qnCr/82fH/MUXZOtvaCHGaSiZkdhp8NSfok6YW7X5hZElYkJquacT0EFc5L\n18yuJV1u2Xnp5iZCHXfvL/vasnG/Bx6+uf6j9J7ux1D3xd1fhP2uwkMG2/I7imFZJ3H3Xlgi74cf\ntIa7Zw+1jcL+B0kfNvUHr+jrz/dHpXNyHdq36vedgSpyv/10rXSy9EsI8KVeWzYuwFGMizQAbBYC\nHACACPEUOgAAESLAAQCIEAEOAECECHAAACJEgAMAECECHACACBHgAABE6P/OMHdMWXS6kgAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11f31ef98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#x = np.matrix('[-2,1; -1,2; 1,5; -1,1; -3,-2; 1,1] ')\n",
    "x = np.matrix('[-0.5,0.5;2,-1;-1,-1;1,1;1.5,0.5]')\n",
    "#y = np.matrix('[0,0,1,0,0,1]').T\n",
    "y = np.matrix('[0,0,1,1,1]').T\n",
    "N = x.shape[0]\n",
    "#A = np.hstack((np.power(x,0), np.power(x,1), np.power(x,2)))\n",
    "#X = np.hstack((x, np.ones((N,1)) ))\n",
    "X = x\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1/(1+np.exp(-x))\n",
    "\n",
    "\n",
    "idx = np.nonzero(y)[0]\n",
    "idxc = np.nonzero(1-y)[0]\n",
    "\n",
    "fig = plt.figure(figsize=(8,4))\n",
    "plt.plot(x[idx,0], x[idx,1], 'rx')\n",
    "plt.plot(x[idxc,0], x[idxc,1], 'bo')\n",
    "fig.gca().set_xlim([-1.1,2.1])\n",
    "fig.gca().set_ylim([-1.1,1.1])\n",
    "print(idxc)\n",
    "print(idx)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAekAAAJOCAYAAACTP/9cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XecJFW5//FPTfekzZEcl5xxXRUx\noSzgVbkoUVC5gooCkgUEJEpclrC7XsMa+Om9Klz2mjOrV1FBgoAokhfJcdkcJvTU74+Zhe1TZ6af\nOVPdVd39fb9evKCLqlOnw3T16efp54niOEZERETypyXrCYiIiIifLtIiIiI5pYu0iIhITukiLSIi\nklO6SIuIiOSULtIiIiI5pYu0iIhITukiLSIiklO6SIuIiORUMesJVNI5ZXQ8fqsJZdsi4iFv+/fp\n8+wz9DEALZ7jKu/jG6dyZTfb/fCN4+5jGdt3rsrzCTuXj+XxyD9bvb6wexI7x6V1Lnfc0LF949jG\n9Z1/+I+R/36kNY4r9HGt3XGWx9pyX2PP2s2yz0t/ffaVOI6neqeRsm2jKF5dixMNeB5+Hcfxe2t4\nytfk/iI9fqsJfPTu48u2tdFddrtAb+K4dmefNroS+xQpDTlu/zjlxxWcY0Ln4xsn/LiSczs5jntf\nazmOj2/skHGyVqJg2Cfsz8wd23Ku3sD52O5H+T5pniut81sea8vj6o4dMq51m+V+1HKcLtor7tNN\nW2Kf66Nzn0xsrJLVwKdrdTLgIphSw9OVyf1FWkREZH0RzXPxUkxaREQkp+rgw0hc8WtY9yvY/n1K\nFfexfE2dPHflr6kt87F8tW09v+Ur+Vp+JV7pGN+5rPL2Fbjla1qc59XyNbF1bPfrzILpmMqPoeWr\n09BzpfU1uf/85Y916FfZybGTf5uhX79b7of7uIU+HyHj+Fhfs5K+OrhIi4iIvC4CWrOeRI3o624R\nEZGc0kpaRETqSjMljuX+fkZUji/7fjrlxkrTiiWH/pTLPZcl/uw7ny3eXb24tSUmbIlt+6T1061a\nCvl5lSWW2z+25SdXlR+z5E+OKs/ZEt/0zyck3ptkiWWH/izKPX81Y/Sh96OWcWv3HS354yr7a1bS\nl/uLtIiIyPoUkxYREZHMaSUtIiJ1pZli0lpJi4iI5FTuP4xExBWTwEILc9gSvtIpimKpwW1JSqtm\ncpkl2a6atbuLpeEXKin0ZlvcpFQc/vl7C9bEsbAEr+Q4btGN4Sebhc7HkpTVP6fsCoxYEvTSLBRi\neYxqmVxW6RhI1vP2Jb7WkmLSIiIikrncr6RFRETWp5i0iIiIZK4OPozEFeOg1WyeYSsCEtJgw9Z0\nIqSYiiVunVa82VJcxBdrtsSSC719hn0q7lJVpWLlOboKxji2Jd5tiW+78UxfsQpb7+rKjUJC+2tn\nXWCk0thpxYStY9cybp08pvJlwddzupbqNSYdRdFZcRzPGs4xWkmLiIhUWRRFM4H9hntcHaykRURE\nXqeYtIiIiKQiiqLpcRwvDDm2WT6MiIhIg8ggJj0liqK717s9P47j+cM4flLoiXN/kfZ1wapWh6vQ\n5LKQoihuQtpgY4cUKvEll1mS7UIKnliSwnwJYJaEr4A6IQBENUwmC5ljryEhrl/5fiXPX6slCS0k\nAS28U1ZY565qdeqyFe+oXeJWmmOHjNPtSfhyC5Mk3z38yYZN5pU4jmcM9j+jKDrOs3lRHMcLR7KK\nhjq4SIuIiORZhVX1tCiKptG/mp40cNG+xzq2LtIiIlJX6uknWHEcL4DXVtsThnu8EsdERESqLI7j\n+XEcbzOcVTTUwUq6v8HG0PHUajbPCIklhxZFsRQq8d+P4TfGsIyTVrzZGrc1xZJD4s1V7MEREv9u\nDfyr8z6OXeWPf68nBGspuJKMbXuaVxSd5hUBhVTAGstOnj8Zpw0rZpI8pnYxYd9YtYxb+9iKlwx/\n3GrL/cUrJVpJi4iI5FSzfBgREZEGUU8x6ZHSSlpERCSntJIWEZG60kxlQevyfqZVhCSkKIo/4Sud\njltpFVxxixNYx3ETxXydqtxEMV9REjcHyZtc5dtmSfAKSRyrZnGTkL8gX7UIwziWJDVfUlqrc77Y\ns4+lwIqbgBZaSMWXcBZSvMSXcOUrlJI8V+26UPkTrNziIclSIWkll7ndqnzvDe7LMaRwSUt9Xkrq\ngh5ZEREZkT04APhmzc6nmLSIiIjBaCbwRg7MehoNSytpEREJtheHE9X4d9OKSedK7I2fVuKL+bpC\niqKk1cwjtMGGv1CKGxOvPMf2Lk/culrxZt9TkVbhktBCJWnFqdMqlGKZj+Wv1TAf33NmKbDiPvch\ncWwYLJZd/np0C6eArQmIG0/1FzxxirIENvxIxps9BWC8cfPhF1gJLYoScm5fyoR7X9e9x23IdmzP\n3tzDjyqOK2Hq4CItIiJ5ExGxN0ezksX8jZ/V+NyKSYuIiAxqB/ZhCltxBzfS611/Sxq0khYRkWFp\nYxRv4jCe5yEW8Zean7+ZYtJaSYuIyLC8kYNpYzS38V9ZT6Xh5f7DSP8npqGTt3yJWi5bgZF0ipCE\ndqqybAstVOImioV2r4rWOhvSTApzx6pWV6zQcXx/LWmdP6WksOD74X5b6dnHTTjzxgSd4yyduyDZ\nvcuScOYrlJIcN6xTliW5rNIxvnMNdr5Kx/nnWD4nt3AJ+N6LfKVKhvdV9Xi2YGdm8ggLWcljQcVP\nxC73F2kREcmP6XySHtZwHzdlNodmShzL5CIdRdF0YBpAHMcLspiDiIgMz2a8lY3Ynb/wTbpYmfV0\nmkJWMelzBi7O0wYu2CIikmMF2pjOMSzhCR5hYaZzWZc4Vqt/slTz80dRdChwF0Acx7Mq7k9cMeZq\nieVaCoyk1eDC27zCENt2C5dYj0urUInbiCE4bhxazCTrAidZjQvpFTOpZmw7YGxLUxBIFlPxhZuT\nxVOSceuSM46vcIoby7bErX1x12TBE1vDD0vRkZBiJj6W4iXuvfM16ihSYmcOYjQbcCfn0EKPYVxJ\nQxYfEt4Er33lPdN3oY6i6DjgOICJW4yp7exERKTMKKayI4fyFLfyCv8AT5JaLTVTTDqrr7sXx3F8\nD7y2si4Tx/H8OI5nxHE8Y8zUjtrPTkREXrM7xxID93ND1lNpOlmspBcDiwb+eyn9K2slj4mI5NAU\n9mBz3s4/+C/W8ErW0wG0kq62BQxkdgMTGIhPi4hIvkS0sBsnsJIXeJgfZj2dplTzlXQcx4uiKFo6\n8DX3ZEvymCskmSp8n3QS0NyiApbkMutxlkIl7U5SWHD3qpBiJtaCH7XsnpU3lr9EX80J9zh/C6Ph\nnyuw4Enisbe+wxgSztLq1OUWSrEkl/lYunJZCqX4hBQz8XNT3oZXuGQLPsQ4tuIOLqLIqmEdW21Z\nZ13XSib3M47j+QP/qa+5RURyqJXxbMexvMLdvMDtWU+naTXLhxERERmG7fkEBUbxIPOynkpChO2b\nldRk+O2cGmyIiEiZcWzHZnyAp/gBq3gy6+k0tbpYSftixZVY4ta1jW2HNtgoP65Y8uxjKFSSiPH5\nPhlWq5iJZRzrcZZxKo2bR6GFUkKKoFhi26HFVULO5TtfWk1APPtY4tZuoRS3SArYGn74CqV0O78x\n9jXNcYug+OLYyXEs8eZkWRY3tl2gxE6cRA/LeIJvUaDkef+yFEmpniiColbSIiLSbDZiJhPZnUf5\nGr2qz525ulhJi4hI9bXQyXacwDIe5Dl+kfV0BhVF0JrtYr5mtJIWEREAtuTjdDCVh5kDxFlPR9BF\nWkREgA42YzOO4jl+yTIeyHo6MqAuv+5Odp2qnHVj614VllxmS0Ar3+ZLFvGO7SSKta1Ndp9xE8US\n3awgmfhg2ceS3OUWN/EdZ+14FZIEFprQkbdkstC/xGp1vQodt5YJZ5ZOXZ45JpLLPMO441i6crlF\nUsBfKMXtTWEpghJSAGW4tuMkYnp4knmJIkppn2ukap44liGtpEVEmtwE3sYk9uEZ5tPN4qynI+tp\nks8iIiLiE1FkS85kDf/iBb5Lso5s/tS8mEmGtJIWEWliG3EUnWzFk1xNnLs4kOT+s0hE7I0nr88S\nb/Yf58aSLXHrsH1CCqcAFHqd4zwPRSLsFVqExN3HF28OiVunWcykUQuc1LKYiWWfahY8SSuWbRnb\nErf2CWj4YYlb9yuP91qKoLiFSyCZ1+LvrVI+jlvwpMhUNuPTLOX3rOBWCgz2Hldwbmf8BxRRDwv+\nVGglLSLSpDbgTCJaeYphNyOUGsn9SlpERNLXyZ5M5BCe55t08XTW0xmeiKa5emklLSLSdCI24gJ6\neJHnmF95d8lMk3wWERGRdSZwMKPYg2c4nT5WZz2d4WuilXRd3s00umKFnss3jqUoiim5zNDhypec\nErkJXr67GrJPaHJZaDETS1KaZZyQfUITt9ISmodTrWImlnOlWfAkZCzLOIHdtEzFXQzJZcUOz3Fr\nA4qgJPPGggqKdNNOC2PZkDNZzV9ZyQ9o83TGqnQuXzEmqY66vEiLiEiYKZxMgcm8yDFZT2VkmuTq\npZi0iEiTaGMbJvJxlnEjXfwj6+mIQZN8FhERkQ24kD7W8DKzs57KyOh30iIi0khGcQBjeCevcC0l\nXs16OmLUsCvpkIpjluQyS6UwS3KZP0nMt638ti85pabVxCwJaKGVw0ISzqpZXayWRZVC/xKrlRRm\nOVdaHa+gaslcg5ThqjyOoZuWZT6R5zBLpbIuypPJfIlaXU4ymVsVrPy4dqZwEV08zHK+XfZ+5L5f\nlQzvcb5z1ZSyu0VEpFGM4USKbMULfJj81cSVoejrbhGRBlZgE8ZyKmv4CWv4c9bTkWHSSlpEpIGN\n4yIiWljGBVlPJT36uru+hMaSLfuEdHuxjOOPPyeLGpg6XLnbfHG4kM5UoYVKQuLWvm2h3ZKqVbyk\nmt8SZt0Fq1qx7bQ6XlVz7MB4c1p5BL48E7d2iaWblq/gScy7GcUhrORK4F+mwiW9Q8a2+4UUUpEw\neqRFRBpSgXHMosTTrGRO1pNJX5P8BEsXaRGRBtTOsbSyK0v4D2BN1tORQLpIi4g0mIiJdHIhXfyR\nLn6c9XTS10QxaWV3i4g0mE4uIGICKzg766nICDXsZxFLwpdlHzcpzdLhKpRbuAQ8SSVpFQ+xjGPZ\nJ80uWGkVM0krucwirXFC/xItx4UUCgk9V0jBE9/Yoclclcb1jW3pphX6OrMc5+mU5Xa283XTKvX6\nC55E0W60t36K3r6vEBf+Xha69SXZlgzFmNwCJyEJtamqs5V0FEXTgWkAcRwvGM6xWkmLiDSQ9uL1\nwBK6ey/KeiryunMGLs7TBi7YZnX0WURERIZSaDmMQss+dPUcDywBRmU8oyqqk+zuKIoOBe4CiON4\n1nCP10paRKQhjKKtOItS37309n0j68k0milRFN293j/HDePYNwGToyiaHkXRWcM9cdOspEOLmVhY\n4tZuQw1T4RIIK0JiieX69nFjl2kVKrEUV/FtSyveXK3iJpZz+1jipGmez3J+yz4hBU+s7zBpxYkr\nzcc3Tmjc2o0T+/IzQougOPv4GnW4tUtKrWfSEm1B19qjKPT1T7i96PzxeVafbvGSNk8zD3cfSwGp\nqqp9TPqVOI5njOD4xXEc3xNF0cwoig4dTly6aS7SIiINq2UrCq1nUer9LnGf6nPX2iAr60VxHC8E\nFgOLBrYtpX9lrYu0iEjTGHUN0EtP97C/Ta1POcvujuN4/hD/ewFw6MB/T2AgPm2lmLSISD0rziRq\nO5jenssgfi7r2YgjjuNFwNKBBLLJw/0JVo4+i4iIyPAUYfQc4tJjlHquy3oytZOzlXQl6620h3WB\nhrq6m+kL6Z4VWiTF7XplKlwCtU0KSytJLeRc1rFD5uhTy4In1Rw35C84rYQrn5DkLt/5QpO5Ks3H\nOo4luSzkXKE8Y0e9QOeJUNgZlv47hSiZoZl43/G88bjvg25xE98+aRVwksr0dbeISD2KpsLoi6Hr\nV9D906xnI1XS1CtpEZG6NeZyiEbBylOznkk26qSYyUhpJS0iUm9a3wgdx8KaOVB6OOvZSBVpJT1M\noQ023OIlpsIlvm1pNcZIqwiJZR9L3Np6XEhcsppx66yFhAarWXAltJhJWg0t0mrUEfK41qzZSwRT\n5kLfS/DqJRD3b2317lv+vlPyvPEUCsNvIpR5TLrOEsdGoknupohIgxj7EejcGxZ/HOIVWc9GqkwX\naRGRehGNgSmzYO0dsPo7Wc8mO1pJi4hI7kz6AhQ3hucOgijOejZSA7pIi4jUg9btYOJpsOxb0HVX\nssFHM4lomuxuXaTXYylUYuF2vDIL7YJVy05ZIftUswtWNZPLKo2bB9XqgpVW4pbvufed35LMFTLv\n0G5aIa8Z3/x8F1K3W5ZvH9/YG18HfWvgpXP65+zu47loFZ05ucVNANoK5U9SyTNQSFEnSYcu0iIi\neTf6fTDm/fDi6VB6KevZZK+JYtL6nbSISJ5FbbDh9dD1ICz5UtazkRprks8iIiJ1auqp0LYdPLU/\n0JP1bPKjSa5eTXI3w5kKlZiKmZTf9jbT8LHE2KrVmCLrBhtu7M56/pBzhYbYsg7N1bLBhuXcaTXv\nSCsmHtpgw40T+16Loc1EhvOctW4MG54Py34Mr94yjAP7RYmYdF9in6JT4MQtbgLJGLSlOZGkQxdp\nEZG82uIqiFrh2dOznkm+NFF2t2LSIiJ5NOatMOVj8Pxs6F6U9WwkI1pJi4jkTgtsORe6n4Hnrhis\nMHfzaqLs7ia5myIidWTqMTBmBjx2JPStyno2kiFdpKvAVzAgwZpMFSI0ecdSTMRNoAktOGIpcFLN\nYiZV6U7kOXe1VauYSS0LnvjGsiSX+YSME/p4pPVcJ+YzHja/Apb9EV68sX+bL/5qKGbi7tPu+btz\nO2P5E8eUKJYVXaRFRPJk64ugOAmeOCnrmeRXE33drcQxEZG8GL0zbPpZeGE+rPpb1rORHGiSzyIi\nInVguzlQWg5PnZ/1TPKvSa5eub+bMRGlHE0z9Ef8xdCQTkhDC8s4tYz3WmOXIWP7YtuWc1nmaBnH\nIq1cA0tjCotqFjNJK5adVqGUtGLJofH34djwQzBpJjxwIqxaXHn/gMItviJKboGTQntyJ/d9TzHq\n2snP1U9EpFm1dMBO18Ly++Gpr0F71hPKuSYqZqKLtIhI1qadCaO2gr/sA7FWqfI6XaRFRLLUsQVs\ncw48dxO8+oesZ1MflN0tIiI1sdPVQAwPnZn1TCSH6vKzSMkJRvRmHJxwO8QEswwTmvBVraSwtLpZ\n+cayFDxJM3EtZJ+0vpkM7Qxl4f55hCaFudIsZlKtLlhpJWz65mzplFXJ1H1g48PhoQtg5dODn9/3\nFuf+fVgSCz3juB36iiXPg1Zwb2b8lbxW0iIiUlVRAd4wF1b/Cx67OuvZSE41yWcREZGcmfYZGL8b\n3HUw9IUsw5tck2R3ayUtIlJrbZNh10vgxYXw/A+zno3kWF2spN0YtO2YnN81a0gnJNxdyyIkacaE\n3RhbLWPr1Yxbh6pmEZRKQgueVLOYSUhjDIvQ++Euft0YtW+fdfb4IhTHwZ2n2AryWOLNlr8pzziW\nQktuDDq1PJxQikmLiEhVTNwTtvs0PPwlWPbPrGcjOdckn0VERHLiTXOhazHcf1HWM6lfTbSSbpK7\nKSKSA9M+DBu8A/7ySehZlvVspA7o624RkVoojoa3XA2L74bHb8h6NlIn6nIlHZZIVrt8fberjFlo\nsk7IcbVMCrN27go5zlLwJK2ksFoWN0lTSI6P5Z0hrU5VvrHSSi6rZsctyz7rJ5NNPwdGbwa/Pxx6\n1nuP8L01uXPyJaBZitQEvIYLvck7Vijk7IXdRF93ayUtIlJt46bBHp+DR74DL9+e9WykjjTJZxER\nkQy99VoodcMdn896Jo1DxUxERGTENj8AtjoI7vkirH4+69lIncn9SjomqhhP9hUuscSg3cYcuSyA\nUq3YabXODbZiIqGxbEvBk2o1CvHJuuCJRVrFTCwx0LRi2VnHrdPS0gp7XQ9LH4W759jP5c7R93bm\njhWYexE5+/hyagrt5TupwUbtZLqSjqLorCzPLyJSVXucBJN2hD+c2v91t8gwZXaRjqJoJrBfVucX\nEamq0RvCWy6EJ34O//pF1rNpLOtW0rX6J0OKSYuIVMM7LodiB9x6WtYzkTqWyUU6iqLpcRwvzOLc\nIiJVt9GbYPdj4d7r+uPRkr5CDf/JUFYL+UnWHWOSCV7JhK/Kj2JoMRP3OPfcVm5yRs2llShVzQ5T\n1SpU4isEEZIUllYxk2q+FkILc6SVFJZWMZOsk8sqHTOkCPadByufh9surfw68r2lGLpXmV6f7YZ9\nEoljnn0S08lZcZMGVvOLtGUVHUXRccBxABO2GFOTeYmIpGK3o2HTt8BPPgY9K7OeTWNqouzuLO7m\ntCiKptG/mp40cNG+Z/0d4jieD8wH2GzGBnEGcxQRGb72cfDuK+GZ2+Af302uZEWGqeYX6TiOF8Br\nq+UJtT6/iEjVvO18GL0B/M8H6A/WSVU00Uo6s+zuOI7nx3G8jbuKFhGpS5N3gDedAvd9E174a9az\nkQZRB59FokQlMEtlMFsyWTp3PzG/YvKzT1wsr+IT+ZKiLKqZvJR1F6yQSmW+fdxEsdA5Viu5zCqt\nhC+LaiWFWRKefGPXMrnMx30NdRj22XcOdK+CW87zJz0OJjQpLK2/V2db0bNPseRszLpudpSDOdRI\nHVykRURybqcDYdsD4JenwKqXs56NNBAVMxERGYliOxx4Hbz0ANz55axnIw1GK2kRkZF4x+kweRv4\n9kzoy7ogQpOos8SxKIoOBZYC0wZ+vWRWR3fzdW68OaTjlWXcupFWnNpyTEgs1xqTrlahkmrG1rMu\nXpLW2Gndj9AOU5ZYdlpx6xC+11kHMH5TeM958PcfwCO/Te7je0tx5+S7H5Zcg4B4s/lv0Z1Ob/lA\nhYKKmVhFUTQdWBTH8T1RFM30/ex4KHV5kRYRyYX3z4KoAD89I+uZNJ/6unpdRX9DqWnDLYmtmLSI\nSIit3w7Tj4Lfz4JX/5X1bKS6pkRRdPd6/xxnPXBg1bwoiqIlwKvDPXF9fRYREcmDqAU+OBeWPAW/\nuzLr2TSf2sekX4njeEbIgVEUTaA/Hn0F8PUoiu6J43iR9XhdpEVEhmvvT8Kmb4D/Ohx61mQ9G8nY\nICvrRQNfbR8HXBHH8dIoihYBhwKzrGPn/iIdE1VM6PL9/7QSxeo2mSxEWklIoYlstSym4habqGUx\nk6wTgK0FRizHuSyJSpaxQ7tXpZUgOZRRE+HAy+DR38MdNw+9r+Wx9iWlWR6PkKQww/Nh6dhXyLoL\nVs6KmVgztuM4XjCcr8qhDi7SIiK5cuDF/Rfqm0/OeiZSB+I4nhVF0VkDq+hJTfETLBGRTGyyK7zr\nBLj1K/Dc37OeTfOqs99Jx3Fs/nrbpexuERGrI+bC6qXwkwuynok0idx/FompXLzEF3+uXlMO37ly\nFByB6jbhCIk5ptlgIyRu7Wt2kFZsu9Ixg41tUa3YdVrzscRbffukVagkrfvha57hM+NQ2OHd8O3P\nwJIlyV7RlmYvkIyl1rLBho+pmEl5g6Bc9MnO/dUrHVpJi4hU0tYJH74GnroP/vD1rGcjTaRJPouI\niIzA+86GyVvA1z4CcV/l/aW6cpbdXU1aSYuIDGXqVv0X6b98Hx79U9azkSajlbSIyFA+Mhv6SnDz\nWVnPRNaps+zukaiDuxklEsNCCpVYEr5847rbTAloRd/86uArspBEJUvClWUf37bQ5LK09klrPpXG\nrbXQwiWW59VyLmsyWRp853cTxQbrcAWw277w5kPg5vPgpWfK97F0swpNkksrcSytvwVHIfMXcfPQ\n190iIj6FInx8Dry4CH55TdazkSZVBytpEZEM7H88bL4LzDoIeny/45PMNNHX3VpJi4i4xk6Bwy+B\n+34Nd/8k69lIE8v9Z5H+BhtDFyYJLVRiK16SThGUXmeX1lr/fKCWzTMsx1jj1JX2SSsmXs1mHqGx\n2yylNWdrE4yQIiih47h8xUwOvwzaR8P8U/0xayvf+d3xfO8Flteeu83ynAXuU8jb6xP0EywRkaa0\n7XTY75Pws7nwzENZz0aaXO5X0iIiNfWZebD8ZbjxkqxnIoNpoph0k9xNERGDd38Edtob5hwDq5dn\nPRsRXaRFRADoHAPHzIKH74TffTvr2chQtJLOD0sXrJoWKjElifn26al4XO4SIdLscBUirYSvtDpl\npZQUFhsfn97QLk8BEvV3DHOM0kruCj0u7aIoHz0PJm8CF3wIeuOhz+07l/VvwXJfQwqe+KTUKatY\ncm/X8MXZ5HJ/kRYRqbpNt4VDT4df3QAP3WlvXynZaKKVtLK7RUROuA6618I3zsl6JiJlmuSziIjI\nIN7yPtjrA/DVz8GSF7OejVjlLTxYJXVwkY5SiUn74s3JoihhBU9MDT+c07dWPGKAO+06eMYSrHGw\nWjYMqFIxE1+82RJb7kkpjt/rGadoeM1Yzt9qiKUWnXG8cevQ58wSyx2u1jY4/jp46mFYMPf1MS0F\nTNz5WIqS+LaF7hNyLos8Fi5pYvX4li8iko4jToHNt4ez3gu9huROyQfFpEVEGtzkjeCY8+HPP4E7\nf531bES8muSziIiI44Qr+7/u/vLpWc9EhksraRGRBrbrXvD+/4DvXQPPPp71bEQGlfvPIrEnccxN\n1LIUKvEJS0ALS1JLSLMQgyW5zN0W2h7XklSSVp2DahZwSCkBzU0U8yWJuUlZvuQun7SSyULGSSSJ\nYZu3m6TW6ksuMyRYeRPO0hJFcPo8eOlZmH859Hn2cX8n7bvvbnJZaMevOkgui5x9CrWstNPkcn+R\nFhFJ1UHHwM4z4JyjYM0qaM96QhKkSX6Cpa+7RaR5jB0PJ10B9/4JfvX9rGcjUpFW0iLSPI6/ECZM\ngRMOyHomMhJNlDiW+7sZE9HlfB/VyZqy22kVKvHtY4l/VxoXoFQs/9IiLiYDYZFvsNw/Q4FCm1VY\njgkoQpJWoRJf/NeN5Xr3CQzxpfXLXre4jmU+vthyIv7uef36iqu4MXDfyz7x9zHcx2ybneDIk2DB\nfHjgvqFP5j5Hvj/70AYb7lgZFcVeAAAgAElEQVSWBhs+Ac1dTPuomEmuNOolQESk3PlzYdUKmPeF\nrGciI6WVtIhIA9nvg7D3TLjks7B0cdazETHTRVpEGlt7B3z+Wnj47/D9r+pdrxFoJS0i0iA+8TnY\nfGs4+j1QKuldT+pK7l+uvmIm3bQNedu6T5tT0cOXFJZMQKucpOZTSmTZeCooWJ6N3D9jHqF1D9Iq\nBOFjGMdSqGSNU9DCUjjEW/Ck8mGpJYmFju0ml/V47oclAc3tlAXJBDNvcpkzlqmT3Cabw6fPgV/c\nDH/6v/5t7p+ir+OVu48lucz6Wgz5Gw5NfrSME6DQ66sAU1uxfictIlLnPn91f4Wxyz+X9UxEgtTj\nukxEpLK93gkfOAKuvxCeeyrr2UiK4ghKTXL10kpaRBpPoQBfnAvP/Au+Nivr2YgEq4vPIpWKjlga\nbITHm9NpwtFbcOZTSEYBvTG2kDh1LWM1aRY+qFYRBUPhlNBCJYniHYZ4szW2bHk4qhWn9r0WQ+bj\nfU1XsTfDa+f76HGw8x7wqUNguS/oXIHbgMb3d+j+nYUW6Klm84yQOLUl/p01raRFROrUxEnw+Uvh\nj7+FX/wg69mIjEiTfBYRkaZx9hdh7Dj4wilZz0SqJI6gt1DLNWZ22exaSYtI49hlDzj603DDf8JD\nD2Q9G5ER00paRBrHZXNhyatw9UVZz0SqKI4iSr4f01dNdw3PVS73F+mYqGISmK0LVvWSwiyJbIlj\nfMUaLAlfvn1Cjksrucz3CnKTbnxCE1HcpBZLskxgwZNEUpglucw3juHUlgSwWubuWM7le+pD76tb\nqMRUBKWj/Gbh0CPgre+k9+RPEb+0tH9cz7lSK1TiHmdNuKpWQR7LPnlLAJOK9HW3iNS/UaNovWw2\n8X1/Jf7vb2U9G5HU5H4lLSJSSevnzqFl083oPfYI6Mu+ZKVUX6nQHHVBtZIWkboWbT2N4iln0nvj\nfxPfcVvW0xFJVe5X0v0NNoaOL6cVSw6NNydj5IbCKUXP56OiZwVgiZ8lxjHsYznON467zRJ/DlXD\n+JkvBurGoL3FTCrchuTdsOzjk3Xc2n3qvbFlwzih99+N+a5rbjL20mugp4cVZ51NuzNQoq8NELl3\nxHdy97g0m72kVagkZB8LQ55HIePYtq/xUqPSSlpE6lbrfvvRdtAHWX35pfQ991zW0xFJXe5X0iIi\nXsUio66dQ+mxx1h7/XVZz0ZqyPern0ali7SI1KWOEz9LcaedWH7QB6A7u9+xilSTLtIiUndaNtiA\nzgsuovuXv6Dn5z/PejqSAV/ti0aU+3sZE9FNW9m2bJPC0imc4hN7no1EkotPNZPJ8s6SmONJ+nG7\nXvmSwhLDGjpcWYp3WAt8ZNkFy3IuS6cs38vOklxW6fxTLr+cqLOT5aeeRmm958VN9lvjGafTuR35\n/jRDksusiWMhnal80ipmogInudaob90i0qDa3vQmxn7iE6yaNYvSI49kPR3JgLK7RUTyKIqYPHcu\nvc8/z6pLL816NiJmURRNd24fGkXRzCiKzhrqOK2kRaRujPnYx+jYay9ePvpo4hUrsp6OZKTeVtJR\nFM0EvgZsM3B7OkAcxwujKJoWRdH0OI7v8R2rlbSI1IVo7FgmXnUVa2+/nZX//d9ZT0fELI7jhcCi\n9TYdASwd+O9FwMzBjs39Str3ianLSSRrc24DdFYtKSyl6maeUki9hWTFsdaQKmChiWS5fzVQ0yQX\nN5nMkqSVZsUxS8KZRVrJZW7Cl6UxlE/ocVPOP5/CBhvw3IEH0hvH/vvlVMBzk8QAep2TuR24vJMM\n7YIVmiRmSH4MOlcDqaeVtMcE4NX1bk8ebMd6eFsWkSbXtsMOTDj1VJbfcANdd9+d9XSk+UyJomj9\nF978OI7n1+LEukiLSO5tdP319K1ezeJzz816KpIDGVQceyWO4xmD/c8oio7zbF408DW3z1Jg0sB/\nTwAWDza2LtIikmtjPvABxrz3vbx82mmUXnop6+mIJASsqm8C1l30pwGDXczzf5GOqRw79sWSu2l3\nbifLBpacUgfV7Kbl3odeTy/UUjEZZWtNqwtWM8etPfFEtzCJWwTDt0/gqVIrZpJ1FyxLoRLLfbW8\nhNb9ZUbt7Wxz/fWs+ec/eflLXzIc6ZzLMwF3W2tooZJQaXXBColBh3blcruLNVn8e6SiKDoUmBFF\n0aFxHC+I4/ieKIpmDGR9Lx0ssxvq4y1XRJrUhqedRvs22/DIfvv5P0lJU/K1MM6zOI4XAAucbabV\nt36CJSK51LrJJmx03nks+eEPWbFw0G8DRRpa/XwUEZGmsulVVxEVizxzxhlZT0VyqM5/gmWmlbSI\n5M7ovfdm8kc/yotXX033E09kPR2RzOR+JR3TkkgCSyvhy+2u5d62jpNICjPNx1M4pej5zFR0Cpz4\nPjxaPlBa9gl5NfiOcc9l6TIEiUIUeeNL3ApJ5qpmMZMsu2JBeIerMi0tbD5vHl1PP80zV17Jur8A\nX0crl/uy8hUqcUPbbkc08HSfCy14Etp1KiT8nlZyWR2ot7KgI5H7i7SINJcNP/EJRk+fzqNHHEHf\n6tVZT0ckU7pIi0huFCZMYKvLL2f5H/7Aq//zP1lPR3LK99PcRqWYtIjkxpYXX0xx4kSePPnkrKci\nkgt1sZJ2Yw9u7NgSk7Y02PDt0+XEw9s8RVE6EzHywEYdnqYbsROTTsTKIPksWmLAlth2aPy7DiSa\nZ3jiedWKCadZzCQkBh36a+OQNwtr/L0IjNl1VzY64QSe+epXWXb//RXHtoR7fed3i5n4itYkCpyk\nVATEK3Sfpv7ZeH39TnoktJIWkVzYYc4cSsuX8/gFF2Q9FZHcyOSjyHrFyLeJ4/jsLOYgIvmxwSGH\nMOk97+HBE06g59VXm2SNJKGaKbu75ivpgVqlCwdKok0buC0iTaqls5Ptr7mGFX/7G8/Or0n3P5G6\nkcUH1mkD/8wHFg38t4g0qWlnnUXnllty99FHE5ca9Ie9krpmWUnX/CLtFBWfTn/LrjIDX4cfB9C+\nxdSKCV5uchckE7zc5C6wddNKqyiKu63Ls09bIVnNo7dQnvrSGtrhKq0uWC5roZIGFdIFK7SYifW4\nSuOEcsfxFS6xdMpaX+cWWzDt7LN57sYbefnWWwcdxyf0Ze4WOPElDba6bylpJm5Vq5iJZRzfZyB3\nm+G+Rk2dtFZbmb2dRlE0HbjH16Jr4EI+H2DcjO3iWs9NRGpjl9mzieOYh888M+upSB1RTLo2Zipp\nTKR5TXn3u9nksMN49PLLWfvMM1lPRySXMrlIR1F0XBzHswb+W4ljIk0mKhTYde5cVi1axOOzZ2c9\nHZHcqvnX3QMX5auiKDobmAQcNtT+MVEqDTYsceLQRh0hhVP84/iabpTfbrXEgEPjzSHHpXmukNh6\nHQiNG1uKqYQWT8naNscfz7hdd+X2D36Qrq4uU2OOTud2aH2RxOPq2cltuhEl016SLPFe63EWljvb\noGKipikLmkXi2EJgYq3PKyL50DZlCjtfcgkv/uY3PPfjH2c9HZFcq8N1iYjUs10uvZTimDH87ZRT\nsp6K1DGVBRURSdnEN7yBrT/1KR6fN48VDz2U9XREcq85PoqISC5MnzePrpdf5p8XX5z1VKSONdNP\nsHJ/kfY9GZbiIZaEr2RRlOQ4o4KSwixdsCoXTgEoFZ0vO5yuWICte1VIp6y0ksvqlKWYiGUfV2h+\nT5odttLgG3eoBLCtjjqKKW97G3d+4hOsXb582OdLq2aPm4DmK2bidsZyC6AAtiIgPiGJYirE1rQa\n5O1URPKsOGYM06++msV33cUTN9yQ9XSkzmklLSKSol3PPZdRm2zCrYccArGKCIpY6SItIlU1dttt\n2en003n829/mlb/8xfSbaJFK9DvpnIipXCzE92S5TTc6WZ3YxxIndsdxG3f0Hzf84iqWfQBKxfJt\nsScmHVkCce7Q1dwnNLad1t+cO3ayb0lNWWLCoc0z3H1Cx6mm6ddeS193N3d//vMjmot7cU+rmIkb\no4ZknDr4g0VoM420YtCWBySkCYfUTO4v0iJSvzZ973vZ/MADuevMM1nzwgtZT0caRH9MujkuX/qd\ntIhURUtrK2++/nqWPfwwD86Zk/V0ROpSc3wUEZGa2+Xkkxm/ww7c8m//Rl9PtX4IJs2ombK7tZIW\nkdR1brQR0y+8kKd/+lOe/dWvsp6OSN3K/Uo6pqVi8pabuNW/zS2Akmxj050YJyzhq1rFVQB6C06S\nXCG5ImlNq8qDpeCJK619rMdZEtcyThSrxLqmDOmCFdpxK20zrriCQlsbt512WtD5fE+rO47lJexL\nCkvkUlmSoiwJV1ZpZfKFFlNJQ9bZiPjfQxuRVtIikqoN3/IWdvz4x/nbtdey/PHHs56OSF3TRVpE\n0hNFvH3ePFY++yx/veyyrGcjUvdy/3W3iNSPHT/+cTZ405tY+JGP0Ltqld5gpCpiIhUzyQtfMZNK\nDTd8+1jizZYGG5ZYsqW4iq8oiqlQiucZa7XEad2QvC9um1ZsO6QoymD7VYkbx09mNdRWNcObIXHr\nEO3jx/OWK6/kuT//mX9+73sjGssyR0uBEV88PNFgw3f+kAepmnHaHMSAJRu5v0iLSH3Y64IL6Jwy\nhR+/971ZT0UanIqZiIgMw6SddmLPk07igW98g5fvvTfr6Yg0jOb4KCIiVfXuOXPoWbWK2887L+up\nSJNolp9g6SItIiOyzUEHseV++/F/J5/MmldeyXo6Ig0l9xfp/mIm5QldbjKXP1Gr/Jg2TzETtzNW\naBeszkQXrOQ4oV2wEoVSOpLpMu1d5akvkS8pbK1zO7RQSGiHq7T2SSu5rYYFT9JKCqtmN62QgiOt\nQKG9nXddey2v/OMf/O0rXyHZoy1M6Msq5L76Cp64Ys/AUWiHK8s+ShQbUjOVBc39RVpE8uuNZ5zB\nhGnTuHnffekLSokWkaHoIi0iQcZsthlvOfdcHlmwgKd/97uspyNNpJlW0sruFpEg75w1i6ilhVs/\n97mspyLSsHK/ku4vZlI+TUucOFEExBvvbXf2qdyoo5oNNkyFUgqefZymG4mGG5Beg42QfawBxrTi\n3Slxo/9pFTxJs8FFVg02NnvHO9jxyCP500UXsfjJJ1/b7j5m1i/A3afVEqa1FCqp6Rfwefy239KE\nI4/zNmiWimNaSYvIsESFAvvNm8eyJ5/kjlmzsp6OSEPL/UpaRPJlj099ig322IMfHXoovWuyLqgq\nzUgVx0REPDomTeKdl17Kk7/7HQ//7/9mPR2RhtccH0VEJBXvuOQS2sePZ+Epp2Q9FWlizZTdnfuL\ndExLIhErJJnLkqjlS/iqVEjFN05axVV8Y3sLpTibEl2xINsOV6FdsCx/g5axPeMUnW2+ZLs1btJN\nxkLzeyyJYpaxN9h9d/b8zGf465e/zAv/+Ifp3JZOVb7zW49zWQqVWBLQepydej2vhdA5ppao5Y6T\n1uu1gZLLGoG+7hYRk/3nzmXtkiXceuGFWU9FpGnkfiUtItnb6fDD2fJd7+IXn/40a5csyXo6Ik3z\ndbdW0iIypNZRo5g5ezYv3Hsv933jG1lPR6Sp5H4lHZOM8VrixJWOgWRc2B8THrqQiu84f4ONonO7\ncnGV/m3lhVq8BVeKzmetoqfNgSFOa4pJdzi3fY0qQgqeDLZfpX1q+GHaF4O0/ADJ8keWZoGTSoZb\n8OTtn/884zbfnAVHHkl33+uvLd/jERq6dMfyPR6Wl6cbg7YUPAlmiQFXM06cpYzzNWIiFTMREZm4\n9da87cwzuf+73+WpP/856+mINJ0hP+RHUTRoUd44jmenPx0RyZP9Z8+mr7eXW846K+upiLymHouZ\nRFE0PY7je9a7fdzAf24Tx/HZgx1XaSW9HLgKWAosc/4RkQY2beZMdjr4YG697DJWPPdc1tMRqVtR\nFM0EbnZuL4zjeD4wbeC215AfReI4nh9F0aFxHCtbRKSJtBSL/NucObz62GP85brrsp6OSEI9ZXfH\ncbwwiqJF622aNvDPfGDRwH97Vfy+II7j/Uc8wxGIaaG75HSrKpSn6/iSudzOWJYOV77ErZBuWm6y\nl+84axcsd5s3Aa7dKZTSvjaxT6sl4cvdlnw4wB06zW5WaXXBMowTOduKnn1ancfDlyQW0inLmnAV\nkisUmoDmHrfXiScydeed+d6BB9Lb5Xux+M+VVhGS0C8y85ZfFaxh7kjDmBJF0d3r3Z4/sAoO4hw7\nHbhpsH1NfwtRFD0G3ALcHMexuruLNLDRU6eyz8UX8+gvf8kjP/tZ1tMRScigLOgrcRzPSHvQKIqm\nA/esH6t2mS7ScRxvG0XRvsB+URRdBTwex/GHU5qniOTIvpdfTuuoUfzq1FOznopIXVgvCWx9i+I4\nXljh0JlDJY2BfSW9FfAG4I1ARH8imYg0mE1mzOANxx7Lbddcw+JHHsl6OiKDylNMOuSr7yiKjovj\neNbAf88c7IJu/Z30IuAI4Mo4jmfEcfyZ4U5IRPItiiL+be5cVr30Erd+8YtZT0ekYURRdCgwY+Df\n67K7r4qi6PEoioass2vNz5gE7AscVvOvu2Mo9ToJXoV0OlyFdNPyJW65Fc984yS7YCXH6TRUTvPf\nV6eamS8JytKZKiRxy/dhNuRcoeevZgLayA/JBWsO0u4f/Sibv/Wt/OjjH6drxYqqnTu4e1RK3CS1\nzHO0Mp9A/am3imNxHC8AFqx3eyEw0XKsdSXdR/+FejL6uluk4bSPHcvMq67imTvu4G/f+U7W0xGR\nAdbFwe+AG4Gz4zh+oorzEZEMvPMLX2Dsxhtz40EHEcdx1tMRGVI9VhwLZc3uTj31XETyYdL227PX\nqady77e+xbN33ZX1dERkPbn/KBLHUSIm3dteucCHGyf2xS+SceKwDlduMRVLNy3fOMFdsNy4tdsV\nC5KdsdKKJacVt7aeP2SOhtBVq2econucofNPaKesWnJjsjOvu46eNWtYeM45qY/tezwsHa7SKu4S\nqjeky1PGnaGkMeX+Ii0i1bPd+97Hdu97H78+/XRWvfRS1tMRMcvTT7CqSa0qRZpUoa2N915/PS8/\n+CB3fulLWU9HRDy0khZpUnudeiqTt9uO/9p/f0o9oVW/RWovg7KgmdFKWqQJjd14Y951/vk89KMf\n8fgtt2Q9HREZRO5X0nFfRNfa8oSqtg4nmaqQTM1xk7ncrlgQ1uEqtJtWMgEsrAuWryhLIgGu3ZNI\nVyxvX+V2gQLSK1QS0nHLev7QpLQK+ySSxEgmk7V6EoMsf0AhnbKqbeZVV9FSLPLr00/PeirBLIlk\nqhPSmOqtmMlIaCUt0mQ233tv9vjYx7ht9myWPKGyByJ5lvuVtIikJ2pp4d/mzmX5M8/wxyuuyHo6\nIsFUzEREGs4exx7LJm98IwuOPJKe1cnf84tIvuT+Iu0rZlKp4QaENc9IKybsFlLxjeMrwGJpuuGL\nwyTvq6fgihMm70iGzcNiyb591jq3LbFl3zZ3Pr7zpVRwxRejL7oxac99dePNITFqsBX48I1tKR6y\nLi7bMWEC+1x+OU/98Y88fOONZfvWY2635bHO/RucBFF2t4g0nHdcdBGdkybxm5NOynoqImKkD5oi\nTWDqLrsw48QTuXf+fF7829+yno7IiGglLSINZb85c+havpw/nH9+1lMRkWHQSlqkwe1w8MFsve++\n/OrEE1mzeHHW0xFJRbOspPN/kY4j+taWJ1R1Obc725NPVrLDVeWkLH8y1/A7XLmFVPrHKS9m0ukt\nilK5M5ZbFMW3zVtwxe2M5XbFgmSClyXhKzRxK7RQSVrnN+zj64xVaRhf4pZlOtVS7Ohgv2uu4aX7\n7+f+r32NIvVR4CP/b0x1wn2y1amr7uhvQaSBzTjzTMZvtRXf22cf4pLeoaUxqOKYiNS9sVtswZvP\nOYcHb7qJp//wh6ynIyIBdJEWaVDvvPpq4jjm/848M+upiEig/H/dHUfQWz7NknO7u5SMAbtNNyzx\n3lo2z/AVRfFtsxRlMRVKaXebcLgVRzwxWEssOTRubClUkpxi8ny+oizucaH3wx3Gt4/hG2RLg42h\nipAMdnuofTbfZx92OPxw/nz++ax4+unKk0yJpbiL775apPVmZTm/97luVPWQpODo/wlW/i9fadBK\nWqTBRIUC7547l2VPPMFfZ8/OejoiMgLN8VFEpIns8ZnPMGW33fjJhz5E71rf1xEi9a9ZfoKllbRI\nA+mYPJm9v/hFnrzlFh770Y+yno6IjJBW0iIN5G2XXkrb2LH83ymnZD0VkappprKg+b9I9wFro7JN\nPU4xE7crFiQ7Y1mSsizJXMGJW4ZOWb7f/SWLsiQzpdwCK/45Osl3vuId7tCW5C5LpyxLchdUr1BJ\nYHKZmzzkK27S6iSOWaZjSRIDf4LZUGNP3nNPdj/uOO6bO5flDz6Y+h+3peNWPUrtcWqUa0YdJpI1\nsvxfpEXEZO9581i7eDF3XHRR1lMRqapmKmaii7RIA9jmyCPZ6O1v57ef/CTdy5ZlPR0RSYku0iJ1\nrjh6NG+5+mpevvtu/nnDDVlPR6QmmuV30vVxLxMVG5wY8FpP8wyn6YYl3us23PAdl1bzDMs4/ce5\nRVk88feQJhxuww1INt3wfZvkbksrbuwb2xdLdrf5YuIpxbYjJyZf9Jyr1dlmiTdb/+gsd6MH2PPc\ncxm96aYsPOwwin3JximWEKM7tu8YUxEQwz4+7tiWc1mamTQ9PSB1T0+hSB0bO20au59xBo9+5zu8\ndPvtWU9HpCaaKbtbv5MWqWN7XXstpe5u7vz857OeiohUgVbSInVq4wMOYKuDDuKOs85i9fPPZz0d\nkZrRSlpEcq2ltZUZ11/Pskce4R9z5mQ9HRGpkvyvpPtIFp5YW54y4nbFgmRnrO5C5aQsW4crS6es\ndIqi+I4L7ZSVSIBr93XKKn+gW0OLmbgPoy8BLHRs96n2fZh2z2cpnGJIkvMVM3ELnhQDumL5puPb\nb/1krh1OOonxO+7ILe97H4Xu7tem6kv4svyRu4VKqtlxKq3ksmodY5bWQq6W78KhXetySL+TrqIo\nig4FlgLT4zielcUcROpVx4YbssuFF/Lcz37Gs7/8ZdbTEZEqqvnX3VEUTQeI43ghsHTdbRGx2f2K\nK2jp6ODe007LeioiUmVZrKSPAG4Z+O9FwEzgngzmIVJ3Jr35zWx9zDE8eNVVrHzssaynI5KJ/sSx\nOvlefoSyuJcTgFfXuz15yL1jPMVMym+6DTcg2XSjVKgc77XEiX0xYbcoiqV5hqUJByQLrPiLspQH\nYX1FUdxt3ti682poDY1fWfZJq1CKJbbtGycgbp1oQAK0Ose5DTcAOp3bbvwXbEVQeqKIN86bx5rn\nn+eRSy+lmJxiVf+gQxpsWGLtvv1CG5WEzMk7jrOTm3tg1hxhU6miXH4UiaLoOOA4ADbcItvJiOTE\nlv/xH0x685u562Mfo3flyqynI5IZ/QSrupYCkwb+ewKw2N0hjuP5cRzPiON4BhOm1nRyInlUHDeO\nXa+8ksW33cZT3/1u1tMRkRrJYiV9EzBj4L+nAQszmINIXdn2ggtonzqVP7///RDHWU9HJHNaSVdJ\nHMf3AERRNBNYuu62iPiN3nFHtjz5ZP71zW+y9K9/zXo6IlJDmcSk4zieb98ZTzET53avJ5nKSSZr\na09m/XSyuvwYbzJX+XHuMRDW4cpfFMVTlMXQ4crSKctUcKWjPIWmvSuZ4hS5SVlpJZdBMgnMUijF\nUqjEV0zFHTswAa3TOa7HU02kx0kmsxb4WLdt5+uvp7RqFQ+fd96QBU58t2vNkpRlTSZL4/zVPFfF\nk49kv1xmC+VHM8Wk9VIQybGpBx7IlAMO4KFTTqH75Zezno6I1Jgu0iI51dLezvbXXcfKBx7gmS9/\nOevpiORG/y9ztZIWkQxtcfrpjNpmG/46cyZxb9ZfZItIFvJ/kTYUM3EbbkCy6Uap5ClmUhh+vNfW\nhCMwJhzYPMM9zlIoxS2SAtDrFHzpLSRj0okmE5ZYri8mbIkThxZKscSt3btvKa7i2cctcuFrwuEW\nOPFN2S140rrppkw77zxe/t//ZcVvf0sr/nhztRpjWFSzmImlUInlvlrm6J2PM3hkzauolvy/U9dY\n81QcU6tKkRzabNYsaGnh8TPOyHoqIpKh5vgoIlJHxrz97Uw66ij+dfHFrH3yyaynI5I7zZTdrZW0\nSJ60tLD5vHl0P/UUT111VdazEZGMaSUtkiNTPvUpRu25J4sOP5y+Ncnf0otIv2ZZSef/It2HoZhJ\n8jC3M1a3r1PW6OEnZVmSwoITt7wdrpyiLJ6MK7fAii+hwt1mSYDr7kgm0rV29ZVvCE3usnTGsiSc\nhXbBshROcffxdLhyi7u4CUcArc4+bpIY9L+ECxMnsulll7Hy979n5c03B3fPcqV1qQ95swgtZmLZ\nJ60uWMEdrioNPNi2kH3SPE4yEUXRdF+FzSiKzorjeNZgx+nrbpGc2PCSSyhMmMBzJ5+c9VREJEUD\nZbBvHmT7fkMdq89iIjnQsdtuTD7+eBZ/5Sus/fvfs56OSK7FRHVVzCSO44VRFC0KOVYraZEc2GTu\nXEpLl/LiBRdkPRURqYGBr78rdoHM/0ra12AjUczEc9za8uBlb2+yMYbbCMNtggG+JhyVm2d0Gppn\nWGLC/cdVbp5hacIRElsveYJ1cbE8Jh1Z4sbWYiaWOLH7ivV9mLac333NWIqrGM7V6cmPcIuF9bgx\n6sMOY8w++/D8Zz5DtGTJa3fRvau+WHYIX1EU91xp1TdLs5hJSKpDaIONRFGaNIuZVCtOXT8LyxGL\na1/MZEoURXevd3v+sBpF+U2y7JT/i7RIA4s6Oxk/ezZr77uPJV//etbTERG/V+I4njHY/4yi6DjP\n5kWDrZStq2jQRVokU2PPPpviFlvwzEc+An19lQ8QESBfP8EKWFVPi6JoGv2r6UmDZX6DYtIimSls\ntRVjzz6b1d/7Hqv/9JFtlagAACAASURBVKespyMiVRJF0aHAjIF/E8fxgjiOFwz87wlDHauVtEhG\nxs+eTVwqseyss7KeikhdqbeyoAMX5AWe7fOBIVfh+b9I+7pgGYqZ0BuV3exem8wMKo1xkrIKYUlZ\nIR2uLIlbvuN8Y7sFT0YZCqVYCq60FTxJcu3lJTU6fAlXjVLMxNJNy9AtyS1w0toFbfvuy6hDDmHF\nuefS8uyzgxY4Ger2YNvS4Euu8hVTsRznsiRz+R6PkGImpn18z1lIl7RqFjMJfae2zFFyTU+ZSK0V\ni4ydM4fexx9n9bXXZj0bkboTE1Hqq5+V9EjoIi1SY50nnEBxl11YetBB0OVbwouI9NNFWqSWpk5l\nzMUX0/XrX9P1k59kPRuR+hRDb29zrKSV3S1SQ20XXEY0ejQrTj0166mISB3I/0ra1wXLVHHMGcbz\nqcvtjNU1OplM5SZh+ZK7kp2qkuO4Xa/8iVuWambJymlu5R1LVbTQ6malovO5ruj5ba+l4pcl4aua\nFcfcsS37+OZsyO56rQrZnm+E//gE3XOvo+2Jh8rO6VYhg/QqjLksSWG+JLG0umCFVhxzH4/gBmzO\na8bXuSyRAJhW4tZg26q1j0X+rwIJcRxR6q3DiQfQSlqkFqIIrpgLr7xM1+WXZD0bEakTzfFRRCRr\nh32E6M17E590DCxfnvVsROpa/0paMWkRScOYMXDhLOJ77oTvfzvr2YhIHcn/Sjq4mIl7TDLo2LXW\nifeOTnavsnXKcmO5yYfVVhTF1z3LLZRSuXuWr+BJosOVoXCKN27e7sTf25MJAa2WQiGW4iGhnbIs\nsWR3H19eg3t+3+vMPc4X3zztPNhoYzjmg1CIvTFQX/csSuU3LXHitOLYlqIkoSzFTCzxZkvBk1G+\nfdziMmkVIQld2KUVf07r3dzyeDTHIjYX8n+RFqlnW20Lnz4dvn8D3HNn1rMRaQwx+rpbRFLwheug\nay1cdk7WMxGROqSVtEi1vPt98J4PwIVnwMsvZj0bkYYRxxG9PVpJi0iotrb+VfTjD8E352U9GxGp\nU/lfScekUszE7YoFyZhGd5ene1V75e5Vlk5VIYlbYOtw5Y7ldrzyjZNW4ZSS5xXUGtodyFJAIqRT\nliVJzVKoxDeOe9y6Yz5xCmy9PXzqvdDSU3Y+b5KY7/Srym9bksIsQ/seVstxteyCZdnHVPDEs9hy\nE/d8+wR1uAp9nftUa5/8v+MbRfT53nwakFbSImmbujEcfz789sfwp19nPRsRqWPN8VFEpJbOuBJa\n2+Cq07OeiUhjigFld4vIsO35VjjoaLjhGnh6UdazEZE6l/+VtKWYiS8mbYhb9zkNNny/u+tttzTG\nKN/mNtPwHWeJCfePVV7gxNI8wzeOW/AktcIpHcnIYHtXefQyssSEfdtCi5m4r2rfOO7rIXSf9V9n\nLS1w/lx46Vn49uWvz9UpShI5twFaPds6LbF1R/JZtTWmCI1lV2JppuFjKVRiilv7ciacbYlmGr7B\nfYu2tGLStYxRp3lcluKoaVbS9fj0iOTT+46BnWfAOUfBmlWV9xcRqUAXaZE0jBkPn74C7v0T/Or7\nWc9GpLHFeH+x04gUkxZJw7EXwfjJcNVJWc9ERBqIVtIiI7X1znDwZ+En8+Hh+7KejUhzMNYbqHf5\nv0j3UbmYiaU7kTe5rPzud3k6ZbV1OAlXBV/CVfk4loQrS+JW/3GVC6W423zjJJPCKhdOcRPifNva\nCp7HrL08cazDmlAT0uEqtJiJ5VyDFSpxjzt1DqxZATd8IbgAS2voG44hmcwdOuvEMcvYlkIl3n2c\nx7bTU6Sm1ZKg6E7IV+zGfV4tRVF8Y2dd8MRlvR9SE/m/SIvk2ds/CG+cCXM+C8sWZz0bkebg+9VP\ng1JMWiRUWweceC0s+jv85KtZz0ZEGpBW0iKhPvQ52HhrOPXdUPL80FlEqqOJVtL5v0inVczEG7cu\nT+HvWetpjOE24SjULiYMtkIpbkMN/zjtzu1kwRN3mz9u7jTY8N3XovMFTXtfYh9T8RBLrM4XK3Tj\ntL64rft6qFSoBMrnN2VzOPQcuPV/4MHfv36873VmeTPx7GON55bx3Fe3MUa9xqTdbYliLxibZ4Q0\nckmzwYYllp3WHCsdM9i2SvJ/5WgY+rpbJMR/XN3/76+fme08RKSh6fOQyHDt8i542xFw44Xw0lNZ\nz0ak+cTYeqc2AK2kRYajpQCfmAsv/Qt+NCvr2YhIg9NKWmQ49v80bLU7zDoEun3JECJSdTGJxjWN\nKv8X6ZjKiWLBxUzc257uVU4yWVt7Mlulk9XlxwQmbrkdryCsUIp7u3+cNc7tsOQ2d5v3vra7j1ny\nwW8NTfhy777vebUUq3DH9s3HfX1MnQxHfRH+8Vu47wf9x1gSFC1ZWYY3HEsSlps4BbDGeYx8Hbcs\n3xxWM3Es0eHKk8zkdq/yFSpxj4t8z2tItzVfcpV7XJpdsCzJZZWO8W2rZjctqQo9HSJWh30ROsfB\nd07JeiYi0iQ/wVJMWsRi8z3hPZ+GW/4Tnnkg69mISJPQSlrE4vC5sHIx/O9FWc9ERFTMJEd8T0al\n4ia+baa4dTKCVnKacJRKniYYTpOJ0JiwJQbs28cdyzK2pXCKG0fv39Y95O3+czkx8o7kPq1dhgIn\nllihpQmHJf491OvjjUfAdu+AGz4FvUvL5+Ue54stWxJcAt9wQmK5PZ5z9fZW3sdtcJEmd46+2Hpi\nH899TcSg02pw4XsNuftYzhV6/qybcDj7xPm/cjQMPdQiQ2kbBR+cDU/9Ff74raxnIyKglbSIDNj/\nHJi4GdxwBMSe1b+ISBXpIi0ymCnTYN8z4c7/gkW3ZT0bEVmniVbSyu4WGcyHroG+Hvjx2VnPRESa\nVP5X0n2EFTNJKbnM7YzV7emU1TW6fNuolBK3INn1yu141T/W8DtcWRK+/F2wKiepJfbxZPjExeRX\nx1FIhytfcpm7zff6qLTPNvvDHh+En58NXc/7C5f4jqtiMRMv5zGLPIl0rc75fQlXvW5SVhVXKW4C\nmI83KcySTGUpMOLuYyl4YkkK840TWmCkWslloYlkebtSaCUt0sRaWuGAOfDyo3Dr9VnPRkSaWN4+\nH4lk782fhSk7wjc/AKXkNw4ikgNaSYs0odEbwrsugkd/AQ/+POvZiEiTy/9K2tdgI61iJqZxyoNT\nvb2rE7u4TTDcGDEkY8CWZhqQbKjhK5SSbLARVvDE0mDDjaX74uiJoigFz+PRnmzp0OHu5itUEhK3\n9o3jHrcuJrzf5VDsgD+cZottu7HkNDvzWGKX7mvWsE8itkuyKIobx/aJDfv4zuVl2c/yeFhi0u5z\nb8lrsMStLefy7WeJW4cWZal0TOA4nl5EUiX5v0iL1MpGb4Ldj4U7ZsGrj2Q9GxEZTIytdVsD0Nfd\nIgBEMHMerHwebr8068mIiABaSYv02/Vo2OQt8POjoXtF1rMRkaHEpBtayjGtpEVax8I7r4Rnb4cH\n/jvr2YiIvCb/K2nfj9YrFTfBcIx5nKjsZvfaZJaJ2+Wpu71y4pal45Vvm68Iils8xV/wxJLwNfyi\nKLZENs8+HcmOY+1d5UEmX2GOoEIlvsSg9Y978/kwegP42YHQHg/vXJZiJpWOSZNvbPfl4FuBuMcZ\nEoMi3+MaypK8lFYREEvCV0iHK9/j4TvOTSYLvR+W5DJLglda+9SSipmINKjiGNhwX2gZ+JAyfgfY\n9VR44Fvw4t3Zzk1ExJH/lbTISBXHwKYHwuaHwUb7wqonYXor3HYi7HYm9K6G287NepYiYtVEK2ld\npKVxtbTDjlfDJkfDy3+Ep2+GO46FnqUw+d9hn+/BqA3h9lNhzctZz1ZEJCH/F+nQBhshcWvDOH2e\nBhsl55f9ve3pxIQBOikvnuIreGJpsGGKExuKooQ0CvHeV0OBk0RxE0jG8wYrVNK5Dex6M6x+FH6z\nNfQs6f9/hYF/Xvk19K3q7xH95A22oii+WG4tM0wtcVHfa9i9HyENR3ws9z00lplWnLaWTTisxUzc\nOfpe59VqsGF5DRmes1LWV446XElHUTQ9juN71r8NTAOI43jBYMcpJi2NZ+rB8Mbb4LlvwANHvH6B\nXt9Op8GYabDiMZiwc+3nKCJNI4qimcDNzuZzBi7O0wYu2F5Zfx4SSU/UCltcDeP/Hf72flgxSCJY\nx6aw2xfgqR9C1ysw6Q3w5F9qO1cRGZk6WknHcbwwiqJF625HUXQocNfA/5s11LFaSUtjaNsSdv5T\n/7/vmj74BRpg16sgKsA9Z8CSe2HSoB9iRUQApkRRdPd6/xw3wvHeBEyOomh6FEVnDbWjLtJS/zp3\nhl1uh8U3waMfgt6lg+87+W2w+Ufgn1fDyidgzQuw7SdrN1cRGbl1Mela/QOvxHE8Y71/5qdwLxav\ni1EPrKy96uPr7koJXqFdsCzJZYlxPIlbToGTtg5PUlahfFvJ2wUrmbFh64LVNuRtSCZ4uQVQfOf3\nFUVxt/kS2dxt/n0899UpcOIWNwFPgZNxO8G2t8CzZ8DS7/cn4bjP9WsPYQvsMQ/WPA2PXtm/PVrp\n7OM7bkDo12uhx7kPm6UrmGUfSzcvn2p+vZhWFyxLEpSbzGXZx5LEaEkA8x2XVsGVGu5TKvrWd32e\nbc1hkJX1ojiOFw5yyGJg3dffS+lfWXuTx+rjIi3i07oDbHoLPHdW/wW6ks0/CePfAPceAaWBrPn2\nDeCp71V3niKSrpxldwesrBcA61bPExiIT/vo626pT63bw0YL4flzYcl3DftPhB0ug8V/gOf/5/Xt\no7bsL24iIlIlA19nz1j3tXYcx4uApQO3Jw/1EyytpKX+FLeDjX4LS86HJd+xHbPdxf0X6n+eXL59\n1Baw9L705ygiMmDgIrzA2bZu9T3oBRrq4SIdWswkrbh1YpwosUuPU+Ck21fwZHRYg42QIihuARRI\nxoXdAij943QPebt/jpaiKMOPkQO0FcoDql3tyZh0x6htYMpvYcWF0PP//MUi3Ods8q6wxfHw7Feh\n5/7+Y9btM3ZLePnH/njiaM+2Smodt3Vfw759LEV73Ji0peBJNVkKlfhY4s0hTTgsxUws+/jmZIll\nW3ImQu+rYZ84EZP2nayGMekYSL49NCR93S11I4qmweTfwYpLYPW37AduPwdKy2DRBcn/17klrHkq\nvUmKiKQo/ytpESCKtqKt43ew/DJY/Q37gRMPgYnvgYePh95Xk/+/cwtYrZi0SF2JqW1J3gxlcpFe\nL119mziOz85iDlJPJtDa8Ut6e2bTunoYSZQtnbDZNbDiPnjWc1zbVOjrgtLK9KYqIpKiml+kB2qY\nLozjeFEURTdHUTRziN+SSdMr0tpxM32lX1Lq/RKtlQ943UZnQfuW8MDH8MbLJr8dlt6Z0jxFpKZy\n9BOsaspiJT1t4J/59P+Ye5q7w8BKu3+13bFF5QQvS1KYpQtW8DhOEZBeXxesyp2qLNssRVDcAij9\n24bf4SqtRDbrfXXnWOwo0RF9ib6oizWcAx2tiQInieIm0P+8tm4JG50NS2+ENX/0FybZcF9Y+tv+\nhB9LhyvfPpakLAtLQo/vtWcpVJJW4piF5TjLu05IcRPfcZbHNTRxy1LMxHJcWsVMLIlrlkQ2z+Pq\neUuTGqn5Rdr50fd04KZB9pkPEI2bEddoapIzbdFnKURvZVXfuxh25uhGs4EYXjxz8H0m7wv3HzWS\nKYpIFnJWzKSaMkscG2jNdc/6/TVF1mlnf9qiz7Gq753AMGPGo98D4w+FF78APc8McoJNoHVqf7xa\nRCSnqnKRNtYxnamkMfEpsjPj+E/W9B1MzHB/HlWEjedA9yJ4Zfbgu03aF5b8H/0fyUWkrmglPTKV\n6phGUXTcuh6aShyT9bUwlYncyAo+Twt3DH+AccdDx67w5Ach9gWsB0zeFxbrZSci+ZZVdvdVURSd\nDUwCDhvyAN8nppCksNS6YPn2Kc857l7rSdxyOmN1t3uqkhk6U1kSvrzVvBIJX2EVx9xt/opjw09S\nA+hiLBvxfVZwE8v4KaMKnjk6Vcg61t+lZQpMvARW/Qa616si5kuWmbwvPHPp6/8vNOEqrUQpy3G+\n5B034Ss0ucwVuk9oVbK0ksnS6oKVVlJWaKUyS+JaSFewwH1Kzrbegu9Bq2EJsCaqOJZF4thCYGKt\nzyv5N4Xr6eVplnF12ABjL4NoNLx8ytD7tW8PcR+seSzsPCIiNaKKY5ILU/ksRbbiRT4UNkDrdBj1\nSVh1PfQ8NPS+4wZ+eiUi9UkVx0RqZxzvZQKH8yIHEHu/rzUYPxf6XoYVFxtOuC+88MOw84iI1FD+\nL9IxYV2w0oo3BxQ86fN1wXILnrT7Cp5UL5bsxoktRVEsMfKRzrmdLdmIS3iM4yiyjPWDb+2++9FR\nHv9v7+qBtqOI2t5GvPJYaF9O5D5nZZ+4W2Hcu+GJz5bHAqtZqMQirQ5Xlrh1WjHpaq5kLMUzLPHV\ntLpghXbKspzfUgTFd35LbD0gbu12vAIoFct7Mbld9TLRJNnd6oIlmYloZ2uu43m+xBoeDBxlDIy6\nmrj3Luj+f4bdD4CuB6DnhcDziYjUTg4+Dkmz2oxzWMsiXkkWnbPrPJeoZRPiFQdj+s3z2CNh+ffD\nzyciUkO6SEsmJvJ+xvJmHqrwC7yhtLAtdJxO3PX/oGT4TXU0Csa8D16qkP0tIvmmYiYi1dPJlmzG\nOTzGJ+hjVfA47S1XA12w+hzbAWMOhDW3Q+mV4HOKiNRS/i/Svk9MId2rUkoKsxVOST6sXU6Bk7YO\nT1JUwVIYJLlPSMJXeIcrdx/ffMrv//rnaqGdnbiCp/kSy3gC1vt/lqIshUJp4Lz708r76ek+k1Jb\neXy5Y7DXy4QjYfX3+5NyRg+yTzWEFJSAZNKPL1ErJLmskRPHLPuEdMGyFEWxjOPbz5IUZhnbl7gW\nkFzm63hVKlZ+j6mpJipmosQxqakdOY2VPM5L/GAEo7Qymqvo5VFKPXNsh7RMgM59YNWPRnBeEZHa\nyv9KWhrGxryX8ezCnRzHqBGM08kJFNmWZRxMu/Xj9KhDYM0tEK8YwZlFJBeaqJiJVtJSE6PZiu05\nkfu5wPsbbasWNmQUZ9PFL+nmFvuBY46ElcrqFpH6kv+VdB+V48KWOHFovDmocEqU2KXHKXDS7Sl4\n0jU6uW2UId5siyU7MfHUmmf49nHnU2Q3LuWfzOdVnh2YYVhRlk4uJaKNJZxPibZEcROAQm/56rq1\nbyNonw6lXwzeUMONUfvUspkG2GLSln1CYtKWVUqts2stj21I04nQgiOWmHBoLNtSKMXS4MMwx9jZ\n5jbTgGRDjVzEpJsku1sraam63TiFpTzIM/x6RON0MJ1RfJiVfIUST9gP7Dwc1v4Y/6cwEZH8yv9K\nWura1uzPOLbjT5w4wpEipvJFSjzPCq4d3qGdR8KKC0d4fhHJlSZZSesiLVUzji3ZjaO5nVPpo2uE\nYx1OB7vzKp8mHs5vq6OtobA1dKnrlYjUH12kpSpaKPIWzuB+bmAlT41wrHFM5vOs4S7WsGB4x7Yd\nBWsX0DSpoCLNoIl+J53/i7SvC5YlESbDLlj+fcqzOnp7Vyd28XWW6apawlcyucxSTCRZzMSfpPYG\nPspyXuJh/sRYz/2ynH/d7Q05nQKTeJJjiZx9CoXkj7kKHet+ZtVCZ9sniZcfkkiYidxrtiXhKi2W\ngiO+/Xz7KHGsnCVxzDKupeBISDcr63Fpdcpy9/Hke7nFS3zJmN3OQL73BqmO/F+kpe5swE5szTv5\nOZ8b8VjtbMtkjmYJN7GWB+gcxrGFlgOI45eISveMeB4ikiP6nbRImFY62JvPcgdfo4vlIx5vI86n\nj9W8NNxkMaDY8hl6S18b8RxERLKilbSk6m0cxfP8nWf564jHGst+jOHtPM/FlHh1WMdGbEGhZS+6\nej+sL+ZEGk0T/U46/xdp35ORu3iz5VzlBU661yaDTN2+phvt/jjtUNtsseSwYiJDFTOZxh5szE78\nhLPpHaJxBkDB8yCtP6cW2tmQL7CaR3mGH8JAIdGC8x2Xexug0D6KURzPGm5idXtEwfN8JJpwpMUS\nu/Tt43tdWWLSln0aJSbtshSAsRwXWvDE0rzCUoSkhk043MIlAN0d5V+ouoVLwNfEJ/+XjkahR1pS\nMYqxvIeP8HO+Rm8KRUM24yja2YyH+CTDDz610sHRLOMDI56HiEiWdJGWVOzLx/gnt/E8jzN2hGO1\nswFb8HFe5Tes4I5hH9/G+ynxCCUeHuFMRCSXmujrbiWOyYjtwtsYw0T+wk9TGW8aJwPwNNcEHd/J\nJ1nLN1OZi4hIlrSSlhEZxxT25kP8gGvpS+E3EeN5AxuwH//i63Tz3LCPL7ItBXagK6UPDCKSQypm\nkiP10AUr4Fx9vi5Ynm1t7cMvZjLchK/Xt1VOUls/KS0iYj+O5S/cwnO8AgP/z5Kk5kv46qHENnyO\nNbzA49xEuyEBzh1nHMeymu/RTfTafFaP9mXvlD8hvj2Cul5ZkpmsCU/uQ5RW4piP5Vwh41ZTaKGS\nLDtlQVDCV1qdsro845SK5Xfu/7d3/+FRVXfix98nk2RCSCAEBBEQCCiiUiBgFasF3aT4+8eK2mqt\niz/o7rf7/XbrturTtcWu21Wxj91ud7cra9en1UdlwSqt7aLQrmXVlhaCv4UqEUFB5VfQEBLI5Hz/\nyASSc08yJzd35t4783k9j4/McO6Zw2QyZ+79fObzMQuXALQ7dOMT2RH9TVpE1lmcTQcp1vNcIPON\n4WIqmczLfMtXrW9FGUP4c3ZxbiDrEUJEmBQzEaJ3xzKaz3A2v+IROq89DUwplUzmZvaygY98bvoV\nXEwrDaTYPuD1CCFEFMiZtOi3YopZwJWs4ld8zL5A5pzKdSQoZzP/7HuOKq5jLz8IZD1CiAgroOzu\n6G/SuWywkcvYdquliL1DgZNU0qXQgL8xZtzJbO4BnfHuOur5iL38gTeotIxxiX93L2ZSxQQmcj7v\nspI9vEdXLNkWtzbnKk6PKeMUihjJPl5gUIa4NUBxmXmfNwvFE+LzWzzDTwwU3F7n5tPvt8GGn5i0\nTS7fOP3GpE0u8eagmnDYjrPFm13i1g5jDhv3pYq9F0/bEj0H2d4bpMFGeKK/SYtIOZ4xzGQa3+dH\ngc05i5s4zAH+xE98zzGMa9jH43RmGgoh8loBnUlLTFo4S1DEFVzCL3iGFg4GMufxfIaRnMrLPMJh\nmn2uq5qhnM8+lgWyJiGEiAo5kxbO5vNp9rKPV3g9kPkSJJnBQvayhUbWUOFznmqu5WNWkWJPIOsS\nQkRcAX1PWs6khZPRDGcuM3iKXwY258lcwWBGsIEH0T4vUyvKqOaL7ObBwNYlhBBREf0z6aC6YLkk\nhWUzAc2hKEpHuyXhy7ivLenS4co2pv9FUbqSshSKa5jPU6xjF23QbX63jlveMUMZzlQuo5Hn2UEj\nkOw1Kay7lnQ3rC6juIBmXuET3qcrc8aWKOZhPtVlLZZBPT+qWwueuDCXE2TimEvCl595bKJW4CTs\nxDE/nbJs41w6ZTmMMZPEwNvhyvb+4dJFzxwTehcsjXxPWogu5zKNdlKsDegyN8BpXEcHHWzgkQHM\nUsRI/oIPeSiwdQkhRJRE/0xahGo4lVzEadzDEwGULOk0llM4nk+zgcdoGcD3rEfwWVI0cYANAa1M\nCBEbkt0tBFzHOTzLRj6kKZD5ikhwFtfyMR/wxgDj22O5lg/5z0DWJYQQURT9M+mgGmyEHG92K4ri\nDSq1tRrNM8osjTESLs0zzDGZ407zGE8lg/gVrx4pcGAWOHFp5nGwW/ColnMZxnE8w/20dmuCAd54\nsy223HXfUKZRQjU7eBEY5BmXiWduazzRjFN700mTxm+QcmmU4RIDhezFpF0Kntj4OXPJ5tmO33ev\noJqi+GnCAd74skss26FQiRl/Bm8M2qV5htsYKWaSK9HfpEUohlLG1czhfn5JKqACIeVUMocLeYfX\n2cbGAc01gS/wLsuQ4iVCFCApZiIK3Q3M5nk28y67A5vzLC6lmFKeY/mA5ilnHFWcyg7+O6CVCSFE\nNMmZtPCYzVhqqObbPBvYnKMYz6mcyXpWs4+PqBzAXOO5mu085audpRAiD8SwmIlSqlZr3dDt9gKg\nCajRWi/t7Tg5kxY9lFPCTZzGv7OOw4F9EVFxDldygP38foBnv6UMYxTnsJ2fBbQ2IYTILqVUHRy9\nhKiUqgUatdZrgMb0bat4nElnSrqKRVKYyxjluetwq5H40epN2Ggb7FLMxBxj73B1I7Ws5wMaaLJ2\nuDILjNgSSMxiCNOZyXHU8DSP8gkaSFoTzsxkLtvco7mS93mOZlqAUnuHK+O+hN/glZHQ0z7Ye+ae\nKjYS+xLeGHmxsURrcpmtWIX5GvHb4colcSzTMX7H2AT12c+lA5mNS+KYy2OZx7kULrEdZ+twZYzx\nX6gkc/cqP2PMRLKci1kxE631GqVUo3H3vUA9nWfSa3o7Vs6kxRGnMoLTGM1DvBLYnEmSnMPFvM9W\nXmX9gOZKkGQ8F/EOKwJanRBC5F76snejUmofsLevsfE4kxZZV0oRf81s/p2NtASYNjmXc6hgCCt4\nEAZYDmUCdezlNQ7wfjCLE0LEU+6zu0copbqfZSztK46ciVKqis549N3AfyilGrTW5pk2IJu0SLue\nybxDE+vYEdicIxjBHM7kZdaxk+0DnK2IE7mMl/huIGsTQoh+2K21nt3bXyqlFlnubuzjMvYi4G6t\ndVP6MvgCYIltYDw36cBiwBnmzeZjuYwBT4ETs7gJeAucmMVNwFbM5OjtiQzmAsbyV/ymRxzYFhM2\nY8B9Neo4j4tpp53VPOuZyz53u3H7aHGT8ZxBC/vZyVYwip5kS8qIu5m3AVJGNZNUsTdunWg3njOH\nuDVYYtd+Y9KmoOYJ8kzGJb4YVBjUT0w6qIIn4IlBa8uYNmNMqthfoRIzvmwb49I8w2WenIvQ96QH\nclattV7RyyYPHWFNkQAAGW1JREFUxHWTFoG6hRN5iK3sC/ArTSdxAlM4gad5hgM0D3A2xae4nI0s\nC2RtQgiRS+mvW81WSi3QWq/QWi9RSt2aPouu7muTl026wNUzikqKeZL3Azs/TZDgYs7jI3bxIn8Y\n8GfuccxC08F7NOToHFoIEWkx+5601noF9Mx41VpbL2+bJLu7gA0mwV8zme+xmVRgPa7gbM5gBNX8\nnFWBlBSdzgJe5okAViaEEPEiZ9IF7CZq+B17eI2PA5tzKBWcy2d5nU28hTVZsV/GMQuA7QP8+pYQ\nIo/E7HvSAxGTTdo4yzOLfsQxKcw5caznv9UsbgLeAidmcRPwFjM5hWHUMYqraDiSFGIWOPFTcORS\n5lJEEU/yP0f+zlZwxGXuYlJM40oaWNnrXC7FTFzYkmW8iWPeMWZRh1TCm92USPRcj0tyGUAi2WGM\n8QzxFkoJuwtWHN44XRLQAuqUZUsKazeOs3WvShX3HNSW8AaNzNenSxESe/GhzGNc5hHZEZNNWgRJ\nAbcziR/xLk0BpkhO5ljOYArP8gJ7Aug/fTwzKCLBVhoyDxZCFA7pgiXy2WUMoxjFSj4MbE6F4gt8\nlr0082t+F8ictVxOA08y0CIoQggRV7JJF5ghJPg6o7mHLYF2Yj6LqYxnJMt5gUMBpF1O4FQSlLCV\nDQGsTggh4ikGl7s7gIPGfcYXceIYb3ZZs+0+M6AFtBkFT8ziJnC0wMlXGcUqmnmZw5hVFbwFTzIX\nHCklSTmlXM4cNrGT59lGufHzscWNWxjU59yncTHreNoTZ/PGpIO55mVrGGDG/FzG2AqemGt2iVsD\nJJI9/23FKcsYI5adaPd+9DJj2bbCKSZrbNsU1xi1j5i0S2w5ZRljK0JixpvbLa8HMwZse125FBjx\nxq0zFzPxOyanCuhydww2aRGUaZQxn0rqAsi67u7PmUUFSR7mxUDmm8hUSkmyRWLRQogCJ5t0gSgC\n7mIUS9jFx3RQGdC8YxlGHSfzGzaxre9mLs7O5HxeZBUSixZCWMWsmMlASEy6QHyeKg6jeYL9gc77\nReZwkEM8EdD3mCdwEmUMYjMbA5lPCCHiTM6kC0A1RdzCCK5je6DnpnMYyymM4Se8QHNAdb+7zqK1\nnEULIfoSxXyHLIjBJu2QOBZ2UphLByE/iWzWxy/xDEmZXbCM4iZ3lFbwZOIAL6HhSOGSzAVPzOIm\ncDQJKkmChcziXfayii10dDvWm1yWuZsWwLFMpIwKNrIJneUq3bYksEysXbCMXyGXxDFbgk+p5UOO\nWZTFmlxm3Gcmm4E94cwzj0MCmveYjEOsXBLXgmLJs3RiJoG5JIDZuBQhsb1mzNenW1JY5iIk2Sx4\nIrIjBpu0GIhPFxVTnyhlHh8EOu8VnMBIyvkOa+kI6Kx3LnWs5TdyFi2EyKxA3iYkJp3HEsD9pRXc\ncegAnwT4ih7JIK5kCmt5jzcCKogyiQlUUMFrvBzIfEIIkQ9kk85jNxaX0YRmeSq4PtEANzINjebH\nvBrYnHXMTZ9FB1liRQgh4k0ud+epYShuLx3MBQcHXkO7u+kcw9mM5WHeYBcHA/kqVw3jGUIlr/JS\nALMJIUT+iMEmbSstY3xBrt2bTJW1KmC2MVntgpV5TIfZBas4xTfKk6xoa+flg8VAsacKWVcFsh73\neSqO9bydQLGI6eykhUfZyiFKnZLCDmboVPU56niGF2ml588xmx2u3I7rmZhjW483wceWkGc+j94X\nkb26W+aOX+Zc1ufMTC6zzWMmpVkKSrlUd3NJUrOxdQELgktyl41ZBczlNWRPLAwmKcxecSyYqmSx\nrDhWQGKwSYv+mlJUxOeTJXyq6UCg817OGCYyhG+znkMBXZY+hckkKWUDr1MWyIxCCJE/JCadh+4Z\nnOS+g4fYo4NLFhtKCTcxkQ3s4oWAMsUViguZxy/5rWR0CyGEhZxJ55n6MsWJCcVVn5jfLR+YRdRQ\nToJ/5fXA5pzFKRyklTd4O7A5hRAin8Rgk+4AWoz7zA5KDjHpOHbKcj6u89+fAO4/Fm77pJVWo4qD\nWeCkbXDmYiaJdEGRKZRzKcfxKB/wltE9y4xbw9G4cXGqjdptT3MoWcnm487uMSYJzGcu/8kztGS5\ncIkLl0Iltpisn7i1La5ujzdn7vjlKXjiOLd3TObYdqbHTh/Y73lsx9mHZJ7L9nP0w6UDmstj22LZ\nQcWb/XS4cnmseBQzKZzi3THYpIWrm6vgo3ZY2RZsEs7tTKCJdh7gPbcDtGbyrnV8ZsvjVLbt4c3j\n5nqGfJZp7GA3W9gR6FqFECKfyCadJ4YWweLhcN57eC80DMD5DGcmQ7iTLXxCKuNXrkZ+8g71b/8H\nY/ZvZlfFeJ6d+lc0VR3fY0wZJZzHbH7AU8EtVAgh8pBs0nni74bDL5rh5TYoCmiTHkQRX2M8r9PM\nSnb1PfbQx5y19TGm71hDW3E5/3PiQt4YPQ+tihhk1F4/n2m8yTbeZ3cwCxVCFBjbV3Pzk2zSeWBS\nCSwcAqduDXbemxjDSEr5Bn/qNfdadaSYseMZztq6jGR7CxvHnEfDhEtoKxlsHT+EMuo5hbt5LNjF\nCiFEHorBJq3xdsEyb1suwrarnrejlhQWYOLYkmr43i74MP21aLO4CXQWOOnOLG4CPQucjKeE6xjN\nSvbwRw7R1T2re8LIpH0NXPr2PzPqwFYaq2awavKX+ahiApV80mPe7gk/FzKL53mHHbQemdM2zna7\n877Mn579JPm4JNQkLZ2qXJLLzCQbs7hJ53GZ/60uCWd+ksR6mzuIx3J9/DD5LVTiMsb2WnTpnOaS\nFOb2Gs5czMRPApqtm1ZuSeKYiIl5VTBzEFyzLdh572Akh9Hcb0nsGnZwJxdu+RHTdq9lX9kolp1y\nB2+OOBOUssx01DEM5mxq+FtWBrtYIYTIU7JJx1gRcH8N3LYT2gKsBTKPwdRTyfd4n13dPq2WpA5S\nv+0h5m5/DI3i2Qk38Idxl9Bu6ZlrcyUzeIZN7Kc1yNw2IUTBkZi0iIHrj4UDHbB8f3BzlgDfYiSN\nHOJhPuq8U2tm7VrN5Vv+ieq2D3lp5Ln8quYv2V820pMU1ptxVDGdMfwNTwa3WCGEyHMx2KQdGmxY\nNwqjQEbU4s0DjElXJOAfxsOlG/EWd231Fnc5bMSkzeIm0FngZCFDmUSSa9lJM6WMbd7ENW/dw5T9\n69k2eAqPTF1MY9XMI8fY4pItxnlygnauYhZP8CZNKKA0cs0zbLFDcz1+49bmv9UW8wuqUInf2Ham\neXu7L9NjufLzsw+KWw5D9hps+C94YhYqyTyPLZZsxqlthUpcmoLklsSkc0IpdavWekmYa4ir2yfC\n6j2w/mOgKpg5R5LgbxnGag7wh8M7+OI79zF3x3IOFA/hpyd8i7XHLaBCmdXfMjuJEUxgGPfxQjAL\nFUKIAhHaJq2UqgPqAdmk+2l8GfzlOPjUi8HO+02qKdWK1R/+mO+//W0GtTfzmzGf5+cTvsKBkqG+\n5/0S03mcVzkcUOcsIUShkzNpEWH3nAg/2AY7vFdWfftScZKrqWTLnqe4YNNXea1qDv81+Vberzhh\nQPOezkgqSPIcW4NZqBBCFJBQNmmlVK3Weo1S6rYwHj/OTh8Kn6mCG14Lbs55+iA/LB2O1prxw+bz\nwPQVPFw1g3JlC5K7U8BNnMQjvESHtKIUQgRKsruzqbqvv1RKLQIWdd46lszFTGyJY8aXfFot3+GN\nYeLYvZNh8SY42NL7GOs8xT0TPdpajxbUOKusHIVCKYUuKqNyaB2HOpopTngvT5tJJbZEoa4Eo/mM\npoUOfss+PIl8OeTSncg2xkyU8ptc5qeblf243CWFBZUk5jchzG+hFBd+OmX5TS5zSSYLKrnMlhRm\nHmdLCvNXzCTsxLHCkZVNOr3JmhrTZ8+1Wus1fR2vtV4KLO2ca6qcgqXNPw5GJuGn24Od9+l2+L9A\nidYcRvO89lbE6q8yiriZydzJKwNfoBBC9CAx6QFJb7K9qVFK1dB5Nl2d3rQbsrGOfKKAe2bBN9+E\nVMAfW9a1p5j/cQufq2zneX2I9Zaylf31BSbwKk28xv4IdIsWQoh4yvnlbq31Cjhyth3Ql4fy39UT\noS0FT+3Mzvzr2lO8oZsDmesYklzBOG5kXSDzCSFET1JxLOu6X9LuWwf+YtLmfZbzOT9xYtvrwrwv\n4Jh0SRHcNR1uXjuAuYt7xuQPW4qZtDQbRUiG+otd3szx/IydbKeD3gqX5DLm6BbzyxxLdhnjUqgk\n6dxgI3Ms2xS1Jhx+x0RN2A02/M5jxpJt85iFSVyK9thi2yI7zFpVIoJuOgne/hie8/a6iJxTGcxs\nqvgpAQfOhRCiAMn3pCOuvBjuqIWLVoW9Eje3Mo4H2EpLDM+WhBBxUTiJY3ImHXFfnQZrd8LG3WGv\nJLP5VJOkiF/yQdhLEUKIvCBn0hFWXQa3TIMzngp7JZklUXyNsdzBO1L8UwiRZZI4FiFZTBwzf8ZB\nJYWVBTPP7WfD8rdgy0f9fHxbkrb5k261FCwwO2WVWRKckj3/IYn083o9I3iVNtaSojyHX7qyJ9S4\nJN1k7upTanS0siXLlBpJYC4JV7bH8luoxK3gSf8TxXKdJOa3e1auBNkFy5TNpDCXYiYuHa5cOm6J\n7JBnOqLGDoEbZsC0R8NeSWbHUMwNDOcKGsNeihCiIEhMWoRs8Vx4YAPsPBD2SjK7hZE8QRPbCuSX\nRgghckXOpCNoynC4dAqc8EM8Jcij5mTKOIcK6ng77KUIIQqGxKQjpANoMe6rNG7bzuDMH6BlTHtJ\nz9u5bJ5he+bTY747D+5bC/v301kPNMC5j972NhzpKO4Zd2pp9saWE0bc+puJYdzPPvZ2e1AzvpjN\nYiZ+m2eYY2wxUfM425rNIg9JvP1D/TTKAG9c2G8MOFtNOFyPy7QeV36bdfjh0lDD5Npgw2WMn2Im\nLvFm2xr9FDyRYia5E4NNurCcNhZOHwdfXBb2SjK7kDKGkeAx9oe9FCFEQZGYtAjJPefBd34NrRG/\nklMK3Fk0lL9nt5QtEUKILJEz6QipPwHGDIGHNoS9ksxuVhX8icP8rycUIYQQ2SYxaZFjSsHd8+Hv\nnoVUxKuBDKeI/6cquLBjNz5Cd0IIIRzFZJM2PzG5FDMxz/BsadIhJo4ZBU8unQlKwxPrA5jbJXHM\nWvCk5/NxuNh7IbuleRB3l5WzjMO80lpCeYWlcInPjdtPko1Lso5LdyJbUpKZLGMWN7EdZ0uoyWbi\nmEunLD+JYkElgLkmewWVSOhnHpfXnd/jXBLQgnoNB5UUZu+CFbViJoUTkw77mRZp374I7nwm7FVk\nNqMowaUlpcxslmQxIYTINtmkI+CS6Z3///lr4a4jEwX8YNBg7mw9yD6tw16OEKKgSUxa5Mjii+Hv\nnw57FZktLE+ggZ8c9l72FUIIYaeUWpT+4ySt9W3p+xYATUCt1npJb8fGYJMOqsGGbYwRpzaLm0D2\nYtLp2xfNhISClb8HBgc0t0tM2mnM0VhVdRH84yi4YH8rbe1H7y+2xK0Tg/sf33Tlp3mGPQ7XM8Zm\nNsoA77rNuBxA0tNgI5hGGbb7/MaJs1UUxSbXjTnC5LdQiZ84td+CJy7xZvM+s0CPfUzYxUziE5NW\nStUBa7TWjUqp5enbewG01muUUjVKqVqtdYPtePmedMgWXwZ3PQVRv3r8j9XwXwfgpfaIp54LIUTw\nRiil1nf7b1HmQ46oAerSf25M376azrPorvvqLMcBsTiTzl/nT4dkCfzMzOiOmNOScMlgmLoda8dP\nIYTIc7u11rP9HKi1XtrtZi2wDJhF+mw6bXhvx8smHaLFl0f/LLoI+LcRcNse2N8hl16EEFEQn8vd\nXZRStUCD1rpBKW/vhN7IJh2S+dOgIgkr/hj2Svq2aAi0aHjY9r1qIYQQ9HL5u1Frvabb7bqupDE6\nL3VXp/9cBezpbe4YbNJBJY7ZyleaBU5ylzi2+DK4azno7st06l5lGWNuoH4Tx4x5jilWfOd4+LPG\no8ebnbLaLIljHraEOAduBRz8dcEyE5NsSTcuCVdmYk52E8eCKXhi4/JYmR7bZd6Bjut5TO6+guO3\neIdLopaNS6ESl4InfpLC7MVMMieg5Va0yoIal7Q9lFKLujK404ljy4Cuy+c1wJrejpWrlyGonwVV\n5bD892GvpG/3joaH98Frtg8GQgghMkpvyvcqpbYopfYBdGVyp/+uqbfMbojFmXT+WXwd3PUEdEQ4\nUfrMSqivgKmbw16JEEKY4hOTTl/yHma5v8+z7y5yJp1jfzYThlfCshfDXknvEsC/TYKv74TmCH+Q\nEEKIfBeDM+lsFjMx77M04TALnAwwJr34WviHn0CHLUTuNyZtHuc3bp2+7yvjYHcbLPvANo/RhMMy\nxIklTu2veUbmMbamF2bxklwWKnGNG7vM7TZPMPHmoIqQBNW8w4Vrgw+TS8ERk2vc2l8jGX8NNoKK\nN7s06sitaMWksykGm3T+OKcWRlXD478GKsNejd2xpXDHBDh7A3KdRQghQiabdA4tXth5Fp2KcMXD\n+ybDgztgcwtQEfZqhBDCJj4x6YGSTTpH5s6AMSPg0V4T7cM3twrOroKTI551LoQQhUI26RxZvBC+\n+3B0z6KLFfzrFPjaW9AiyWJCiEiTmHSEZDNxzMzesiSOmQVOfCSOnT4DJh4Lj6yEI3ksLglgDnNb\nj3MZYySS/c1U2NYCT27PsB7Pfd4CMH4vQqXKjK4+CX+df8wxtkQllzF+CpWYyWa9HZdpHvBXqCTs\nTlkuj2UTZGGUsLgW+MhlFyw/89iSKM15bMmYIjtisEnH39dvhO8/Bu0RfY+ZMBhuOxnO+F3YKxFC\nCBeFE5OW/N0sm3Q8zPs0/Hhl2Cvp3dLTYcmbsMX2tTAhhBChkTPpLPvaQnjgcThgu9oeAdfXQHUp\n3P8mMDTs1QghhAuJSUeI7bJGptvg/QHafqAucWsjTm0WN4Fe48bDh8E1F8HUcy1LtHWVcolJuxzn\nMqYYRg2CJTPgc6sgdRCCq0+QueBJqt1SnMG4z4xRA6SSmWNsZiy5lDbLmFJjjC2W3P9CJeZj28a4\nxITtx4Ubbw4qBp3N+HMu49Z+m0yE2WDDJW5tize7rEdkh1zuzqKvXA9P/Dd8uCvsldj9cA48uBle\n3pt5rBBCiNyTj0NZUlYG/+dLMO+qsFdid+l4+NQwuO63Ya9ECCH6q3ASx2STzpLrF8C6l2DT22Gv\nxGtoKfzLHLjmOWiLaMa5EEII2aSzoqgIbrkZbvxG2CuxW3IW/GIb/K+tgYYQQkRe4SSOKa112Gvo\nk1LqEyBuXY1HALvDXoQPcVy3rDl34rhuWXPuTNFa56R1kFJqFZ3PU67s1lqfl8PHOyIOm/R6rfXs\nsNfRH3FcM8Rz3bLm3InjumXNuRPXdUedZHcLIYQQESWbtBBCCBFRcdikl4a9AB/iuGaI57plzbkT\nx3XLmnMnruuOtMjHpIUQQohCFYczaSGE6DelVG3YaxBioGSTzhGl1K1hr0FEU9Q3E6XUAqVUXZxe\nw0qpOmB52OvoL6XUovR/94a9Flfp10ZdnNYcJ7HapLteBEqpRWGvpT/Sbxj1Ya/DVVx/6eL6BkeE\nN5OuDxBa6zVAU9Q/UHRJr7cx7HX0R/q1sEZrvRSoSd+OtPQar0w/37VxeX3ESaw2aWCRUmoLMfvl\ni5O4/tLF8Q0OYrGZXA00pf/cCMTieY2pGo4+v43p25GmtV6jtf5y+maN1roh1AXlobht0jdrrSel\n39hiQSlVG6f1xviXLnZvcDFRBXTvkzY8rIXkO6310vSHTIBaYH2Y6+mPdCjkyxkHin6L2yZdE7fY\nGFAd9gL8iNsvXZzf4IToLn31qiFGH5DRWi8BvqyUqgp7LfkmVg020i8ElFL1Sqm6KJyh9hIfb9Ra\nr4nbWXR3WuslSqnl6VJ/TZmPiIaovcH19frI+WL8aeLoB80qYE+IaykUdVrr28JehItuOQsNdF7B\nWgQsCXVReSZSm3SGDW8RsFdrvYLON4pIXM7sdvZmU6OUqqHzTa46vWmHvnlk+mAB0fylc9zwIvUG\nl+H1EQfLgK56zDVALD5cKKUWALOVUgvS7xmxoJRa1O1kJBInIhnUAV3vaVXAH0NcS16K1Cad4Q1t\nPUcTbCYBD2R/RQPT9eaQ3lwicxkow/Mc2V+6TBteDN/gIr+ZaK0blFKz04l4TVH4kOki/VxG7vns\nS/o5vlcpdRudH+yvDHlJLpYCV3V9gI7iazjuYlVxrOtsms6Epkic3eWbdEzpqvTNWd2SyCKt21eZ\n9pJ+g4vDJi2EEH2J1SYthBBCFJK4ZXcLIYQQBUM2aSGEECKiZJMWQgghIko2aSGEECKiZJMWQggh\nIko2aSFCopS6Vym1Wim1IV30RgghepBNWogQpCu71Wqt64GbiUFxHiFE7skmLUSWpM+Qq9L/6a6z\n5XS71TpgNRwpwTq7j6mEEAUqUmVBhcgzy+is3raXzlKrC5RSDXTWvx5OhEquCiGiSc6khcieFUB9\n+r/buv15Od4mMZGp7S6EiA7ZpIXIEq11I50b8ex0HfFqOrt0raHzbLoejsSnpc64EMJDLncLkV3r\nOdqP+cif092lGpRSq9N/F4tGJkKI3JIGG0IIIUREyeVuIYQQIqJkkxZCCCEiSjZpIYQQIqJkkxZC\nCCEiSjZpIYQQIqJkkxZCCCEiSjZpIYQQIqL+PwQUw/ua1BLHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11dacf668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from itertools import product\n",
    "\n",
    "def ellipse_line(A, mu, col='b'):\n",
    "    '''\n",
    "    Creates an ellipse from short line segments y = A x + \\mu \n",
    "    where x is on the unit circle.\n",
    "    '''\n",
    "    N = 18\n",
    "    th = np.arange(0, 2*np.pi+np.pi/N, np.pi/N)\n",
    "    X = np.mat(np.vstack((np.cos(th),np.sin(th))))\n",
    "    \n",
    "    Y = A*X\n",
    "    ln = plt.Line2D(mu[0]+Y[0,:],mu[1]+Y[1,:],markeredgecolor='w', linewidth=1, color=col)\n",
    "    return ln\n",
    "\n",
    "\n",
    "left = -5\n",
    "right = 3\n",
    "bottom = -5\n",
    "top = 7\n",
    "step = 0.1\n",
    "W0 = np.arange(left,right, step)\n",
    "W1 = np.arange(bottom,top, step)\n",
    "\n",
    "LLSurf = np.zeros((len(W1),len(W0)))\n",
    "\n",
    "# y^\\top X w - \\mathbf{1}^\\top \\text{logsumexp}(0, X w)\n",
    "vmax = -np.inf\n",
    "vmin = np.inf\n",
    "for i,j in product(range(len(W1)), range(len(W0))):\n",
    "    w = np.matrix([W0[j], W1[i]]).T\n",
    "    p = X*w\n",
    "    ll = y.T*p - np.sum(np.log(1+np.exp(p)))\n",
    "    \n",
    "    vmax = np.max((vmax, ll))\n",
    "    vmin = np.min((vmin, ll))\n",
    "    \n",
    "    LLSurf[i,j] = ll\n",
    "\n",
    "fig = plt.figure(figsize=(10,10))\n",
    "plt.imshow(LLSurf, interpolation='nearest', \n",
    "           vmin=vmin, vmax=vmax,origin='lower',\n",
    "           extent=(left,right,bottom,top),cmap=plt.cm.jet)\n",
    "plt.xlabel('w0')\n",
    "plt.ylabel('w1')\n",
    "plt.colorbar()\n",
    "\n",
    "\n",
    "W0 = np.arange(left+2,right-5, 12*step)\n",
    "W1 = np.arange(bottom+1,top-10, 12*step)\n",
    "\n",
    "for i,j in product(range(len(W1)), range(len(W0))):\n",
    "    w = np.matrix([W0[j], W1[i]]).T\n",
    "    #w = np.mat([-1,1]).T\n",
    "\n",
    "    p = sigmoid(X*w)\n",
    "    dw = 0.2*X.T*(y-p)\n",
    "    #print(p)\n",
    "    S = np.mat(np.diag(np.asarray(np.multiply(p,1-p)).flatten()))\n",
    "    H = X.T*S*X\n",
    "    dw_nwt = 0.08*H.I*X.T*(y-p)\n",
    "\n",
    "    C = np.linalg.cholesky(H.I)\n",
    "#    plt.hold(True)\n",
    "    ln = ellipse_line(C/3., w, 'w')\n",
    "    ax = fig.gca()\n",
    "    ax.add_line(ln)\n",
    "    ln2 = plt.Line2D((float(w[0]), float(w[0]+dw[0])), (float(w[1]), float(w[1]+dw[1])),color='y')\n",
    "    ax.add_line(ln2)\n",
    "    ln3 = plt.Line2D((float(w[0]), float(w[0]+dw_nwt[0])), (float(w[1]), float(w[1]+dw_nwt[1])),color='w')\n",
    "    ax.add_line(ln3)\n",
    "    plt.plot(w[0,0],w[1,0],'.w')\n",
    "\n",
    "    \n",
    "    \n",
    "    #print(C)\n",
    "    #print(S)\n",
    "\n",
    "ax.set_xlim((left,right))\n",
    "ax.set_ylim((bottom,top))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0]\n",
      " [0]\n",
      " [1]\n",
      " [1]\n",
      " [1]]\n",
      "[[-0.5  0.5]\n",
      " [ 2.  -1. ]\n",
      " [-1.  -1. ]\n",
      " [ 1.   1. ]\n",
      " [ 2.   1. ]]\n",
      "[[1]\n",
      " [2]]\n",
      "[[ 0.62245933]\n",
      " [ 0.5       ]\n",
      " [ 0.04742587]\n",
      " [ 0.95257413]\n",
      " [ 0.98201379]]\n",
      "[[ 0.          0.59561717]\n",
      " [ 0.          0.30966921]\n",
      " [ 1.          0.32737446]\n",
      " [ 1.          0.67262554]\n",
      " [ 1.          0.66660954]]\n",
      "[[-0.02719403]\n",
      " [ 0.74727817]]\n"
     ]
    }
   ],
   "source": [
    "print(y)\n",
    "print(X)\n",
    "\n",
    "#w = np.random.randn(3,1)\n",
    "\n",
    "w = np.mat('[1;2]')\n",
    "\n",
    "print(w)\n",
    "\n",
    "print(sigmoid(X*w))\n",
    "eta = 0.1\n",
    "\n",
    "for i in range(10000):\n",
    "    pr = sigmoid(X*w)\n",
    "    w = w + eta*X.T*(y-pr)\n",
    "    \n",
    "print(np.hstack((y,pr)))\n",
    "print(w)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---------------------------\n",
    "# Optimization Frameworks\n",
    "---------------------------\n",
    "## CVX -- Convex Optimization\n",
    "\n",
    "CVX is a framework that can be used for solving convex optimization problems. \n",
    "\n",
    "Convex optimization includes many problems of interest; for example the minimization of the negative loglikelihood of the logistic regression is a convex problem. S\n",
    "\n",
    "Unfortunately, many important problems and interesting problems "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from cvxpy import *\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pylab as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selecting relevant features with regularization\n",
    "\n",
    "Below we generate a dataset with some irrelevant features that are not informative for classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Maximize\n",
    "\n",
    "$$\n",
    "\\mathcal{L}(w) + \\lambda \\|w\\|_p\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 380,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(x):\n",
    "    return 1/(1+np.exp(-x))\n",
    "\n",
    "# Number of data points\n",
    "N = 1000\n",
    "\n",
    "# Number of relevant features\n",
    "K = 10\n",
    "\n",
    "# Number of irrelevant features\n",
    "Ke = 30\n",
    "\n",
    "# Generate random features\n",
    "X = np.matrix(np.random.randn(N, K + Ke))\n",
    "\n",
    "# Generate parameters and set the irrelevant ones to zero\n",
    "w_true = np.random.randn(K + Ke,1)\n",
    "w_true[K:] = 0\n",
    "\n",
    "p = sigmoid(X*w_true)\n",
    "u = np.random.rand(N,1)\n",
    "y = (u < p)\n",
    "y = y.astype(np.float64)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 389,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAD7CAYAAAC/mkrPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3W1sXNd95/HfoSSKpmJr9GQrjSPS\n1IvUctLAJKeNu7uopiYLY4HCyIIa2WARbLbNaBn0RRdYUJCIwLAB2csBDBTboKqZhwKB2azGBOJ0\n3wQhpWu/qAPvkAyMjeWmCzGioqqOZckjOaZoyubZF/cOOcPHmXt55+l+PwAxc87MmXPmzJl7/zz3\nzL3GWisAAACEp6naDQAAAGh0BFwAAAAhI+ACAAAIGQEXAABAyAi4AAAAQkbABQAAEDICLgAAgJAR\ncAEAAISMgAsAACBk26vdgEL79++37e3t1W4GAADApqampt631h4o5bk1FXC1t7drcnKy2s0AAADY\nlDFmttTnckgRAAAgZARcAAAAISPgAgAACFlNreECAABb7+7du7p69arm5+er3ZS61NLSogcffFA7\nduzw/RoEXAAANLirV6/q3nvvVXt7u4wx1W5OXbHW6saNG7p69aoeeugh36/DIcUoS6clxynOcxw3\nHwDQMObn57Vv3z6CLR+MMdq3b1/g2UECriiLx6Vkcjnochw3HY9Xt10AgC1XTrA1Oiq1t0tNTe7t\n6GhozaoLWxGockgxyhIJKZPRsy++Ko1f0jPfOSVlMm4+ACCSRkelVEqam3PTs7NuWpL6+6vXrnrH\nDFfUJRK6+KWv6OLl96WBAYItAIi4oSE32Nrz+Nva8/jbktz00JD/18zlcjLGqKura+mvt7d3w+eP\njY1JksbGxpQOsNSl8LVKdfjwYd/1rWdLAi5jTOcGjw17t6mtqAtbzHGka9ektjbp7NnVa7oAAJFy\n5Yp723z/bTXff3tVvl8dHR2amppa+hsfH1/3uTdv3tS5c+ckSX19fRocHPRdb+FrVVPggMsY0yPp\nlQ2ekjLGXJI0E7QubLH8mq0jR9yD9JlM8ZouAEDkHDpUXn5QMzMz6u3tVW9vr44dO6ZcLqfh4WFN\nTExobGxMY2NjOnnypCYmJpaec/jwYaXTafX29qqrq0vT09OStPQ6vb29S7Naha8lSSdOnFiaZcuX\ny+VyRW0IQ+CAy1o7oY2DqW9Yaw97z0MtyWbdICsWc9Pemi5ls9VtFwCgas6ckVpbi/NaW938IGZm\nZooOKZ44cUKSe8iws7NT4+PjOnHihG7evKmTJ0+qp6dHfX19q17nlVde0fDwsMbHxzU+Pq5Tp07p\n3LlzmpmZ0YkTJzQ+Pq7h4WG99NJLklT0WiMjI5KkqakpnT9/fim4euGFF9Tb27v0ejdv3gz2ZtdQ\niUXzHd4sWKe1lvMN1JL8FO2//Gw5L5FgHRcARFh+YfzQeWl+3l1xcuZM8AXz+UOKK6VSqaWAp6Oj\nQ8PDw+sGPJ2d7gqmWCymjo6Opfu5XE579+5dCsLWMzU1pcnJyaVAK+ZNOExPT2t4eLiojq0WesCV\nD7KMMb3GmJ6VM13e2q6UJB0Ka74SAABIkt59/nl9/M4/b/icfy/p7L+567eO/If7pJ9Isz9Z//k7\nH/5dHTx92ld7MpmMjh8/ruHhYaXTaY2MjKw5s7WZF154QV1dXUqlUpqYmFgKoAp1dXUpFostPZZf\njN/Z2amJiQl1dnYuHWbcaqEGXF4wddNaOybphqSOlc+x1o5IGpGk7u5uG2Z7AABAaVqbt23p6+UP\nKRY6f/68uru7dezYMXV0dCwFQ3v37tX09HRZvy48fvy4Tp48qfHxcXV0dGhmZkbT09Pq6OhYeq1U\nKqVjx44t/UIyf1jz1KlTOnbsmMbHx4tmz7aSsTZ4jGOMGbfW9hakY9banPfrxRnv/kuSXrLWrhs6\ndnd328nJycDtQXmOv+QeUjx34rEqtwQAEIZ33nlHDz/8cLWbUdfW6kNjzJS1truU8lvxK8U+Sd3e\nbd55SfKCq6T32KWNgi0AAIBGFfiQone4cGxFXlfB/ZGgdQAAANQzzjQPAAAQMgIuAACAkBFwAQAA\nhIyACwAALEunV1/izXHcfPhGwAUAAJbF48XX1c1fdzce9/2S+esXHj58WHv27FFXV1do1yysVZW4\ntA8AAKgX3nV1n33xVWn8kp75zin3OrsBLvuWv67hyMiILl26tOZZ4BsdM1wAAKBYIqGLX/qKLl5+\nXxoYCOUau2NjYzpx4oQOHz6sb3/720uX2ZFUdEb6/OxYV1dXaJfdqQQCLgAAUMxxpGvX3CtXnz27\nek3XFpmcnNSlS5d08ODBNR8fGXFP5Tk1NaXz58/X9WFIDikCAIBl+TVb//0HUizmHk5MJgMfVlxL\nT0/Pho9PTU1pcnJyKdCKxWJbWn8lEXA1gnTaXcxY+EVwHCmblQYHq9cuAEDNeff55/XxO/+8/hN+\n/WvpK4/pP4+7665mP3uf9JXHpGefk77/92sW2fnw7+rg6dOB2nXjxg1JKrpgdVdX19IFrSUVHXas\nNxxSbAQh/KIEABBRn/+8FIuptXmbWpu3uXmxmJsfkr6+Pk1PT6u3t1fZbHYpP5VKaWZmRr29vert\n7VVHR0dobQibsdZWuw1Luru77eTkZLWbUZ8cx/1Fye99qexflBx/6WeSpHMnHguzhQCAKnnnnXf0\n8MMPV7sZdW2tPjTGTFlru0spzyHFRpFI6OJP35Uuz4b2ixIAAOAPhxQbRYV+UQIAAMpHwNUI8mu2\njhyR2tuXf1FC0AUAQE0g4GoE2awbZOV/LuudJVgFCw8BANFWS2u2681W9B1ruBpB/tQP//Kz5bxE\ngnVcAABJUktLi27cuKF9+/bJGFPt5tQVa61u3LihlpaWQK9DwAUAQIN78MEHdfXqVV2/fr3aTalL\nLS0tevDBBwO9BgEXAAANbseOHXrooYeq3YxIYw0XAABAyAi4AAAAQkbABQAAEDICLgAAgJARcAEA\nAISMgAsAACBkBFwAAAAh25KAyxjTucFjfcaYHmPM4FbUBQAAUG8CB1zGmB5Jr6zzWKckWWsnJOU2\nCswAAAAaVeCAywumZtZ5+LiknHd/RlJP0PoAAADqTdhruGKSbhak94VcHwAAQM1h0TwAAEDIwg64\ncpL2evdjkm6sfIIxJmWMmTTGTDbUVczTaclxivMcx80HAACREkrAZYyJeXfPSerw7ndImlj5XGvt\niLW221rbfeDAgTCaUx3xuJRMLgddjuOm4/HqtgsAAFTcVvxKsU9St3ebd16SrLXT3nN6JOXy6UhI\nJKRMRs+++KqePf1dN9jKZNx8AAAQKduDvoC1dkzS2Iq8roL7I0HrqFuJhC7+9F3p8qw0MECwBQBA\nRLFoPkyOI127JrW1SWfPrl7TBQAAIoGAKyz5NVtHjkjt7e7hxMI1XQAAIDIIuMKSzbpBVsz7/YC3\npkvZbHXbBQAAKo6AKyyDgxq9ltCbb0qvv+5Oco1eS0iD4VxScnRUxXWNhlsOAACUjoArJKOjUiol\nzc+76dlZNx1GQOO3rkq2EQCAKCPgCsnQkDQ3J23f85G27/lIkpseGqqduirZRgAAoizwaSGwtitX\n3NumHZ+umV8LdVWyjQAARBkzXCE5dKi8/GrUVck2AgAQZQRcITlzRmptLc5rbXXza6WuSrYRAIAo\nI+AKSX+/NDIiNXk93Nbmpvv7a6euSrYRAIAoYw1XiPr7pf/x/9z7//dybdZVyTYCABBVzHABAACE\njIALAAAgZARcAAAAISPgAgAACBkBFwAAQMgIuAAAAEJGwAUAABAyAi4AAICQEXABAACEjIALAAAg\nZARcKF86LTlOcZ7juPkAAGAVAi6ULx6XksnloMtx3HQ8Xt12AQBQo7h4NcqXSEiZjJ598VVp/JKe\n+c4pKZNx8wEAwCoEXPAnkdDFn74rXZ6VBgYItgAA2ACHFOGP40jXrkltbdLZs6vXdAEAgCUEXChf\nfs3WkSNSe7t7OLFwTRcAAChCwIXyZbNukBWLuWlvTZey2eq2CwCAGhU44DLG9Bljeowxg+s8Puzd\npoLWhRoxOLh6zVYi4eYDAIBVAgVcxphOSbLWTkjK5dMrpIwxlyTNBKkLAACgXgWd4TouKefdn5HU\ns8ZzvmGtPewFZQAAAJETNOCKSbpZkN63xnM6NjrkCAAA0OhCXzRvrU17s1v7jDGrZsCMMSljzKQx\nZvL69ethNwcAAKDiggZcOUl7vfsxSTcKH/SCqT4veUNSx8oXsNaOWGu7rbXdBw4cCNgcAACA2hM0\n4Dqn5SCqQ9KEJBljvPMFaDKfJ+mwlwYAAIiUQAGXtXZakrxDhbl8WtL5gseT3izXpYLHUedGR6U3\n35Ref9099+noaLVbBABA7Qp8LUVr7cgaeV0bPY76NjoqpVLSvU+66dlZNy1J/f3VaxcAALWKM82j\nbEND0tyctH3PR9q+5yNJbnpoqMoNAwCgRhFwlSKdXn2dQMdx8yPoyhX3tmnHp2ra8emqfAAAUIyA\nqxTxePHFmfMXb47Hq9uuKjl0qLx8AACijoCrFN7FmZ998VU9e/q7brCVyay+nmBEnDkjtbYW57W2\nuvkAAGC1wIvmIyOR0MWfvitdnpUGBiIbbEnLC+OHpqTFRamtzQ22WDAPAMDamOEqleNI16650cXZ\ns6vXdEVMf790733S7ph0+TLBFgAAGyHgKkV+zdaRI+5JpzKZ4jVdAAAAGyDgKkU2q4lURm/+Muae\n6PPrCU2kMlI2W+2WAQCAOkDAVYLRzw3qyb9OaH7eTc/OSk/+dUKjnxusbsMAAEBdIOAqASf6BAAA\nQfArxRIUnuhzrXwAAICNMMNVAk70CQAAgiDgKgEn+gQAAEEQcJWgv18aGZGavN5qa3PTnHuqAriO\nJQCgARBwlYgTfVZJPK75J5N6+qCjpibp6YOO5p+M7nUsAQD1iYALNW30WkJfXcio7Yuv6ok//q7+\n52+S+upCRqPXontpJQBA/eFXiqhpQ0PS7McJ3Tr0a31+/prOakA/+Tihd4aYZQQA1A9muFDTrlyR\njsrRZxfe09VdBzWgszoqh1NyAADqCgEXatrx+x1llNQv72vXr3d9VklllFFSx+/nOpYAgPpBwIWa\nNpjI6ms7M7rdfK8k6TUl9LWdGQ0muI4lAKB+EHChpj36w0H92fcSRafk+LPvJfToD7mOJQCgfhBw\noeZxSo7gRkel9nb3XHLt7W4aAFA5BFxAgxsdlVIpaXZWsta9TaXCDboI8IrRH8HRh9hIPYwPAi6g\nzpS7YRkakubmpD2Pv609j78tyU0PDYXXvkoHeLWM/giOPqxPlQqC6mV8EHABdcTPhiV/Co3m+2+r\n+f7bq/K3WqUDvFpXL/1RyzME9dKHWFbJIKhexgcBF1BH/GxYDh0qLz+oSgd4ta4e+qPWZwjqoQ9R\nrJJBUL2MDwIuoI742bCcOSO1thbntba6+WGodIBX6+qhP2p9hiBIH9byzF0jCxIElfuZ1cN3TNqC\ngMsY02eM6THGrPk7/c0eB6LKz47Az4alv18aGZFaWtx0W5ubLuXXnn7a6DfAa9QdY6UDXj9qfYYg\nyJiq5Zm7RuY3CPLzmdXDd0wKGHAZYzolyVo7ISmXT5f6OBCKdFoTQ07RzntiyJHS6U3LyVlxBnun\ntHLl1vfzp9N6+c+doo3Ky3/u6OdPb1zXjx5L64mdxW18YqejHz22Qbl0Wv2/4+gP/kD6oz/yTq3x\nO5u/L79t7P/XtH78V05RgPfjv3LU/69b3x++PrMAn7Ofcn76o9JtPLM7raMqLndUjs7sro02+upD\nSVf+Mq3fn3OKZu5+f87Rlb+s83FV4fHhp5yvbZX8fWZ+x4fv/vAp6AzXcUk57/6MpJ4yHwe23MSt\nuL78fFIPzTqyVnpo1tGXn09q4lZ844LxuJRMLn8BHcdNxzcu56e+tBPXDz5O6k8f/XvtefxtHZWj\nH3ycVNrZuK5HU3H9qDmpxAdTkqSnHnD0o+akHk1tUM57X4/8cqqs9+W3jYrH1TOS1H+JTbkB3t87\n6hnZuL4gdZX9mcXjmn8yqacPOmpqkp4+6Gj+yc37w+/48NMfQeryUy7+zbgySurf/cYdI0flXlIr\n/s0aaaOfPpQ0nnPfV8I6ar7/9tL7Gs+F08ZK9kclx4efcr62VfL5mfkcH777wydjrfVf2JiXJL1k\nrZ02xvRI6rXWniz18ZW6u7vt5ORkyfX/4E+e1sHcu2qzH5XV7lmzS5LKLvfLbbslSV/49FaoZSR/\nbfRbVyXLVaLvb9+Wtn+yoNjdnOa2tar10znldsT0yfZm3XffxmWbFha0PZfTQss92vnxHd3dHdNi\nc/OW15fLSc2LC4p9ktOdHS26Z2FeuR0xLTQ1KxYrrY0f72xRy8J8SW30U6aSbdyKukr9zBYWpE8/\nWtDuuznd2X6P7rl7R7eaY9q2q1mbdIkW5xbU8luv3Cd3NP+ZmJpaNymkYJ9ZOWMxSLml97ajRffc\nnS/pvQXtj3La6KcPl76bK8ZVWNuCSn5mlR4ftf6Z+amrsNz8Z3br4feuSZmMlEhsWi7PGDNlre0u\n5blVXzRvjEkZYyaNMZPXr18vq+xts0PzTdvLrnO+abuvci2Ln6hl8ZPQy0j+2ui3rkqWq0TfLy5K\nC03NmtvWqs98+pHmtrVqoalZi4sllG1udr+wd+b06T2tpe2ofNTX1OSWubP9Hu1auLNUpqmEb+Ri\nc7MWWu7RPfN3Sm+jjzKVbGPQusr5zObnpY9Ns+7saNGuu3Oa29aqj02z5uc3rmdhQfrwbkG5plZ9\neLdZCwultdHPZ1buWAxSrqnVbeOuhTtabG3dNHAK2h9lf8989GFLi3R3W/G4uruteenQUxhtrNRn\nNi+371vuuH0/r3DHR61/Zn7qKnxfe66/Kw0MlBVslc1a6/tP0rCkHu9+n6TBch5f+dfV1WXL8cVn\nfmK/+MxPyioTpFwl1UMb/ajE+2prs/aoLtj3tN8+q2/Z97TfHtUF29ZWQuELF+ytz8TsK//x69bu\n32/thQuh1Pfyy9Y+sbO4zBM7L9iXXy6tjXb/fmu/9a2S2+inTCXbGLSucj4zY9zP6/qOPTbd/l+X\nPi9jNq4m/zm/vzNmX3zk62WPKz+fWbljMWi5ctoYtD/KbqOfPrTWjp++YN9v2m+f07fs+0377fjp\nMsp5763Ucn7KWGvL7o/896Ww78P6vgQqV8nPzGddvvvDI2nSlhozlfrENQtLnZJS3v1BSZ3e/dhG\nj6/3R8C1rB7a6Ecl3tf46QtLG39pORja9EvrfWGf/W9/Y5N/98byF3iTL6Cv+i5csHfu3W+fesDd\n0T/1gJve9Mu+sk2ltNFPmSq00W9dd+7db5964m/sA0+/UVK5px5Y+/N66oGN60p4z/vTR79v9zz+\ni6VyCYX3mZU7FoOWK6eNQfujrDYGGMN+yuW/0//pj91xVcp32k+ZfJv8juGVdW02hv3UlS9X059Z\nwLrK/r4UKCfgCnRI0Vo7LUne+qxcPi3p/CaPA6Hp2Z3VW6cz+lVbQsZIv2pL6K3TGfXszm5cMJuV\nMhm9/YUuN51IuMfzsxuX81VfNquWH2f0w3cTWlyUfvhuQi0/3ryufBuXpr1LaaOfMlVoo5+6fj6S\n1VcXMnL2uJ/Z//pNQl9dyOjnI+uXG0xk9bWdGb0mt32vKaGv7cxoMLFxXb2xrJLK6H///Ov64Pwj\nek0JJZVRbyyc/phIZfT9XJdef11q/3pCE6nS+t5vuXLbGLQ/yvqeBRjDfspl/9Z9b//0gNvG/HvL\n/u365fyUkfyN4UO/cetyTEIL7923VNeh32x9Xe6b8zGuKvmZBayr3G2+b6VGZpX4Y4ZrWT200Y96\neF/Jv3vD/W8HNa+tzZ2nf+DpN+wDT79h3ZNK2E0Pa738svscY9zbUg7FvPyyta2tdqkOyU2XdBin\nTPm6Ct9XKXX5LRekjX77o5a/Z8a472fP47+wex7/xdL72+iws58y1vobw/kyK/82G/dBvi+VGlfV\nEGQsqlIzXECjGR2V3nxT7n9x7ZwgsdblT8y58N59WnjvvlX56+nvd89HtrjonZeshJPA5k8g29Ym\nGVPeCWTLlT/ze6FSzvzut5wfleyPSsufnPOD84/og/OPrMrfqjKSvzHs90Sffr8vlRxXjaz8n4sB\nDSp/huN7n3TT+TMcS42xE2lEhw65n1PhDi6fH4b+/sqMhcId41r5W13Or0r1R6WdOeN+9wuDjM0C\nGj9lJH9jON/nQ0PuZ3vokFvPZp+F3+9LpcdVo2KGCxUzOip9eFu6lavN2SP+i6s/9XJJj3L5nS3x\nW67San0m2c/snd8ZP79j2M8srd+66mVc+VHJsUjAhYrIzx7lz01Vi9c08zvdjupp1MNafneM9RCA\n5rcF+fOe1eK2QPJ/2LmWD1VXOiisdZUeiwRcqIh6mD1q5P/iGpmfnVyt87tjrIcAtB62BZVWyTFc\n60FhJVV6LLKGCxWRnyVavLttzfxa4HcNBhAGv+ujan1dFeuB6lOtjys/Kj0WCbhQEfnFmp98sGtV\nfq3wuxAVQOkq/UMHYD2VHoscUkRF1MsagEY8PAXUknrZFqDxVXosEnChIvJrAPIXIG2UNQAAytOo\n64FQfyo9FjmkWGvSaU3ciuvDO+4sS3u79N1+x71MzOBgtVsXSH+/9I+/de+f+4fqtgVA9TTieiDU\np0qORWa4aszErbi+/HxSn5n/UJL00KyjLz+f1MSteJVbBgAA/CLgqjF/MepehPQLty/r8x/9mzJK\nKqmM/mI0Ue2mAQAAnwi4asyVK+5V5u/O7dSf/OoNndWAXlOCn0wDAFDHCLhqzKFD0lE5+t6PnpM5\nf1ADOqujcvjJNAAAdYxF8zXmu/3umq2kMnpNCTlKKKOk3urPSOKwIgAA9YgZrhrTszurt05n9Ku2\nhIyRftWW0FunM+6vFAEAQF1ihqvWDA6qR9LlohOvJcTsFgAA9YsZLgAAgJARcKEy0mnJcYrzHMfN\nBwCgwRFwoTLicSmZlHI5N+04bjrOCV0BAI2PgAuVkUhImYx08aJ7Vehk0k0nWJsGAGh8BFyonERC\nRz63W0eyjjQwQLAFAIgMAi5UjuPome+c0jN/eFA6e3b1mi4AABoUARcqI79mK5ORnnvOvU0mCboA\nAJFAwIXKyGaL12zl13RlOaErAKDxceJTVMbg4Oq8RIJ1XACASKjbGa7RUenD29KtnNTe7qYBAABq\nUV0GXKOjUiolLS666dlZN03QBQAAalHggMsY02eM6THGrHHMSDLGDHu3qaB15Q0NSXNzxXlzc24+\nAABArQkUcBljOiXJWjshKZdPr5AyxlySNBOkrkJXrri3i3e3afHutlX5AAAAtSToDNdxSd61WjQj\nqWeN53zDWnvYC8q2xJndaR2Vo08+2KVPPtglSToqR2d2c10+AABQe4IGXDFJNwvS+9Z4TscmhxxT\nxphJY8zk9evXS6o0/s24Mkoq5sV6R+Uoo6Ti3+S6fAAAoPaEvmjeWpv2Zrf2GWNWzYBZa0estd3W\n2u4DBw6U9Jo9ZxJ663RGj+ii2nVZY01JvXU6o54znGIAAADUnk3Pw7XOYveZ/LotSXu9vJikG2uU\nvWmtHfMe6wjW3GU9ZxL6p9OX9Hv/x9G+oQH1PEewBQAAatOmAZe1dmSDh89J6vbud0iakCRjTMxa\nm5M0qeXF8oclveS/qSt41+XTwIB7XT5OogkAAGpUoEOK1tppSfIOFebyaUnnCx5PGmP6JF0qeDwY\nrssHAADqSOBL+6w1A2at7dro8cA2ui4fs1wAAKDG1Oe1FLkuHwAAqCN1eWkfAACAekLABQAAEDIC\nLgAAgJARcAEAAISMgAsAACBkBFwAAAAhI+ACAAAIGQEXAABAyAi4AAAAQkbABQAAEDICLgAAgJAR\ncAEAAISMgAsAACBkBFwAAAAhI+ACAAAIGQEXAABAyCIXcI2OSh/elm7lpPZ2Nw0AABCmSAVco6NS\nKiUtLrrp2Vk3TdAFAADCFKmAa2hImpsrzpubc/MBAADCsr3aDaikK1fc28W729bMBwAACEOkZrgO\nHXJvP/lglz75YNeqfAAAgDBEKuA6c0ZqbS3Oa2118wEAAMISqYCrv18aGZFaWtx0W5ub7u+vbrsA\nAEBji9QaLskNrv7xt+79c/9Q3bYAAIBoiNQMFwAAQDVsScBljOnc4LE+Y0yPMWZwK+oCAACoN4ED\nLmNMj6RX1nmsU5KstROSchsFZgAAAI0qcMDlBVMz6zx8XFLOuz8jqSdofQAAAPUm7DVcMUk3C9L7\nQq4PAACg5lR90bwxJmWMmTTGTF6/fr3azQEAANhym54WwhiTWiN7xjuUuJmcpL3e/ZikGyufYK0d\nkTQiSd3d3baE1wQAAKgrmwZcXkBUFmNMzFqbk3ROUreX3SGplCANAACgoWzFrxT7JHV7t3nnJcla\nO+09p0dSLp8GAACIksBnmrfWjkkaW5HXVXC/7BkyAACARlL1RfMAAACNjoALAAAgZARcAAAAISPg\nAgAACBkBFwAAQMiiFXCl05LjFOc5jpsPAAAQkmgFXPG4lExKOe962o7jpuPx6rYLAAA0tGgFXImE\nlMlIFy9Kly+7wVYm4+YDAACEJFoBlyQlEjryud06knWkgQGCLQAAELroBVyOo2e+c0rP/OFB6ezZ\n1Wu6AAAAtli0Aq78mq1MRnruOfc2mSToAgAAoYpWwJXNFq/Zyq/pymar2y4AANDQjLW22m1Y0t3d\nbScnJ6vdDAAAgE0ZY6astd2lPDdaM1wAAABVQMAFAAAQMgIuAACAkBFwAQAAhIyACwAAIGQ19StF\nY8x1SbNlFtsv6f0QmlOv6I9i9Ecx+qMY/bEafVKM/ihGfxT7grX23lKeuD3slpTDWnug3DLGmMlS\nf5IZBfRHMfqjGP1RjP5YjT4pRn8Uoz+KGWNKPpcVhxQBAABCRsAFAAAQskYIuEaq3YAaQ38Uoz+K\n0R/F6I/V6JNi9Ecx+qNYyf1RU4vmga1gjOm01k4XpPsk5SR1WmvT1WtZdazRH8PW2pPGmJS1lo0n\nAJTJGDOY35+Uuo+p6xkuY0wDoMwHAAACvUlEQVSfMabHGDNY7bbUAmPMsHebqnZbqsUY0yPplYJ0\npyRZayck5fLpqFjZH56UMeaSpJkqNKnqjDEp72+4IC+y25J1+iOy2xJvHPQwPlzr9Edkx4e0tF3t\n9e6XvI+p24Ar6jvSdUR6RyotjYfC939c7n8e8vJ7Kt6oKlqjPyTpG9baw95jkeJtKCe8mb0Ob0cS\n2W3JWv3hPRTJbYn3/o95Y6HTGNPJ+CjuD++hSI6PdZS8j6nbgEsR35GuI7I70g3EJN0sSO+rVkNq\nSEdU/1uX1KHlbcWMl47ytmSt/pAiui2x1k5Ya094yQ7vUHxkx8c6/SFFdHxIS0s0Ct93yfuYeg64\n2JGuFuUdKUpkrU17G4x9BTMakWCtHSlYt9YpaVIR3pas0x9SxLcl3vvOBxqRHR95K/pDivb42Ou3\nYD0HXFghyjvSDeS0/AWJSbpRxbZUnbdWp89L3tDyjEakeIdGpgt/TBBlK/sj6tsSb+HzCWNMrNpt\nqQUr+yOq42ON2S2pjH1MTZ1pvkzsSAt4ixdvWmvHFOEd6RrOScqfFblDUuSmwFeY1PK6i8OSXqpi\nW6qpx1p70rvPtqSgP6K8LSlYrzUt93uSUoTHx1r9YYzJKaLjQ+7MXofc8bDX65+S9zH1PMN1Tssf\nNDtSd0ea74PDWj40ECne7E13fhYn/x+7919YLmozGuv0R9JLX4paf0huQFHwc+4eRXxbskZ/RHlb\n0qPi4GpG0R4fa/VHZMeHtXbMCzQltz/K2sfU9Xm4vP/EZuQu5ov8+YTy/5nK7Y/InW8K2EzBaTJu\nyt2RHLPWTkR1W7JJf0RuW+IdMkt6ya78gvEIj4+N+iNy4yOoug64AAAA6kE9H1IEAACoCwRcAAAA\nISPgAgAACBkBFwAAQMgIuAAAAEJGwAUAABAyAi4AAICQEXABAACE7P8DUyKaMksu5yMAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11efefba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Regularization coefficient\n",
    "lam = 100.\n",
    "\n",
    "zero_vector = np.zeros((N,1))\n",
    "# Construct the problem.\n",
    "w = Variable(K+Ke)\n",
    "objective = Minimize(lam*norm(w, np.inf ) -y.T*X*w + sum_entries(log_sum_exp(hstack(zero_vector, X*w),axis=1)))\n",
    "prob = Problem(objective)\n",
    "\n",
    "# The optimal objective is returned by prob.solve().\n",
    "result = prob.solve()\n",
    "\n",
    "# The optimal value for x is stored in x.value.\n",
    "#print(w.value)\n",
    "\n",
    "plt.figure(figsize=(10,4))\n",
    "plt.stem(w.value, markerfmt='ob')\n",
    "plt.stem(w_true, markerfmt='xr')\n",
    "plt.gca().set_xlim((-1, K+Ke))\n",
    "plt.legend(['Estimated', 'True'])\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimization with pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_np, y_np, w_true_np, M, N = generate_toy_dataset(number_of_features=3, number_of_datapoints=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Descent for Logistic Regression: Reference implementation in numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.96195283 -0.21886467  0.83477378]\n"
     ]
    }
   ],
   "source": [
    "# Initialization \n",
    "w_np = np.ones(M)\n",
    "\n",
    "# Learnig rate\n",
    "eta = 0.01\n",
    "MAX_ITER = 100\n",
    "\n",
    "for epoch in range(MAX_ITER):\n",
    "    sig = sigmoid(np.dot(X_np,w_np))\n",
    "\n",
    "    # Gradient dLL/dw -- symbolically derived and hard coded\n",
    "    w_grad = np.dot(X_np.T, y_np-sig)\n",
    "    \n",
    "    # Gradient ascent step\n",
    "    w_np = w_np + eta*w_grad\n",
    "\n",
    "print(w_np)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Descent for Logistic Regression: First implementation in pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.autograd\n",
    "from torch.autograd import Variable\n",
    "\n",
    "#sigmoid_f = torch.nn.Sigmoid()\n",
    "def sigmoid_f(x):\n",
    "    return 1./(1. + torch.exp(-x))\n",
    "\n",
    "X = Variable(torch.from_numpy(X_np).double())\n",
    "y = Variable(torch.from_numpy(y_np.reshape(N,1)).double())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.96195283]\n",
      " [-0.21886467]\n",
      " [ 0.83477378]]\n"
     ]
    }
   ],
   "source": [
    "# Implementation \n",
    "w = Variable(torch.ones(M,1).double(), requires_grad=True)\n",
    "\n",
    "eta = 0.01\n",
    "MAX_ITER = 100\n",
    "for epoch in range(MAX_ITER): \n",
    "    sig = sigmoid_f(torch.matmul(X, w))\n",
    "\n",
    "    # Compute the loglikelihood\n",
    "    LL = torch.sum(y*torch.log(sig) + (1-y)*torch.log(1-sig))    \n",
    "    # Compute the gradients by automated differentiation\n",
    "    LL.backward()\n",
    "    \n",
    "    # The gradient ascent step\n",
    "    w.data.add_(eta*w.grad.data)\n",
    "    \n",
    "    # Reset the gradients, as otherwise they are accumulated in w.grad\n",
    "    w.grad.zero_() \n",
    "    \n",
    "print(w.data.numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 334,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"shell_port\": 65415,\n",
      "  \"iopub_port\": 65416,\n",
      "  \"stdin_port\": 65417,\n",
      "  \"control_port\": 65418,\n",
      "  \"hb_port\": 65419,\n",
      "  \"ip\": \"127.0.0.1\",\n",
      "  \"key\": \"40c24992-b940437a6d68edf64080bfde\",\n",
      "  \"transport\": \"tcp\",\n",
      "  \"signature_scheme\": \"hmac-sha256\",\n",
      "  \"kernel_name\": \"\"\n",
      "}\n",
      "\n",
      "Paste the above JSON into a file, and connect with:\n",
      "    $> jupyter <app> --existing <file>\n",
      "or, if you are local, you can connect with just:\n",
      "    $> jupyter <app> --existing kernel-ab030b1c-c549-4b31-8e5e-c11e1befeaa2.json\n",
      "or even just:\n",
      "    $> jupyter <app> --existing\n",
      "if this is the most recent Jupyter kernel you have started.\n"
     ]
    }
   ],
   "source": [
    "%connect_info"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}