markov.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "## 2. Марковский процесс принятия решения (markov decison process)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Задание\n",
    "#### 1. Приведите жизненный пример марковского процесса принятия решения (это может быть какаю-нибудь игра и т.п.).\n",
    "Очень простой пример - шашки. Конечное, пусть и очень большое, количество состояний $S$, конечное множество действий $ A \\subset S \\times S$. Среда - наш противник - отвечает вполне себе случайно. Если рассматривать среду именно как противника, то она, конечно, отдаёт сильное предпочтение тем действиям, которые она считает выигрышными. Награда для каждого допустимого действия из $\\; S \\times A \\;$ нулевая за исключением последнего хода, по результатам которого присуждается награда $\\pm 1$. Кроме того, очевидно, что для принятия решения информация о предыдущих состояниях нерелевантна. Таким образом, принятие решения в шашках можно смоделировать с помощью марковского процесса.\n",
    "#### 2. Можете ли вы привести пример игры, где принятие решения нельзя смоделировать с помощью марковского процесса?\n",
    "Шахматы, однако, не являются марковским процессом. Например, там есть такое понятие, как рокировка: перестановка короля и ладьи, если они не двигались в течение игры. Однако чтобы это проверить это условие, требуется хранить историю игры, что противоречит определению марковского процесса.\n",
    "#### 3. Выведите следующие значения через $p(s_{t+1}, r_{t+1}|s_t, a_t)$, для простоты все распределения можно считать дискретными\n",
    "  * $r(s_{t}, a_{t}) = \\mathbb{E}[R_{t+1}|S_t = s_t, A_t = a_t]$ - средняя награда за действие $a_t$ в $s_t$ \n",
    "  * $p(s_{t+1} | s_t, a_t) = \\Pr\\{S_{t+1} = s_{t+1} | S_t = s_t, A_t = a_t \\}$ - вероятность попасть в $s_{t+1}$ из $s_t$, сделав $a_t$.\n",
    "  * $r(s_t, a_t, s_{t+1}) = \\mathbb{E}[R_{t+1}|S_{t+1} = s_{t+1}, S_t = s_t, A_t = a_t]$ - средняя награда при переезде из $s_t$ в $s_{t+1}$, сделав $a_t$.\n",
    "  \n",
    "Раскрываем просто по определению матожидания и по базовым свойствам вероятности:\n",
    "$r(s_{t}, a_{t}) = \\mathbb{E}[R_{t+1}|S_t = s_t, A_t = a_t] = \\sum_{s,r} r \\cdot p(s, r \\;|\\; s_t, a_t)$\n",
    "\n",
    "$p(s_{t+1} | s_t, a_t) = \\Pr\\{S_{t+1} = s_{t+1} | S_t = s_t, A_t = a_t \\} = \\sum_{r} p(s_{t+1}, r \\;|\\; s_t, a_t)$\n",
    "\n",
    "$r(s_t, a_t, s_{t+1}) = \\mathbb{E}[R_{t+1}|S_{t+1} = s_{t+1}, S_t = s_t, A_t = a_t] = \\sum_{r} r \\cdot p(s_{t+1}, r \\;|\\; s_t, a_t)$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Смоделируем среду:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math\n",
    "from scipy.ndimage.filters import gaussian_filter1d\n",
    "\n",
    "\n",
    "def log_progress(sequence, every=None, size=None):\n",
    "    from ipywidgets import IntProgress, HTML, VBox\n",
    "    from IPython.display import display\n",
    "\n",
    "    is_iterator = False\n",
    "    if size is None:\n",
    "        try:\n",
    "            size = len(sequence)\n",
    "        except TypeError:\n",
    "            is_iterator = True\n",
    "    if size is not None:\n",
    "        if every is None:\n",
    "            if size <= 200:\n",
    "                every = 1\n",
    "            else:\n",
    "                every = int(size / 200)     # every 0.5%\n",
    "    else:\n",
    "        assert every is not None, 'sequence is iterator, set every'\n",
    "\n",
    "    if is_iterator:\n",
    "        progress = IntProgress(min=0, max=1, value=1)\n",
    "        progress.bar_style = 'info'\n",
    "    else:\n",
    "        progress = IntProgress(min=0, max=size, value=0)\n",
    "    label = HTML()\n",
    "    box = VBox(children=[label, progress])\n",
    "    display(box)\n",
    "\n",
    "    index = 0\n",
    "    try:\n",
    "        for index, record in enumerate(sequence, 1):\n",
    "            if index == 1 or index % every == 0:\n",
    "                if is_iterator:\n",
    "                    label.value = '{index} / ?'.format(index=index)\n",
    "                else:\n",
    "                    progress.value = index\n",
    "                    label.value = u'{index} / {size}'.format(\n",
    "                        index=index,\n",
    "                        size=size\n",
    "                    )\n",
    "            yield record\n",
    "    except:\n",
    "        progress.bar_style = 'danger'\n",
    "        raise\n",
    "    else:\n",
    "        progress.bar_style = 'success'\n",
    "        progress.value = index\n",
    "        label.value = str(index or '?')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Построим такую модель:\n",
    "Неформально определим её так: агент может пытаться двигаться на целочисленном отрезке $\\mathbb{N} \\cap [0;49]$ вправо или влево. При выборе направления движения агент может случайным образом сдвинуться в соответсвующую сторону на $1$ с вероятностью $\\frac{5}{16}$, на $2$ с вероятностью $\\frac{4}{16}$, на $3$ с вероятностью $\\frac{3}{16}$, на $4$ с вероятностью $\\frac{2}{16}$, на $5$ с вероятностью $\\frac{1}{16}$. Кроме того, с вероятностью $\\frac{1}{16}$ агент может остаться на месте.\n",
    "\n",
    "В 40-ом состоянии агент получает награду $10$. В 42-ом он получает штраф $100$. Во всех прочих состояниях он ничего не получает.\n",
    "\n",
    "Игра бесконечна."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "class Environment:\n",
    "    def __init__(self, states, actions=2):\n",
    "        self.states = states\n",
    "        self.actions = actions\n",
    "        self.transition_probas = np.zeros((states, actions, states))\n",
    "        self.rewards = np.zeros(states)\n",
    "        self.good_state = states - 10\n",
    "        self.bad_state = states - 8\n",
    "        self.rewards[self.good_state] = 10\n",
    "        self.rewards[self.bad_state] = -100\n",
    "        for state in range(states):\n",
    "            for i in range(state+1, min(states, state+6)):\n",
    "                self.transition_probas[state, 0, i] = 6 - (i - state)\n",
    "            self.transition_probas[state, 0, state] = 1\n",
    "            for i in range(state-1, max(-1, state-6), -1):\n",
    "                self.transition_probas[state, 1, i] = 6 - (state - i)\n",
    "            self.transition_probas[state][1, state] = 1\n",
    "            \n",
    "            self.transition_probas[state, 0] /= self.transition_probas[state, 0].sum()\n",
    "            self.transition_probas[state, 1] /= self.transition_probas[state, 1].sum()\n",
    "                \n",
    "        \n",
    "    \n",
    "    def step(self, state, action):\n",
    "        new_state = np.random.choice(self.states, p=self.transition_probas[state, action])\n",
    "        return (new_state, self.rewards[new_state])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class PolicyIterationStrategy:\n",
    "    def __init__(self, env, discount):\n",
    "        self.env = env\n",
    "        self.discount = discount\n",
    "        self.policy = np.ones((env.states, env.actions))\n",
    "        self.policy /= env.actions\n",
    "    def learn(self):\n",
    "        self.state_values = np.random.randn(self.env.states)\n",
    "        self.state_action_values = np.zeros((self.env.states, self.env.actions))\n",
    "        i = 0\n",
    "        while True:\n",
    "            while True:\n",
    "                epsilon = 0\n",
    "                for state in range(self.env.states):\n",
    "                    old_value = self.state_values[state]\n",
    "                    val_sum = 0\n",
    "                    for action in range(self.env.actions):\n",
    "                        action_sum = 0\n",
    "                        for new_state in range(self.env.states):\n",
    "                            action_sum += self.env.transition_probas[state, action, new_state] * \\\n",
    "                                (self.env.rewards[new_state] + \\\n",
    "                                    self.discount*self.state_values[new_state])\n",
    "                        val_sum += self.policy[state, action] * action_sum\n",
    "                    self.state_values[state] = val_sum\n",
    "                    epsilon = max(epsilon, abs(old_value - self.state_values[state]))\n",
    "                i += 1\n",
    "                if epsilon < 0.0001:\n",
    "                    break\n",
    "                \n",
    "            stable = True    \n",
    "            for state in range(self.env.states):\n",
    "                comp_values = self.state_action_values[state]\n",
    "                old_best = np.argwhere(comp_values == comp_values.max())\n",
    "                for action in range(self.env.actions):\n",
    "                    action_sum = 0\n",
    "                    for new_state in range(self.env.states):\n",
    "                        action_sum += self.env.transition_probas[state, action, new_state] * \\\n",
    "                            (self.env.rewards[new_state] + \\\n",
    "                                self.discount*self.state_values[new_state])\n",
    "                    self.state_action_values[state, action] = action_sum\n",
    "                comp_values = self.state_action_values[state]\n",
    "                best = np.argwhere(comp_values == comp_values.max())\n",
    "                self.policy[state] = 0\n",
    "                self.policy[state][best] = 1\n",
    "                self.policy[state] /= self.policy[state].sum()\n",
    "                if not np.array_equal(old_best, best):\n",
    "                    stable = False\n",
    "            if stable:\n",
    "                break\n",
    "        return i\n",
    "                    \n",
    "                \n",
    "    def choose(self, state):\n",
    "        return np.random.choice(self.env.actions, p=self.policy[state])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "class ValueIterationStrategy:\n",
    "    def __init__(self, env, discount):\n",
    "        self.env = env\n",
    "        self.discount = discount\n",
    "        self.policy = np.ones((env.states, env.actions))\n",
    "        self.policy /= env.actions\n",
    "    def learn(self):\n",
    "        self.state_values = np.random.randn(self.env.states)\n",
    "        self.state_action_values = np.zeros((self.env.states, self.env.actions))\n",
    "        i = 0\n",
    "        while True:\n",
    "            epsilon = 0\n",
    "            for state in range(self.env.states):\n",
    "                old_value = self.state_values[state]\n",
    "                for action in range(self.env.actions):\n",
    "                    action_sum = 0\n",
    "                    for new_state in range(self.env.states):\n",
    "                        action_sum += self.env.transition_probas[state, action, new_state] * \\\n",
    "                            (self.env.rewards[new_state] + \\\n",
    "                                self.discount*self.state_values[new_state])\n",
    "                    if action == 0:\n",
    "                        max_val = action_sum\n",
    "                    else:\n",
    "                        max_val = max(max_val, action_sum)\n",
    "                self.state_values[state] = max_val\n",
    "                epsilon = max(epsilon, abs(old_value - self.state_values[state]))\n",
    "            i += 1\n",
    "            if epsilon < 0.0001:\n",
    "                break\n",
    "                \n",
    "        for state in range(self.env.states):\n",
    "            res = np.zeros(self.env.actions)\n",
    "            for action in range(self.env.actions):\n",
    "                action_sum = 0\n",
    "                for new_state in range(self.env.actions):\n",
    "                    for new_state in range(self.env.states):\n",
    "                        action_sum += self.env.transition_probas[state, action, new_state] * \\\n",
    "                            (self.env.rewards[new_state] + \\\n",
    "                                self.discount*self.state_values[new_state])\n",
    "                res[action] = action_sum\n",
    "            best = np.argwhere(res == res.max())\n",
    "            self.policy[state] = 0\n",
    "            self.policy[state][best] = 1\n",
    "            self.policy[state] /= self.policy[state].sum()\n",
    "        \n",
    "        return i\n",
    "                \n",
    "    def choose(self, state):\n",
    "        return np.random.choice(self.env.actions, p=self.policy[state])\n",
    "    \n",
    "class RandomStrategy:\n",
    "    def __init__(self, env, discount):\n",
    "        self.env = env\n",
    "        self.discount = discount\n",
    "        self.policy = np.ones((env.states, env.actions))\n",
    "        self.policy /= env.actions\n",
    "    def learn(self):\n",
    "        return 0\n",
    "    def choose(self, state):\n",
    "        return np.random.choice(self.env.actions, p=self.policy[state])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "class MarkovPlayer:\n",
    "    def __init__(self, states, actions, steps, strategy_class, discount):\n",
    "        self.states = states\n",
    "        self.actions = actions\n",
    "        self.strategy_class = strategy_class\n",
    "        self.steps = steps\n",
    "        self.discount = discount\n",
    "    def evaluate(self, games=1, progressbar=True,hold=False, show=True,\n",
    "                 color='b', label=' '):\n",
    "        rewards = np.zeros(self.steps)\n",
    "        if progressbar:\n",
    "            games_range = log_progress(range(games), every=1)\n",
    "        else:\n",
    "            games_range = range(games)\n",
    "        avg_iterations = 0\n",
    "        for game in games_range:\n",
    "            state = 0\n",
    "            self.state = state\n",
    "            env = Environment(self.states, self.actions)\n",
    "            strategy = self.strategy_class(env, self.discount)\n",
    "            self.strategy = strategy\n",
    "            avg_iterations += strategy.learn()\n",
    "            for i in range(self.steps):\n",
    "                state, r = env.step(state, strategy.choose(state))\n",
    "                rewards[i] += r\n",
    "                self.state = state\n",
    "        avg_iterations /= games\n",
    "        discount = self.discount\n",
    "        print(avg_iterations, 'steps to converge')\n",
    "        for i in range(1, self.steps):\n",
    "            rewards[i] = rewards[i-1] + discount * rewards[i]\n",
    "            discount *= self.discount\n",
    "        for i in range(1, self.steps):\n",
    "            rewards[i] /= games\n",
    "        if show:\n",
    "            x = np.arange(1, self.steps+1)\n",
    "            plot_rewards = rewards\n",
    "            plt.plot(x, plot_rewards, color, label=label)\n",
    "            plt.title('Total reward', fontsize=16)\n",
    "            if not hold:\n",
    "                plt.show()\n",
    "        return rewards.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Сравним эффективность и скорость сходимости алгоритмов:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## ATTENTION: Не запускать следующий код с исходным games_num. Долго работает. Для проверки работы достаточно games_num ~ 50 или даже меньше"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel/__main__.py:14: MatplotlibDeprecationWarning: pyplot.hold is deprecated.\n",
      "    Future behavior will be consistent with the long-time default:\n",
      "    plot commands add elements without first clearing the\n",
      "    Axes and/or Figure.\n",
      "/usr/local/lib/python3.5/dist-packages/matplotlib/__init__.py:917: UserWarning: axes.hold is deprecated. Please remove it from your matplotlibrc and/or style files.\n",
      "  warnings.warn(self.msg_depr_set % key)\n",
      "/usr/local/lib/python3.5/dist-packages/matplotlib/rcsetup.py:152: UserWarning: axes.hold is deprecated, will be removed in 3.0\n",
      "  warnings.warn(\"axes.hold is deprecated, will be removed in 3.0\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0 steps to converge\n",
      "===\n",
      "432.204 steps to converge\n",
      "===\n",
      "110.322 steps to converge\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel/__main__.py:24: MatplotlibDeprecationWarning: pyplot.hold is deprecated.\n",
      "    Future behavior will be consistent with the long-time default:\n",
      "    plot commands add elements without first clearing the\n",
      "    Axes and/or Figure.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEYCAYAAABcGYHrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4FNX6wPHv2bRND4EkEHoXLr03aTYQUcSCoBQFQRHB\nglwULojlB0hVRKXjBQELooggNiKKApcIKEWlo4SSmJBeNtnz+2OWGGoSssnsJu/neeZJpu57mDDv\nnjNnziitNUIIIYTF7ACEEEK4BkkIQgghAEkIQgghHCQhCCGEACQhCCGEcJCEIIQQApCEIEyklNIF\nmI4X4nieSqkXlVKdixDTdqXUF9e7vztSSlkd/9bjzY5FmMvT7ABEmdb+kvl1wF7gxTzLMgtxPE9g\nMpANbC1SZEKUQZIQhGm01tvzziulMoG4S5e7A6WUt9Y6y+w4rkYp5aO1LkxyFWWQNBkJt6GUelgp\n9atSKlMpFauUWqaUCnesswLpjk1fztPkNN6xvr1Sap1S6i+lVLpS6jel1BSllM91xHGD49jDlFJz\nlFKngQyllK9jfR2l1BqlVJxSKkMpFa2U6pVn/46O/VvlWfacY9nEPMsaO5bd5JivpJRapJQ67CjD\nSaXUf5VSFS+Jb5pSKlsp1Ugp9Y1SKgX4r2Odp2P9WaVUmlLqG6B+Yf8NROkkCUG4BaXUaGApsAfo\nA0wE7gS2OC7EmUAXx+YLMJqj2uO4EAI1gP8BjwE9gfnASMe212sKUAUYBtwDZCulagE7gBuA0cBd\nwAFgvVLqNsd+O4FUoHueY3XHSGiXLssEtjnmKwDJwDjgNmA80BjYqpTyuiQ2BXwKfAn0dpQXYCrw\nHMa/ZR/gO+CT6yq9KH201jLJ5BITcBxYeYXl3sDfwBeXLL8Z0MBwx7zVMT8xn89RGM2lwzDuNwTm\nWbf90s+5wv43OD7nxyusew+IAYIvWb4V2J5nfjOwyfG7J8aFfhZGUvBxLP8EiLpGHJ5AXUcsPfMs\nn+ZYNuKS7cMcx597yfLJju3Hm/03IJO5k9QQhDtoBIQCK/Mu1Fp/DZzln5rBVSmlyimlZimljmJ8\n67YBiwAPoPZ1xnWlb9Y9gM+AVEfzjKdSyhPjm3prR9MWwLdAJ8c3+zYYyWwaRrLqoJSyOMq1JU8Z\nlFJqtKPZLMVRhj8cq6/U7LPukvnmjs/54JLlawpWXFHaSUIQ7iDU8fP0FdadybP+WlYCDwNzMGoW\nrYFnHOusV9spHxfFo5TycMQyHONinXd6GeP/WznH5luAAEcc3YBdWutYjBpKN4yLdwhG4rhgLDAX\n+By4GyORXEiGl5bBrrU+d8mySo6fZy9Zfum8KKOkl5FwB/GOnxWvsK4isP9aOyulgjDuG4zTWs/L\ns7x1EeO6aOx4rXWOUioR44I95yr7xDl+RgNJGPcJuvHPhf9b4FaMJqR0jPsRFzwAbNRa5z4voJRq\nUJDYHC4ksAjgSJ7lEVc5hihjJCEId7APIyk8gNFGD4Cj900EEOVYlIVxIfS9ZH9fjKYYW559FTC4\nGGL9AmgK/Kqv0c3TkTy2YjQxtQT+z7HqW4wb5jnANn1xV1Y/8pTB4eFCxLYbyADuB37Ms/yBQhxD\nlGKSEITL01pnKaWmAK8rpZYB7wPVgFcxevCsdGxnV0r9DtyllPoWSAT+0lqfUUrtAcYrpeKA8xjN\nOhWKIdwXML7VRyml3gJOYjQTNQEitdaP5dl2C8aN5Ez+uUDvwLjod3YcK68vgCeVUuOAnzF6GvUp\naGBa61il1JvAM0qpdIzk047iSYzCDck9BOEWtNZvAEOBVhjdKV/FaJrpprVOz7Pp4xg9hzZidDMd\n4lh+H/ArRjfTpcAxjO6Xzo7zqCPG34DpwFfAW0BH/qnJXHDhhvH2C2XQWtuAHy5Zf8F/gGWOuD/G\n6GHUi8J5HpiJ8W/5KdCVQiQVUbopreUVmkIIIaSGIIQQwkESghBCCEASghBCCAdJCEIIIQA363Za\noUIFXaNGjULtk5qair+/f/EEVMKkLK5JyuKaSlNZoGjliY6OjtNah+W3nVslhBo1arBr165C7RMV\nFUXXrl2LJ6ASJmVxTVIW11SaygJFK49S6kRBtpMmIyGEEIAkBCGEEA6SEIQQQgCSEIQQQjhIQhBC\nCAFIQhBCCOEgCUEIIQTgZs8hCCGEq9EabDZIT4fMTMjIMH5mZkJWFmRnG5PNZkxZWcbPC8vzTpcu\nvzBvs0H9+pe+98n5JCEIIdye1saFNj39n+nYMT98fSE52bhIX7gYp6UZU3r6PxfvjIyLp0vXpaYa\nU97l2dmQk2McsyTeIjB9uiQEIUQplp0NCQnGlJhoXLyTki5flpLyz8+UFOOCnppqbHthm8svym0K\nFINSYLWC1arx8QFfq8bXqvHxsuPjpbF65VA5OBv/sGx8vWxYLVn4KBue2PCwZ+NjsWH1sOFjseFr\nycTqkYU3Nnx0Bt46Ey+dhac9C09tc8xn4q2NeQ97Fp52Gx52G146K3dbL3smHtmZeNkz8bRn4ZGT\nxe6AgRgv3is+khCEEE6htXFhjouDmBg4fhxOnIDz5y+/0P/9t7Hd+fP5H9ffHwICIDAQAgM1/r52\nyvnZqBxsI6hWFsHWTAK9MvFT6fhiTFadTtLpo9QJsxKQfR5fWxJemSl4ZaXgn/43fhnx+KbHY81M\nxDMjBZWeBkmpxld+Z/PyMiZPT+Onh8c/8x4e//zMO11Y7+kJ3p7g4YO2FP8tX0kIQoh8aQ3nzsGx\nY3D0qPHz5En48084fRpiY40LfGbm5fv6+V24oGvKBdkpF5hNzXqZhDVLpbx3MuW9kynnkUhwTgKB\nmXEEZsRSLuss5TJOE5R+Fkt6qlEtSEyEEwlG209BWSxGJgkIMAK5kF1CA8G/Jvj6GpOfnzFZreDt\nbVyQvb3Bx+efnz4+/2xvVCn+WX7hwp73wu/jY/yulFPOQXJUlFOOcy2SEIQQZGQYF/jo6HKcOAGn\nThnzx44Z3/RPnjS2yatCBU21SjaqlEuneaU0wqwphHnEE6bPEWH7i5pZv1Mt5QC+cX/CmTNwpADV\nAS8vKFcOgoONKTAQQisbF/GQEGO6sC4oyLjAX7hIBwQY845lW3ftovOttzrtglwWSEIQoozIyIAD\nB2DfPmM6dAj++sv4ln/27IWtmuZuXyE4ixohiTT2PUvvOqeobj9GzexD1Mo4QI2UffjF/QVxV/mw\nwECoVAkiIqBRI7j5Zihf/p+LdkiIceG/cIEPDDR+9/d32gXc7uMjyaCQJCEIUcqkp8PvvxsX/99+\nM6Z9++CPP/5pIvf2slMnPJmqPmdpZj1BtWonqJ5zlEqJv1Ij5Tcqcwq/xHRIxGgOCQ+HihWhSphx\nIQ/tbcxXrGhc6C98qw8LMybf4u8RI5xPEoIQbkproynn559h9+5/vvkfOQJ2u7GNxaKpFZ5Cg+AY\n7qm3jybJ22h8+kvq2H7D81SO0cZdr55xYQ8OJiZLE9lmIFStCjVrGlOVKkZSEKWeJAQh3ER8PGzb\nBtu3Q3S0McU5mmwsFk29yqk0DT1N/8b7+VfcVhqc+pq69j+wnsmEs8q4uDdrCAPvhEYvGE059esb\nNz8d/oiKIrIUvVRGFI4kBCFcTEoK7N8Pv/5qNPccPmw09xw8aKz39LDzr9DT3Kl30dL/B1qmbqWJ\n/Rd8/8yAPzGad1q3hmF9oUED46Jft67RPi/ENUhCEMJEWhsX+++/h59+gh07jLb/Cw9ZWb1zqF0u\nnroeJxgQ+j03xn9Cm5yd+CbaoX17aNgQKt0BkcONpp/69Y02fLmZKq6DJAQhSlhCAnz5JWzYAJs3\nG334AcoH22hb+S/ubRhNs7ivaXz2K2pmHcNyVkP16tC+FbTsAW0mQYcOcuNWOJ0kBCGKkd1uNPns\n3g07d0JUFOzerdFaUSEwg9sq76dr0FfcePI96iXuQyUC1arBjW2g1XBo2RJatIDQULOLIsoASQhC\nOJHWRvPPxo1G88+uXcY9AQAfz2zal/uNyd6fcUvmZ7RN3oHHcW/jon/XrdDO8c2/cmVzCyHKLEkI\nQjjBsWOwYgW8+64xtIOXFzRramdQl5O0OruRZnuW0zB7Lz6e5eGh26HzY9BiIdxwg9H1UwgXIH+J\nQlynkyfh88/hnXea8csvxrLu/zrLlH47uTtjDf7frIddKcaDWyMfhEFvG80/csNXuChJCEIUkNZG\nV9C1a2HdOuN3gNp+ybzqN58H0xZSff9J2I/xMNdDD0Hv3nDTTRf19RfCVUlCECIfcXFGU9CSJcaz\nABaL5saIP5hl/S+9Mj6ipscZvO/sDbdPhcaNjR5BQUFmhy1EoUlCEOIKsrONrqHLlsH69cZbsdrX\njeWdeiu4+49phCemwn33wIDX+c7Liy433WR2yEIUmSQEIfLQGj74AMaPN4Z9ruCXysiAtQyNf41G\nh/YbXUJnjINhw4zROQFdAuPUC1ESJCEIgZEItmyBSZM027YpmgYfY63lOe5IW4936w5w+yDo1s24\nKSwDvYlSShKCKNNsNli1CmZPz+KXg95EeP7NIsbzsMd6PMY+DI8egDp1zA5TiBIhCUGUSdnZsHIl\nvDwxg6OnrDTidxYzlwcb/Ip19HB4cJ4MDSHKHEkIosz57jsY9ZiNfb950ZwDfBo6j97P3YC6Z7wx\nKqgQZZQkBFFmnDqRzbj+f7Lqp5pU5xQfWsZxz9PVUZPfMF7hKEQZJwlBlHoZGTB71FH+b1lFsu2V\n+E/IG4wfnoDfsFelRiBEHpIQRKmVnAwLp5xm9nxvYjJqcbfvF8ycCbUef1KGjxDiCkxLCEopK7AV\n8HHE8ZHWerJZ8YjS488/4c2X4ln4rjfnbZXo5rGVFQ+fpPv8e+RGsRDXYGYNIRPorrVOUUp5AT8o\npTZprbebGJNwY6dPw/NPJrPyYz+0Dqavx3qeG3yaNrMfgNDOZocnhMszLSForTXgGCkeL8ekzYpH\nuC+bDeZNjmPyjACysr0Z4zGP0UOSqf5/I4z3CwshCkRpbd41WCnlAUQDdYD5Wut/X2Gb4cBwgIiI\niJZr1qwp1GekpKQQEBDghGjNJ2W53JHvM5k9oy4HkmvRU23i+Vs24DGsK1lhYU6IsmDkvLim0lQW\nKFp5unXrFq21bpXvhlpr0ycgBNgCNLrWdi1bttSFtWXLlkLv46qkLP/IOX1WP9v0K63I0ZX5S3/c\ne6m2x5x2TnCFJOfFNZWmsmhdtPIAu3QBrsWW60o3Tqa1Po+REHqYHYtwcVlZ2GfMYkS1TczaezPD\nG27jwG8W7l7/MKpSRbOjE8KtmZYQlFJhSqkQx+++wC3Ab2bFI9zA8ePojp14cpyVxbbBTBgRx9v7\nbiSofiWzIxOiVDCzl1El4F3HfQQL8IHWeoOJ8QgXZl+/gQ0DVjErYzZb6cRzz8HL0yvI4wRCOJGZ\nvYx+AZqb9fnCTdjt/PrEO/R7pysHWUW1SBvznocnnpBny4RwNnlSWbiupCQ+vOltHt71BEG+NtYs\nyOKe/t54yl+tEMVC/msJl5S27Wcm9NrL3MR/077madb+UJFKkVIlEKI4uUQvIyFyac2PY96nWadA\n5iY+zKi+MWw5WEmSgRAlQGoIwnWcPs3yW1cxdN9TVPWN45tVSXTvE2l2VEKUGZIQhGvYuJG379/C\nyNQZ3HLDn6zdUYXAIKkVCFGSpMlImE5/vpFpvbcxMnUGvbsls353VUkGQphAagjCVMkbv+fhOzNZ\na3+VB/pm8e7qQLy9zY5KiLJJagjCNL8u3kHrOyL4xN6bmVNSWfWRtyQDIUwkNQRR4rSG76bFM21z\nT4I9Uvnm/fN0uaeC2WEJUeZJQhAlSts1j7TYw/K9fbk1dBf/3V6PiLpBZoclhECajERJstuZ3eVT\nlu9tzuhaa9gU01SSgRAuRBKCKBk5OWy94zX+/cMd9K3zC30WRWDx8TI7KiFEHpIQRPGz2/nzgefo\nt2kwtUITWbarMcoi3UqFcDWSEETx0ppv+7xBq4/+TYp3KGujyhMULMlACFckCUEUG23XzOiygVs+\ne5Ly5WHnbm8aNzY7KiHE1UhCEMVCa5jUdSvjvu/NPXV+YeexcBo0lJqBEK5MEoJwOq1h8q0/8cr3\nXRhadytrDjYlIFCSgRCuThKCcLpX7vmZl79uz9Ca37Jwf0csnvJnJoQ7kP+pwqnmjD3FpHUtGFxp\nMwsP3ojFy8PskIQQBSQJQTjN4jfSeGZWZe61bmBxdAt5zkAINyMJQTjF9h/tjBjjQw/1Be9tCsWz\nUpjZIQkhCkkSgiiytDQYfEccVfmTD149jHfXDmaHJIS4DpIQRJFN6BnNHwnhLO3zGYHjnzA7HCHE\ndZKEIIpk6+RveH1rc56o/QXdPxoJSrqXCuGuJCGI63bgg330fakptawxTNveFTykR5EQ7kwSgrgu\nR7ef4+b+FfDysPNFlC8BFaxmhySEKCJJCKLQEk5ncHNXG5l2L756P4E6bcubHZIQwgkkIYjCyclh\nzk0bOJZZmQ2v7KXRPfXNjkgI4SSSEETBaU384xN4/eAt3NvkD9pP6G52REIIJ5J3KouCmzOHOYv8\nSSKYSSuDzY5GlLCkpCTOnTuHzWYzO5QCCQ4O5uDBg2aH4TRXK4+Xlxfh4eEEBRX9dbSSEETBHD5M\n/PjXeN3zKPf10TRuLN1Ly5KkpCTOnj1L5cqV8fX1RblB9+Lk5GQCAwPNDsNprlQerTXp6emcOnUK\noMhJQZqMRP60htGjmcrzJGf7MWmy618MhHOdO3eOypUr4+fn5xbJoKxQSuHn50flypU5d+5ckY9n\nWkJQSlVVSm1RSh1QSu1XSo0xKxZxbfrT9UzZ1JqZtjE88gg0amR2RKKk2Ww2fH19zQ5DXIWvr69T\nmvLMbDLKBp7VWv+slAoEopVSX2mtD5gYk7iEPTWdMYPO8yZTGDLIzoIFUqksq6Rm4LqcdW5M+9+t\ntT6ttf7Z8XsycBCobFY84nJaw8i20byZPJhn7v+TJcsseMpdJyFKLaW1NjsGlFI1gK1AI6110iXr\nhgPDASIiIlquWbOmUMdOSUkhICDAOYGarCTLojUsm2BlxU/tePyGT7jvrRCnDlMk58U1Xa0swcHB\n1KlTx4SIrl9OTg4epWg4lfzKc/jwYRITE6+4rlu3btFa61b5fojW2tQJCACigb75bduyZUtdWFu2\nbCn0Pq6qJMsy5ZkEDVqPDl+t7RmZTj++nBfXdLWyHDhwoGQDcYKkpKQS/bwZM2bo6tWrF9vx8yvP\ntc4RsEsX4HpsaoOwUsoLWAu8p7X+2MxYxD+OHMjkpdkBPOi5hrk/tEL5eJsdkhCiBJjZy0gBS4CD\nWuvZZsUhLqE1r9yxHS9szJjvj6rrXs0EQlxLVlaW2SG4NDNrCB2BgUB3pdQex3S7ifEI4PCouaw4\n1pHHO/5KpeG9zQ5HiCLp2rUrjz/+OGPHjiUsLIyOHTsye/ZsmjRpgr+/P5UrV2bYsGGcP38+d5/l\ny5cTEBDAN998Q6NGjfD396dbt24cO3bsomO/9tprVKxYkYCAAAYNGkRKSspF6+12Oy+//DJVq1bF\nx8eHxo0b8+mnn+auP378OEop1qxZQ5cuXfD19aV58+b88ssv7Nu3jw4dOuDv70+nTp0u++ziYlqf\nEa31D4D0Y3MlH33Ey2+F4u2Rw7gPW5sdjXBxTz31FHv27CnRz2zWrBlz584t1D4rV65k+PDhfP/9\n92it2bx5M3PnzqVWrVqcOHGCJ598kieffJIVK1bk7pOZmcnUqVNZunQpVquVwYMH89hjj7F582YA\nPvjgAyZOnMi8efPo1q0bH374IdOnTyc0NDT3GK+//jozZszgnXfeoVWrVqxcuZK+ffsSHR1Ns2bN\ncrebPHkyc+bMoVatWjz++OP079+f8PBwXn31VcLDwxk8eDCjR49m1apVRfzXy590IhSGhAT+GDGL\nlfzAU09CxUqSq0XpULNmTWbNmpU736BBg9zfa9SowWuvvcZdd93Fu+++i8ViNJpkZ2czf/586tc3\nRvMdO3YsjzzyCFprlFLMnTuXwYMHM2LECAAmTJjAli1bOHz4cO6xZ86cydixYxkwYAAAL730Elu3\nbmXmzJmsXLkyd7tnnnmG2283GkeeffZZevfuzcsvv0y3bt0AGDVqFKNGjSqOf5rLSEIQANjGPs+Q\n+NkEBMC48aWnq54oPoX9pm6Wli1bXjT/7bffMnXqVA4ePEhiYiI5OTlkZWVx5swZIiMjAfDx8clN\nBgCRkZFkZWWRkJBAaGgoBw8eZNiwYRcdt3379rkJISkpiZiYGDp27HjRNp06dWLjxo0XLWvSpEnu\n7xEREQA0btz4omWpqamkpaUV+9hM8tipgK1beXFpVX6iPQsWeeD4mxSiVPD398/9/cSJE/Tq1YsG\nDRrw4YcfEh0dzdKlS4GLbzh7XvIE5oUnge12e5HjufSpYi8vr8vWXWmZMz47P5IQyjqbjW8fWspU\nnueRQTYeeMDsgIQoPrt27SIrK4s5c+bQvn176tWrR0xMTKGP06BBA7Zv337RsrzzQUFBREZGsm3b\ntou2+eGHH2jYsOH1BV8CpMmojEt7cymD/nyF+lVSeeOt0jNUsBBXUrduXex2O3PnzqVv375s3779\nupq+xowZw6BBg2jdujVdu3blo48+YseOHRfdVH7uueeYNGkSdevWpWXLlqxcuZLvv/+en3/+2ZlF\ncipJCGVZUhJv/yeGU1Rh63uaPDVrIUqlJk2a8PrrrzN9+nQmTpxIhw4dmDlzJv369SvUcfr168fR\no0eZMGECaWlp3HnnnTzzzDMsX748d5vRo0eTnJzMuHHjOHv2LPXr12ft2rU0bdrUyaVyooI8zuwq\nkwxdscWpx0t+booO46y+pe15px63IOS8uCYZusJ1lcTQFdesISilQq+1Xmsd79TsJErOqVPMn5NF\nLOFMmWN2MEIIV5Bfk1E0oDEeIKsGJDh+DwFOAjWLNTpRbJJGT+S17Jn07JJG+/Z+ZocjhHAB1+xl\npLWuqbWuBXwN9NZaV9BalwfuAL4siQBFMVi3jnkfRxJPeabMkGQghDAUtNtpO6117tMUWutNQIfi\nCUkUq/h4kkY8xyyPcfS63U5rGaFCCOFQ0F5GMUqpicCF560fBArfeVeY7+mnmfd3fxLswbw4xexg\nhBCupKAJoT8wGViHcU9hq2OZcCc//kjSf9cxy3qGO26GVvm/P0kIUYbkmxCUUh7AC1rrMSUQjyhO\nEycyz/95ElL9ePFFs4MRQriafBOC1jpHKdWpJIIRxejbb0ne8j9m+W2kd2+4ZLwvIYQocJPRbqXU\neuBDIPXCQi2vvXQPWsOECSwOHktCopWJE80OSAjhigqaEKzA30D3PMs0IAnBHWzciG37LuaU+4ou\nXaBNG7MDEsI9DBkyhLi4ODZs2HDF+dKmQAlBa/1wcQciikl8PDzxBO+Hj+bPcwG8/ZzZAQlRcoYM\nGcK7774LGENaV61alb59+zJlypSLhsUuqNdffx1jJIjSqUAJQSllBYYC/8KoLQCgtX6kmOISzqA1\nPPww+lQMM2q8RMMK0LOn2UEJUbJuvvlmVqxYgc1m4/vvv2fYsGGkpqby9ttvF/pYwcHBxRCh6yjo\ng2krgIrAbcB3QBUgubiCEk4yZw6sX89XQ9fwy2F/xo4Fi7wBQ5QxPj4+VKxYkapVqzJgwAAefPBB\nPvnkEwC2bt1K27ZtsVqtRERE8PTTT1/0opxLDRkyhDvuuCN3XmvNrFmzqFu3Lj4+PlSpUoXnn38e\ngO7du1/26sukpCT8/Pz4+GPXbG0v6D2EOlrr+5RSd2mt31VKrQK+L87ARBHt3Qv//je2u+7lhV13\nExkJjle7CuEcTz0Fe/aU7Gc2awZFfHWnr68vNpuNU6dO0bNnTwYOHMjy5cs5cuQIw4YNw2KxXPQO\n5mt54YUXePvtt5k9ezadO3cmNjaW3bt3A/Doo4/yxBNPMGvWLHx8fABYvXo1AQEB9O7du0hlKC4F\n/b5oc/w8r5RqBAQD4cUTkigyreGJJyAkhBmN3iU6WvHGG+D4mxSizNq5cyerVq3ipptu4q233iIy\nMpK33nqLBg0acMcddzBt2jTefPNN0tLS8j1WSkoKc+bMYdq0aTzyyCPUqVOH9u3bM3LkSAD69u2L\nxWJh3bp1ufssXbqUQYMGXfSKTFdS0BrCQqVUOeA/wHogwPG7cEUrVsC2beybspYXX/Hj/vvhnnvM\nDkqUOkX8pl5SvvjiCwICAsjOzsZms3HXXXcxb948HnvsMdq1a4clTztqp06dyMrK4vDhwzRp0uSa\nxz1w4ACZmZncdNNNV1zv4+PDwIEDWbp0KQ888AD79+9n586dF71Ex9UUtJfRYsev3wG1ii8cUWSJ\niTBuHNltOvDwZ3cTEgJvvml2UEKYp3PnzixcuBAvLy8iIyML9O38wovti2rYsGE0adKEkydPsnTp\nUtq3b0+DBg2ccuziUKAmI6XUEaXUe0qpx5RS/yruoEQRvPQSnDvH+71XsmuX0VQUFmZ2UEKYx8/P\njzp16lC9evWLkkGDBg3Yvn07drs9d9kPP/yAt7c3tWvXzve4DRo0wMfHh2+++eaq2/zrX/+ibdu2\nLFq0iJUrV/LII67dMbOg9xAaAguA8sAMR4JYl88+oqQlJMA776AHDmL2uprccAPcf7/ZQQnhmkaO\nHElMTAwjR47k4MGDfP7554wfP55Ro0bh55f/e0ICAwMZM2YMzz//PMuWLePIkSPs3Lnzsu6sjz76\nKK+99hqpqamFfndzSSvoPYQcjBvLOYAdOOeYhCtZtgzS0vjuxon8/F9YuFC6mQpxNZUrV2bTpk08\n99xzNGvWjJCQEAYMGMD//d//FfgYU6dOpVy5crz88sv89ddfREREMGjQoIu26devH6NHj+a+++4j\nMDDQ2cVwqoImhCTgV2A2sEhr/XfxhSSuS04OzJ8PnToxe30dwsLgoYfMDkoIc+V3A7dz587s2LGj\nwPtfOm+xWBg/fjzjx4+/6jHOnz9Peno6Q4cOzS9c0xX0+2N/jHcgjATWKKWmKKWufGtdmGPTJjh6\nlN/7Ps81Ek1oAAAdjElEQVRnn8HIkeDra3ZQQpRdNpuNM2fO8MILL9C8eXM6duxodkj5Kmgvo0+B\nT5VSNwA9gaeAcYBcclzFvHkQGcnc327Dx8dICEII82zbto1u3bpRt25dPvjgA7PDKZCCjmW0FmgK\nHMGoKQwCrl7PEiXr99/hyy85N24my9/wYOBACJfHBoUwVdeuXd1uILyC3kOYCuzWWuc488OVUkuB\nO4BzWutGzjx2mfLWW+DlxRtZI8jMhOdkRFMhxHUo6D2EA8DzSqmFAEqpukqpO/LZpyCWAz2ccJyy\nKzkZli8n+e5BzF8ewN13Q716ZgclhHBHBU0Iy4AsoINj/hTwSlE/XGu9FYgv6nHKtJUrISmJRRUn\ncv48/PvfZgckhHBXBU0ItbXWr+EY5E5rnQY459lucf20hvnzyWreltlrq9O1q7wNTQhx/Qp6DyFL\nKeWL8dpMlFK1gcxiiyoPpdRwYDhAREQEUVFRhdo/JSWl0Pu4qkvLErJnD83272dmzw84tVvx5JO/\nEBXlHhWu0nxe3NnVyhIcHExysnu9AiUnJ8ftYr6W/MqTkZFR9L9DrfU1J4yawCCMge1igfeA40DX\n/PYtyATUAPYVZNuWLVvqwtqyZUuh93FVl5Xlnnt0drkKum6dHN28udZ2uylhXZdSfV7c2NXKcuDA\ngZINxAmSkpLMDsGp8ivPtc4RsEsX4Bqbb5OR42DPAX2BIcBqoJXWuoipSBTJ6dPwySd80GEuhw5b\nmDgRnDRAoxAij8WLFxMSEmJ2GCWioPcQfgZqaa0/11pv0FrHOePDlVKrgZ+A+kqpv5RSrv9st6tY\nuhR7jp1X/7iXhg2hTx+zAxLCtfTr1++q7yo4ePAgSim+/PLLEo7KtRX0HkJb4EGl1AkgFaMZSWut\nr/0GiXxorfsXZf8yKycHFi3i08b/Yf+vPrz3ngxiJ8SlBg0axIABAzh+/Dg1atS4aN2SJUuoXr06\nN998sznBuaiCXkZuA2oD3YHeGA+TueZLQcuCr75CnzjBKyljqFNHhrgW4kpuu+02IiIiWLZs2UXL\nbTYbK1as4JFHHsFisTB27Fjq1auHr68vNWvWZPz48WRmXr3PzMSJE2nWrNlFy67UrPTpp5/SokUL\nrFYrNWvW5D//+Q9ZWVnOK2AxKOhYRieKOxBRCAsW8EVQP34+FsrixeBZ0HqeEE701FOwZ0/Jfmaz\nZgV/c6enpyeDBw9m+fLlTJ48OfdVmZ999hlxcXE8/PDDAAQFBbF8+XIiIyPZv38/I0aMwNfXl8mT\nJ193nBs3bmTQoEG8/vrr3HjjjZw4cYIRI0Zgs9mYNm3adR+3uElDg7uJiUGv/4yXA6ZTrRoMHGh2\nQEK4rqFDh3Ly5Em+/vrr3GVLlizh1ltvpWrVqgBMmjSJDh06UKNGDXr16sX48eNZvXp1kT73lVde\nYfz48QwZMoTatWvTvXt3pk6detnLc1yNfLd0N2+/TZT9Rn6Kqc78+eDtbXZAoqwq6Dd1M9WtW5cu\nXbqwdOlSbr31VmJiYti8eTNr1qzJ3eb999/njTfe4MiRI6SkpJCdnZ1bm7he0dHR7N69m1dffTV3\nmd1uJz09ndjYWMJc9L22khDciEdKCsybxythW6noAS7+elYhXMLQoUN59NFHiY+PZ/ny5YSGhnLX\nXXcBxjuUH3zwQaZMmcKtt95KSEgI69at44UXXrjq8SwWy2WjmNpstovmtdZMmTKFvn37XrZ/aGio\nE0pVPCQhuJHKn3zCj4kN+ZYmzJoFVqvZEQnh+u69916efPJJVq5cydKlSxk0aBBeXl6A8c6C6tWr\nM2HChNztjx8/fs3jhYWFcebMGbTWKMfDP3suuZnSvHlzfv/9d+rUqePcwhQzSQjuIiWFyh98xIMh\n24nwgREjzA5ICPfg6+vLgAEDePHFF0lISLjoVZb16tXj5MmTrF69mjZt2rBp06Z8X2bTrVs3xowZ\nw/Tp07nvvvv49ttvWbdu3UXbTJ48mbvuuouqVaty33334eHhwa+//kp0dLTcVBZO8M47fJDcm+3n\nb2DaNPD3NzsgIdzHsGHDSEhIoEOHDjRo0CB3+d13383TTz/N6NGjadasGVFRUUyZMuWax2rUqBFv\nvvkmb731Fk2aNGHLli2XvVP59ttv57PPPuOrr76idevWtGnThtdee41q1aoVS/mcpiDjW7jKVGbH\nMsrM1EnhtXWE1zndpo3WOTlmB1R0peK8OJSFsshYRuYribGMpMnIHXz+Oa+eG8ZZwvj0DXkqWQhR\nPOTS4gYOvbmZOTzNbbfG0Lat2dEIIUorSQguTp85y+hv+2D11gwfcdzscIQQpZgkBBf32Qs/8QU9\nmPLMeUJDXXscFCGEe5OE4MLS0zRPrWxJQ9+jPPFSRbPDEWWcvuRhLOE6nHVuJCG4sFnPxnDMVpV5\nI/bjeI5GCFN4eXmRnp5udhjiKtLT03MftisKSQguKiYGpi6uwD2WdXSffKPZ4YgyLjw8nFOnTpGW\nliY1BReitSYtLY1Tp04RHh5e5ONJt1MXNXFcFtnZML3vTxByt9nhiDIuKCgIgJiYmMvG7XFVGRkZ\nWEvR+C5XK4+XlxcRERG556goJCG4oJ9/huWrvHiWmdR+Xt5+I1xDUFCQUy46JSUqKormzZubHYbT\nlER5pMnIxWgNzz6rKW9JYELzTdCqldkhCSHKCKkhuJhvv4WoKMU8JhEyepDZ4QghyhCpIbiYadOg\nojWBYeU+hn79zA5HCFGGSEJwIbt2wddfw9OZ07EOfRB8fc0OSQhRhkiTkQuZPh2CvVJ5zLIUxvxs\ndjhCiDJGaggu4tAhWLtWM9L2BkFjHoYqVcwOSQhRxkgNwUVMnw7eysaYwGUwfofZ4QghyiCpIbiA\nI0dg+XLNo/YFRLwwFMqVMzskIUQZJDUEF/DKlGy87Nm8ELEUntxmdjhCiDJKaggmO3QI/rvSwkg9\nn0rL/g/8/MwOSQhRRklCMNmUETFYdTrjhsZDz55mhyOEKMMkIZjop/WxrNpSkVEV3ifizf+YHY4Q\nooyThGCS4/+LpU9fRS11jH+vawelaFRGIYR7koRggqRjf9P7xvNk5Xiw4b8JhHZqaHZIQghhbi8j\npVQP4HXAA1istZ5mZjzF4s8/4ccf4cABOHiQ2H1nufe3Vzio27P5tV+44SEZzVQI4RpMSwhKKQ9g\nPnAL8BfwP6XUeq31AbNiciq7HebMgRdegKwssFjYUakP9/79EbGWEFZMPs5Nz7UwO0psNhuJiYmU\nL18epZTZ4QghTGRmDaENcFhrfRRAKbUGuAtwekKYP3QNv/6Y6uzDXsauPTiVWokjyTWJTS6Psg8m\nx+MRsq1eaIuF9NNWQkKSeGzgKv6ynmPGjMId/8iRI/zvf/+75jZaa5KTkzlz5gyxsbHY7XYA0tLS\niI2NJS4ujuzsbAAyMzNJTEwEoF27dixZsoSGDQvefJWRkcGpU6cIDAx0yuv7hBDmUma9H1UpdS/Q\nQ2s9zDE/EGirtR51yXbDgeEAERERLdesWVOoz0lJSeGdAQl8njzQOYHnowKxNGUvVTnMT2Tz+0Vr\nE4GZwPlijcFisRASEkJISAgeHh4AeHt7ExwcTHBwcO7LuD08PAgODsbDw4MPP/yQtLQ0+vTpQ1hY\nGFprMjIySElJyX2Prtaa1NRUzp49y7lz5zh/3ihHuXLlWLZsGcHBwQWOMSUlhYCAAOcX3gRSFtdU\nmsoCRStPt27dorXW+bZPu3xCyKtVq1Z6165dhfqcqKgoWvyrBRkpGUWKtyCUMkasVgrw9gbHhddZ\ntm7dSufOnfPdzmq15iaCgjp37hxjxozh0oQbEBBAUFAQFovR/yAwMJDq1atTtWpVqlatSkhICM8+\n+yz3338/K1euLPDnRUVF0bVr10LF6KqkLK6pNJUFilYepVSBEoKZTUangKp55qs4ljldUFgQQWHu\n8y7Yq/H19cXf379Yjh0eHs7q1atZsmQJOTk5uZ/n6Zn/n8jff//NlClT6N+/P7169SqW+IQQxc/M\nhPA/oK5SqiZGIngAGGBiPALwu46hM1544QXWrl3LiBEjWLJkCRaLhezsbFJSUkhNTaVu3bq0adMm\nt6lKCOGaTEsIWutspdQoYDNGt9OlWuv9ZsUjrp+3tzdLly6lQ4cO9OjR44rbBAUF0atXLxYsWEBg\nYGAJRyiEKAhTn0PQWm8ENpoZg3CO1q1bc/jwYf766y/AuGEdGBiI1Wplz549bN68mSVLlpCVlcWH\nH35ocrRCiCuR4a+F01SvXp3q1atftrx27drcc8891K9fn7FjxzJ16lQ6dOhgQoRCiGuRoStEiXnm\nmWcYMGAAEydO5LPPPiMjo/h7fgkhCk4SgigxSikWLVpE69atmT17NhUrVmTkyJEkJyebHZoQAmky\nEiXMz8+PH3/8kblz57Jnzx4WLlzI8ePHWb9+fYG6uAohio/UEESJ8/DwoGXLlqxYsYK33nqLTZs2\n8dRTT2HWQ5JCCIN8JROmGj58OIcOHWLmzJn4+vryxBNPUKNGDbPDEqJMkhqCMN306dN56KGHmDlz\nJjVr1qRt27Z8/fXXZoclRJkjCUGYzmKxsGLFCo4cOcL06dOJj4/nlltu4bHHHiM2NpaYmBgOHTqU\nO3KrEKJ4SEIQLqNWrVqMGzeOX375hWeffZaFCxcSHh5O5cqVqVevHrfeeivx8fFmhylEqSX3EITL\n8fX1ZebMmdx///1s2bKFkJAQEhISmDRpEm3btmX9+vU0aNDA7DCFKHUkIQiX1aZNG9q0aZM7f+ON\nN9K3b18aNWpEixYt6Ny5Mw888ACtW7c2MUohSg9pMhJuo2PHjkRHRzNhwgT8/f2ZP38+bdq0oUeP\nHvzwww9mhyeE25OEINxKlSpVeOmll4iKiiI2Npbp06eze/dubrzxRrp168aWLVvkeQYhrpMkBOG2\nAgMDGTduHMeOHWPOnDn8/vvvdO/enY4dO/Lpp59KryQhCkkSgnB7fn5+PPXUUxw9epT58+dz+vRp\n+vTpQ+3atWnSpAm1atWiWbNmPProoyxevJi0tDSzQxbCJUlCEKWG1Wpl5MiRHDp0iFWrVtG4cWNq\n165Np06dqFSpEh9//DGPPvooXbp04cyZM2aHK4TLkV5GotTx9PSkf//+9O/f/6LlWms++eQTHnro\nIdq1a8eGDRto1KiRSVEK4XqkhiDKDKUUd999N9999x2ZmZk0bdqU7t27884777Bnzx4ZhluUeVJD\nEGVOq1atiI6OZsGCBbz//vs8/vjjuevCw8OpXbs2tWrVokOHDvTp04fIyEgToxWi5EhCEGVSZGQk\nU6ZM4cUXX+S3335j//79HD58mMOHD3Ps2DG2bt3Ke++9x6hRo2jatCk+Pj5kZ2eTlZVFRkYGdrud\ntm3b0qNHDzw9PcnOzpb3OQi3J3/BokxTStGgQYMrDoVx4MAB1q5dy/fff49SCk9PT3x8fLBardhs\nNr766itWrVoFwKBBg6hRowZDhgxhzJgxBAQElHRRhCgySQhCXEXDhg1p2LDhVdfb7Xb27NnD6tWr\n8fb2ZteuXUycOJG5c+cyevRoevToQfPmzaXmINyG3FQW4jpZLBZatGhBr169ePXVV9m8eTM7duyg\nRYsWTJo0iTZt2lC+fHkefPBBvvjiC7Kzs80OWYhrkoQghBO1adOGzZs3c/r0adasWcP999/Ppk2b\n6NmzJ1WqVGHUqFFs3bqVtLQ0GWJDuBxJCEIUg4oVK9KvXz8WLVrE6dOnWbt2LR07dmTJkiV06dIF\nf39/rFYr1atX59FHH2Xjxo0kJCRIkhCmksZNIYqZj48Pffv2pW/fvqSkpLBp0yaOHj1KfHw8R48e\n5f3332fx4sWA8VBdxYoV6dmzJwMGDKBz585YLPK9TZQMSQhClKCAgADuu+++i5ZlZmayZcsWDh48\nSGxsLEeOHGHVqlUsWrQIHx8fwsPDc5+PaNCgAY0aNaJt27ZUqVIFpZRJJRGlkSQEIUzm4+NDjx49\n6NGjR+6y1NRUNmzYQHR0NLGxsZw5c4bo6Gg++uij3FFcK1WqRNWqVQkKCiIsLIzmzZvTqlUr2rVr\nh6+vr1nFEW5MEoIQLsjf359+/frRr1+/i5ZnZGSwb98+duzYwc6dOzl79ixJSUn88ccfrF69GoCQ\nkBAGDhzI0KFDady4sTQ5iQKThCCEG7FarbRq1YpWrVrxxBNPXLQuLi6OHTt28N5777FgwQLmzZuH\nv78/jRo1onPnzgwaNEgG8xPXJAlBiFKiQoUK9OrVi169ehEXF8f69evZu3cve/fuZc6cOcyYMYMW\nLVrQsWNHGjVqRM2aNQkMDCQgIACr1YqPjw9JSUnY7XapVZRRkhCEKIUqVKjAI488kjsfGxvLqlWr\n+OCDD1i2bBkpKSlX3ddisVC+fHmsVisWiwVvb2/CwsIIDw+nXr16tG3blnbt2smgf6WQKQlBKXUf\n8CLQAGijtd5lRhxClBVhYWGMGTOGMWPGYLfbOXnyJH/++SepqamkpKSQkZFBRkYGe/bsoVy5csTF\nxZGZmUlOTg4ZGRnExcVx6NAhPv/8c2w2G2C837pt27Y0b96cevXqUb9+fRo1aiS1CzdmVg1hH9AX\nWGDS5wtRZlksFmrUqEGNGjUuWxcVFUXXrl2vuu+FpLF9+3Z27NjBjh07WLt2be76yMhI7r33Xjp1\n6oTWmpycHMqVK0elSpWoWLEiYWFhkjBcmCkJQWt9EJA+1EK4GavVSrt27WjXrl3ustTUVA4fPsze\nvXtZt24dCxYs4I033rji/l5eXlSqVIkqVapQtWpVKleuTFBQEAEBAURERFCrVi2qVauGn59f7siy\nHh4eJVW8Mk+Z+ai8UioKGHutJiOl1HBgOEBERETLNWvWFOozUlJSSs1QxFIW1yRluVhaWhpnzpzB\nYrGglCI5OZn4+Hj+/vtv4uLiiI2NJTY2lnPnzhEfH09GRsY1j+fr64u/vz/e3t65Q5AHBQURGBhI\nhQoViIiIoFKlSlSrVo3IyMjcBFKazgsUrTzdunWL1lq3ym+7YqshKKW+BipeYdUErfWnBT2O1noh\nsBCgVatW+lrV2SvJrwrsTqQsrknKUjQ5OTmkpqZy+vRpjh49ysmTJ8nIyCArK4vU1FQSExNJSkoi\nMzMTm81GamoqCQkJxMTEsHPnTtLS0nKP5e3tTc2aNalVqxaenp7Uq1cPPz8/goKCCA0NpXz58lSr\nVo2aNWsSEhJSouUsqpI4N8WWELTWNxfXsYUQpYeHhwdBQUEEBQVRv379Qu2rtebvv//myJEjHDx4\nkIMHD3LkyBGOHj3KsWPH+Oabby5KGHn5+fkRGhqaO5UrV46QkBACAwMJCgoiPDycypUrU7FiRcqX\nL0/58uUJDQ0t1fdApNupEMJtKaWoUKECFSpUoG3bthetu/CNWmtNSkoK8fHxxMbGcuLECY4dO8aZ\nM2dym7ISEhL4448/OH/+PMnJyaSkpOQOEZKXp6cnkZGRVKpUicDAQPz9/S+aAgMDCQwMxM/PD09P\nTzw9PbFarfj6+uY+6+Hj44O3tzdeXl54e3vj5+eXO/n7+5uacMzqdno3MA8IAz5XSu3RWt9mRixC\niNJNKZV7oa5evTqtWuXblI7dbicuLo5Tp05dlDhOnz6duywlJYVz586Rmpqa2303NTW1yEOYW61W\nvLy8Lnplq9Vq5fHHH3ffJqNr0VqvA9aZ8dlCCJEfi8WSO8psYdjtdlJTU0lPTycnJwebzUZGRgbp\n6elkZGSQmZlJZmYmWVlZ2Gw2srKySE9PJy0tLTexpKamkp2dfdH6jIwM/Pz8iqm0/5AmIyGEcBKL\nxZJbG3G2qKgopx/zUqX37ogQQohCkYQghBACkIQghBDCQRKCEEIIQBKCEEIIB0kIQgghAEkIQggh\nHCQhCCGEAEwe/rqwlFKxwIlC7lYBiCuGcMwgZXFNUhbXVJrKAkUrT3WtdVh+G7lVQrgeSqldBRkH\n3B1IWVyTlMU1laayQMmUR5qMhBBCAJIQhBBCOJSFhLDQ7ACcSMrimqQsrqk0lQVKoDyl/h6CEEKI\ngikLNQQhhBAFIAlBCCEEUIoTglKqh1Lqd6XUYaXUeLPjKQylVFWl1Bal1AGl1H6l1BjH8lCl1FdK\nqUOOn+XMjrWglFIeSqndSqkNjvmaSqkdjvPzvlLK2+wYC0opFaKU+kgp9ZtS6qBSqr27nhul1NOO\nv7F9SqnVSimru5wbpdRSpdQ5pdS+PMuueB6U4Q1HmX5RSrUwL/LLXaUsMxx/Y78opdYppULyrHve\nUZbflVJOe/1wqUwISikPYD7QE2gI9FdKNTQ3qkLJBp7VWjcE2gFPOOIfD3yjta4LfOOYdxdjgIN5\n5qcDc7TWdYAEYKgpUV2f14EvtNY3AE0xyuV250YpVRkYDbTSWjcCPIAHcJ9zsxzoccmyq52HnkBd\nxzQceLuEYiyo5Vxelq+ARlrrJsAfwPMAjmvBA8C/HPu85bjmFVmpTAhAG+Cw1vqo1joLWAPcZXJM\nBaa1Pq21/tnxezLGBacyRhnedWz2LtDHnAgLRylVBegFLHbMK6A78JFjE3cqSzDQGVgCoLXO0lqf\nx03PDcZrdH2VUp6AH3AaNzk3WuutQPwli692Hu4C/qsN24EQpVSlkok0f1cqi9b6S611tmN2O1DF\n8ftdwBqtdabW+hhwGOOaV2SlNSFUBv7MM/+XY5nbUUrVAJoDO4AIrfVpx6ozQIRJYRXWXGAcYHfM\nlwfO5/ljd6fzUxOIBZY5msAWK6X8ccNzo7U+BcwETmIkgkQgGvc9N3D18+Du14RHgE2O34utLKU1\nIZQKSqkAYC3wlNY6Ke86bfQXdvk+w0qpO4BzWutos2NxEk+gBfC21ro5kMolzUNudG7KYXzbrAlE\nAv5c3mzhttzlPORHKTUBoxn5veL+rNKaEE4BVfPMV3EscxtKKS+MZPCe1vpjx+KzF6q5jp/nzIqv\nEDoCdyqljmM03XXHaIMPcTRTgHudn7+Av7TWOxzzH2EkCHc8NzcDx7TWsVprG/Axxvly13MDVz8P\nbnlNUEoNAe4AHtT/PDRWbGUprQnhf0BdR28Jb4wbMOtNjqnAHG3sS4CDWuvZeVatBwY7fh8MfFrS\nsRWW1vp5rXUVrXUNjPPwrdb6QWALcK9jM7coC4DW+gzwp1KqvmPRTcAB3PDcYDQVtVNK+Tn+5i6U\nxS3PjcPVzsN6YJCjt1E7IDFP05JLUkr1wGhqvVNrnZZn1XrgAaWUj1KqJsaN8p1O+VCtdamcgNsx\n7swfASaYHU8hY++EUdX9BdjjmG7HaHv/BjgEfA2Emh1rIcvVFdjg+L2W44/4MPAh4GN2fIUoRzNg\nl+P8fAKUc9dzA0wBfgP2ASsAH3c5N8BqjHsfNoya29CrnQdAYfQ8PAL8itGzyvQy5FOWwxj3Ci5c\nA97Js/0ER1l+B3o6Kw4ZukIIIQRQepuMhBBCFJIkBCGEEIAkBCGEEA6SEIQQQgCSEIQQQjhIQhCi\ngJRSTyml/MyOQ4jiIt1OhSggx9PWrbTWcWbHIkRxkBqCEFeglPJXSn2ulNrreFfAZIzxfrYopbY4\ntrlVKfWTUupnpdSHjrGnUEodV0q9ppT6VSm1UylVx7H8Psex9iqltppXOiGuTBKCEFfWA4jRWjfV\nxrsC5gIxQDetdTelVAVgInCz1roFxpPLz+TZP1Fr3Rh407EvwCTgNq11U+DOkiqIEAUlCUGIK/sV\nuEUpNV0pdaPWOvGS9e0wXr60TSm1B2PcnOp51q/O87O94/dtwHKl1KMYL6MRwqV45r+JEGWP1voP\nx2sWbwdeUUp9c8kmCvhKa93/aoe49Het9WNKqbYYLwuKVkq11Fr/7ezYhbheUkMQ4gqUUpFAmtZ6\nJTADY4jrZCDQscl2oGOe+wP+Sql6eQ7RL8/Pnxzb1NZa79BaT8J4yU7eIYyFMJ3UEIS4ssbADKWU\nHWMEyscxmn6+UErFOO4jDAFWK6V8HPtMxBhhF6CcUuoXIBO4UIuYoZSqi1G7+AbYWzJFEaJgpNup\nEE4m3VOFu5ImIyGEEIDUEIQQQjhIDUEIIQQgCUEIIYSDJAQhhBCAJAQhhBAOkhCEEEIA8P/o3Oog\nOzaV+AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fb3cc058eb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 22min 29s, sys: 1.63 s, total: 22min 31s\n",
      "Wall time: 22min 35s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# == IMPORTANT PARAMETER ==\n",
    "games_num = 500\n",
    "# =========================\n",
    "steps = 120\n",
    "discount = 0.95\n",
    "states = 50\n",
    "actions = 2\n",
    "random_player = MarkovPlayer(states=states, actions=actions, steps=steps,\n",
    "                           strategy_class=RandomStrategy, discount=discount)\n",
    "policy_iteration_player = MarkovPlayer(states=states, actions=actions, steps=steps,\n",
    "                           strategy_class=PolicyIterationStrategy, discount=discount)\n",
    "value_iteration_player = MarkovPlayer(states=states, actions=actions, steps=steps,\n",
    "                           strategy_class=ValueIterationStrategy, discount=discount)\n",
    "plt.hold(True)\n",
    "random_player.evaluate(games=games_num, hold=True, color='k', label='random')\n",
    "print('===')\n",
    "policy_iteration_player.evaluate(games=games_num, hold=True, color='r', label='Policy')\n",
    "print('===')\n",
    "value_iteration_player.evaluate(games=games_num, hold=True, color='b', label='Value')\n",
    "plt.xlabel('steps')\n",
    "plt.ylabel('reward')\n",
    "plt.grid(True)\n",
    "plt.legend(loc='right', fontsize=14)\n",
    "plt.hold(False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "Как можно заметить, оба алгоритма показывают схожие результаты (что логично, так как они должны находить одну оптимальную стратегию), однако **value iteration strategy** сходится всё же гораздо быстрее.\n",
    "\n",
    "### Теперь посмотрим, как зависит значение $V_\\gamma^*(s)$ для пяти произвольных состояний:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel/__main__.py:11: MatplotlibDeprecationWarning: pyplot.hold is deprecated.\n",
      "    Future behavior will be consistent with the long-time default:\n",
      "    plot commands add elements without first clearing the\n",
      "    Axes and/or Figure.\n",
      "/usr/local/lib/python3.5/dist-packages/matplotlib/__init__.py:917: UserWarning: axes.hold is deprecated. Please remove it from your matplotlibrc and/or style files.\n",
      "  warnings.warn(self.msg_depr_set % key)\n",
      "/usr/local/lib/python3.5/dist-packages/matplotlib/rcsetup.py:152: UserWarning: axes.hold is deprecated, will be removed in 3.0\n",
      "  warnings.warn(\"axes.hold is deprecated, will be removed in 3.0\")\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel/__main__.py:18: MatplotlibDeprecationWarning: pyplot.hold is deprecated.\n",
      "    Future behavior will be consistent with the long-time default:\n",
      "    plot commands add elements without first clearing the\n",
      "    Axes and/or Figure.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEKCAYAAAAFJbKyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VNX5wPHvG8i+k0BYEhZB2TFCyiKiUBStu9UKLsUF\npaLoz9oiWm1dqq1W61YVRevaqrWuqIgiagNEkMVg2GUnyWQhkJB9mTm/P+4kmWwwgWRmknk/z3Of\ne+feM3PfO+i8Oefcc64YY1BKKaXcFeDtAJRSSnUsmjiUUkq1iiYOpZRSraKJQymlVKto4lBKKdUq\nmjiUUkq1iiYOpZRSraKJQymlVKto4lBKKdUqXb0dQHuIj483/fv393YYSinVYaxbt+6AMaa7O2U7\nZeLo378/a9eu9XYYSinVYYjIXnfLalOVUkqpVtHEoZRSqlU0cSillGqVTtnH0Zzq6moyMzOpqKjw\ndiidQkhICImJiQQGBno7FKWUh/lN4sjMzCQyMpL+/fsjIt4Op0MzxlBQUEBmZiYDBgzwdjhKKQ/z\nm6aqiooK4uLiNGm0AREhLi5Oa29K+Sm/SRyAJo02pN+lUv7LrxKHUkp1dAvWLOCtjLe8GoMmDi97\n6qmnKCsra7Nyzdm2bRvXXHMNDoeDCRMmNDj217/+lUGDBjF48GC++OKLY/p8pZTn/OP7f/D+lve9\nGoMmDi/zROJYvnw5p59+OhkZGYwYMaJu/+bNm3nnnXfYtGkTS5Ys4eabb8Zutx/TOZRSnmErsdEr\nopdXY/Cbu6q8rbS0lMsvv5zMzEzsdjt//OMfyc3NJTs7mylTphAfH88333zDnDlzWLNmDeXl5Vx2\n2WU88MADPPPMM03Kffnll9x3331UVlYycOBAXn31VSIiIhqcc/ny5dx6663s27ePhIQEiouLCQgI\nICUlhbVr1/Lxxx8zY8YMgoODGTBgAIMGDeL7779vUitRSvmG8upyCisKNXF4w+233056enqbfmZy\ncjJPPfVUi8eXLFlC7969+eyzzwAoKioiOjqaJ554gm+++Yb4+HgAHn74Ybp164bdbmfq1Kn8+OOP\n3HbbbQ3KHThwgIceeoivvvqK8PBwHn30UZ544gn+9Kc/NTjnpEmTSE9PZ8KECaSlpXH99dfz+9//\nnuHDhwOQlZXF+PHj68onJiaSlZXVpt+LUqrt5JTkANAr0ruJQ5uqPGTkyJEsXbqU+fPns3z5cqKj\no5st9+677zJ69GhOOeUUNm3axObNm5uUWbVqFZs3b2bixIkkJyfz+uuvs3dv8/OTlZWVERwcjIjw\n008/MXjw4Da9LqWU59QlDq1xeN6Ragbt5aSTTmL9+vUsXryYe++9l6lTpzapIezevZvHH3+cNWvW\nEBsby7XXXtvsWAljDGeddRZvv/32Ec954YUXsnXrVgoLCxk1ahR79uwhJSWFu+++m+nTp9OnTx/2\n799fVz4zM5M+ffq0zQUrpdqcrcQGaI3Db2RnZxMWFsbVV1/NvHnzWL9+PQCRkZEUFxcDcPjwYcLD\nw4mOjiY3N5fPP/+87v2u5caPH8/KlSvZsWMHYPWfbN++vck5Fy1axI033siCBQt45plnuOmmm0hP\nT2f69OmAlVjeeecdKisr2b17Nz/99BNjx45t1+9BKXXsbMXOxKE1Dv+QkZHBvHnzCAgIIDAwkAUL\nFgAwe/ZszjnnHHr37s0333zDKaecwpAhQ0hKSmLixIl1729c7rXXXuOKK66gsrISgIceeoiTTjqp\nyXlTU1OZOXMmCxcu5IwzzmhwbPjw4Vx++eUMGzaMrl278txzz9GlS5d2/BaUUsfDVmKji3She7hb\nz1tqN2KM8WoA7SElJcU0fpDTli1bGDp0qJci6pz0O1XKs2Z9PIslO5eQdUfb38QiIuuMMSnulNWm\nKqWU6iB8YQwHaOJQSqkOw1Zio2dET2+HoYlDKaU6Clux1jiUUkq5qcZRQ15pntdvxQUPJA4ReUVE\n8kRko8u+x0Rkq4j8KCIfikhMC+/dIyIZIpIuImubK6OUUv4grzQPg/GbGsdrwDmN9i0FRhhjRgHb\ngbuP8P4pxphkd3v7lVKqM6obw+EPNQ5jTCpwsNG+L40xNc6Xq4DE9o7DV3lzWvWlS5cyZswYRo4c\nyZgxY/j666/rjq1bt46RI0cyaNAgbrvtNjrjbdtKdSR1o8b9pMZxNNcDn7dwzABfisg6EZntwZg8\nxpvTqsfHx/PJJ5+QkZHB66+/zq9//eu6Y3PmzOGll17ip59+4qeffmLJkiXHdG6lVNvwpRqHV0eO\ni8g9QA3w7xaKnGaMyRKRHsBSEdnqrME091mzgdkAffv2bZd4j4cvTqt+yimn1JUdPnw45eXlVFZW\ncvDgQQ4fPlw3c+7MmTP56KOP+MUvfuHR70wpVa92gkNfuB3Xa4lDRK4FzgemmhbaQYwxWc51noh8\nCIwFmk0cxpiFwEKwRo4f8eS33w5tPK06ycnQwaZVd/X+++8zevRogoODycrKIjGxvvVQp1tXyvts\nJTbiQuMI6hLk7VC801QlIucAdwIXGmOabX8RkXARiazdBqYBG5sr2xH48rTqmzZtYv78+bz44ovH\nd5FKqXZjK7H5RDMVeKDGISJvA5OBeBHJBO7DuosqGKv5CWCVMeYmEekNvGyMORdIAD50Hu8KvGWM\naZuGdp1WvW6G3MzMTC655BLeeOMNBg4cCECfPn3IzMys+xydbl0p7/OVwX/ggcRhjLmimd3/bKFs\nNnCuc3sXcHI7huZR2dnZdOvWjauvvpqYmBhefvlloH669Pj4+GanVZ88eXKTcuPHj+eWW25hx44d\nDBo0iNLSUrKysprMjrto0SIee+wxTjjhBOLi4li8eDF/+9vf6o4XFhZy3nnn8cgjjzSYibdXr15E\nRUWxatUqxo0bxxtvvMGtt97a/l+SUqpFthIbg+N940FsvnBXlV/IyMhg7NixJCcn88ADD3DvvfcC\n9dOlT5kyhZNPPrluWvUrr7yy2WnVp0yZQvfu3eumVR81ahQTJkxg69atzZ43NTWV0047jeXLlzeZ\nVv3ZZ59lx44dPPjggyQnJ5OcnExeXh4Azz//PDfccAODBg1i4MCB2jGulBcZY8gpyaFnuPc7xkGn\nVVfHQb9TpTyjoKyA+MfiefLsJ7l9/O3tcg6dVl0ppToRXxr8B5o4lFLK5zUY/LdxI+zb59V49NGx\nSinl4xrUOC6ZDr17w6efei0erXEopZSPa1DjyM62EocXaeJQSikfZyuxEREUQURACOTlQS/v9nVo\n4lBKKR9X96zx3FwwRmsc/s6b06p///33deM3Tj75ZD788MO68rX7k5OTiYqK4ikvjLZXSllySnLq\nm6lAE4e/8+a06iNGjGDt2rWkp6ezZMkSfvOb31BTU8PgwYNJT08nPT2ddevWERYWxiWXXHJM51ZK\nHb+66UZsVl+Ht5uq9K4qD/HFadXDwsLqylZUVOCcF6yBZcuWMXDgQPr169fu35FSqnl1TVV7fKPG\n4ZeJ4/Ylt5Oe07bTqif3TOapczretOqrV6/m+uuvZ+/evbz55pt07drwP4l33nmHK65obroxpZQn\nlFSVUFJVUt9UJQI9eng1Jm2q8hBfnVZ93LhxbNq0iTVr1vDXv/61wWy8VVVVLFq0iF/96lfHceVK\nqeNRdytubVNVQgJ09e7f/H5Z4zhSzaC9+Oq06rWGDh1KREQEGzduJCXFmq7m888/Z/To0SQkJBzn\n1SuljlXt4L+eET19YgwHaI3DY7KzswkLC+Pqq69m3rx5rF+/HqifLh1odlr1Wq7lxo8fz8qVK9mx\nYwdg9Z9s3769yTkXLVrEjTfeyIIFC3jmmWe46aabSE9Pr0sau3fvpqamBoC9e/eydetW+vfvX/f+\nt99+W5uplPKyBoP/bDavd4yDn9Y4vCEjI4N58+YREBBAYGAgCxYsAOqnS+/duzfffPNN3bTqSUlJ\nzU6rXluudlr1yspKAB566KEmz+MAa1r1mTNnsnDhwibTqq9YsYJHHnmEwMBAAgICeP755+v6WkpL\nS1m6dKk+FVApL2sw3Uh2NkWjJhHpgAAv/tmv06qrY6bfqVLtb/7S+Ty1+ikq5pcgwcH8rNd+4kf1\nwaVBok3otOpKKdVJ2Eps9IzoieTlgTHYyqLp6eXnOWniUEopH1Y3hiM7GwdCbnGYfyQOEXlFRPJE\nZKPLvm4islREfnKuY1t47zXOMj+JyDWeiFcppXyFrdhWN4bjAPHU2AO83j/uqRrHa8A5jfbdBSwz\nxpwILHO+bkBEugH3AeOAscB9LSUYpZTqjOpqHDYbNqyM4ReJwxiTChxstPsi4HXn9uvAxc289Wxg\nqTHmoDHmELCUpglIKaU6pcqaSg6WH6xrqrJJH8BPEkcLEowxzhm7yAGaG2XWB9jv8jrTua8JEZkt\nImtFZG1+fn7bRqqUUl6QW5oL1I/hyIk80Xrtx4mjjrHuCT6u+4KNMQuNMSnGmJTu3bu3UWTtz5vT\nqtfat28fERERPP744wDs37+fKVOmMGzYMIYPH87TTz99TOdVSh2fBtONZGdji9DEkSsivQCc67xm\nymQBSS6vE537Og1vTqte64477uAXv/hF3euuXbvy97//nc2bN7Nq1Sqee+65ZufMUkq1r7rBf87O\ncVtQX6KiwGVia6/w5sjxRcA1wCPO9cfNlPkC+ItLh/g04G7PhNe2fHFadYCPPvqIAQMGEB4eXve+\nXr160cv5J01kZCRDhw4lKyuLYcOGee4LU0o1meDQFtbH67UN8FDiEJG3gclAvIhkYt0p9QjwrojM\nAvYClzvLpgA3GWNuMMYcFJE/A2ucH/WgMaZxJ3ur3X47pLftrOokJ8ORHpLni9Oql5SU8Oijj7J0\n6dK6ZqrG9uzZww8//MC4cePa4FtSSrWGrcSGIHQPioG8PGyJ3enlA4/G8UjiMMa0NFPe1GbKrgVu\ncHn9CvBKO4XmMSNHjuR3v/sd8+fP5/zzz2fSpEnNlnv33XdZuHAhNTU12Gw2Nm/ezKhRoxqUcZ1W\nHazpz5vru4AjT6t+//3389vf/rZJTaVWSUkJl156KU899RRRUVHHctlKqeNgK7bRI7wHXfMLrFHj\n5dGM85cah6/xxuOzfXFa9dWrV/Pee+9x5513UlhYSEBAACEhIcydO5fq6mouvfRSrrrqKn75y1+2\n6XehlHKPrcRWd0eVAWyHI7w+ahz8NHF4Q3Z2Nt26dePqq68mJiaGl19+GaifLj0+Pr7ZadUnT57c\npNz48eO55ZZb2LFjB4MGDaK0tJSsrKwms+MuWrSIxx57jBNOOIG4uDgWL17M3/72t7rjy5cvr9u+\n//77iYiIYO7cuRhjmDVrFkOHDuWOO+5o/y9HKdUs1+lGDhNFeVUX/+njUL45rXpLVq5cyZtvvsnI\nkSNJTk4G4C9/+Qvnnnvu8X4NSqlWsBXbSE5Itu6o8pFR46DTqqvjoN+pUu3H7rAT9FAQd592Nw99\nG8C3D61givmar76CqU16h4+fTquulFIdXH5ZPg7jqB/8F2Xd2OILNQ5NHEop5YMaPzLWV0aNgyYO\npZTySTklOYDLdCPB/QkOhpgYLweGJg6llPJJTaYbCehNr15w6aW/5JZbbvFqbJo4lFLKB9U2VfUM\njoP8fGz2HvTqBevWraO4uNirsWniUEopH2QrsREbEktIQZE1arwilp49DVlZWSQmJno1Nk0cXuaL\n06oDPP3004wYMYLhw4fzlDeG2ivl5+pGjWdnW68PhxMVVYbdbicpKeko725fmji8zBenVd+4cSMv\nvfQS33//PRs2bODTTz9lx44dx3RupdSxsRXb6BnRE2w2KgimsDSI4GBrjldv1zh05LiHdKRp1bds\n2cK4ceMIc076f8YZZ/DBBx9w5513eu4LU8rP2UpsTEyaCNnZ5GBNUNWli/XYIm/XOPwycfx0+0+U\npJe06WdGJEdw4lMntni8I02rPmLECO655x4KCgoIDQ1l8eLFpKS4NaBUKdUGjDHklORYt+Km26xn\njRuw2zMBrXH4jY40rfrQoUOZP38+06ZNIzw8nOTkZLp06XI8l6+UaoVdh3ZRUVPBoG6DIHsNtugh\nUAhlZTsJCQkhLi7Oq/H5ZeI4Us2gvXS0adVnzZrFrFmzAPjDH/7g9b9wlPInafvTADg16VTI/hhb\n5OlQCIcPbyMxMRER8Wp8fpk4vKEjTasOkJeXR48ePdi3bx8ffPABq1ataudvSClVK21/GlHBUQzv\nMdyabiR4AAEBkJ+/2ev9G6CJw2M60rTqAJdeeikFBQUEBgby3HPPEeML8xwo5SfSMtOYkDiBAAmw\nRo3H9iEhAbKy9rXq/+P2otOqq2Om36lSba+ooojYR2O5f/L9/OnUuyEoiHMHbSc3ahAbNgQyf/58\nHn744TY/b4eYVl1EBotIustyWERub1RmsogUuZT5U0ufp5RSncHqrNUYjNW/kZsLgK0ylpiYSux2\nu0/0N3qtqcoYsw1IBhCRLkAW8GEzRZcbY873ZGxKKeUtafvTCJAAxvYZCxu2ApBTEkFS+GHA+2M4\nwHdGjk8Fdhpj9rbnSTpjs5y36HepVPtI25/GyB4jiQqOApsNOwHkFQUTFFQAeH8MB/hO4pgBtHRv\n6QQR2SAin4vI8JY+QERmi8haEVmbn5/f5HhISAgFBQX6g9cGjDEUFBQQEhLi7VCU6lTsDjurMldZ\nzVQA2dnk0QOHQwBrtlxfqHF4/a4qEQkCLgTububweqCfMaZERM4FPgKaHYRhjFkILASrc7zx8cTE\nRDIzM2kuqajWCwkJ8Ym/fJTqTDblb6K4qrhB4qgdNV5dvY+QkBC6devm3SDxgcQB/AJYb4zJbXzA\nGHPYZXuxiDwvIvHGmAOtPUlgYCADBgw4zlCVUqr9NBj4B9YYjpihcAhKS3eQlJTk9cF/4BtNVVfQ\nQjOViPQU57ckImOx4i3wYGxKKeUxafvT6BnRkwExzj9ys7OxRVnjsw4d2uwztXyv1jhEJBw4C/iN\ny76bAIwxLwCXAXNEpAYoB2YY7aRQSnVSafvTODXp1PpaRXY2tmArieTl/ciIEROP8G7P8WriMMaU\nAnGN9r3gsv0s8Kyn41JKKU/LLcll56GdzEmZU7/TZsMWn0i3bgabbQ+JidO9FV4DvtBUpZRSfq9J\n/0Z1NeTlYXMk0L17jU88+a+WJg6llPIBafvTCOoSxOheo60dzlHjOZXdiIy0nv7pK30cmjiUUsoH\npGWmkdI7heCuwdaO2meNl0YSGloI+MYYDtDEoZRSXldZU8na7LWcmnhq/c7sbAxgOxRC167W+DOt\ncSillAJgvW09Vfaq+v4NAJuNQ8RSVR2Aw5FFaGjoEQf/FaYWkvNGjgei1cShlFJeV9sxPiHJ5RHQ\ntaPGgcrKPUd98l/uv3LZNX9Xu8ZZSxOHUkp5WVpmGifEnkDPiJ71O7OzscUOA6xHxh6tf8PUGCTQ\nM6PKNXEopZQXGWNYuW8lE5MaDe6z2bBFDQbg4MFNR+3fMDUG6aqJQymlOr3dhbvJLc1t2L8BVo0j\npHbU+Iaj1ziqNXEopZRfaDLwr5bNhq1rEmFhDhyOIvdqHNpUpZRSnV/a/jQigyIZ3t3lcUNlZZCf\nTw496datEjj6GA5HtcM3axwiEu58zKtSSqk2kLY/jfGJ4+kS4PLTuno1GINNehERUQocfQyHz/Rx\niEiAiFwpIp+JSB6wFbCJyGYReUxEBnkkSqWU6oQOVx4mIy+jaTNVaiqIYCuPITj4IHD0GocvNVV9\nAwzEejpfT2NMkjGmB3AasAp4VESubucYlVKqU/py55c4jINJfSc1PJCaCsnJ2HK7EBCQQ2hoKLGx\nsUf8LE92jh9tWvUzjTHVjXcaYw4C7wPvi0hgu0SmlFKd3IvrXqRvdF8m959cv7OqCr77jtLr5lL8\nA9TU7HfryX8+01RVmzREZKCIBDu3J4vIbSIS41pGKaWU+34q+Imvdn3F7NGzG/ZvrFsH5eXYhk0F\noKxsl1tzVJkaQ0CgZ+53cvcs7wN2Z5/GQiAJeKvdolJKqU7uxXUv0jWgK7NGz2p4IDUVAFvfcQAU\nFW11L3H44DgOhzGmBrgE+IcxZh7Qq/3CUkqpzqu8upxX01/lkiGXNJxmBGD5chg6FFt5DGCNGndn\nOnVf6hyvVS0iVwDXAJ8692nfhlJKHYP3Nr/HwfKD3JRyU8MDdjusWAGTJmGzWbscjky3m6p8rcZx\nHTABeNgYs1tEBgBvtkUAIrJHRDJEJF1E1jZzXETkGRHZISI/isjotjivUkp5y4K1Czgp7iSm9J/S\n8EBGBhQVwemnk5MDXbs6gAL3ahw+dFcVAMaYzcBtLq93A4+2YRxTjDEHWjj2C+BE5zIOWOBcK6VU\nh7MhZwPfZX7HE9OeaHqnlLN/g9NPx7YUoqMrKChw7wFOPtNUJSKfiMgFzd1yKyIniMiDInJ9+4UH\nwEXAG8ayCogREe1fUUp1SC+sfYGQriFck3xN04OpqdC/PyQlYbNBWNhhwL1HxvrSlCM3ApOArSKy\nRkQWi8jXIrILeBFYZ4x55ThjMMCXIrJORGY3c7wPsN/ldaZzXwMiMltE1orI2vz8/OMMSSml2l5x\nZTH/yvgX04dPp1too6f5GWMljtNPB8Bmg8DAA24N/gPP9nEcsanKGJMD3AncKSL9se6kKge2G2PK\n2iiG04wxWSLSA1gqIluNMamt/RBjzEKsW4VJSUkxbRSbUkq1mX9n/JuSqhLmpMxpenD7dsjPh0nW\nKHKbDaKibG4N/gMfaqoCEJEznZsDjTHfGWPS2zBpYIzJcq7zgA+BsY2KZGGNG6mV6NynlFIdhjGG\nBWsXkNwzmbF9Gv/M0aB/o6wMDhyAqqo9bvVvgO+N4zhDRCYCk9v65M7ZdiNrt4FpwMZGxRYBM513\nV40HiowxtraORSml2tOqzFX8mPsjc1LmNF+DSE2FhAQ48UTWrLF2lZevcKt/A3xo5LiI3AcEA18B\nQSLypzY+fwKwQkQ2AN8DnxljlojITSJSe4PzYmAXsAN4Cbi5jWNQSql2t2DtAiKDIrly5JXNF6jt\n3xAhzXq2EwcPLna/xuFDfRwPOO+auh/IM8a82pYnN8bsAk5uZv8LLtsGuKUtz6uUUp5UUFbAu5ve\nZdYps4gIimhaYO9e2LcP5s0DIC0NBg2qZseOA+7XOHylqUpEngP6GGMeBZq5WqWUUkfz9OqnqbRX\nNh0pXsulf8MYK3EMGXIIcG8MB/hW5/h24FwR2QMkicgp7R+SUkp1HpvzN/PIike4YsQVjEwY2Xyh\n1FSIiYERI9i+HQ4ehMTEvYB7YziMw4DBN2ocxpinjTETgDOAAuAVEdkqIveJyEkeiVAppTooh3Fw\nw6IbiAqO4ulznm654PLlcNppEBBQ178RFbUJcHPUeLU1AsEnEkctY8xeY8yjxphTgCuAi4Et7RqZ\nUkp1cAvWLOC7zO948uwn6R7evflCubmwbVvdwL+0NIiNhaqqDMLCwtwe/Af4TFMVACLS1Tn1yL+B\nz4FtwC/bNTKllOrA9hXt465ldzFt4DSuHnWEJ2wvX26tXRLHhAmQlbWfxMREtwb/OaodgI/UOETk\nLBF5BWuajxuBz7AGAs4wxnzsiQCVUqqjMcYw57M5OIyDF8574cg//qmpEBYGo0dTWAibN8Opp8L+\n/ftbNYYDfCRxAHcDacBQY8yFxpi3jDGlHohLKaU6rP9s+g+Lf1rMQ1MeYkDsgCMXTk21MkVgIKtW\nWbtOPRUyM917Dgf4WFOVMebnxpiXjTGHPBKNUkp1cAVlBdz2+W38rPfPuG3cbUcuXFgIP/5YNz9V\nWhp06QIjR1aQnZ3dqjEc4Lkah1vP41BKKeWe3335Ow5VHOKrC7+iS0CXIxf+4ANrVtyf/xywEsfJ\nJ8P333+Nw+Fg4sSJbp2ztsbhE1OOKKWUct+XO7/k9Q2vM3/ifEYljDpyYYcDHn/cyhQTJ1JTA6tX\nW81Un3zyCeHh4UyePNmt8/paH4dSSik3bMrbxBXvX8GQ+CHce/q9R3/D55/Dli3w+9+DCBs3QkkJ\nTJhg+PTTTznrrLMICQlx69w+OY5DKaVUy/YU7mHav6YR1CWIz678jJCubvzgP/449OkD06cD1A38\ni4nZTGZmJhdccIHb5/d057j2cSil1HHILcnlrDfPoqy6jNRrUzkh9oSjv2ntWvj2W3jsMQi0nsyd\nlga9e8PatR8gIpx33nlux+DppipNHEopdYyKKoo459/nkHU4i69mftXyXFSN/f3vEBkJN95Ytyst\nzerf+PTTTxg7diwJCQlux6FNVUop1QGUV5dzwdsXsClvEx9M/4BTk05174179sB//wu/+Q1ERwPW\nY2J374YRIw6zZs2aVjVTgTZVKaWUz6u2V3P5e5ezYt8K3rr0Lc4ZdI77b376aRCB2+rHeHz3nbWu\nqvoWgPPPP79V8fjUlCNKKaUaqqipYOZHM/l0+6c8d+5zzBgxw/03HzoEL70EM2aAy+C+tDQIDoaM\njDdISkpi1Kij3MrbiN6Oq5RSPmp/0X5Of/V03tn4Do9MfYQ5P5vTug948UUoLbVuwXWRlgZjxjhY\ntuxzLrjgArcmNnTlU1OOKKWUsny751vGLBzD1gNb+XD6h8w/bX7rPqCyEp55Bs46yxr051RRAevW\nQa9euykrK2t1/wb4Uee4iCSJyDcisllENonI/zVTZrKIFIlIunP5kzdiVUr5L2MMT373JGe+cSbd\nQrvx/Y3fc/GQi1v/QW+/bfWCN6ptrF8PVVVQWrq0VaPFG8To4SlHvNk5XgP8zhizXkQigXUistQY\ns7lRueXGmNb1FCmlVBsoqy7jxk9u5K2Mt7h4yMW8fvHrRAVHtf6DjLEG/I0aZdU4XNQO/Pvxxxdb\nNVq8wcf7yzgOY4wNsDm3i0VkC9AHaJw4lFLK49Zmr2XWollk5Gbw0JSHuHvS3QTIMf5F/8knsGkT\nvPGGdUeVi7Q0SEqqZP/+dC644NZj+ni/aapyJSL9gVOA1c0cniAiG0TkcxEZfoTPmC0ia0VkbX5+\nfjtFqpT6OynqAAAdlUlEQVTq7A6VH+Lmz25m7EtjySvN47MrP+Oe0+859qSRkwOzZ8OQIXXTi9Qy\nxkoccXFbWz1avMHn+Ns4DhGJAN4HbjfGHG50eD3QzxhTIiLnAh8BJzb3OcaYhcBCgJSUFNOOISul\nOiFjDG9seIN5S+dRUF7AbeNu44HJDxAdEn3sH2q3w1VXweHD8NVXEBTU4PDOndYjx0NDP2/1aPEG\nsftLUxWAiARiJY1/G2M+aHzcNZEYYxaLyPMiEm+MOeDJOJVSnVtGbgY3L76ZFftWMCFxAl+e9yXJ\nPZOP/4P//Gf4+mt45RUYMaLJ4bfestZ79ixk1qzrj/k0ftNUJdaNyv8EthhjnmihTE9nOURkLFa8\nBZ6LUinVmW09sJVrPrqGU148hS35W3j5gpdZcf2Ktkkay5bBgw/CzJlw7bVNDtfUWGMBhw/PBHYf\n0224tfypqWoi8GsgQ0TSnfv+APQFMMa8AFwGzBGRGqAcmGGM0WYopdRxSc9J5y/L/8J7m98jpGsI\nc8fO5Y+n/5G4sLi2OUFOjtVENWQIPP98kw5xgMWLITMTevV685hGi7vy9JQj3ryragVwxKs0xjwL\nPOuZiJRSnd13+7/j4eUP89lPnxEVHMVdp93F7eNvp0d4j7Y7id0OV15p9WssWwbh4c0We+EF6NXL\nsHHjI1x33dWtHi3uyq/6OJRSqr0VVxbzn03/4eX1L7M6azVxoXH8ecqfmTt2LjEhMW1/wgcfhG++\ngVdfheHN3wi6ezcsWQJnn72GJUsOc/HFxzCg0IU/NVUppVS7MMawOms1L69/mXc2vkNpdSnDug/j\nqbOfYtboWUQERbTPib/4wuoQv/baZvs1ai1cCCKG1NRfc/bZZ3PmmWce12k93TmuiUMp1WnsPLiT\n97e8z+sbXmdz/mbCA8OZMWIGN4y+gXF9xh1Xc9BRvfuu1RE+fDg823ILe1UV/POfhu7dV1FcnMkL\nL3xx3HHV1Ti6aOJQSqkjMsawOX8z7295n/e3vM+PuT8CMK7POF664CWmD59OZHBkewdhPQJ2/nyY\nOBE+/rjFfg2ADz+E/HwBHuDJJx+mf//+xx9CjUG6SvsmRheaOJRSHUplTSUr96/kix1f8NG2j9he\nsB1BmNh3Ik9Me4JfDv0l/WL6eSaYmhq49Varp3v6dHjtNTjKXFP/+EcVAQE2xowp5NZbj22KkcZM\ntfFYMxVo4lBK+ThjDJvyN7F051K+3PUl/9vzP8pryuka0JXJ/Sfz2/G/5aLBF9ErspdnAysutpLF\n55/DXXfBww9DwJGHxm3dCitXBhEQ8CL//OdLdOnSpU1CMTXGYx3joIlDKeVjahw1bMjZwIp9K1i5\nfyUr9q3AVmIDYEj8EG4YfQPTBk7jjH5ntH8zVEuysuD88yEjw3o40+zZbr3tD3/YA/Tm//4vipEj\nR7ZZOFrjUEr5lZySHNZlr+P7rO9ZuX8lqzJXUVpdCkDf6L5M7j+ZqQOmctbAs+gb3de7wToc8Prr\nVn9Gebk16+0vfuHWW/PzS/j441giI5fy17/+tk3Dqu3j8BRNHEopjzDGkF2cTXpOOuts61hnW8fa\n7LVkF2cDECABnJxwMtclX8fEvhOZmDSRpOiko3yqB61bB3PnwqpVMH68dU9tK2oNV121CIfjSh5+\nOIng4OA2DU2bqpRSHV5hRSGb8jaxMW8jGXkZ1pKbwaGKQwAIwuD4wfx8wM8Z02sMY3qN4ZRep7Tf\n+IrjcfAg3HOP1STVvbs1sG/mzKP2Z7hatmwZS5eeQExMDnPnHvvUIi1xVDu0xqGU8n3V9mr2Fe1j\nW8E2th3YxtYDW9lWYK1zS3PrykUFRzGixwguH345I3uMZFTCKJJ7Jnuvf8JdFRXWXVL33guFhXDb\nbXD//RDTutHmL774Irfc8jKwhrvuqmhu2qrjpjUOpZRPMMZQWFHInsI97C3ay86DO9lxcAc7D+1k\n56Gd7C3ci93Y68rHhcYxJH4I5590PoPjBjOs+zBGJowkKSrJY+ML2sS+fdbttS+9BAcOwOmnwz/+\nYT32tRWqqqq4/fbbWbBgAXFx66iqMsye3frHwrpD+ziUUh5RWVNJVnEWmYcz65b9RfvZU7SHvYV7\n2VO4h+Kq4gbv6RbajYGxAxnbZyxXjLiCgbEDOSnuJAbHDyY+LN5LV9IGjIFvv7VGfH/0kbXvwgut\nPo2f/7zZ2W2PJD8/n8suu4zU1FTOO+9tPvtsNM8+C7GxbR866F1VSqnjVFlTSV5pHjklOeSU5JBd\nnF232EpsZBdnk1WcRV5pXpP3RgdH0z+mPwNiBzCl/xT6xfSjf0x/+kX3Y2C3ge0zKaC3GGPdTvvx\nx/DOO7B5M3TrBvPmwZw50O/YBhGmp6dz0UUXkZeXx4IF73HvvZcyYYL1ke3F1BgCAj33eCVNHEr5\nOLvDzsHygxwoO0B+WT55pXnkl+aTX5ZPfmk+eWV55JbkkluaS25Jbl0HtCtB6BHeg96Rvekd2ZuU\n3ikkRSWRGJVIYlQiSdFJ9Ins4/v9DserpgaWL7eSxccfw549Vm1i/HjrKX0zZkBo6DF9dHl5Oa+8\n8grz5s0jLi6OFStW8NRTYzh82Gr1akVfeqtpU5VSnZTdYaewopDCikIOVRziUPkhDpYfbLhUHKSg\nrIADZQcoKLfWh8oPYWj++WWxIbF0D+9OQngCI3qM4MwBZ5IQkUBCeAIJEQn0iuhF78jeJEQk0DXA\nD/93r6qC9eth5Upr+fZbOHQIgoPhzDPhD3+ACy6Anj2P+RT5+fk8//zzPPvssxw4cIDJkyfzzjvv\nkJ6ewL/+BX/8Y4uzq7cZbapSygc5jIPSqlKKq4opqiiiqLKIw5WHG2wXVhRSVFFEYaVz7UwStYni\ncOXhI54jIiiCbqHd6BbajfiwePrF9CMuNI74sPi6dffw7vQI70H3sO7Eh8UT2CXQQ99AB2C3w86d\nsHEjfP+9lSjWrIHKSuv4CSfARRdZiWLaNIg4vlt/t2/fzpNPPslrr71GRUUFF1xwAb///e+ZNGkS\nZWXCTTfB4MFWbmpveleVUm2gyl5FaVUppdWllFSVUFplrRsvxVXF1rqymOIq51JZvz5cebhuu6W/\n+l1FBUcRExJDTEgM0cHRJEUnMSphFLEhscSGxhIbEktMSEzddlxYXF2yCOoS5IFvphOoqLCamHbu\nhC1brH6KjRutPoqKCqtMYCCMHg233GLNWHvqqcdVqwBwOBxkZGSwbNkyvvjiC5YuXUpQUBAzZ87k\njjvuYMiQIXVl77vPCjE19ahzHrYJU+1HiUNEzgGeBroALxtjHml0PBh4AxgDFADTjTF7PB2najvG\nGCpqKiivKae8urxuXVZdRnmNtS6rLqvbV7uUVpc2eV1aVdrgmGuiqHHUuB1TUJcgIoMiiQyOJDIo\nkoigCGJCYugb3ZeooCgigyOJCo4iKjiKyKBIokOiiQqOIjrYuXa+jgyKpEtA20xa57ccDsjPh+xs\na8nKspY9e2DXLmvJzm74nt69rRHct9wCI0ZY28OGHXNfRS273c6uXbv49ttvWbZsGV9//TX5+fkA\nDB48mHvuuYe5c+eSkJDQ4H3r1sGTT8JvfgOTJh1XCG4zNYaAUD/oHBeRLsBzwFlAJrBGRBYZYza7\nFJsFHDLGDBKRGcCjwHTPR9t5GGOosldRaa+koqaCyppKKu2VVNY4Xzv31x6r3W58rHYpry6nwt7o\ntUtiaJwkKmoqjinuLtKF8KBwwgLDCAsMIzzQ2g4PCqdbaLe6feFB4XXriKCIBtu1S21yiAiKIDwo\nXP/Sby8VFdbAOdelqMgaiX3gQMMlP99acnKsDmxXItCnj9XUNG2atT7hBBgwAIYMse6EOkYOh4OC\nggIyMzPZtm0bW7duZcuWLWzdupXt27dT4azB9O7dm3POOYepU6cydepUEhMTm/28mhq48UZISIBH\nHz3msFrNn5qqxgI7jDG7AETkHeAiwDVxXATc79x+D3hWRMQYc/Q2Ay8wxmA3dqrsVVTbq6myV9Ut\n1Y5Grxsdb7xU2ivrt2sqm+5zbtf+8Dcu55oQXNdV9qo2udagLkGEdg0lpGtI3RIaGEpwl2BCA0Pp\nHtad0EDreG25sMAwQruGEhoY2mBdmwxCA0PrytS+rk0Q2pZ/DBwOqK62lpqa+u3apaqq+aWiwuoX\nqKhouJSXQ1lZ/bp2u6TEmmK8pKThdnX1keOLjob4eGvp0wdOPtmqPTReEhIgqPnkbrfbKS8poby8\nnPLycsrKyigvL6ekpITCwkKKiooaLAcPHsRms5GTk4PNZiM3Nxe7vX4QY0BAAAMGDGDIkCGcddZZ\nDBkyhIkTJzJkyBC3BjE+8gj88AO8/751eZ7iT1OO9AH2u7zOBMa1VMYYUyMiRUAccKA9Anp82J0Y\nRwBGwAAOAYcY57ax9otzP8a5bXBQv7+9iIFAE0AwIQSYUAIMBBhpfg0EOMRa1x1rWK5LM++r31d7\n3Fq6OGqP15c/vkutdi5NO4vLncvBY/hUcePviWbjbuZ9jcs1eN2ofN0xU/u60XFn+ZbKictnivN4\n3TFjGuyrLSvNlKnbV/u6uWs9Tg4R7BKKQ8KxBwh2CbD2BQRQ43xdEyDYwwKoEaEmIIBq51ITIFRL\nANUiVAaI9f9NhcHsN5h9BoxVAzCmFIfZhnFsdb42OBwO7A47DocDh91hvbbbcRiH27F3CehCr+Be\nDAwdSFhYGKHhoYSNCCM0NJSw8DCio6KJio6qf0ZGIbDKWrax7aifv2MHFP0PFpwAwz6DrZ81KuDu\nn7vS/LZr4jLGgB2Mw2DshtINpYQOOr6mudboNJ3jIjIbmA3Qt++xTb3cL+tkAmuC6/6tpO5/8GZe\nG5f9R3vtxue4dQ4v8cnqXTswLWxb3PsXcOe7aqlMc/tb3CfNl3E9VpuKmruuJmuX97i+z7Wccf0R\nQ8BRe6xRomz0l3lXXH5oBMSItW7wq2i9NmLq9rt+jojzsagCElj/iFRB6o41XgK6BFjrgID699d+\nPwYocy5112gooohjUVEB5QUwKRjiK+HQly3893K0/4xcvsoGDSuN/xEDQALEesZ4AIScEELslHYa\nlt4MbyaOLMB1zuRE577mymSKSFcgGquTvAljzEJgIUBKSsox/db9quiqY3mbUsqPrVwJ558Fw8bA\nN99AZCcfQwlWq4a3rAFOFJEBIhIEzAAWNSqzCLjGuX0Z8LWv9m8opfxPRob1IMCkJOsJsv6QNMCL\nNQ5nn8Vc4Aus23FfMcZsEpEHgbXGmEXAP4E3RWQHVrP3DG/Fq5RSrnbvhrPPhrAw+PJL61Ed/sKr\nfRzGmMXA4kb7/uSyXQH8ytNxKaXUkeTmWncGV1RYU18d43yIHZY3m6qUUqrD2b3bmuYqKws+/bT9\n56HyRZo4lFLKTcuWQUoKZGbCJ59YM5n4I00cSil1FMZY04hMmwa9ellzJ06d6u2ovEcTh1JKHUF5\nOfz613DHHXDxxfDddzBokLej8i5NHEop1YJ9++C00+Ctt+Chh+C99/znltsj6TQjx5VSqq0YA//6\nF/z2t9aUW598Aued5+2ofIfWOJRSysXWrfDzn8PMmXDiidYzoTRpNKSJQymlsPoy/vhHGDUK0tPh\nxRet6UQGD/Z2ZL5Hm6qUUn7viy/g5put50T9+tfw+OPQo4e3o/JdWuNQSvmt//3PGsx3zjnW02a/\n/hreeEOTxtFo4lBK+RVjrIF8Z5wBkydbjyv/+99hwwaYMsXb0XUMmjiUUn7BGFiyBCZOtGoZO3fC\nM89YU4jccQcEB3s7wo5D+ziUUp3aoUNW89PChbB5M/TtCwsWwHXXabI4Vpo4lFKdjjGQlmbdGfXf\n/1qz2I4dC6+8Aldd1eIjzJWbNHEopTqNvXutRPHqq1btIjLSqlnMng3Jyd6OrvPQxKGU6tD27rWm\nAnn3XWuwHli1i5dfhhkzIDzcu/F1Rpo4lFIdijHWnVCffw7vv1+fLEaPhkcegcsug4EDvRtjZ6eJ\nQynl8woL4auvrLuiliyxHqIEmiy8RROHUsrnFBdbndupqfDtt7B6NdjtEBMDZ51lDdg7+2zo08fb\nkfonryQOEXkMuACoAnYC1xljCpsptwcoBuxAjTEmxZNxKqU8IzcXVq2yEkVqKvzwg5UounaFMWPg\n7rutZDFunLVPeZe3/gmWAncbY2pE5FHgbmB+C2WnGGMOeC40pVR7KimB9eutvonaZe9e61hwMIwf\nD3/4A5x+urUdEeHdeFVTXkkcxpgvXV6uAi7zRhxKqfaVk2NN5ZGeXr/etg0cDuv4gAFWcrjtNutO\nqJ/9TAfldQS+UOm7HvhPC8cM8KWIGOBFY8xCz4WllHLXwYPWuInaZdMmyMiwmqBq9esHJ58Ml19e\nnyS6d/dezOrYtVviEJGvgJ7NHLrHGPOxs8w9QA3w7xY+5jRjTJaI9ACWishWY0xqC+ebDcwG6Nu3\n73HHr5RqqLramtdp+/aGy5YtVs2iVlgYDBtm9UmccoqVLE4+GWJjvRe7altijPHOiUWuBX4DTDXG\nlLlR/n6gxBjz+NHKpqSkmLVr1x53jEr5m+JiKzns3Gk9m2LXLmt7505rv91eXzY2Fk46CYYOtRLF\n8OHWum9fCNDpUzscEVnn7g1I3rqr6hzgTuCMlpKGiIQDAcaYYuf2NOBBD4apVKficEB+PmRmwr59\nVoe067Jnj9Xk5ComxhofkZxsNTGddFL9EhfnlctQPsBbfRzPAsFYzU8Aq4wxN4lIb+BlY8y5QALw\nofN4V+AtY8wSL8WrlE+rrASbDbKz65esLGvZv99asrKgqqrh+8LDrb6Hfv2sfod+/axEccIJ1qLN\nS6o53rqralAL+7OBc53bu4CTPRmXUr7EbocDByAvz+pkzs21+hJqF5utfrugoOn7AwOtAXJJSTBh\nAiQmWtu16/79oVs3sP42U8p9vnBXlVJ+obra+oE/cMBa5+c3XfLy6tcHDtTftuoqOBh69YKePeHE\nE2HSJOjdu37p08dax8VpUlDtQxOHUq1UU2M9HOjQIatPoHZ98KCVEBqva5PF4cMtf2ZMjHVravfu\nVlPRqadCQoL17OuEhPrtXr0gKkoTgvIuTRzK79jt1o94UZG1FBbWr12XQ4earg8dsu48OpKYGKsJ\nKC7OWp90EsTHW6/j4+uXuDgrUcTHW81KSnUUmjhUh2G3Wz/axcXWD3/turmlqKjhtutSUnL0c4WH\nWx3DtUu/ftadRbVJITa26Touzjqucympzk7/E1ftwhjrcZ0lJVBaaq1rl+Lipq9rl8avXZfycvfO\nHRZmNedER1vrqCiriSc6uvklNtb6wY+Jqd+nP/5KtUz/9/BjDof1415aCmVl1rrx9tGW2sTQ3HZz\nHbvNEbEmsouMbLhOTLS2a5eoqIbbta8bb+uPvlLtS/8X80G1f62XlVl/ZZeX12+XlTXdPtK+5pba\n5FB21PH6TYWGWs044eHWj3tEhLXdrVv9du1+19fh4U0TQ+0SGqojjZXqSDRxHIUx1qCpigrrh7nx\n+kjb7iyuycF1ORaBgdaPsOuPe1iYtfTs2fz+8PCm+5vbrn2tP/BKKU0cLsaMsTpPGyeI45nOKzi4\n/se88RIdXf+DHhpq/TC7rhvvO9q23pmjlPIETRwuhg61kkRoKISEtG7deAkJsRb9C10p1dlo4nDx\nr395OwKllPJ9+vewUkqpVtHEoZRSqlU0cSillGoVTRxKKaVaRROHUkqpVtHEoZRSqlU0cSillGoV\nTRxKKaVaRczxzKfho0QkH9jbirfEAwfaKRxfptftX/S6/Utrr7ufMaa7OwU7ZeJoLRFZa4xJ8XYc\nnqbX7V/0uv1Le163NlUppZRqFU0cSimlWkUTh2WhtwPwEr1u/6LX7V/a7bq1j0MppVSraI1DKaVU\nq/hN4hCRc0Rkm4jsEJG7mjkeLCL/cR5fLSL9PR9l23Pjuu8Qkc0i8qOILBORft6Isz0c7dpdyl0q\nIkZEOsWdN+5ct4hc7vx33yQib3k6xvbgxn/rfUXkGxH5wfnf+7neiLMticgrIpInIhtbOC4i8ozz\nO/lRREa3yYmNMZ1+AboAO4ETgCBgAzCsUZmbgRec2zOA/3g7bg9d9xQgzLk9pzNct7vX7iwXCaQC\nq4AUb8ftoX/zE4EfgFjn6x7ejttD170QmOPcHgbs8XbcbXDdpwOjgY0tHD8X+BwQYDywui3O6y81\njrHADmPMLmNMFfAOcFGjMhcBrzu33wOmioh4MMb2cNTrNsZ8Y4wpc75cBSR6OMb24s6/OcCfgUeB\nCk8G147cue4bgeeMMYcAjDF5Ho6xPbhz3QaIcm5HA9kejK9dGGNSgYNHKHIR8IaxrAJiRKTX8Z7X\nXxJHH2C/y+tM575myxhjaoAiIM4j0bUfd67b1Sysv046g6Neu7PanmSM+cyTgbUzd/7NTwJOEpGV\nIrJKRM7xWHTtx53rvh+4WkQygcXArZ4Jzata+xvgFn3muAJARK4GUoAzvB2LJ4hIAPAEcK2XQ/GG\nrljNVZOxapipIjLSGFPo1aja3xXAa8aYv4vIBOBNERlhjHF4O7COxl9qHFlAksvrROe+ZsuISFes\nqmyBR6JrP+5cNyJyJnAPcKExptJDsbW3o117JDAC+FZE9mC1/y7qBB3k7vybZwKLjDHVxpjdwHas\nRNKRuXPds4B3AYwx3wEhWPM5dWZu/Qa0lr8kjjXAiSIyQESCsDq/FzUqswi4xrl9GfC1cfYudWBH\nvW4ROQV4EStpdIa27lpHvHZjTJExJt4Y098Y0x+rf+dCY8xa74TbZtz5b/0jrNoGIhKP1XS1y5NB\ntgN3rnsfMBVARIZiJY58j0bpeYuAmc67q8YDRcYY2/F+qF80VRljakRkLvAF1t0XrxhjNonIg8Ba\nY8wi4J9YVdcdWJ1NM7wXcdtw87ofAyKA/zrvBdhnjLnQa0G3ETevvdNx87q/AKaJyGbADswzxnTo\n2rWb1/074CUR+S1WR/m1Hf2PQxF5G+uPgHhn3819QCCAMeYFrL6cc4EdQBlwXZuct4N/b0oppTzM\nX5qqlFJKtRFNHEoppVpFE4dSSqlW0cShlFKqVTRxKKWUahVNHMpvicj9IvJ75/aDzoGQ3ool+Xhn\naxWRGBG5ua1iUqolmjiUAowxfzLGfOXFEJKx7rc/IuesBi2JwZrlWal2pYlD+RURuUdEtovICmCw\ny/7XROQy5/YjLs8oedy5L0FEPhSRDc7lVOf+O0Rko3O53bmvv+vzEUTk9yJyv3P7WxF5VES+d8Yx\nyTnS+UFguoiki8j0RjFfKyKLRORrYJmIRIj17JT1IpIhIrWzwD4CDHR+xmPO984TkTXOa3mgXb5U\n5Xf8YuS4UgAiMgZrRoBkrP/21wPrGpWJAy4BhhhjjIjEOA89A/zPGHOJiHQBIpyfdx0wDut5B6tF\n5H/AoaOE0tUYM9bZNHWfMeZMEfkT1vNA5rbwntHAKGPMQWet4xJjzGHnlCGrRGQRcBcwwhiT7LyW\naVhzUI11xrdIRE53TsWt1DHTGofyJ5OAD40xZcaYwzSdywis6fQrgH+KyC+xpmkA+DmwAMAYYzfG\nFAGnOT+v1BhTAnzgPMfRfOBcrwP6uxn7UmNM7XMXBPiLiPwIfIU1TXZCM++Z5lx+wEqSQ+j4kxkq\nH6A1DqVcOOc8Gos1Gd5lwFyspNEaNTT8oyyk0fHaGYjtuP//YKnL9lVAd2CMMabaObtv43OAlWD+\naox50c1zKOUWrXEof5IKXCwioSISCVzQuICIRADRxpjFwG+Bk52HlmE9WhcR6SIi0cBy5+eFiUg4\nVhPXciAX6CEicSISDJzvRmzFWFO9uyMayHMmjSlA7XPiG3/GF8D1zmtCRPqISA83z6FUi7TGofyG\nMWa9iPwH63nUeVhTcTcWCXwsIiFYf7Hf4dz/f8BCEZmFVVOYY4z5TkReA753lnnZGPMDWLf3Ovdn\nAVvdCO8b4C4RSceqJfznCGX/DXwiIhnA2trPN8YUiPVUv43A58aYec7pw79zznxcAlztvHaljpnO\njquUUqpVtKlKKaVUq2jiUEop1SqaOJRSSrWKJg6llFKtoolDKaVUq2jiUEop1SqaOJRSSrWKJg6l\nlFKt8v8mFqhmcRhLiAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fb3cf832208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "states = 50\n",
    "actions = 2\n",
    "env = Environment(states, actions)\n",
    "gammas = [x*0.01 for x in range(1, 98) if x % 2 == 1]\n",
    "gammas = gammas + [0.985, 0.986, 0.99]\n",
    "strategies = []\n",
    "states = {}\n",
    "for gamma in log_progress(gammas):\n",
    "    strategies.append(ValueIterationStrategy(env, gamma))\n",
    "    strategies[-1].learn()\n",
    "plt.hold(True)\n",
    "for state, color in zip([0, 20, 37, 42], ['k', 'r', 'g', 'b']):\n",
    "    states[state] = [strategy.state_values[state] for strategy in strategies[:-3]]\n",
    "    plt.plot(gammas[:-3], states[state], color, label='state #' + str(state))\n",
    "states[49] = [strategy.state_values[49] for strategy in strategies]\n",
    "plt.plot(gammas, states[49], 'm', label='state #' + str(49))\n",
    "\n",
    "plt.hold(False)\n",
    "plt.xlabel('discount rate')\n",
    "plt.ylabel('V*(s)')\n",
    "plt.legend(loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Начальное положение кривых относительно друг друга понятно: при низком значении дисконтирования очень сильно учитывается награда, полученная в самом начале, и почти не оценивается награда, полученная в самом конце. Из-за этого начальная ценность всех состояний, кроме 42-го при попадании в которое начисляется штраф, близка к нулю. Затем при росте $\\gamma$ заметна следующая тенденция: чем ближе к \"выигрышнному\" состоянию мы находимся, тем скорее текущее состояние набирает ценность при росте $\\gamma$ в начале, однако затем они более-менее выравниваются. Это происходит из-за того, что более близкие к выигрышному состояния раньше начинают получать выигрыш, и он, соответственно, в меньшей степени подвергается дисконтированию. Потом этот разрыв уменьшается, так как все состояния рано или поздно достигают выигрышного, и начальная фора начинает выражаться просто константой\n",
    "\n",
    "Интересным исключением в данной модели является только 49-ое состояния. Так как от выигрышного состояния его отделяет большой штраф, при низких значениях $\\gamma$ от просто не успевает скомпенсировать его, в результате чего ему становится выгоднее не рисковать и идти в противоположную сторону, где есть лишь нулевая награда. Однако с определённого момента последующая после штрафа награда уже начинает перевешивать штраф, и происходит взрывной рост ценности"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2+"
  },
  "widgets": {
   "state": {
    "14390153cf4d4dffba6fd0f4609d1f0d": {
     "views": [
      {
       "cell_index": 11
      }
     ]
    },
    "419d2770d3634f7ba1d67bc518fcec8a": {
     "views": [
      {
       "cell_index": 11
      }
     ]
    },
    "7489f39d74ad45ab8e0d7a7e1dd5d09a": {
     "views": [
      {
       "cell_index": 11
      }
     ]
    },
    "808b49c25b4b4279a2394eb3f0a52d00": {
     "views": [
      {
       "cell_index": 13
      }
     ]
    }
   },
   "version": "1.2.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}