understanding-word-vectors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding word vectors\n",
    "\n",
    "... for, like, actual poets. By [Allison Parrish](http://www.decontextualize.com/)\n",
    "\n",
    "In this tutorial, I'm going to show you how word vectors work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "pd.set_option('max_rows', 25)\n",
    "plt.style.use('ggplot')\n",
    "plt.rcParams[\"figure.figsize\"] = (10, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why word vectors for poetry?\n",
    "\n",
    "Tzara proposed a method of composing a Dada poem: cut out the words of a text, shake them in a bag, then write down the words as you remove them at random from the bag. The very idea caused a riot and sundered the avant garde in twain (or so the story goes). For poets, word vectors are (for better or worse) a tool to help soften the blow of cut-up techniques: instead of selecting words at random, we might select units of text that are *close in meaning* to other units. This can yield poetic juxtapositions with subtle effects impossible to achieve with other techniques.\n",
    "\n",
    "Also, it's fun!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Animal similarity and simple linear algebra\n",
    "\n",
    "We'll begin by considering a small subset of English: words for animals. Our task is to be able to write computer programs to find similarities among these words and the creatures they designate. To do this, we might start by making a spreadsheet of some animals and their characteristics. In Python, you'd define such a spreadsheet like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "animals = [\n",
    "    {'name': 'kitten', 'cuteness': 95, 'size': 15},\n",
    "    {'name': 'hamster', 'cuteness': 80, 'size': 8},\n",
    "    {'name': 'tarantula', 'cuteness': 8, 'size': 3},\n",
    "    {'name': 'puppy', 'cuteness': 90, 'size': 20},\n",
    "    {'name': 'crocodile', 'cuteness': 5, 'size': 40},\n",
    "    {'name': 'dolphin', 'cuteness': 60, 'size': 45},\n",
    "    {'name': 'panda bear', 'cuteness': 75, 'size': 40},\n",
    "    {'name': 'lobster', 'cuteness': 2, 'size': 15},\n",
    "    {'name': 'capybara', 'cuteness': 70, 'size': 30},\n",
    "    {'name': 'elephant', 'cuteness': 65, 'size': 90},\n",
    "    {'name': 'mosquito', 'cuteness': 1, 'size': 1},\n",
    "    {'name': 'goldfish', 'cuteness': 25, 'size': 2},\n",
    "    {'name': 'horse', 'cuteness': 50, 'size': 50},\n",
    "    {'name': 'chicken', 'cuteness': 25, 'size': 15}\n",
    "]\n",
    "animal_lookup = {item['name']: (item['cuteness'], item['size']) for item in animals}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then display it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>cuteness</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>kitten</td>\n",
       "      <td>95</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>hamster</td>\n",
       "      <td>80</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tarantula</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>puppy</td>\n",
       "      <td>90</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>crocodile</td>\n",
       "      <td>5</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>dolphin</td>\n",
       "      <td>60</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>panda bear</td>\n",
       "      <td>75</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>lobster</td>\n",
       "      <td>2</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>capybara</td>\n",
       "      <td>70</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>elephant</td>\n",
       "      <td>65</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mosquito</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>goldfish</td>\n",
       "      <td>25</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>horse</td>\n",
       "      <td>50</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>chicken</td>\n",
       "      <td>25</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          name  cuteness  size\n",
       "0       kitten        95    15\n",
       "1      hamster        80     8\n",
       "2    tarantula         8     3\n",
       "3        puppy        90    20\n",
       "4    crocodile         5    40\n",
       "5      dolphin        60    45\n",
       "6   panda bear        75    40\n",
       "7      lobster         2    15\n",
       "8     capybara        70    30\n",
       "9     elephant        65    90\n",
       "10    mosquito         1     1\n",
       "11    goldfish        25     2\n",
       "12       horse        50    50\n",
       "13     chicken        25    15"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(animals, columns=['name', 'cuteness', 'size'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This table associates a handful of animals with two numbers: their cuteness and their size, both in a range from zero to one hundred. (The values themselves are simply based on my own judgment. Your taste in cuteness and evaluation of size may differ significantly from mine. As with all data, these data are simply a mirror reflection of the person who collected them.)\n",
    "\n",
    "These values give us everything we need to make determinations about which animals are similar (at least, similar in the properties that we've included in the data). Try to answer the following question: Which animal is most similar to a capybara? You could go through the values one by one and do the math to make that evaluation, but visualizing the data as points in 2-dimensional space makes finding the answer very intuitive:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgQAAAHjCAYAAABcqwcxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABNHUlEQVR4nO3de3zP9f//8dsOmc1h3juwtunI2GbOTCSnoWIslNPSkkpCloQckj6FfBgJ61uSaonEHNLBEkIhQ6M5y4fNZrONNpvZ3u/fH+r982YOs7134H69XLpc3u/X+/V8vR7vx/Tefa/X8/162ZhMJhMiIiJyR7Mt7QJERESk9CkQiIiIiAKBiIiIKBCIiIgICgQiIiKCAoGIiIigQCAiIiKAfWkXcCsSExNveaybmxupqanFWI2A+mpN6q11qK/Wob4WP09PzxLZj44QiIiIiAKBiIiIKBCIiIgICgQiIiKCAoGISKmZMWMGw4YNK/J2Tpw4gZeXF3l5ecVQldypFAhEROSalixZQkhISGmXISVAgUBEREQUCERErC0pKYnnn3+egIAAWrRowYIFCwpcb+fOnXTr1g1fX1+CgoLYunWr+bVevXoxZcoUunTpQt26dXn22WdJT0+3GL98+XKaNWtGvXr1mD17tnn5rl27CA4OxtfXl0aNGjFu3Dhyc3PNr3t5efHZZ5/RqlUr/Pz8eOONNzCZTBw6dIixY8eyc+dOateuja+vbzF3RsoSBQIRESsyGo2EhYXh5+fHzp07WbJkCR9//DEbNmywWO/UqVMMGDCAV155hX379jFx4kSef/55zpw5Y15n2bJlzJgxg9jYWOzt7Zk4caLFNnbs2MGmTZtYsmQJs2bN4tChQwDY2dkxadIk4uLiWLVqFZs3b2bRokUWY2NiYli7di3r1q1j9erVbNiwgdq1azNlyhSaNGnCoUOHiI+Pt06TpExQIBARsaLdu3dz5swZwsPDqVChAvfeey/9+vVj5cqVFustX76c9u3b06FDB2xtbXnkkUdo0KABP/30k3mdnj17UrduXZycnBg1ahSrV68mPz/f/Hp4eDiOjo74+/vj5+fHn3/+CUD9+vVp0qQJ9vb21KxZk9DQUH777TeL/Q8dOhRnZ2e8vLxo2bIl+/bts2JXpCwql5cuFhEpL06ePElycrLF4fb8/HwCAwPx9va2WO/bb78lJibGvOzixYu0bNnS/PzyS9h6e3tz8eJF0tLSzMuqV69ufuzo6EhWVhYAR44c4a233uKPP/4gOzubvLw86tevb1Gnu7t7gWPlzqFAICJiRZ6entSsWZMtW7Zc9dqMGTMs1uvZsyfTp0+/5rYuv49LQkICd911Fy4uLje8v8vYsWOpV68e8+bNo3Llynz00Ud8++23N1W/jY3NTa0n5Z9OGYiIWFGjRo2oUqUKc+fOJTs7m/z8fPbv38/u3bst1uvRowfr1q1jw4YN5Ofnk5OTw9atWy1+2X/zzTccPHiQ7Oxspk+fTpcuXbCzs7thDVlZWVSpUoVKlSpx+PBhPvvss5uu393dnVOnTllMQpTbkwKBiIgV2dnZ8emnn7Jv3z4eeughAgICeO211zh37pzFel5eXnzyySfMmTOH+vXr06xZM+bPn4/JZDKv06tXL8LDw2nYsCEXLlzg7bffvqkaJkyYwIoVK/Dx8WHUqFF069btputv1aoVPj4+NGrUiHr16t30OCl/bEyX/2srJ3T747JHfbUe9dY6yltfe/XqRY8ePejXr19pl3Jd5a2v5UFJ3f5YcwhERIqBMSUJVkZhykjDppoLdO+PrbtHaZclctMUCEREisiYkoQpYiKkJAFgAjh6AGP4ZIUCKTcUCEREimpllDkMmP1zxIBBI4tlF8uWLSuW7YhciyYViogUkSkjrVDLRcoiBQIRkSKyqeZSqOUiZZECgYhIUXXvD1fOFXD3uLRcpJzQHAIRkSKydffAGD5Z3zKQck2BQESkGNi6exTbBEKR0qBTBiIiIqJAICIiIgoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIgLYl9SO1qxZw/r167GxsaFmzZoMGTKE3NxcIiIiSElJwd3dnfDwcCpXrlxSJYmIiMg/SuQIQVpaGt999x1Tp05lxowZGI1Gtm7dSnR0NAEBAbz//vsEBAQQHR1dEuWIiIjIFUrslIHRaCQ3N5f8/Hxyc3MxGAzs2LGDNm3aANCmTRt27NhRUuWIiIjIZUrklIGLiwvBwcG89NJLVKhQgQYNGtCgQQPOnj2LwWAAwGAwcO7cuZIoR0RERK5QIoEgMzOTHTt2MHfuXJycnJg5cyabNm266fExMTHExMQAMHXqVNzc3G65Fnt7+yKNl4Kpr9aj3lqH+mod6mv5VSKBIC4ujurVq1O1alUAAgMDOXjwIM7OzqSnp2MwGEhPTze/fqWgoCCCgoLMz1NTU2+5Fjc3tyKNl4Kpr9aj3lqH+mod6mvx8/T0LJH9lMgcAjc3Nw4dOsSFCxcwmUzExcXh5eVF06ZN2bhxIwAbN26kWbNmJVGOiIiIXKFEjhDUrl2bFi1aMHr0aOzs7LjvvvsICgoiJyeHiIgI1q9fj5ubG6+++mpJlCMiIiJXsDGZTKbSLqKwEhMTb3msDmdZh/pqPeqtdaiv1qG+Fr/b6pSBiIiIlG0KBCIiIqJAICIiIgoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiIC2JfUjrKysoiMjOTEiRPY2Njw0ksv4enpSUREBCkpKbi7uxMeHk7lypVLqiQRERH5R4kFgoULF9KwYUNGjhxJXl4eFy5cYMWKFQQEBBASEkJ0dDTR0dGEhoaWVEkiIiLyjxI5ZXD+/Hni4+Np3749APb29lSqVIkdO3bQpk0bANq0acOOHTtKohwRERG5QokcITh9+jRVq1Zl3rx5HD9+nAceeICwsDDOnj2LwWAAwGAwcO7cuZIoR0RERK5QIoEgPz+fY8eOMXDgQGrXrs3ChQuJjo6+6fExMTHExMQAMHXqVNzc3G65Fnt7+yKNl4Kpr9aj3lqH+mod6mv5VSKBwNXVFVdXV2rXrg1AixYtiI6OxtnZmfT0dAwGA+np6VStWrXA8UFBQQQFBZmfp6am3nItbm5uRRovBVNfrUe9tQ711TrU1+Ln6elZIvspkTkE1apVw9XVlcTERADi4uLw9vamadOmbNy4EYCNGzfSrFmzkihHRERErlBi3zIYOHAg77//Pnl5eVSvXp0hQ4ZgMpmIiIhg/fr1uLm58eqrr5ZUOSIiInIZG5PJZCrtIgrr3yMNt0KHs6xDfbUe9dY61FfrUF+L3211ykBERETKNgUCERERUSAQERERBQIRERFBgUBERERQIBAREREUCERERAQFAhEREUGBQERERFAgEBERERQIREREBAUCERERQYFAREREUCAQERERFAhEREQEBQIRERFBgUBERERQIBAREREUCERERAQFAhEREUGBQERERFAgEBERERQIREREBAUCERERQYFAREREUCAQESsLDAxk06ZNpV2GiNyAAoGIiIgoEIhI2ZeXl1faJYjc9hQIRMTq/vzzT4KCgqhbty6DBw8mJycHgKioKFq1aoW/vz9hYWEkJSWZx3h5efHpp5/SqlUrHn74YUwmE2+++Sb169enbt26BAUFsX//fgAuXLjA5MmTadasGQ0aNGD06NFkZ2eXynsVKa8UCETE6lavXs0XX3zBr7/+Snx8PEuXLmXz5s1MmTKFyMhIYmNj8fb2ZsiQIRbjvv/+e9asWcPPP//Mxo0b2bZtG7/88gvx8fHMnz8fg8EAwDvvvMPRo0f58ccf2bJlC0lJScyaNasU3qlI+aVAICJWN3DgQDw8PDAYDHTs2JF9+/axYsUK+vTpQ0BAAA4ODowdO5adO3dy4sQJ87ihQ4diMBhwdHTE3t6ezMxMDh8+jMlkonbt2tSoUQOTycSXX37JpEmTMBgMVK5cmWHDhrFy5cpSfMci5Y99aRcgIrc/d3d382NHR0eSk5NJT08nICDAvLxSpUoYDAaSkpKoWbMmAJ6enubXH374YZ599lnGjRtHQkICjz32GBMmTODChQtkZ2fz2GOPmdc1mUzk5+eXwDsTuX3oCIGIlIoaNWpw8uRJ8/Pz58+Tnp6Oh4eHeZmNjY3FmOeee47vv/+en3/+maNHjzJ//nxcXFyoWLEi69evJz4+nvj4ePbv38+hQ4dK7L2I3A4UCESkVISEhLBkyRL27t3LhQsXmDp1Ko0aNTIfHbjS7t27iY2N5eLFizg5OeHg4ICdnR22trb079+fSZMmkZqaCsCpU6fYsGFDCb4bkfJPgUBESkXr1q0ZNWoUL7zwAo0bN+avv/5i3rx511z/77//5vXXX8fPz4/mzZtjMBgYPHgwAG+88Qb33XcfwcHB1KlThz59+nDkyJGSeisitwUbk8lkKu0iCisxMfGWx7q5uZn/ipDio75aj3prHeqrdaivxe/yuTTWpEmFIlJkxpQkWBmFKSMNm2ou0L0/tu4eNx4oImWGAoGIFIkxJQlTxERIuXRRIRPA0QMYwycrFIiUI5pDICJFszLKHAbM/jliICLlhwKBiBSJKSOtUMtFpGxSIBCRIrGp5lKo5SJSNikQiEjRdO8PV84VcPe4tFxEyg1NKhSRIrF198AYPlnfMhAp5xQIRKTIbN09YNDI0i5DRIpApwxEREREgUBEREQUCERERAQFAhGR6xoxYgTTpk274XpeXl4cO3bsprZ5vXWXL19O3759C1WjSHHQpEIRkTKkR48e9OjRo7TLkDuQjhCIiIhI4QLBH3/8wfz585k6dSoAR44cYe/evVYpTESkNOzdu5fOnTvj4+PD4MGDuXDhgvm1qKgoWrVqhb+/P2FhYSQlJRW4jREjRjB69Gj69OmDj48PPXv25OTJkxbr/PLLL7Rq1Qo/Pz/eeOMN/r0T/ZIlSwgJCTGv5+XlxWeffVbguiLF6aYDwXfffcdHH33E3XffTXx8PAAVKlTgq6++slpxIiIlKTc3l4EDB9KzZ0/27dtH165dWbt2LQCbN29mypQpREZGEhsbi7e3N0OGDLnmtlasWMGIESOIi4vD39+foUOHWrweExPD2rVrWbduHatXr2bDhg3X3FZh1hW5VTcdCNauXcuECRMICQnB1vbSMC8vLxITE61WnIhISYqNjSUvL4/nn3+eu+66i65du9KgQQPg0i/4Pn36EBAQgIODA2PHjmXnzp2cOHGiwG116NCBFi1a4ODgwOjRo9m5cycJCQnm14cOHYqzszNeXl60bNmSffv2XbOuwqwrcqtuOhBkZ2fj5uZmsSwvLw97e81LFJHbQ3JyMh4eHtjY2JiXeXt7m1/79zFApUqVMBgM1zxt4OnpabFutWrVSE5ONi9zd3c3P3Z0dCQrK+uadRVmXZFbddOBwNfXl+joaItl3333Hf7+/sVdk4hIqahevTpJSUkW5+j//au+Ro0aFvMAzp8/T3p6Oh4eBd+z4fKjp1lZWWRkZFCjRg0rVS5SdDcdCAYOHMj27dt5+eWXycnJ4ZVXXuG3337jmWeesWZ9IiIlpkmTJtjZ2bFgwQLy8vJYu3Ytu3fvBiAkJIQlS5awd+9eLly4wNSpU2nUqBE1a9YscFvr169n+/bt5Obm8t5779GoUSO8vLxK8N2IFM5NH+83GAxMmTKFI0eOkJKSgqurK7Vq1TLPJxARKe8qVKjAxx9/zKhRo3jvvfdo3749jz32GACtW7dm1KhRvPDCC5w9e5YmTZowb968a24rJCSEmTNnsnPnTgICAvjggw9K6m2I3BIb001+f2XZsmU0a9aMe++912J5dHS0xVdkSkJRJjK6ubmRmppajNUIqK/WpN5ahzX7OmLECO6++25Gjx5tle2XZfr3Wvwun49iTTf95/0333zDf/7zH3799VeL5StWrCj2okRERKRk3fQpg7vuuotx48Yxffp0jh8/Tp8+fQB0gQwRKVeMKUmwMoq0rL8xVqoC3ftj617wxECRO8lNBwIbGxvuu+8+pkyZwsyZM3nvvfcYPny4xddzRETKMmNKEqaIiZCSxMV/Fx49gDF8crGGglmzZhXbtkRKyk2fMvj3SEDVqlWZMGEC1apVY+zYseTl5VmtOBGRYrUyClKuuG7AP0cMRO50Nx0I2rZta35sZ2fHCy+8wOOPP46Pj4816hIRKXamjLRCLRe5k9z0KYOBAwdetaxjx4507NixWAsSEbEWm2ouFDTryaaaS4nXIlLWXDcQfPjhh7z44osA1/0O7ZU37RARKZO694ejByxPG7h7XFoucoe7biCoXr26+bEuuSki5Z2tuwfG8MmwMgr7rL/J07cMRMyuGwieeOIJ82NfX1+qV69O9erVSU9PJyoqCltbW/r162f1IkVEioutuwcMGomLLqAjYuGmJxUuWLDAfJnizz77jPz8fGxsbPjwww+tVpyIiIiUjJueVJiWloabmxv5+fns3r2b+fPnY29vb55jICIiIuXXTQcCR0dHMjIyOHHiBDVr1qRixYrk5eXpOgQiIiK3gZs+ZfDoo48yduxY3n//fTp37gzA/v37dTvPcmLJkiUWN6GqXbs2x48fBy7diGXatGmlVJmI3KkCAwPZtGlTocedOHECLy8v/UFazG76CEFISAjNmzfH1tYWD49LM3JdXFwYPHiw1YoT6zl06FBplyAiImXITQcCuPoWjCV1S8bbWV5eHvb2hfoxiIhICcnPz8fOzq60yygRN33KQAovISGBQYMGERAQgL+/P+PGjWPJkiV0796dN998E39/f2bMmMG5c+cYPnw4AQEBNG/enFmzZmE0Gs3biYqKok2bNvj4+NC2bVvi4uKAS3/l9+rVC19fX9q1a8ePP/5oHpOWlkZYWBh16tShS5cu5tMD//Ly8uLYsWMF1r1u3To6duyIr68v3bp1488//7RCd0SkLAoMDGTOnDm0bdsWPz8/wsPDycnJASAjI4MBAwYQEBCAn58fAwYMIDEx0Ty2V69eTJo0ie7du+Pj40Pfvn1JS/v/l4VetmwZzZs3x9/fn9mzZ1vsd9euXQQHB+Pr60ujRo0YN24cubm51631q6++onHjxjRq1IjIyEjzcqPRyAcffEDLli3x9/fnxRdfJD093fz6Cy+8QMOGDalbty49evTgwIED5tdGjBjBmDFjePrpp6lVqxZbtmy5tUaWQwoEVpKfn88zzzyDl5cX27ZtY+fOnXTv3h249A//3nvvZc+ePQwfPpzx48fz999/8+uvv/LNN9+wbNkylixZAsDq1auZOXMms2fP5sCBAyxcuBCDwcDFixcJCwvjkUceYc+ePbz99tsMHTqUw4cPAzBu3DgcHBzYtWsXM2bMMG/vRuLi4hg5ciTTpk1j7969hIaG8uyzz3LhwgXrNEpEypwVK1YQFRXFli1bOHr0qPmXt9FopHfv3mzfvp3t27dTsWJFxo8fbzH2q6++YubMmezZs4eLFy+af1EfPHjQPA8tNjaW9PR0Tp06ZR5nZ2fHpEmTiIuLY9WqVWzevJlFixZdt86tW7eyefNmvvzyS+bOnWuej7BgwQK+//57li1bRmxsLM7OzowbN848rn379mzevJk9e/ZQr169q662Gx0dzfDhwzl48CDNmze/9UaWMwoEVrJr1y6Sk5OZMGECTk5OVKxY0fwPq0aNGgwcOBB7e3sqVKjA6tWrGTt2LJUrV6ZmzZq8+OKLfPPNNwAsXryYl156iYYNG2JjY8P999+Pt7c3sbGxZGVlMXToUCpUqMDDDz9MUFAQK1euJD8/n7Vr1zJq1CicnJyoW7cuvXr1uqm6o6KiCA0NpXHjxtjZ2fHUU09RoUIFYmNjrdYrESlbwsLC8PLywmAwMHz4cFauXAlcmjfWpUsXHB0dqVy5MsOHD+e3336zGPvMM8/w4IMP4ujoSNeuXdm3bx8A3377LUFBQbRo0QIHBwdef/1187VtAOrXr0+TJk2wt7enZs2ahIaGXrXtK7366qs4OTnh6+tL7969zXV+8cUXjB49Gk9PTxwcHBg5ciTffvuteRJinz59qFy5svm1P//8k3Pnzpm326lTJ5o1a4atrS0VK1YsekPLCZ28tpLExES8vb0LnB9w+dyLtLQ0cnNz8fb2Ni/z9vY2J+fExETuvffeq7aRlJSEp6enxf9Q3t7eJCUlcebMGfLy8iz24+3tzbZt225Yd0JCAl9//TULFy40L8vNzSU5OfmGY0Xk9nDlZ8e///9nZ2fz5ptvsmHDBs6ePQtAZmamxXn2yy9z7+joSFZWFvD/P7P+5eTkhMFgMD8/cuQIb731Fn/88QfZ2dnk5eVRv379QtW5f/9+AE6ePMmgQYMsPh/t7OxISUmhevXqTJs2jTVr1nDmzBnzOmlpaVStWvWq7d5JFAisxNPTk4SEhAInDdrY2Jgfu7i4cNddd3Hy5EnzraQTEhK4++67zdu58vw/gIeHB4mJiRiNRvM/6ISEBB544AFcXV2xt7cnMTGRWrVqmV+7GXfffTfDhw/nlVdeKfybFpHbwuXzAhISEsy/5CMjIzl69Chr1qyhevXq7N27l86dO2MyFXQPSUs1atSw+HZTdna2xXn9sWPHUq9ePebNm0flypX56KOP+Pbbb29Y5+Wfcf/W6enpycyZM2nWrNlVY5YtW8YPP/zAV199Rc2aNTl37hx+fn4W7+Hyz+g7iU4ZWEmjRo2oXr067777LufPnycnJ4cdO3ZctZ6dnR3BwcFMmzaNzMxMTp48yf/93//Ro0cPAPr27UtkZCR//PEHJpOJY8eOcfLkSRo1aoSTkxPz5s3j4sWLbN26lXXr1tG9e3fs7Ox47LHHmDFjBtnZ2Rw8eJCvv/76puru378/n3/+ObGxsZhMJs6fP09MTAyZmZnF2h8RKbsWLVpEYmIi6enpzJkzh+DgYACysrKoWLEiVatWJT09nYiIiJveZpcuXYiJiWH79u3k5uYyffp0i8nTWVlZVKlShUqVKnH48GE+++yzG25z1qxZZGdnc+DAAZYsWUK3bt0AePrpp5k2bRonT54E4MyZM/zwww/ApSMaFSpUwGAwkJ2dzdSpU2/6PdzuFAisxM7OjkWLFvHXX3/RrFkzmjZtyqpVqwpc9+2338bJyYmHHnqIkJAQnnjiCfr06QNAcHAww4cP5+WXX8bHx4fnnnuOjIwMKlSowMKFC/n5558JCAjgjTfeYPbs2ea0/M4775CVlUXDhg0JDw+nd+/eN1V3gwYNmD59OuPHj8fPz49WrVqxdOnS4mmKiJQLISEh9OvXj5YtW3LPPfcwYsQIAAYNGkROTg4BAQEEBwfTtm3bm95mnTp1eOedd3j55Zdp1KgRzs7O5iOhABMmTGDFihX4+PgwatQo8y/362nRogWtWrWid+/eDB48mDZt2pjr7NixI3379sXHx4fg4GDzPKgnn3wSb29vmjRpQtu2bWncuPHNN+Y2Z2O6mWM9xcRoNDJmzBhcXFwYM2YMmZmZREREkJKSgru7O+Hh4VSuXPmG27n8cFZhuekOZ1ahvlqPemsd6mvBAgMDmT59Oo888sgtjVdfi19JzWko0TkEa9euxcvLi+zsbODSVzsCAgIICQkhOjqa6OhoQkNDS7KkIjOmJMHKKEwZadhUc9G91UWkzNPnlhSkxE4ZnDlzhtjYWDp06GBetmPHDvMhnjZt2hR4jr0sM6YkYYqYiGnbRjgQh2nbRkwREy/9zyYiUgbpc0uupcQCwaeffkpoaKjF7M2zZ8+av3ZiMBgsvgdaLqyMgiv/J/oneYuIlEk3+Nzatm3bLZ8ukPKtRE4Z7Ny5E2dnZx544AHzRSoKIyYmhpiYGACmTp2Km5vbLddib29fpPGXS8v6m4sF7SPrb1yKaR/lRXH2VSypt9Zxp/bV2p9bd2pfbwclEggOHDjA77//zq5du8jNzSU7O5v3338fZ2dn0tPTMRgMpKenmy8KcaWgoCCCgoLMz4syYaU4J7wYK1UpcHlepSp33KQaTSSyHvXWOu7Uvlr7c+tO7as1ldSkwhI5ZdCvXz8iIyOZO3cuI0aMoF69egwfPpymTZuyceNGADZu3FjgRSTKtO794cqJOO4el5aLiJRF+tySayjVKxWGhIQQERHB+vXrcXNz49VXXy3NcgrN1t0DY/hkzdYVkXJDn1tyLSV6HYLiousQlD3qq/Wot9ahvlqH+lr8bqtTBiIiIlK2KRCIiIiIAoGIiIgoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiIigQCAiIiIoEIiIiAgKBCIiIoICgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgICgQiIiKCAoGIiNyEEydO4OXlRV5eXmmXIlaiQCAiIiIKBCIiUrJ0lKFsUiAQESmnEhISGDRoEAEBAfj7+zNu3Dj++usvnnzySfz9/alXrx5Dhw7l7Nmz5jGBgYHMmTOHtm3b4ufnR3h4ODk5OQC0b9+eH3/80bzuxYsXqVevHnv37jUv++qrr2jcuDGNGjUiMjLSvHzXrl0EBwdTvXp1GjVqxLhx48jNzTW/7uXlxaeffkqrVq14+OGHAZg4cSJNmzalTp06PProo2zbts1qvZIbUyAQESmH8vPzeeaZZ/Dy8mLbtm3s3LmT7t27YzKZGDZsGLGxsWzcuJHExERmzJhhMXbFihVERUWxZcsWjh49yuzZswHo1asXy5cvN6+3fv16atSoQb169czLtm7dyubNm/nyyy+ZO3cumzZtAsDOzo5JkyaRmJjIqlWr2Lx5M4sWLbLY7/fff8+aNWv4+eefAWjQoAE//vgj+/btIyQkhBdffNEcTqTkKRCIiJRDu3btIjk5mQkTJuDk5ETFihVp3rw5999/P4888ggODg64urrywgsv8Ntvv1mMDQsLw8vLC4PBwPDhw1m5ciUAPXr0YP369fz9998ALFu2jJ49e1qMffXVV3FycsLX15fevXubx9avX58mTZpgb29PzZo1CQ0NvWq/Q4cOxWAw4OjoCEDPnj1xcXHB3t6ewYMHk5uby5EjR6zSL7kx+9IuQERECi8xMRFvb2/s7S0/xlNTU5kwYQLbt28nMzMTo9GIs7OzxTqenp7mx97e3iQnJwPg4eFBs2bNWLt2LY8++ig///wzkydPvu7Y/fv3A3DkyBHeeust9u7dS1ZWFnl5edSvX/+aYwEiIyNZvHgxycnJ2NjY8Pfff5OWlnaLHZGi0hECEZFyyNPTk4SEhKsm6E2ZMgUbGxvWrVvHgQMHmDNnDiaTyWKdxMRE8+OEhARq1Khhfv7kk0/yzTffsGbNGpo0acLdd999U2PHjh1LrVq12LdvHwcOHGDMmDFX7dfGxsb8eNu2bcybN4/IyEj+/PNP4uPjqVq16lVjpOQoEIiIlEONGjWievXqvPvuu5w/f56cnBx27NhBZmYmlSpVwtnZmVOnTjF//vyrxi5atIjExETS09OZM2cOwcHB5tc6d+7M3r17WbBgAb169bpq7KxZs8jOzubAgQMsWbKEbt26AZCVlUWVKlWoXLkyhw8f5rPPPrtu/ZmZmdjb2+Pq6kpeXh4RERHmUxVSOhQIRETKITs7OxYtWsRff/1Fs2bNaNq0KatWreLVV18lLi6OunXrMmDAAB577LGrxoaEhNCvXz9atmzJPffcw4gRI8yvOTo68vjjj/O///2Pxx9//KqxLVq0oFWrVvTu3ZvBgwfTpk0bACZMmMCKFStwdXVl1KhR5qBwLW3btqVdu3a0bt2awMBAHBwcrjqlICXLxlQOj89cfsiqsNzc3EhNTS3GagTUV2tSb63jTu1rYGAg06dP55FHHrnmOhERERw9epQ5c+YUevt3al+tqaSCkiYVioiUUcaUJFgZhSkjDZtqLtC9P7buHlbdZ3p6OosXL+b999+36n6k7NEpAxGRMsiYkoQpYiKmbRvhQBymbRsxRUy8FBKsJCoqimbNmtGuXTtatGhhtf1I2aQjBCIiZdHKKLjyl/8/RwwYNPKWN3u9qwH279+f/v373/K2pXzTEQIRkTLIlFHw9/GvtVykqBQIRETKIJtqLoVaLlJUCgQiImVR9/5w5QRCd49Ly0WsQHMIRETKIFt3D4zhk0v8WwZy51IgEBEpo2zdPYo0gVCkMHTKQERERBQIRERERIFAREREUCAQERERFAhEREQEBQIRERFBgUBERERQIBAREREUCERERAQFAhEREUGBQERERFAgEBERERQIREREBAUCERERQYFAREREUCAQERERFAhEREQEBQIRERFBgUBERERQIBAREREUCERERAQFAhEREUGBQERERFAgEBERERQIREREBAUCERERQYFAREREUCAQERERFAhEREQsBAYGMmfOHNq2bYufnx/h4eHk5OSwZMkSQkJCLNb18vLi2LFjAIwYMYLRo0fTp08ffHx86NmzJydPnrRYd8GCBTz00EPUq1ePt99+G6PRyIULF/D39yc+Pt68bmpqKg8++CBnzpwpkfcMCgQiIiJXWbFiBVFRUWzZsoWjR48ye/bsmx43YsQI4uLi8Pf3Z+jQoRavf/fdd6xdu5YffviBH374ga+++goHBwe6devG8uXLzetFR0fz8MMP4+rqWqzv63oUCERERK4QFhaGl5cXBoOB4cOHs3Llypsa16FDB1q0aIGDgwOjR49m586dJCQkmF9/+eWXMRgMeHl5MWjQIKKjowF48sknWbFiBUajEYBvvvmGnj17Fvv7uh4FAhERkSt4enqaH3t7e5OcnFzocZUqVaJatWoWY6+13caNG+Pk5MSvv/7K4cOH+euvv+jUqVNR30ah2Jfo3kRERMqBxMRE8+OEhARq1KiBk5MT2dnZ5uWnT5++7risrCwyMjKoUaOGxet16tSx2O6/nnzySZYvX467uztdunShYsWKxfqebkRHCERERK6waNEiEhMTSU9PZ86cOQQHB+Pn58fBgwfZu3cvOTk5zJgx46px69evZ/v27eTm5vLee+/RqFEjvLy8zK/Pnz+fjIwMEhISWLBgAd26dTO/1rNnT77//nuWL19Or169SuR9Xk6BQERE5AohISH069ePli1bcs899zBixAgefPBBRowYQZ8+fXj44Ydp3rx5geNmzpyJv78/cXFxfPDBBxavd+7cmccee4xOnTrRoUMH+vbta37N09OTevXqYWNjQ2BgoNXf45V0ykBEROQKDRo0YNiwYVctf+WVV3jllVfMz6+c+Ofi4sK0adOuud327dvz3HPPXfN1Ly8vGjdujI2NzS1UXTQlEghSU1OZO3cuGRkZ2NjYEBQUxOOPP05mZiYRERGkpKTg7u5OeHg4lStXLomSREREypQTJ07w3Xff8cMPP5TK/kskENjZ2fH000/zwAMPkJ2dzZgxY6hfvz4bNmwgICCAkJAQoqOjiY6OJjQ0tCRKEhGRO5QxJQlWRmHKSMOmmgt074+tu0ep1vTee+/x0UcfMXToUO65555SqcHGZDKZSnqn7733Ho8++igLFixg0qRJGAwG0tPTmTRp0k1d/OHyWZyF5ebmRmpq6i2Pl4Kpr9aj3lqH+modZb2vxpQkTBETISXp/y9098AmfHKph4JrufyritZU4pMKT58+zbFjx6hVqxZnz57FYDAAYDAYOHfuXEmXIyIid5KVUZZhAC49XxlVOvWUISU6qfDfr2mEhYXh5OR00+NiYmKIiYkBYOrUqbi5ud1yDfb29kUaLwVTX61HvbUO9dU6ynpf07L+5mIBy+2z/salDNddEkosEOTl5TFjxgxat25t/jqFs7Mz6enp5lMGVatWLXBsUFAQQUFB5udFORxV1g9nlVfqq/Wot9ahvlpHWe+rsVKVApfnVapSZuu+rU4ZmEwmIiMj8fLyomvXrublTZs2ZePGjQBs3LiRZs2alUQ5IiJyp+reH66cK+DucWn5Ha5EjhAcOHCATZs2cc899zBq1CgA+vbtS0hICBEREaxfvx43NzdeffXVkihHRETuULbuHhjDJ5e5bxmUBaXyLYOi0rcMyh711XrUW+tQX61DfS1+t9UpAxERESnbFAhEREREgUBEREQUCG5aYGAgmzZtuu46W7dupUmTJiVUkVxpyZIlhISEXPP10NBQli5desPt3MzPWkSksK712dKuXTu2bt1aChVZUiAoAxQkSsYXX3zBU089VdpliIhY+Pnnn2nZsiUAM2bMuOoui23btuXjjz+2eh0KBLeBvLy80i5BRETKOQWCQrpw4QITJ06kcePGNG7cmIkTJ3LhwgWLdd5//33q1atHYGAgy5cvNy//6aefaNu2LT4+PjRp0oTIyEjOnz/P008/TXJyMrVr16Z27dokJSVhNBr54IMPaNmyJf7+/rz44oukp6cDl26R6eXlxeLFi2nWrNkd+VdvQkICgwYNIiAgAH9/f8aNG2d+bfLkyfj5+dGiRQvWr19vXt6rVy++/PJL8/OoqCjatGmDj48Pbdu2JS4u7qr9HD58mBYtWrBy5UoA1q1bR8eOHfH19aVbt278+eef5nUDAwOJjIwkKCiIunXrMnjwYHJycqzx9kWknLv8s+XfUwk///wzc+bMYdWqVdSuXZugoCCmTp3KL7/8wtChQ6lcuTJDhw4FYP/+/XTs2BEXFxfq1KljcTo0LCyMl19+mS5dulClShUCAwM5cuTIDWtSICik999/n9jYWH788UfWrVvH7t27Le7QmJKSQlpaGjt37mTWrFm8/vrrHD58GIDXXnuNadOmcfDgQX766SdatWqFk5MTn3/+OTVq1ODQoUMcOnQIDw8PFixYwPfff8+yZcuIjY3F2dnZ4pcewK+//srGjRuJirqzbsqRn5/PM888g5eXF9u2bWPnzp10794dgF27dvHggw8SFxfHSy+9xGuvvUZBl9pYvXo1M2fOZPbs2Rw4cICFCxeab7T1r7i4OPr27cvbb79N9+7diYuLY+TIkUybNo29e/cSGhrKs88+axEIV69ezRdffMGvv/5KfHz8Tc1ZEJE7y5WfLf9q164dw4YNo1u3bhw6dIiYmBjGjBlD69at+eCDD8jMzOSDDz4gKyuLjh070q9fP06fPs3ixYsZMmQI+/btM29r8eLFvPnmm6Snp1OrVq2rfn8URIGgkFasWEF4eDhubm64urry6quv8s0331is8/rrr+Pg4MBDDz1Ehw4dWL16NXDpph8HDx7k77//plq1agQEBFxzP1988QWjR4/G09MTBwcHRo4cybfffmtxemDkyJE4OTnh6OhonTdbRu3atYvk5GQmTJiAk5MTFStWpHnz5gB4e3vTv39/7OzseOqpp0hOTiYlJeWqbSxevJiXXnqJhg0bYmNjw/3334+3t7f59e3btxMWFsasWbPo2LEjcOmIQmhoKI0bNzZvv0KFCsTGxprHDRw4EA8PDwwGAx07drT4H1REpKDPlsJas2YN9913H88++yz29vY0btyYnj17smzZMvM6PXr0oHnz5tjb29O/f3927959w+2W6N0ObwfJyckWvzi8vb1JTk42P3d2dra4k+Plr3/00UfMnj2bKVOm4Ovry9ixY2natGmB+zl58iSDBg3C1vb/ZzY7OzuLX24ldfWqsiYxMRFvb2/s7a/+5+vu7m5+/G9QysrKKnAb99577zX38fnnn9OiRQtatWplXpaQkMDXX3/NwoULzctyc3Mtfv5X7v/y10RECvpsKazjx4+zbds2qlWrZl6Wl5fH008/bX7u4fH/L8Xs5OREZmbmDberIwSFVKNGDU6ePGl+npCQQI0aNczPz549y/nz5wt8vWHDhixcuJA9e/bQuXNnBg8eDICNjc1V+/H09OSLL74gPj7e/N/Ro0e5++67zesUNO5O4OnpSUJCQpEmU3p6enL8+PFrvj516lQSEhJ48803zcvuvvtuhg8fbvEzOXLkyHW/6igicrmCPlsuV9Dn+pXLatasSZs2bcjIyDD/l5mZyfz584tUmwJBIXXv3p3Zs2dz5swZ0tLSiIiIoEePHhbr/Pe//yU3N5dt27YRExNDcHAwubm5LF++nHPnznHXXXdRpUoV7OzsgEt/VWZkZHDu3DnzNp5++mmmTZtmDh9nzpzhhx9+KLk3WoY1atSI6tWr8+6773L+/HlycnLYsWNHobbRt29fIiMj+eOPPzCZTBw7dswi6FWqVImoqCi2bdvGu+++C0D//v35/PPPiY2NxWQycf78eWJiYm4qeYuIQMGfLZdzc3PjxIkTGI1G87IaNWpw9OhR8/OuXbty8OBBPv/8cy5evMjFixfZsWMH8fHxRapNgaCQXnnlFRo0aEBQUBAdOnQgICCAV155xfy6u7s7zs7ONG7cmKFDhzJ16lRq1aoFwDfffEOLFi2oU6cOn3/+OXPmzAGgVq1adO/enYceeghfX1+SkpIYNGgQHTt2pG/fvvj4+BAcHGxxrvpOZmdnx6JFi/jrr79o1qwZTZs2ZdWqVYXaRnBwMMOHD+fll1/Gx8eH5557joyMDIt1nJ2dWbx4MevXr+e9996jQYMGTJ8+nfHjx+Pn50erVq00aVBECu3Kz5bLde3aFYB69erRuXNn4NLvnWXLlmEwGBg+fDhVqlThxx9/5KuvvsLT0xMPDw9Gjx591TfeCkt3O5Riob5aj3prHeqrdaivxa+k5otpUuFljClJukd2GaGfhYiUN+X9c0uB4B/GlCRMERMhJQkAE8DRAxjDJ5erH+jtQD8LESlvbofPLc0h+NfKKPMP0uyftCclTD8LESlvboPPLQWCf5gy0gq1XKxHPwsRKW9uh88tBYJ/2FRzKdRysR79LESkvLkdPrcUCP7VvT9ceZ7H3ePScilZ+lmISHlzG3xuaVLhP2zdPTCGTy7XM0RvF/pZiEh5czt8buk6BFIs1FfrUW+tQ321DvW1+JXUdQh0ykBEREQUCERERESBQERERFAgEBERERQIREREBAUCERERQYFAREREUCAQERERFAhEREQEBQIRERFBgUBERERQIBAREREUCEREpAQEBgayadOm0i5DrkOBQEREbnsjRoxg2rRppV1GmaZAICIicgN5eXmlXYLVKRCIiEiJ+PPPPwkKCqJu3boMHjyYnJwcMjIyGDBgAAEBAfj5+TFgwAASExPNY3r16sW0adPo1q0btWvX5plnniEtLY2hQ4dSp04dHn/8cU6cOAGAyWTizTffpH79+tStW5egoCD279/PF198wYoVK5g/f755GwBJSUk8//zzBAQE0KJFCxYsWGDe74wZM3j++ecZNmwYderUYenSpSXbrFKgQCAiIiVi9erVfPHFF/z666/Ex8ezdOlSjEYjvXv3Zvv27Wzfvp2KFSsyfvx4i3GrVq3i/fffZ+fOnRw/fpxu3brx1FNPsW/fPmrVqsXMmTMB2LhxI9u2beOXX34hPj6e+fPnYzAYCA0N5YknnuCll17i0KFDLFq0CKPRSFhYGH5+fuzcuZMlS5bw8ccfs2HDBvN+f/zxR7p06UJ8fDxPPPFESbaqVCgQiIhIiRg4cCAeHh4YDAY6duzIvn37cHFxoUuXLjg6OlK5cmWGDx/Ob7/9ZjHuqaee4r777qNq1aq0a9eO++67j0ceeQR7e3u6du3K3r17AbC3tyczM5PDhw9jMpmoXbs2NWrUKLCW3bt3c+bMGcLDw6lQoQL33nsv/fr1Y+XKleZ1mjRpwqOPPoqtrS2Ojo7Wa0wZYV/aBYiIyJ3B3d3d/NjR0ZHk5GSys7N588032bBhA2fPngUgMzOT/Px87OzsrhpXsWJF3NzcLJ6fP38egIcffphnn32WcePGkZCQwGOPPcaECROoUqXKVbWcPHmS5ORkfH19zcvy8/MJDAw0P/f09Cymd14+KBCIiEipiYyM5OjRo6xZs4bq1auzd+9eOnfujMlkuqXtPffcczz33HOkpqYyePBg5s+fz+uvv46NjY3Fep6entSsWZMtW7Zcc1tXjrnd6ZSBiIiUmqysLCpWrEjVqlVJT08nIiLilre1e/duYmNjuXjxIk5OTjg4OFgcZfjf//5nXrdRo0ZUqVKFuXPnkp2dTX5+Pvv372f37t1FfUvllgKBiIiUmkGDBpGTk0NAQADBwcG0bdv2lrf1999/8/rrr+Pn50fz5s0xGAwMHjwYgD59+nDw4EF8fX0ZOHAgdnZ2fPrpp+zbt4+HHnqIgIAAXnvtNc6dO1dM76z8sTHd6nGZUnT5V1IKy83NjdTU1GKsRkB9tSb11jrUV+tQX4tfSc1l0BwCEREpMmNKEqyMIi3rb4yVqkD3/ti6e5R2WVIICgQiIlIkxpQkTBETISWJi/8uPHoAY/hkhYJyRHMIRESkaFZGQUqS5bJ/jhhI+aFAICIiRWLKSCvUcimbFAhERKRIbKq5FGq5lE0KBCIiUjTd+8OVcwXcPS4tl3JDkwpFRKRIbN09MIZPhpVR2Gf9TZ6+ZVAuKRCIiEiR2bp7wKCRuOg6BOWWThmIiIiIAoGIiIgoEIiIiAgKBFIEJ06cwMvLi7y8vNIuRUREikiBoIwJDAxk06ZNpbLvGTNmMGzYsFLZt4iIlC4FgttIfn5+aZcgIiLllAJBGTJs2DASEhJ49tlnqV27NvPmzeOFF16gYcOG1K1blx49enDgwAHz+iNGjGDMmDE8/fTT1KpViy1bthATE0OnTp2oU6cOTZs2ZcaMGeb1/z3Ev3TpUpo1a0a9evWYPXs2AD///DNz5sxh1apV1K5dm6CgIODqIxbXO4qwZMkS2rRpg4+PDw899BCff/65NdokIiJWoEBQhsyZMwcvLy8WLlzIoUOHGDJkCO3bt2fz5s3s2bOHevXqMXToUIsx0dHRDB8+nIMHD9K8eXOcnJyYPXs28fHxfPbZZ3z22Wd8//33FmN27NjBpk2bWLJkCbNmzeLQoUO0a9eOYcOG0a1bNw4dOkRMTEyh63d1dWXRokUcOHCAmTNnMmnSJOLi4orUk+Lm5eXFsWPHCnxtyZIlhISEmJ/v2LGDVq1aUbt2bb7//ntCQ0NZunRpkfYhIlJW6cJEZVyfPn3Mj0eOHImfnx/nzp2jatWqAHTq1IlmzZoBULFiRVq2bGle38/Pj+7du/Prr7/y6KOPmpeHh4fj6OiIv78/fn5+/Pnnn9SuXbvItf57VAHgoYceok2bNmzbto2AgIAib7s0TJ8+nWeffZZBgwYBWPRQROR2o0BQhuXn5zNt2jTWrFnDmTNnsLW9dEAnLS3NHAg8PT0txsTGxvLuu+9y4MABLl68SG5uLl26dLFYp3r16ubHjo6OZGVlFUu969evZ+bMmRw7dgyj0Uh2djZ169Ytlm2XhoSEBHx8fEq7DBGREqFTBmWMjY2N+fGKFSv44Ycf+Oqrr9i/fz+//fYbACaTqcD1AYYOHUqnTp3YsWMH+/fvJzQ09Jb2/S8nJydycnLMz0+fPl3g2AsXLvD8888zePBgdu/eTXx8PO3bt7eotTjFxcXRqVMnfHx8eOGFFxg8eDDTpk0DICoqilatWuHv709YWBhJSUkFbiMtLY2wsDDq1KlDly5dOH78uPm1li1bcvz4cfN8jgsXLtCrVy++/PJLAI4dO0bPnj2pW7cu9erVY/DgwRbb/uWXX2jVqhV+fn688cYbVuuDiEhxUSAoY9zc3Pjf//4HQGZmJhUqVMBgMJCdnc3UqVNvOD4zM5Nq1apRsWJFdu3aRXR0dKH2feLECYxGo3mZv78/K1eu5OLFi+zZs4dvv/22wLG5ubnk5ubi6uqKvb0969evZ+PGjTe978LIzc3lueee46mnnmLfvn2EhISY50ls3ryZKVOmEBkZSWxsLN7e3gwZMqTA7YwbNw4HBwd27drFjBkzWLJkifm1rVu3WszncHBwsBg7ffp0HnnkEf78809+//13Bg4caPF6TEwMa9euZd26daxevZoNGzYUbxNERIqZAkEZM2zYMGbPno2vry8ZGRl4e3vTpEkT2rZtS+PGjQHYs2cPrVu3LnD8u+++y3//+198fHyIiIggODj4pvfdtWtXAOrVq0fnzp0BGDVqFMePH8fPz4///ve/PPHEEwWOrVKlCm+//TaDBw/Gz8+PFStW0KlTp8K89ZsWGxtLfn4+zz33HHfddRePP/44DRs2BC4dVenTpw8BAQE4ODgwduxYdu7cyYkTJyy2kZ+fz9q1axk1ahROTk7UrVuXXr163XQN9vb2JCQkkJSURMWKFWnevLnF60OHDsXZ2RkvLy9atmzJvn37ivy+RUSsSXMIypjOnTubfxkX5MknnwSwmA0fGBho/ou1a9eu5l/sV6pZsyYJCQkWy5YtW2Z+7OLictURhXvvvZc1a9bc1PbCwsIICwu7Zu3FJTk5GQ8PD4tTHP/OpUhOTraYxFipUiUMBgNJSUnUrFnTvPzMmTPk5eVZzMHw9vZm27ZtN1XD+PHjmT59Ol27dsXZ2ZkXX3zRYgKou7u7+XFxztMQEbEWBYISZkxJgpVRmDLSsKnmonuG34Lq1auTlJSEyWQyh4LExETuvfdeatSowcmTJ83rnj9/nvT0dDw8LHv876mNxMREatWqBXBVWLpRDdOnTwdg+/bt9OnTh8DAQO6///6ivj0RkVKhUwbXEBgYyPz58wkKCqJWrVqMHDmSlJQUQkND8fHxoXfv3mRkZADw448/0q5dO3x9fenVqxeHDh0yb2fu3Lk0adIEHx8fWrdsyS+vD8G0bSM5f+4h/P8W4d88kLatH2b+/Pk0adLEPO7K77KPGDHCPGlu69at5nULupjRjWoq75o0aYKtrS0LFy4kLy+PH374gd27dwOXjpwsWbKEvXv3cuHCBaZOnUqjRo0sjg4A2NnZ8dhjjzFjxgyys7M5ePAgX3/99U3XsHr1ahITEwFwdnbGxsYGOzu7YnuPIiIlTYHgOr799lsWL17ML7/8wrp16wgNDWXMmDHExcVhNBr55JNPOHLkCEOGDOGtt97ijz/+oH379oSFhZGbm8vhw4dZuHAh3377LQcPHuSLJ4LwvnDp0HHEoUSOn7/A5kf8+SIkqFC/jC5X0MWMrlfT7aBChQp8/PHHLF68GF9fX7755huCgoKoUKECrVu3ZtSoUbzwwgs0btyYv/76yxySrvTOO++QlZVFw4YNCQ8Pp3fv3jddw549ewgODqZ27do8++yzvPXWW9xzzz3F9RZFREqcThlcx8CBA83nggMDA3F1daVevXoAPPbYY2zevBkbGxs6dOjAI488AsDgwYNZsGABv//+O3fffTe5ubkcPHgQV1dXvG3yodKl2eprTqXzTr17qFbBnmqmiwwcOJCIiIhiqXvVqlXXrOnyCxeVZw0aNGDdunXm5127dqVjx44ADBgwgAEDBhQ47vLTAq6urnz22WfX3MeV8wkun28xfvx4xo8ff8N9AMyaNeua+xARKSt0hOA63NzczI8rVqxoMVGsYsWKZGVlkZycjLe3t3m5ra0td999N0lJSdx///289dZbzJw5kwYNGvDyul9Jyrn0V/rpCxe5u2IFAGyquVhso6iuV9Pt4tdff+X06dPk5eWxdOlS4uPjadu2bWmXJSJSbikQFNGVk9hMJhOnTp0yT2J74okniI6OZtu2bdh438eUv9IAqO5wF6dycsHdA7r3v+qvSkdHR7Kzs83PU1JSrlnDlRcUulFN5YExJQnjxzPI/+84jB/PuDQZ8zJHjhyhY8eO1K1bl//7v//jww8/pEaNGqVUrYhI+adAUETBwcH89NNP/PLLL1y8eJEPP/yQChUq0LRpUw4fPszmzZu5cOECDg4OODpXw86/ETaBbeji78Pc09mcfe41ki4aWbhwocV2/f39iY6OJj8/n59//tl8lcKCXH4xoxvVVB4YU5IwRUzEtG0jHIjDtG0jpoiJFqEgNDSUPXv2cPjwYWJiYizuoyAiIoWnQFBEtWrVYs6cOUyYMIGAgADWrVvHp59+SoUKFcjNzWXKlCkEBATQqFEjUlNTGTvpLWwHjeS1r6LxbtqCVl27069fP3r27Gmx3cmTJ7Nu3Tp8fX1Zvnz5da9NcPnFjCIjI69bU7mwMgquOCLAP1/XFBER67AxlcOLrP/7da9b4ebmRmpqajFWUzy2bt3KsGHD2LlzZ2mXckuKs6/5/x0HBwq4bXKdAOxee6dY9lGelNV/s+Wd+mod6mvxu/ImdtZyx37LQBcIKrtsqrlQUEq1qeZS4rWIiNwp7shA8O856n8PS5sAjh7AGD5ZoaAs6N4fjh6wPG3wz+RLERGxjjtzDkEZPEfdsmXLcnu6oLjZuntgEz4Zm8A2UCcAm8A22CisiYhY1R15hMCUkVao5VLybN09YNDI0i5DROSOUeqBYPfu3SxcuBCj0UiHDh0s7uJnLTpHLSIiYqlUTxkYjUYWLFjAG2+8QUREBFu2bLG4oI7VdO9/6Zz05XSOWkRE7mCleoTg8OHDeHh4mK8w17JlS3bs2FGsl/EtiK27B8bwyfqWgYiIyD9KNRCkpaXh6upqfu7q6lrgbXpjYmKIiYkBYOrUqRb3GCgse3v7S+Pd3MB3yi1vRyyZ+yrFTr21DvXVOtTX8qtUA0FB10S68rr8AEFBQRaXpi3KRS900QzrUF+tR721DvXVOtTX4ldSFyYq1TkErq6unDlzxvz8zJkzGAyGUqxIRETkzlSqgeDBBx/k1KlT5tvYbt26tdzcgEdEROR2UqqnDOzs7Bg4cCDvvPMORqORdu3aUbNmzdIsSURE5I5U6tchaNy4MY0bNy7tMkRERO5od+ali0VERMSCAoGIiIgoEIiIiIgCgYiIiKBAICIiIigQiIiICAoEIiIiggKBiIiIoEAgIiIiKBCIiIgIYGMq6B7EIiIicke5444QjBkzprRLuC2pr9aj3lqH+mod6mv5dccFAhEREbmaAoGIiIjceYEgKCiotEu4Lamv1qPeWof6ah3qa/mlSYUiIiJy5x0hEBERkavZl3YBJWX37t0sXLgQo9FIhw4dCAkJKe2Syq3U1FTmzp1LRkYGNjY2BAUF8fjjj5OZmUlERAQpKSm4u7sTHh5O5cqVS7vccsdoNDJmzBhcXFwYM2aM+loMsrKyiIyM5MSJE9jY2PDSSy/h6empvhaDNWvWsH79emxsbKhZsyZDhgwhNzdXvS2H7ohTBkajkVdeeYXx48fj6urK2LFjeeWVV/D29i7t0sql9PR00tPTeeCBB8jOzmbMmDGMGjWKDRs2ULlyZUJCQoiOjiYzM5PQ0NDSLrfcWbNmDUeOHDH39osvvlBfi+iDDz7A19eXDh06kJeXx4ULF1ixYoX6WkRpaWlMmDCBiIgIKlSowMyZM2ncuDEnT55Ub8uhO+KUweHDh/Hw8KBGjRrY29vTsmVLduzYUdpllVsGg4EHHngAAEdHR7y8vEhLS2PHjh20adMGgDZt2qjHt+DMmTPExsbSoUMH8zL1tWjOnz9PfHw87du3B8De3p5KlSqpr8XEaDSSm5tLfn4+ubm5GAwG9bacuiNOGaSlpeHq6mp+7urqyqFDh0qxotvH6dOnOXbsGLVq1eLs2bMYDAbgUmg4d+5cKVdX/nz66aeEhoaSnZ1tXqa+Fs3p06epWrUq8+bN4/jx4zzwwAOEhYWpr8XAxcWF4OBgXnrpJSpUqECDBg1o0KCBeltO3RFHCAo6K2JjY1MKldxecnJymDFjBmFhYTg5OZV2OeXezp07cXZ2Nh99keKRn5/PsWPH6NSpE++99x4ODg5ER0eXdlm3hczMTHbs2MHcuXP58MMPycnJYdOmTaVdltyiO+IIgaurK2fOnDE/P3PmjDm9yq3Jy8tjxowZtG7dmsDAQACcnZ1JT0/HYDCQnp5O1apVS7nK8uXAgQP8/vvv7Nq1i9zcXLKzs3n//ffV1yJydXXF1dWV2rVrA9CiRQuio6PV12IQFxdH9erVzb0LDAzk4MGD6m05dUccIXjwwQc5deoUp0+fJi8vj61bt9K0adPSLqvcMplMREZG4uXlRdeuXc3LmzZtysaNGwHYuHEjzZo1K60Sy6V+/foRGRnJ3LlzGTFiBPXq1WP48OHqaxFVq1YNV1dXEhMTgUu/xLy9vdXXYuDm5sahQ4e4cOECJpOJuLg4vLy81Nty6o74lgFAbGwsixYtwmg00q5dO3r06FHaJZVb+/fvZ+LEidxzzz3mUy99+/aldu3aREREkJqaipubG6+++qq+anSL9u3bx+rVqxkzZgx///23+lpEf/31F5GRkeTl5VG9enWGDBmCyWRSX4vB0qVL2bp1K3Z2dtx3330MHjyYnJwc9bYcumMCgYiIiFzbHXHKQERERK5PgUBEREQUCERERESBQERERFAgEBERERQIREREBAUCkTvWpEmT+Omnn0q7DBEpIxQIRERERBcmErkdpKam8umnnxIfH4/JZKJVq1ZUqVKFpKQkhg8fDly669/QoUNZvHgxS5cuJTo6Gnt7e2xtbWnbti3PPfccCQkJfPLJJxw9epSqVavSu3dvWrZsCcDcuXNxcHAgJSWF+Ph4vL29GT58OB4eHgDXHRsbG8vnn3/OmTNncHR0pEuXLnTr1o1z584xb9489u/fj42NDTVr1mTSpEnY2upvFZGSdkfc3EjkdmY0Gpk2bRr+/v7MnTsXW1tbjh49yh9//HHNMX379uXAgQO0bt2aDh06AJfuXvmf//yHp556ijfeeIPjx4/zzjvvULNmTWrWrAnAli1bGDduHPfffz9z587lq6++YsSIETccGxkZSXh4OL6+vmRmZnL69GkA1qxZg4uLCx9//DEAhw4d0p1IRUqJYrhIOXf48GHS0tJ4+umnqVixIhUqVKBu3bqF3k5sbCzu7u60a9cOOzs7HnjgAQIDA/ntt9/M6wQGBlKrVi3s7Ox4+OGH+euvv25qrJ2dHSdPnuT8+fNUrlzZfItnOzs7MjIySE1Nxd7eHl9fXwUCkVKiIwQi5Vxqairu7u7Y2dkVaTspKSkcOnSIsLAw87L8/HweeeQR8/Nq1aqZHzs4OJCTk3NTY0eOHMny5cv58ssvueeee+jfvz8+Pj5069aNr7/+mv/85z8ABAUFERISUqT3ISK3RoFApJxzc3MjNTWV/Px8i1BQsWJFcnNzzc8zMjIsxl35l7irqyt+fn5MmDCh0DXcaGytWrV4/fXXycvL4/vvvyciIoL58+fj6OjIgAEDGDBgACdOnOCtt97iwQcfJCAgoNA1iEjR6JSBSDlXq1YtDAYDUVFR5OTkkJuby/79+7nvvvuIj48nNTWV8+fPEx0dbTHO2dmZ5ORk8/MmTZpw6tQpNm3aRF5eHnl5eRw+fJiTJ0/esIbrjc3Ly+OXX37h/Pnz2Nvb4+TkZJ40uHPnTpKSkjCZTDg6OmJra6sJhSKlREcIRMo5W1tbRo8ezSeffMKQIUOwsbGhVatWDBw4kIceeojXXnuNKlWq0L17d37//XfzuMcff5y5c+eybt06WrduzcCBAxk/fjyLFi1i0aJFmEwm7r33Xp555pkb1uDo6HjdsZs2beKTTz7BaDTi6enJsGHDADh16hSffPIJ586do1KlSnTq1Al/f3/rNEpErktfOxQRERGdMhAREREFAhEREUGBQERERFAgEBERERQIREREBAUCERERQYFAREREUCAQERERFAhEREQE+H95nX3Ao9btoAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 8))\n",
    "plt.scatter([item[0] for item in animal_lookup.values()],\n",
    "            [item[1] for item in animal_lookup.values()])\n",
    "plt.xlabel('cuteness')\n",
    "plt.ylabel('size')\n",
    "for label, (cute, size) in animal_lookup.items():\n",
    "    plt.text(cute+1, size+1, label, fontsize=12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot shows us that the closest animal to the capybara is the panda bear (again, in terms of its subjective size and cuteness). One way of calculating how \"far apart\" two points are is to find their *Euclidean distance*. (This is simply the length of the line that connects the two points.) For points in two dimensions, Euclidean distance can be calculated with the following Python function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "def distance2d(a, b):\n",
    "    return math.sqrt((a[0] - b[0])**2 + (a[1] - b[1])**2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(The `**` operator raises the value on its left to the power on its right.)\n",
    "\n",
    "So, the distance between \"capybara\" (70, 30) and \"panda\" (74, 40):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11.180339887498949"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distance2d(animal_lookup['capybara'], animal_lookup['panda bear']) # panda and capybara"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... is less than the distance between \"tarantula\" and \"elephant\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "104.0096149401583"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distance2d(animal_lookup['tarantula'], animal_lookup['elephant']) # tarantula and elephant"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modeling animals in this way has a few other interesting properties. For example, you can pick an arbitrary point in \"animal space\" and then find the animal closest to that point. If you imagine an animal of size 25 and cuteness 30, you can easily look at the space to find the animal that most closely fits that description: the chicken.\n",
    "\n",
    "Reasoning visually, you can also answer questions like: what's halfway between a chicken and an elephant? Simply draw a line from \"elephant\" to \"chicken,\" mark off the midpoint and find the closest animal. (According to our chart, halfway between an elephant and a chicken is a horse.)\n",
    "\n",
    "You can also ask: what's the *difference* between a hamster and a tarantula? According to our plot, it's about seventy five units of cute (and a few units of size).\n",
    "\n",
    "The relationship of \"difference\" is an interesting one, because it allows us to reason about *analogous* relationships. In the chart below, I've drawn an arrow from \"tarantula\" to \"hamster\" (in blue):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfMAAAHjCAYAAAA32fbQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABWwUlEQVR4nO3deXxM9/7H8dckIYslsoostloTQWyxtWijqrWkpbVWVbVXFRVLUdX1KuoSy8/S26pqaatVonTlUqpaVGhRe1ASQRZLSMRk5veHmgpBIpPl8H4+Hh6dOXO+53zzKfPO+Z7vOcdktVqtiIiIiGE5FHUHREREJH8U5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg3Mq6g7cjoSEhNtu6+3tTVJSkh17c/dSLe1HtbQf1dI+VEf7yW8t/f39b7mOjsxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXETEjqZMmcLgwYPzvZ2jR48SEBCA2Wy2Q6/kTqcwFxG5yyxevJjIyMii7obYkcJcRETE4BTmIiK3ITExkWeffZbQ0FCaNm3KvHnzclxv69atdOrUidq1axMREcHGjRttn3Xt2pUJEybwyCOPUKtWLZ5++mlSU1OztV+6dCmNGzemTp06TJ8+3bZ827ZtdOzYkdq1axMWFsbYsWPJzMy0fR4QEMBHH31EixYtCA4O5uWXX8ZqtbJ//37GjBnD1q1bqV69OrVr17ZzZaQoKMxFRPLIYrHQt29fgoOD2bp1K4sXL+b999/nxx9/zLbe8ePH6dOnDy+++CK7du3i1Vdf5dlnnyU5Odm2zpIlS5gyZQqxsbE4OTnx6quvZtvGli1bWL9+PYsXL2batGns378fAEdHR15//XV27NjBV199xYYNG1iwYEG2tqtXr+abb75h1apVrFixgh9//JHq1aszYcIEGjZsyP79+9m9e3fBFEkKlcJcRCSPtm/fTnJyMlFRUZQsWZJKlSrRs2dPli9fnm29pUuXcv/99/PAAw/g4ODAfffdR7169fjf//5nW6dLly7UqlULNzc3Ro4cyYoVK8jKyrJ9HhUVhaurKyEhIQQHB/Pnn38CULduXRo2bIiTkxNBQUH07t2bX3/9Ndv+Bw0ahLu7OwEBATRv3pxdu3YVYFWkKBnydq4iIkXp2LFjnDhxItsQdVZWFuHh4QQGBmZb7+uvv2b16tW2ZZcuXaJ58+a291ffqjMwMJBLly6RkpJiW+br62t77erqyvnz5wE4ePAgb7zxBn/88Qfp6emYzWbq1q2brZ8+Pj45tpU7j8JcRCSP/P39CQoK4ueff77usylTpmRbr0uXLkyePPmG27r6WRPx8fGUKFECT0/PWz6DYsyYMdSpU4fZs2dTunRp3nvvPb7++utc9d9kMuVqPTEODbOLiORRWFgYZcqUYdasWaSnp5OVlcWePXvYvn17tvUee+wxVq1axY8//khWVhYZGRls3LgxW1B/+eWX7Nu3j/T0dCZPnswjjzyCo6PjLftw/vx5ypQpQ6lSpThw4AAfffRRrvvv4+PD8ePHs02YE2NTmIuI5JGjoyMffvghu3btolmzZoSGhjJixAjOnj2bbb2AgAA++OADZs6cSd26dWncuDFz5szBarXa1unatStRUVHUr1+fixcv8tZbb+WqD+PGjWPZsmXUqFGDkSNH0qlTp1z3v0WLFtSoUYOwsDDq1KmT63ZSfJmsV/+tMgg9ArV4UC3tR7W0HyPVsmvXrjz22GP07NmzqLtyHSPVsbgrjEeg6py5iEgOLKcSYfkirKdTMJXzhM69cPDxK+puieRIYS4icg3LqUSs0a/CqUQArABxe7FEvalAl2JJYS4icq3li2xBbvP3kTr9h9ttN0uWLLHbtuTupglwIiLXsJ5OydNykaKmMBcRuYapnGeelosUNYW5iMi1OveCa8+N+/hdXi5SDOmcuYjINRx8/LBEvanZ7GIYCnMRkRw4+PjZdbKbSEHSMLuIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicE6FtaOVK1eyZs0aTCYTQUFBDBw4kMzMTKKjozl16hQ+Pj5ERUVRunTpwuqSiIjIHaFQjsxTUlL49ttvmThxIlOmTMFisbBx40ZiYmIIDQ1lxowZhIaGEhMTUxjdERERuaMU2jC7xWIhMzOTrKwsMjMz8fDwYMuWLbRq1QqAVq1asWXLlsLqjoiIyB2jUIbZPT096dixI88//zwlS5akXr161KtXjzNnzuDh4QGAh4cHZ8+eLYzuiIiI3FEKJczT0tLYsmULs2bNws3NjalTp7J+/fpct1+9ejWrV68GYOLEiXh7e992X5ycnPLVXv6hWtqPamk/qqV9qI72Uxi1LJQw37FjB76+vpQtWxaA8PBw9u3bh7u7O6mpqXh4eJCammr7/FoRERFERETY3iclJd12X7y9vfPVXv6hWtqPamk/qqV9qI72k99a+vv733KdQjln7u3tzf79+7l48SJWq5UdO3YQEBBAo0aNWLduHQDr1q2jcePGhdEdERGRO0qhHJlXr16dpk2bMmrUKBwdHalcuTIRERFkZGQQHR3NmjVr8Pb2ZtiwYYXRHRERkTuKyWq1Wou6E3mVkJBw2201dGQ/qqX9qJb2o1rah+poP3fMMLuIiIgUHIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIG51RYOzp//jxz587l6NGjmEwmnn/+efz9/YmOjubUqVP4+PgQFRVF6dKlC6tLIiIid4RCC/P58+dTv359hg8fjtls5uLFiyxbtozQ0FAiIyOJiYkhJiaG3r17F1aXRERE7giFMsx+4cIFdu/ezf333w+Ak5MTpUqVYsuWLbRq1QqAVq1asWXLlsLojoiIyB2lUI7MT548SdmyZZk9ezZHjhyhatWq9O3blzNnzuDh4QGAh4cHZ8+eLYzuiIiI3FEKJcyzsrI4dOgQ/fr1o3r16syfP5+YmJhct1+9ejWrV68GYOLEiXh7e992X5ycnPLVXv6hWtqPamk/qqV9qI72Uxi1LJQw9/LywsvLi+rVqwPQtGlTYmJicHd3JzU1FQ8PD1JTUylbtmyO7SMiIoiIiLC9T0pKuu2+eHt756u9/EO1tB/V0n5US/tQHe0nv7X09/e/5TqFcs68XLlyeHl5kZCQAMCOHTsIDAykUaNGrFu3DoB169bRuHHjwuiOiIjIHaXQZrP369ePGTNmYDab8fX1ZeDAgVitVqKjo1mzZg3e3t4MGzassLojIiJyxyi0MK9cuTITJ068bvmrr75aWF0QERG5I+kOcCIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBeR2xYeHs769euLuhsidz2FuYiIiMEpzEWkSJnN5qLugojhKcxFJF/+/PNPIiIiqFWrFgMGDCAjIwOARYsW0aJFC0JCQujbty+JiYm2NgEBAXz44Ye0aNGCli1bYrVaee2116hbty61atUiIiKCPXv2AHDx4kXefPNNGjduTL169Rg1ahTp6elF8rOKFFcKcxHJlxUrVrBw4UJ++eUXdu/ezeeff86GDRuYMGECc+fOJTY2lsDAQAYOHJit3XfffcfKlStZu3Yt69atY9OmTfz000/s3r2bOXPm4OHhAcD48eOJi4vjhx9+4OeffyYxMZFp06YVwU8qUnwpzEUkX/r164efnx8eHh60bduWXbt2sWzZMrp3705oaCjOzs6MGTOGrVu3cvToUVu7QYMG4eHhgaurK05OTqSlpXHgwAGsVivVq1enfPnyWK1WPvnkE15//XU8PDwoXbo0gwcPZvny5UX4E4sUP05F3QERMTYfHx/ba1dXV06cOEFqaiqhoaG25aVKlcLDw4PExESCgoIA8Pf3t33esmVLnn76acaOHUt8fDzt27dn3LhxXLx4kfT0dNq3b29b12q1kpWVVQg/mYhx6MhcROyufPnyHDt2zPb+woULpKam4ufnZ1tmMpmytXnmmWf47rvvWLt2LXFxccyZMwdPT09cXFxYs2YNu3fvZvfu3ezZs4f9+/cX2s8iYgQKcxGxu8jISBYvXszOnTu5ePEiEydOJCwszHZUfq3t27cTGxvLpUuXcHNzw9nZGUdHRxwcHOjVqxevv/46SUlJABw/fpwff/yxEH8akeJPYS4idnfvvfcycuRInnvuORo0aMDhw4eZPXv2Ddc/d+4cL730EsHBwTRp0gQPDw8GDBgAwMsvv0zlypXp2LEjNWvWpHv37hw8eLCwfhQRQzBZrVZrUXcirxISEm67rbe3t+03fMkf1dJ+VEv7US3tQ3W0n/zW8ur5JTeiCXAiclOWU4mwfBHW0ymYynlC5144+PjduqGIFBqFuYjckOVUItboV+HU5Ru+WAHi9mKJelOBLlKM6Jy5iNzY8kW2ILf5+0hdRIoPhbmI3JD1dEqelotI0VCYi8gNmcp55mm5iBQNhbmI3FjnXnDtuXEfv8vLRaTY0AQ4EbkhBx8/LFFvaja7SDGnMBeRm3Lw8YP+w4u6GyJyExpmFxERMTiFuYiIiMEpzEVERAxOYS4id52hQ4cyadKkW64XEBDAoUOHcrXNm627dOlSevTokac+iuSFJsCJiBSwxx57jMcee6youyF3MB2Zi4iIGFyewvyPP/5gzpw5TJw4EYCDBw+yc+fOAumYiIi97Ny5k3bt2lGjRg0GDBjAxYsXbZ8tWrSIFi1aEBISQt++fUlMTMxxG0OHDmXUqFF0796dGjVq0KVLF44dO5ZtnZ9++okWLVoQHBzMyy+/zJUnTC9evJjIyEjbegEBAXz00Uc5rityO3Id5t9++y3vvfceFSpUYPfu3QCULFmSzz77rMA6JyKSX5mZmfTr148uXbqwa9cuOnTowDfffAPAhg0bmDBhAnPnziU2NpbAwEAGDhx4w20tW7aMoUOHsmPHDkJCQhg0aFC2z1evXs0333zDqlWrWLFiBT/++OMNt5WXdUVuJddh/s033zBu3DgiIyNxcLjcLCAggISEhALrnIhIfsXGxmI2m3n22WcpUaIEHTp0oF69esDlcO7evTuhoaE4OzszZswYtm7dytGjR3Pc1gMPPEDTpk1xdnZm1KhRbN26lfj4eNvngwYNwt3dnYCAAJo3b86uXbtu2K+8rCtyK7kO8/T0dLy9vbMtM5vNODlpDp2IFF8nTpzAz88Pk8lkWxYYGGj77MprgFKlSuHh4XHDoXZ/f/9s65YrV44TJ07Ylvn4+Nheu7q6cv78+Rv2Ky/ritxKrsO8du3axMTEZFv27bffEhISYu8+iYjYja+vL4mJidnOSV85mi5fvny2894XLlwgNTUVP7+c7z1/9Ujk+fPnOX36NOXLly+gnovkXq7DvF+/fmzevJkXXniBjIwMXnzxRX799VeeeuqpguyfiEi+NGzYEEdHR+bNm4fZbOabb75h+/btAERGRrJ48WJ27tzJxYsXmThxImFhYQQFBeW4rTVr1rB582YyMzN55513CAsLIyAgoBB/GpGc5XqM3MPDgwkTJnDw4EFOnTqFl5cX1apVs50/FxEpjkqWLMn777/PyJEjeeedd7j//vtp3749APfeey8jR47kueee48yZMzRs2JDZs2ffcFuRkZFMnTqVrVu3Ehoayv/93/8V1o8hclMmay6vh1iyZAmNGzemUqVK2ZbHxMRku+SiMORn0p23tzdJSUl27M3dS7W0H9XSfgqqlkOHDqVChQqMGjXK7tsujvR30n7yW8ur52rcSK4Pq7/88kv+/e9/88svv2RbvmzZsrz3TEREROwm18PsJUqUYOzYsUyePJkjR47QvXt3AN3oQESKBcupRFi+iJTz57CUKgOde11+FrvIXSDXYW4ymahcuTITJkxg6tSpvPPOOwwZMiTb5R4iIkXBcioRa/SrcCqRS1cWxu3FEvWm3QJ92rRpdtmOSEHI9TD7lSPwsmXLMm7cOMqVK8eYMWMwm80F1jkRkVxZvghOXXNt+N9H6iJ3g1yHeevWrW2vHR0dee6553j44YepUaNGQfRLRCTXrKdT8rRc5E6T62H2fv36Xbesbdu2tG3b1q4dEhHJK1M5T3KavWMq51nofREpCjcN83fffZd//etfADe9nvLahw2IiBSqzr0gbm/2oXYfv8vLRe4CNw1zX19f22vdslBEiisHHz8sUW/C8kU4nT+HWbPZ5S5z0zB/9NFHba9r166Nr68vvr6+pKamsmjRIhwcHOjZs2eBd1JE5FYcfPyg/3A8dbMTuQvlegLcvHnzbLdu/eijj8jKysJkMvHuu+8WWOdERETk1nI9AS4lJQVvb2+ysrLYvn07c+bMwcnJyXZOXURERIpGrsPc1dWV06dPc/ToUYKCgnBxccFsNus6cxERkSKW62H2hx56iDFjxjBjxgzatWsHwJ49e/T4P4NYvHhxtgfiVK9enSNHjgCXHyAxadKkIuqZiNwJwsPDWb9+fZ7bHT16lICAAB0Y5lOuj8wjIyNp0qQJDg4O+PldniHq6enJgAEDCqxzUnD2799f1F0QERE7yXWYw/WPYcvNY9nk5sxmM05OefrfICIityErKwtHR8ei7kaByPUwu+RdfHw8/fv3JzQ0lJCQEMaOHcvixYvp3Lkzr732GiEhIUyZMoWzZ88yZMgQQkNDadKkCdOmTcNisdi2s2jRIlq1akWNGjVo3bo1O3bsAC4fXXft2pXatWvTpk0bfvjhB1ublJQU+vbtS82aNXnkkUdsQ+pXBAQEcOjQoRz7vWrVKtq2bUvt2rXp1KkTf/75ZwFUR0QKWnh4ODNnzqR169YEBwcTFRVFRkYGAKdPn6ZPnz6EhoYSHBxMnz59SEhIsLVt27Yt77zzDp07d6ZGjRr06NGDlJR/bo+7ZMkSmjRpQkhICNOnT8+2323bttGxY0dq165NWFgYY8eOJTMz86Z9/eyzz2jQoAFhYWHMnTvXttxisfB///d/NG/enJCQEP71r3+Rmppq+/y5556jfv361KpVi8cee4y9e/faPhs6dCijR4/mySefpFq1avz888+3V0gDUJgXkKysLJ566ikCAgLYtGkTW7dupXPnzsDlv+iVKlXi999/Z8iQIbzyyiucO3eOX375hS+//JIlS5awePFiAFasWMHUqVOZPn06e/fuZf78+Xh4eHDp0iX69u3Lfffdx++//85bb73FoEGDOHDgAABjx47F2dmZbdu2MWXKFNv2bmXHjh0MHz6cSZMmsXPnTnr37s3TTz/NxYsXC6ZQIlKgli1bxqJFi/j555+Ji4uzBa/FYqFbt25s3ryZzZs34+LiwiuvvJKtbUxMDFOnTuX333/n0qVLtpDdt2+fbQ5VbGwsqampHD9+3NbO0dGR119/nR07dvDVV1+xYcMGFixYcNN+bty4kQ0bNvDJJ58wa9Ys2/n3efPm8d1337FkyRJiY2Nxd3dn7Nixtnb3338/GzZs4Pfff6dOnTrX3ZE0JiaGIUOGsG/fPpo0aXL7hSzmFOYFZNu2bZw4cYJx48bh5uaGi4uL7S9S+fLl6devH05OTpQsWZIVK1YwZswYSpcuTVBQEP/617/48ssvAfj00095/vnnqV+/PiaTiSpVqhAYGEhsbCznz59n0KBBlCxZkpYtWxIREcHy5cvJysrim2++YeTIkbi5uVGrVi26du2aq34vWrSI3r1706BBAxwdHXniiScoWbIksbGxBVYrESk4ffv2JSAgAA8PD4YMGcLy5cuBy3OeHnnkEVxdXSldujRDhgzh119/zdb2iSee4J577sHV1ZUOHTqwa9cuAL7++msiIiJo2rQpzs7OvPTSS7b7kADUrVuXhg0b4uTkRFBQEL17975u29caNmwYbm5u1K5dm27dutn6uXDhQkaNGoW/vz/Ozs4MHz6cr7/+2jZhrnv37pQuXdr22Z9//snZs2dt233wwQdp3LgxDg4OuLi45L+gxZRO1haQhIQEAgMDczwffvVcg5SUFDIzMwkMDLQtCwwMtP2Wm5CQQKVKla7bRmJiIv7+/tn+AQUGBpKYmEhycjJmsznbfgIDA9m0adMt+x0fH88XX3zB/PnzbcsyMzM5ceLELduKSPFz7ffAlX/L6enpvPbaa/z444+cOXMGgLS0tGznla++pberqyvnz58H/vn+ucLNzQ0PDw/b+4MHD/LGG2/wxx9/kJ6ejtlspm7dunnq5549ewA4duwY/fv3z/Zd5+joyKlTp/D19WXSpEmsXLmS5ORk2zopKSmULVv2uu3eyRTmBcTf35/4+PgcJ7iZTCbba09PT0qUKMGxY8dsj5ONj4+nQoUKtu1ce74bwM/Pj4SEBCwWi+0vcHx8PFWrVsXLywsnJycSEhKoVq2a7bPcqFChAkOGDOHFF1/M+w8tIsXO1efB4+Pjbc/ZmDt3LnFxcaxcuRJfX1927txJu3btsFpzev5cduXLl892RUx6enq289hjxoyhTp06zJ49m9KlS/Pee+/x9ddf37KfV39fXemnv78/U6dOpXHjxte1WbJkCd9//z2fffYZQUFBnD17luDg4Gw/w9Xft3cyDbMXkLCwMHx9fXn77be5cOECGRkZbNmy5br1HB0d6dixI5MmTSItLY1jx47x3//+l8ceewyAHj16MHfuXP744w+sViuHDh3i2LFjhIWF4ebmxuzZs7l06RIbN25k1apVdO7cGUdHR9q3b8+UKVNIT09n3759fPHFF7nqd69evfj444+JjY3FarVy4cIFVq9eTVpaml3rIyKFY8GCBSQkJJCamsrMmTPp2LEjAOfPn8fFxYWyZcuSmppKdHR0rrf5yCOPsHr1ajZv3kxmZiaTJ0/ONmn3/PnzlClThlKlSnHgwAE++uijW25z2rRppKens3fvXhYvXkynTp0AePLJJ5k0aRLHjh0DIDk5me+//x64PJJQsmRJPDw8SE9PZ+LEibn+Ge40CvMC4ujoyIIFCzh8+DCNGzemUaNGfPXVVzmu+9Zbb+Hm5kazZs2IjIzk0UcfpXv37gB07NiRIUOG8MILL1CjRg2eeeYZTp8+TcmSJZk/fz5r164lNDSUl19+menTp9t+sx0/fjznz5+nfv36REVF0a1bt1z1u169ekyePJlXXnmF4OBgWrRoweeff26foohIoYuMjKRnz540b96cihUrMnToUAD69+9PRkYGoaGhdOzYkdatW+d6mzVr1mT8+PG88MILhIWF4e7ubhtNBBg3bhzLli2jRo0ajBw50hbMN9O0aVNatGhBt27dGDBgAK1atbL1s23btvTo0YMaNWrQsWNH2xyexx9/nMDAQBo2bEjr1q1p0KBB7gtzhzFZczOmYicWi4XRo0fj6enJ6NGjSUtLIzo6mlOnTuHj40NUVBSlS5e+5XauHjbKK289UcluVEv7US3tR7X8R3h4OJMnT+a+++7Lc1vV0X7yW8vcnPcv1HPm33zzDQEBAaSnpwOXLxkIDQ0lMjKSmJgYYmJi6N27d2F2KV8spxJh+SKsp1MwlfPU85NFpNDpe0igEIfZk5OTiY2N5YEHHrAt27Jli20opVWrVjmeUy6uLKcSsUa/inXTOti7A+umdVijX738D0tEpBDoe0iuKLQw//DDD+ndu3e2mYVnzpyxXc7g4eGR7drAYm/5Irj2H8zfvyGLiBSKW3wPbdq06baG2MV4CmWYfevWrbi7u1O1alXbTQfyYvXq1axevRqAiRMn4u3tfdt9cXJyylf7K1LOn+NSTts/fw5PO2zfCOxVS1Et7eluqmVBfg/dTXUsaIVRy0IJ87179/Lbb7+xbds2MjMzSU9PZ8aMGbi7u5OamoqHhwepqam2i/yvFRERQUREhO19fiYS2GtSh6VUmRyXm0uVuWsmjWiCjP2olvZzN9WyIL+H7qY6FrTCmABXKMPsPXv2ZO7cucyaNYuhQ4dSp04dhgwZQqNGjVi3bh0A69aty/GmAMVW515w7SQTH7/Ly0VECoO+h+RvRXoHuMjISKKjo1mzZg3e3t4MGzasKLuTJw4+flii3tQsUhEpMvoekisK9Tpze9F15sWDamk/qqX9qJb2oTrazx0zzC4iIiIFR2EuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuIiJicApzERERg1OYi4iIGJzCXERExOAU5iIiIganMBcRETE4hbmIiIjBKcxFREQMTmEuInIXOnr0KAEBAZjN5qLuitiBwlxERMTgFOYiIpIvOrovegpzEZFiIj4+nv79+xMaGkpISAhjx47l8OHDPP7444SEhFCnTh0GDRrEmTNnbG3Cw8OZOXMmrVu3Jjg4mKioKDIyMgC4//77+eGHH2zrXrp0iTp16rBz507bss8++4wGDRoQFhbG3Llzbcu3bNlCx44dqV27NmFhYYwdO5bMzEzb5wEBAXz44Ye0aNGCli1bAvDqq6/SqFEjatasyUMPPcSmTZsKrFaSncJcRKQYyMrK4qmnniIgIIBNmzaxdetWOnfujNVqZfDgwcTGxrJu3ToSEhKYMmVKtrbLli1j0aJF/Pzzz8TFxTF9+nQAunbtytKlS23rrVmzhvLly1OnTh3bso0bN7JhwwY++eQTZs2axfr16wFwdHTk9ddfZ8eOHXz11Vds2LCBBQsWZNvvd999x8qVK1m7di0A9erV44cffmDXrl1ERkbyr3/9y/aLhRQshbmISDGwbds2Tpw4wbhx43Bzc8PFxYUmTZpQpUoV7rvvPpydnfHy8uK5557j119/zda2b9++BAQE4OHhwZAhQ1i+fDkAjz32GGvWrOHcuXMALFmyhC5dumRrO2zYMNzc3KhduzbdunWztW3QoAENGzbEycmJoKAgevfufd1+Bw0ahIeHB66urgB06dIFT09PnJycGDBgAJmZmRw8eLBA6iXZORV1B0REBBISEggMDMTJKfvXclJSEuPGjWPz5s2kpaVhsVhwd3fPto6/v7/tdWBgICdOnADAz8+Pxo0b88033/DQQw+xdu1a3nzzzZu23bNnDwD79u1j6NCh/PHHH6Snp2M2m6lbt+4N2wLMnTuXTz/9lBMnTmAymTh37hwpKSm3WRHJCx2Zi4gUA/7+/sTHx183mWzChAmYTCZWrVrF3r17mTlzJlarNds6CQkJttfx8fGUL1/e9v7xxx/nyy+/ZOXKlTRs2JAKFSrkqu3gwYOpVq0aGzZsYO/evYwePfq6/ZpMJtvrTZs2MXv2bObOncuff/7J7t27KVu27HVtpGAozEVEioGwsDB8fX15++23uXDhAhkZGWzZsoW0tDRKlSqFu7s7x48fZ86cOde1XbBgAQkJCaSmpjJz5kw6duxo+6xdu3bs3LmTefPm0bVr1+vaTps2jfT0dPbu3cvixYvp1KkTAGlpaZQpU4ZSpUpx4MABPvroo5v2Py0tDScnJ7y8vDCbzURHR9uG96XgKcxFRIoBR0dHFixYwOHDh2ncuDGNGjXiq6++YtiwYezYsYNatWrRp08f2rdvf13byMhIevbsSfPmzalYsSJDhw61febq6srDDz/MX3/9xcMPP3xd26ZNm9KiRQu6devGgAEDaNWqFQATJ05k2bJl1KhRg5EjR9pC/kZat25NmzZtuPfeewkPD8fZ2fm6YXgpOCarAcdArh4Wyitvb2+SkpLs2Ju7l2ppP6ql/dxttQwPD2fy5Mncd999N1wnOjqauLg4Zs6cmevt3m11LEj5rWVufinSBDgRkUJiOZUIyxdhPZ2CqZwndO6Fg49fge4zNTWVTz/9lBkzZhTofqRoaZhdRKQQWE4lYo1+FeumdbB3B9ZN67BGv3o54AvIokWLaNy4MW3atKFp06YFth8pejoyFxEpDMsXwbXB/feROv2H3/Zmb3aXtV69etGrV6/b3rYYh47MRUQKgfV0ztdb32i5SF4ozEVECoGpnGeelovkhcJcRKQwdO4F10528/G7vFwkn3TOXESkEDj4+GGJerPQZ7PL3UFhLiJSSBx8/PI12U3kRjTMLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgFOYiInJHCQ8PZ+bMmbRu3Zrg4GCioqLIyMhg8eLFREZGZls3ICCAQ4cOATB06FBGjRpF9+7dqVGjBl26dOHYsWPZ1p03bx7NmjWjTp06vPXWW1gsFi5evEhISAi7d++2rZuUlMQ999xDcnJyofzMCnMREbnjLFu2jEWLFvHzzz8TFxfH9OnTc91u6NCh7Nixg5CQEAYNGpTt82+//ZZvvvmG77//nu+//57PPvsMZ2dnOnXqxNKlS23rxcTE0LJlS7y8vOz6c92IwlxERO44ffv2JSAgAA8PD4YMGcLy5ctz1e6BBx6gadOmODs7M2rUKLZu3Up8fLzt8xdeeAEPDw8CAgLo378/MTExADz++OMsW7YMi8UCwJdffkmXLl3s/nPdiMJcRETuOP7+/rbXgYGBnDhxIs/tSpUqRbly5bK1vdF2GzRogJubG7/88gsHDhzg8OHDPPjgg/n9MXLNqdD2JCIiUkgSEhJsr+Pj4ylfvjxubm6kp6fblp88efKm7c6fP8/p06cpX758ts9r1qyZbbtXPP744yxduhQfHx8eeeQRXFxc7Poz3YyOzEVE5I6zYMECEhISSE1NZebMmXTs2JHg4GD27dvHzp07ycjIYMqUKde1W7NmDZs3byYzM5N33nmHsLAwAgICbJ/PmTOH06dPEx8fz7x58+jUqZPtsy5duvDdd9+xdOlSunbtWig/5xUKcxERueNERkbSs2dPmjdvTsWKFRk6dCj33HMPQ4cOpXv37rRs2ZImTZrk2G7q1KmEhISwY8cO/u///i/b5+3ataN9+/Y8+OCDPPDAA/To0cP2mb+/P3Xq1MFkMhEeHl7gP+PVNMwuIiJ3nHr16jF48ODrlr/44ou8+OKLtvfXTlLz9PRk0qRJN9zugxUr8q/33sMcEgIm03WfBwQE0KBBA0w5fFaQCiXMk5KSmDVrFqdPn8ZkMhEREcHDDz9MWloa0dHRnDp1Ch8fH6KioihdunRhdElERCTPPPv3x9dsxhwQQEaHDqS3b8+lhg3BwYGjR4/y7bff8v333xd6vwolzB0dHXnyySepWrUq6enpjB49mrp16/Ljjz8SGhpKZGQkMTExxMTE0Lt378LokoiIGJjlVCIsX4T1dAqmcp7QuRcOPn4Fvl+T2QyAY3w8pf77X0q/+y5ZXl687O/PrH37eGHwYCpWrFjg/bhWoZwz9/DwoGrVqgC4uroSEBBASkoKW7ZsoVWrVgC0atWKLVu2FEZ3RETEwCynErFGv4p10zrYuwPrpnVYo1+9HPDApk2buO+++/K83WnTpjFq1Kgbfn58xw6q/f3aBJisVgAck5OZuGsXaRcvMuHddym5eXOe951fhX7O/OTJkxw6dIhq1apx5swZPDw8gMuBf/bs2cLujoiIGM3yRfB3cNv8faRO/+G3bm+x4HD8OE5xcdf/OXz4trpk+vtmMaa0NExFkGWFGuZXLgXo27cvbm5uuW63evVqVq9eDcDEiRPx9va+7T44OTnlq738Q7W0H9XSflRL+yjOdUw5f45Lf782ZVlwOn8Rp/MZOP/vJ9wOJmHatw/T/v2Y7HxfdKurK6arrlO3LTeZwMEBS+/eZL30EmWqVaPMVZ8XRi0LLczNZjNTpkzh3nvvtU3Zd3d3JzU1FQ8PD1JTUylbtmyObSMiIoiIiLC9T0pKuu1+eHt756u9/EO1tB/V0n5US/so9DparTgkJdmOkB3j4nA6eND23pSVZVv1xmfG42B97k7XZpUvj7lq1ev+ZFWsCCVL3rCd77334hQXd7nLJhM4OnKhe3fSBg0iKyjo8krX1C2/tbz6rnM3UihhbrVamTt3LgEBAXTo0MG2vFGjRqxbt47IyEjWrVtH48aNC6M7IiJSkDIzcfzrrxyHsR1zeVvV3LKawFzKhUse7mTd346sOqGY77kHc9WqWLy9c7x8LD8s5cpd3m+JEpx/8knSnn8eSy7CtqAVSpjv3buX9evXU7FiRUaOHAlAjx49iIyMJDo6mjVr1uDt7c2wYcMKozsiIpJLptOnLwfxVUfJV46cHTIy7LqvLA8Psq4+Uv47lLMqV8bq6ppt3aKazX566lRcVq7kQs+eWK66lWtRM1mtf0/HM5Cr752bVxqCsx/V0n5US/tRLW8gKwvHY8euC2SnuDicrnpmt72YK1fOcRjbUqECONxdNx+9Y4bZRUTEfkxpaTgdOvRPGF/1x8HOM6ktpUvbjpDNVatePnK+5x7MlStjLVPm1huQQqEwFxEpKlbrjS+ROnTI7rszBwT8M4x9dUAHBIBT9jjQCIexKMxFROwhIwOnw4dzHMZ2tPclUi4umKtUuS6UzVWqYPX0tOu+xBgU5rkUHh7O5MmTb3pXoY0bNzJ48GC2bt1aiD27Oy1evJhPP/2UmJiYHD/v3bs3nTp14oknnrjpdnLz/1XuMlYrDsnJ2c8pXzUB7MrtPO0ly9c3eyBfGcoOCgJnZ7vuS27fjb4r2rRpw/jx42nevHkR9ewyhXkxoF8C7G/hwoVF3QUpDi5duv4SqYMHL59vTky8dfs8sDo6Zg/jqyd9+fjY/RIpKR7Wrl1rez1lyhQOHz7MzJkzbcu6du1Knz59sj33vCAozO8AZrMZJyf9r5Q7m+nMmeuOkm2XSF1zV678XvV73SVSV0K6SpXrLpESKQ6UAHl08eJFxo8fz8qVKwHo0KEDY8eOxfmq4bAZM2bw3//+l1KlSjFq1Cgee+wxAP73v//x1ltvkZCQQJkyZXj22Wfp06cPTz75JBcvXqR69eoA/PTTT/j6+jJ79mw++eQTzpw5Q8uWLZk4cSIeHh4cPXqUpk2b8p///IepU6cSFBTE0qVLC78YhSQ+Pp7XXnuNTZs2YbFYiIyMpG7dugC8+eabfPbZZ5QtW5a3336b+++/H7j82/Bjjz1Gz549AVi0aBH//e9/OX78OP7+/sycOZPQ0NBs+zlw4AC9e/dmzJgxdO7cmVWrVvHOO+9w7NgxqlevzsSJEwkODgYuD7k9/fTTLFmyhGPHjtG6dWumTZuGi4tLIVbGoLKycExIsIVytqHso0ftvjvbJVJXzjFXrUrWPfeQdRdeIiX2cfV3xdtvv83kyZPJyspi5syZWK1WvvvuOypVqkRERASbNm0iNjaW4cOH88QTTzB+/HgOHDjAK6+8wo4dO/D09GTkyJG2I/ehQ4fi5ubG0aNH2bRpEzVq1OCLL77gnnvuuWmfFOZ5NGPGDGJjY/nhhx8wmUw8/fTTTJ8+nZdeegmAU6dOkZKSwtatW4mNjeXJJ5+kbt26VKtWjREjRjB37lzCw8M5ffo0R48exc3NjY8//vi6Yfb33nuP7777jiVLluDl5cW4ceMYO3Yss2fPtq3zyy+/sG7dOkx38PBdVlYWTz31FC1atGDTpk04ODjwxx9/cOjQIbZt28bjjz/Ojh07WLhwISNGjGDr1q3X1WPFihVMnTqVefPmUa9ePQ4fPkyJEiWyrbNjxw769evH22+/Tdu2bdmxYwfDhw/nww8/pF69enz55Zc8/fTTrF+/3vaL24oVK1i4cCHOzs5ERkby+eef06dPn0KrTXFgunDhuttuOh06hNPBgwVziVROw9hVqmC96lbQmoUtBena74q3334buHzufPDgwdcNs//222/ZhtkvXLhA9+7dGTlyJAsXLmT37t306NGDmjVrUrNmTQCWL1/OwoULCQ0NZejQoYwdO5bPPvvspv1SmOfRsmXLeOutt2w3zR82bBijRo2yhTnASy+9hLOzM82aNeOBBx5gxYoVREVF4eTkxL59+wgODqZcuXKU+/u2gDlZuHAh//73v203Cxg+fDhNmjTBfNXkm+HDh+fpgTVGtG3bNk6cOMG4ceNspxKaNGnCoUOHCAwMpFevXgA88cQTvPzyy5w6dQpfX99s2/j00095/vnnqV+/PgBVqlTJ9vnmzZv59NNPmTFjBi1atAAuH8n37t2bBg0a2LY/c+ZMYmNjadasGQD9+vXDz+/yHafatm3Lrl27CqYIhcFqxSEx8fpzywV1iZS/f87D2EFB110iJVJc5PRdkVerVq0iKCiIbt26ARAaGsrDDz/M119/bQvz9u3bExYWBsCjjz7K+PHjb7ld/avJoxMnThAYGGh7HxgYyImr7jXs7u6eLWCv/vy9995j+vTpTJgwgdq1azNmzBgaNWqU436OHTtG//79cbhqGNDR0ZFTp07Z3ufmrkBGl5CQQGBgYI5zAnx8fGyvXf8+j3n+/Pkct1GpUqUb7uPjjz+madOm2f5xxsfH88UXXzB//nzbsszMzGz/r6/d/wk733P6tmVk4HTkSLZALrBLpJydrxvCvjIzW5dIyZ0mp++KvIqPj2fbtm3Url3btsxsNtOlSxfb+2u/W9LS0m65XYV5HpUvX55jx47ZfoOKj4+n/FX35z1z5gwXLlywBXp8fLxt3fr16zN//nwuXbrE/PnzGTBgAL/99luOw+T+/v5MnTo1x4fPHP37vOKdPLx+hb+/P/Hx8fma5Ofv78+RI0du+PnEiROZNWsWr732Gm+88QYAFSpUYMiQIbz44ou3tU+7sFpxSEm5HMLX3BfbKS4O06VLt95GHmT5+Fx/l6+qVTFXrKhLpETI+bviarn5Tvb396dp06a3HDbPK4V5HnXu3Jnp06dTv359TCYT0dHRtgluV/znP/9h9OjRbNu2jdWrVzNixAgyMzNZuXIlERERlC1bljJlyuDo6Ahc/i3s9OnTnD171vYY2CeffJJJkyYxbdo0AgMDSU5O5rfffqNdu3aF/jMXpbCwMHx9fXn77bcZMWIEDg4O7NixI0/b6NGjB2+88QZNmjQhNDTUds78yghLqVKlWLRoEd26dePtt9/m5ZdfplevXjzzzDPce++9hIWFkZ6ezsaNG2natCmlS5fO+w9y6RKOR49e97AKp7g42yVS9hpnsZpM1wfyladI6RIpkduW03fF1by9vVm/fj0Wi8U2qurj48Ohq05VRURE8Pbbb7NkyRI6d+4MwK5duyhVqpRtEvTtUJjn0YsvvkhaWprt+eodOnTIdvTm4+ODu7s7DRo0wNXVlYkTJ1KtWjUyMzP58ssveeWVV8jKyuKee+6xTZKoVq0anTt3plmzZlgsFtauXUv//v2xWq306NGDEydO4O3tTceOHe+6MHd0dGTBggWMGzeOxo0bYzKZePTRR6lTp06ut9GxY0dSU1N54YUXSExMJCgoiBkzZmQ7XeLu7s6nn37K448/jpOTEy+99BKTJ0/mlVde4dChQ7i4uNC4cWOaNm2K6cwZTJmZOG/YQJlNm3CKi8Ptl19wSUnB385XFVjKlct+Xvnqp0jd4fMlRIqja78rrtahQweWLl1KnTp1CAoK4vvvv+eZZ55h+PDhvPvuu3Tp0oW33nqLTz75hDfeeIM33ngDi8VCcHAwr732Wr76paemyW0zfC0tFhzj46+/01dcHE5//WX33ZkrVbp+wtffl0h5+/oau5bFiOH/XhYTqqP96Klphaiono0rl+W3/rZLpHJ4WIXD6dP27WupUraj5JtdIiUixmLkHFCYc/l/oDX6VTh1+dylFSBuL5aoNw3zP9LIstXfasV08RIlNm6kROMHKHEqKduRs8nOA0m6REpEwPg5oG8rgOWLbP8Dbf7+DY3+w4umT3eT5Ytw/OsIFdbuJNvUrO9/zfUmcrxE6u9hbIuHhyZ9icjNGTwHFOaA9XRKnpaLfVlPp2BxciS9ggdux1PJKunEpVIumAMqYOn8xD/hXLEi6HapIlIAjJ4DCnPAVM6TnAZvTeV004vCYCrnibWEE8lhVUkOu2p5eCsc+g8suo6JyF3D6DmgpwwAdO4F154T8fG7vFwKnuovIkXN4N9DOjIHHHz8sES9adhZjEan+otIUTP695DC/G8OPn6GmORwp1L9RaSoGfl7SMPsIiIiBqcwFxERMTiFuYiIiMEpzEVERAxOYS4iImJwCnMRERGDU5iLiIgYnMJcRETE4BTmIiIiBqcwFxERMTiFuYiIiMEpzEVERAxOYS4iIrkWHh7O+vXri7obcg2FuYiIGMbQoUOZNGlSUXej2FGYi4jIXcNsNhd1FwqEwlxERPLkzz//JCIiglq1ajFgwAAyMjI4ffo0ffr0ITQ0lODgYPr06UNCQoKtTdeuXZk0aRKdOnWievXqPPXUU6SkpDBo0CBq1qzJww8/zNGjRwGwWq289tpr1K1bl1q1ahEREcGePXtYuHAhy5YtY86cObZtACQmJvLss88SGhpK06ZNmTdvnm2/U6ZM4dlnn2Xw4MHUrFmTzz//vHCLVUgU5iIikicrVqxg4cKF/PLLL+zevZvPP/8ci8VCt27d2Lx5M5s3b8bFxYVXXnklW7uvvvqKGTNmsHXrVo4cOUKnTp144okn2LVrF9WqVWPq1KkArFu3jk2bNvHTTz+xe/du5syZg4eHB7179+bRRx/l+eefZ//+/SxYsACLxULfvn0JDg5m69atLF68mPfff58ff/zRtt8ffviBRx55hN27d/Poo48WZqkKjcJcRETypF+/fvj5+eHh4UHbtm3ZtWsXnp6ePPLII7i6ulK6dGmGDBnCr7/+mq3dE088QeXKlSlbtixt2rShcuXK3HfffTg5OdGhQwd27twJgJOTE2lpaRw4cACr1Ur16tUpX758jn3Zvn07ycnJREVFUbJkSSpVqkTPnj1Zvny5bZ2GDRvy0EMP4eDggKura8EVpgg5FXUHRETEWHx8fGyvXV1dOXHiBOnp6bz22mv8+OOPnDlzBoC0tDSysrJwdHS8rp2Liwve3t7Z3l+4cAGAli1b8vTTTzN27Fji4+Np374948aNo0yZMtf15dixY5w4cYLatWvblmVlZREeHm577+/vb6efvPhSmIuISL7NnTuXuLg4Vq5cia+vLzt37qRdu3ZYrdbb2t4zzzzDM888Q1JSEgMGDGDOnDm89NJLmEymbOv5+/sTFBTEzz//fMNtXdvmTqRhdhERybfz58/j4uJC2bJlSU1NJTo6+ra3tX37dmJjY7l06RJubm44OztnO7r/66+/bOuGhYVRpkwZZs2aRXp6OllZWezZs4ft27fn90cyFIW5iIhcJzMzb+v379+fjIwMQkND6dixI61bt77tfZ87d46XXnqJ4OBgmjRpgoeHBwMGDACge/fu7Nu3j9q1a9OvXz8cHR358MMP2bVrF82aNSM0NJQRI0Zw9uzZ296/EZmstzsGUoSuvtwhr7y9vUlKSrJjb+5eqqX9qJb2o1rm37ffuvDCCx7Uq5fJI49k0L59BgEBWUXdLcPK79/J3Jzz15G5iIjYWE4l8vtHW7l40cRvW0rw2mvuNGlSnnbtvJkxozQHDmiqVXGkMBcREeBykFujX6XsyV2X31v/iYhdu0owaVJZWrXy5d57fZg+vXRRdVNyoF+xRETuEpmZcOSIE3FxTsTFOf7938t/Tp50BPyBlTm2tVr/mREeF1eCGTMcefHFtMLpuNySwlxExIBSUkzZwvjKn0OHnMjIKJhLsRwcrFgsJu69N4NhwxTkxYnCXESkCJnNcOyYYw7B7Eh8vP2/oqtUMVO16vV//Pws8MEUrJvW8XNSI3psnmtr42CyYLE60KbNRYYOPUeDBpfs3i/JH4W5iIidnDtn4tCh7MPYBw9efp+WZt8pSmXLWq4L5HvuMVO5chalS9/eRUqWzr0gbi/uZ879vcQKmGjb+ixRoy4RGqoQL64U5iIi17BY4PhxhxyHsQ8ftv/XZmDglUDOsoVy1apmAgKy+PteKYXCwccPS9SbVF68hLaJv+PsYmXI6yUJaV6u8Doht0VhLiJ3tPR0OHzYKdtR8pUj59RU+yali4vFFshXh3KVKmY8PIxxSw8HHz/cBw1i5eu6Xt9IFOYiYghWKyQlOVwXyFdeZ2VdPekr/w/WKF8+K8dzyxUrZlGyZL43L2JXCnMRKXSZmfDXX9dfHhUX58SJE/Y9WnZ0tGY7Sr56KNvLy8Jd8AwOuQsozEUkX1JTc75EKi7OkYwM+0768vTMynEYu1IlM1ceU63bucrdSGEut+Xo0aMEBARw5MgRnJz01+hOkJWV0yVSl98fO2b//8eVK199tJz9EikH3ZtSJE/0LVyMhIeHM3nyZO67775C3/eUKVM4fPgwM2fOLPR9i/2lpV25RCp7OB886MS5c/ZNyjJlLDmEchaVK5spU8YYk75EjE5hfofIysqyPe9X7hxWa86XSB08WDiXSF35ExCQhQZgRIov/fMsJgYPHkx8fDxPP/00Dg4OREVFsX37djZv3kxGRgbBwcFMmDCBmjVrAjB06FBcXFyIj4/nl19+4YMPPiAzM5N33nmHI0eOUKZMGXr06MHw4cOBy8PiTZs2JTo6msmTJ5Oens6zzz7Liy++yNq1a5k5cyZWq5XvvvuOSpUqsXr16utGCm529L548WJmz57N8ePH8fLyYuDAgTz55JOFV0ADyMj45xKp7JdJOZKScuW+2PZx7SVSV/5UqWLG01NHyyJ3GoV5MTFz5kw2b96cLTw/++wzpk6dSokSJRg/fjyDBg1i1apVtjYxMTF8/PHHLFiwgMzMTGJjY5k+fTo1a9Zkz5499OjRg5CQEB566CFbmy1btrB+/Xri4uLo0KEDDz/8MG3atGHw4MH5Gmb38vJiwYIFVKpUiV9//ZXevXtTv359QkND81eYfAgICGDDhg1UqVLlus8WL17Mp59+SkxMDHC5LkOHDuXkyZPMnDmThQsX0qlTJ5544onr2lqtkJx8+Wj50Ud96NVrG8nJNW0hbTbbd3p0zpdIZVGxolmXSIkIoDAv1rp37257PXz4cIKDgzl79ixly5YF4MEHH6Rx48YAuLi40Lx5c9v6wcHBdO7cmV9++SVbmEdFReHq6kpISAjBwcH8+eefVK9ePd99jYiIsL1u1qwZrVq1YtOmTUUa5rlx6RL89ZcjY8ZMoUaNf9GixWDef9+JQ4eeYu1aR6Kibr2NRYtKA663XO/KJVI5DWPXru1JcrJmYIvI7VGYF1NZWVlMmjSJlStXkpycjMPf03tTUlJsYe7vn31YNjY2lrfffpu9e/dy6dIlMjMzeeSRR7Kt4+vra3vt6urK+fPn7dLfNWvWMHXqVA4dOoTFYiE9PZ1atWrZZdu5deaMKdsNRQD69vXk2DG/HC6R8gBKUrnylRomsHt3I6BUrvbl4XH5EqmtW+G5587RsGHK38PYWbi65n0YW9c6i0h+KMyLEdNV3+jLli3j+++/57PPPiMoKIizZ88SHByM1WrNcX2AQYMG0bdvXxYuXIiLiwuvvvoqqamped73FW5ubmRkZNjenzx5Mse2Fy9e5Nlnn2X69Om0a9eOEiVK0K9fv2x9zYusLIiP/2cW9ubNO1mz5gUuXDiI1foQ4ABUB/4NvAdMAlKAlsBcwMe2rQMHSvy9fjLwNPAjUAtoZ1vHyekezOZDODp2xNHRkfff38u0aY/Tvftj9OrVk0OHDjFixAh27dqFk5MTLVu2ZO7cy0+UCgiAKlW+Z8KEd0lNTSUyMpLx48fnWE8RkYKiMC9GvL29+euvvwBIS0ujZMmSeHh4kJ6ezsSJE2/ZPi0tjXLlyuHi4sK2bduIiYmhVatWud73+vXrsVgstlGAkJAQli9fTps2bfjzzz/5+uuvadOmzXVtr4wCeHl54eTkxJo1a1i3bh01a9bk/HkThw45XnMLzst/zp7NzSVSmUBPYBgwEFgBdAdeAtYAY4AfgBBKlBhOyZLdaNv2e6pWzWLqVJg3L5kWLUrx0ksDsFisREfH8tdff9GrVy+CgoKIiUkAfrpust+cOf8cLV9Z/sUXX5CZmckff/yRrYerV6/mm2++IS0tjYceeoi2bdvmWCcRkYKiWzMUI4MHD2b69OnUrl2b06dPExgYSMOGDWndujUNGjSwrbdp0ya+/fbb69q//fbb/Oc//6FGjRpER0fTsWPHXO+7Q4cOANSpU4d27S4ftY4cOZIjR44QHBzMf/7zHx588DFOnnTgo4/cmDatNACtWvlSs2YNLJYZdO06kMDAEJ588nsyMjozY0YZatSoQLt2vgwc6Ml//lOWpUvd2L695C2D3N/fTMuWF2nXbi1lylzio496sXFjCkeONKVx4/oMGXKO7t3f4/nnnyA+3o/4+GR27XqRixd/ZfToHQwffvkRjjVrmnFzM/PNN98wcuRI3NzcqFWrFl27ds11bZycnIiPjycxMREXFxeaNGmS7fNBgwbh7u5OQEAAzZs3Z9euXbnetoiIPejIvBhp166dLUhz8vjjjwNQpUoV9u7da1t+5aiyQ4cOtlC+VlBQEPHx8cA/l0j167eSuDgnhg1zIi7Oi7i4XzhzxpEzZy4PH1++VCoWgDVr/tnWhg0A5QArhw9fWfrC33+yc3a2ZpvodfVTpHJzidTy5XGcOFGeBx7ItC27MlfgxIkT2SbYlSpVCg8PDxITEwkKCrItT05Oxmw2Z5tjEBgYyKZNm265f4BXXnnFVl93d3f+9a9/ZZuc6OPzz7C+PechiIjklsK8EFlOJcLyRVhPp2Aq5wmde+Hg43fb27tyidTFiyZ+/NGZDRvKZBvGvnTJvudtfX2zz8CuX98NH59UgoLMODvbdVdX7dOXxMRErFar7Tx0QkIClSpVonz58hw7dsy27oULF0hNTcXPL3tNrwz/JyQkUK1aNQDbLza57cPkyZMB2Lx5M927dyc8PDzHS95ERIqCwvwGwsPD6du3L19++SWHDx+mc+fOjB49mqioKDZv3kxYWBjvvvsu5cqV44cffmDChAkkJiYSEhLChAkTbJd7zZo1iw8++IBzZ89S3snEv2v509K7LOlZFl7+6HNWnTyDr58fXbt24/335zF58g7i4px4881yhIXt4vjxmiQmOgJ9gUAuT/r6EegNHAOeBI7x7rs9AUfgVS6fT/6Ky+eT44H6wBygNgAODtarjpSzB7SPT+6fIuXt7UpSktke5b6hhg0b4uDgwPz58+nTpw//+9//2L59O82aNSMyMpKBAwcSGRlJ9erVmThxImFhYdmOygEcHR1p3749U6ZMYerUqRw9epQvvvjiuvVuZMWKFTRs2BB/f3/c3d0xmUy6256IFCsK85v4+uuv+fTTTzGbzbRr146dO3cyZcoUqlevTu/evfnggw/o3LkzAwcO5IMPPqBZs2a899579O3bl7Vr1/LXX38xf/58vv76a3xXLuKvtT/w46kmPBc7nTTzv4GfgeWcOXeeCRPaAw707etl2/+2bSW5HNA3Vq7cAi5c+ImmTf+PZs3uo2pVMyVL/sLAgd2z9WnRoodZu3YtJQ12l5GSJUvy/vvvM2LECCZMmECbNm2IiIigZMmS3HvvvYwcOZLnnnuOM2fO0LBhQ2bPnp3jdsaPH09UVBT169enWrVqdOvWjY0bN+aqD7///juvv/46Z8+excfHhzfeeIOKFSva88cUEckXhflN9OvXz3Y+NDw8HC8vL+rUqQNA+/bt2bBhAyaTiQceeMA2C3rAgAHMmzeP3377jQoVKpCZmcm+ffsol5xEkJszZ831STOXBj4HZgOef/8ZgqPjm7RqlUGVKmbmzYPo6BRatDhBhQpZDBt2gQoVzjFqVAIbNyYxeHAWW7cm/N23LJ5/Po377ksDIDp66Q37dPWNZYyiXr162e5816FDB9q2bQtAnz596NOnT47trh5K9/Ly4qOPPrrhPq49f75kyRLb61deeYVXXnnllvsAmDZt2g33ISJSUDSb/Sa8vb1tr11cXLJNdHJxceH8+fOcOHGCwMBA23IHBwcqVKhAYmIiVapU4Y033mDq1Kk0+O9iXtgWR5eA9/jr4UY4Oxxm1b3DOfrGcOLjE1i0qDQ+Pll8/HEKb755FoDGjS8REJCV58dB3qxPRvTLL79w8uRJzGYzn3/+Obt376Z169ZF3S0RkWJDYZ5P107CslqtHD9+3DYJ69FHHyUmJoZfVv+AydmVCXsuH8n5OpfguHMp6NwLuP4Iz9XVlfT0dNv7U6dO3bAP196g5FZ9Ko4spxKxvD+FrP+MxfL+lMuTBf928OBB2rZtS61atfjvf//Lu+++S/ny5YuwtyIixYvCPJ86duzI//73P3766ScuXbrEu+++S8mSJWnUqBEHDhxgw4YNXLx4EdeAirjUa4Sjrx/UDKVDg7rMSs3ibAkXEhISmD9/frbthoSEEBMTQ1ZWFmvXruXXX3+9YR+uvtnMrfpUHFlOJWKNfhXrpnWwdwfWTeuwRr9qC/TevXvz+++/c+DAAVavXp3tPvAiIqIwz7dq1aoxc+ZMxo0bR2hoKKtWreLDDz+kZMmSZGZmMmHCBEJDQwkLCyM57Txj5n2M44jxDF/wGYGVq9CsWTN69uxJly5dsm33zTffZNWqVdSuXZulS5fe9Przq282M3fu3Jv2qVhavghOXXMK4O/L+ERE5NZM1tu9gXYRSkhIuO223t7eJCUVv6dTbdy4kcGDB7N169ai7kqu2auWWf8ZC3t3XP9BzVAcR4zP9/aNoLj+vTQi1dI+VEf7yW8tr32oVk7u2tns9r6Bi9w+UzlPcvqN0lTOs9D7IiJiRHdlmF85R3tlaNcKELcXS9SbCvSi0LkXxO3NPtTu42ebHCgiIjd3d54zL4bnaJs3b26oIXZ7cvDxwxT1JqbwVlAzFFN4K0z6xUpEJNfuyiNz6+mUPC2Xgufg4wf9hxd1N0REDKnIw3z79u3Mnz8fi8XCAw88QGRkZIHvU+doRUTkTlKkw+wWi4V58+bx8ssvEx0dzc8//5ztZicFpnOvy+dkr6ZztCIiYlBFemR+4MAB/Pz8bHfzat68OVu2bMl2K9KC4ODjhyXqTc1mFxGRO0KRhnlKSgpeXv88JczLy4v9+/dft97q1atZvXo1ABMnTsx2z/S8cnJyutze2xtqT7jt7chVtZR8Uy3tR7W0D9XRfgqjlkUa5jndr+ba+4wDREREZLuFZ34uvteNEOxHtbQf1dJ+VEv7UB3tpzBuGlOk58y9vLxITk62vU9OTsbDw6MIeyQiImI8RRrm99xzD8ePH7c93nLjxo3F9mEgIiIixVWRDrM7OjrSr18/xo8fj8VioU2bNgQFBRVll0RERAynyK8zb9CgAQ0aNCjqboiIiBjW3Xk7VxERkTuIwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYNTmIuIiBicwlxERMTgTNacnkMqIiIihnHXHZmPHj26qLtwx1At7Ue1tB/V0j5UR/spjFredWEuIiJyp1GYi4iIGNxdF+YRERFF3YU7hmppP6ql/aiW9qE62k9h1FIT4ERERAzurjsyFxERudM4FXUHCtP27duZP38+FouFBx54gMjIyKLukiEkJSUxa9YsTp8+jclkIiIigocffpi0tDSio6M5deoUPj4+REVFUbp06aLuriFYLBZGjx6Np6cno0ePVi1v0/nz55k7dy5Hjx7FZDLx/PPP4+/vr1rehpUrV7JmzRpMJhNBQUEMHDiQzMxM1TIXZs+eTWxsLO7u7kyZMgXgpv+mly1bxpo1a3BwcODpp5+mfv36+e+E9S6RlZVlHTRokDUxMdF66dIl64gRI6xHjx4t6m4ZQkpKivXgwYNWq9VqvXDhgnXIkCHWo0ePWj/++GPrsmXLrFar1bps2TLrxx9/XIS9NJYVK1ZYp02bZp0wYYLVarWqlrdp5syZ1tWrV1utVqv10qVL1rS0NNXyNiQnJ1sHDhxovXjxotVqtVqnTJliXbt2rWqZS7t27bIePHjQOmzYMNuyG9Xu6NGj1hEjRlgzMzOtJ06csA4aNMialZWV7z7cNcPsBw4cwM/Pj/Lly+Pk5ETz5s3ZsmVLUXfLEDw8PKhatSoArq6uBAQEkJKSwpYtW2jVqhUArVq1Uj1zKTk5mdjYWB544AHbMtUy7y5cuMDu3bu5//77AXBycqJUqVKq5W2yWCxkZmaSlZVFZmYmHh4eqmUuBQcHXzdicaPabdmyhebNm1OiRAl8fX3x8/PjwIED+e7DXTPMnpKSgpeXl+29l5cX+/fvL8IeGdPJkyc5dOgQ1apV48yZM3h4eACXA//s2bNF3Dtj+PDDD+nduzfp6em2Zapl3p08eZKyZcsye/Zsjhw5QtWqVenbt69qeRs8PT3p2LEjzz//PCVLlqRevXrUq1dPtcyHG9UuJSWF6tWr29bz9PQkJSUl3/u7a47MrTlM2jeZTEXQE+PKyMhgypQp9O3bFzc3t6LujiFt3boVd3d320iH3L6srCwOHTrEgw8+yDvvvIOzszMxMTFF3S1DSktLY8uWLcyaNYt3332XjIwM1q9fX9TduiPllEX2cNccmXt5eZGcnGx7n5ycbPutSW7NbDYzZcoU7r33XsLDwwFwd3cnNTUVDw8PUlNTKVu2bBH3svjbu3cvv/32G9u2bSMzM5P09HRmzJihWt4GLy8vvLy8bEc5TZs2JSYmRrW8DTt27MDX19dWq/DwcPbt26da5sONandtFqWkpODp6Znv/d01R+b33HMPx48f5+TJk5jNZjZu3EijRo2KuluGYLVamTt3LgEBAXTo0MG2vFGjRqxbtw6AdevW0bhx46LqomH07NmTuXPnMmvWLIYOHUqdOnUYMmSIankbypUrh5eXFwkJCcDlQAoMDFQtb4O3tzf79+/n4sWLWK1WduzYQUBAgGqZDzeqXaNGjdi4cSOXLl3i5MmTHD9+nGrVquV7f3fVTWNiY2NZsGABFouFNm3a8NhjjxV1lwxhz549vPrqq1SsWNF2aqJHjx5Ur16d6OhokpKS8Pb2ZtiwYbpsJQ927drFihUrGD16NOfOnVMtb8Phw4eZO3cuZrMZX19fBg4ciNVqVS1vw+eff87GjRtxdHSkcuXKDBgwgIyMDNUyF6ZNm8aff/7JuXPncHd354knnqBx48Y3rN3SpUtZu3YtDg4O9O3bl7CwsHz34a4KcxERkTvRXTPMLiIicqdSmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRe5Sr7/+Ov/73/+KuhsiYgcKcxEREYPTTWNE7gBJSUl8+OGH7N69G6vVSosWLShTpgyJiYkMGTIEuPyUsUGDBvHpp5/y+eefExMTg5OTEw4ODrRu3ZpnnnmG+Ph4PvjgA+Li4ihbtizdunWjefPmAMyaNQtnZ2dOnTrF7t27CQwMZMiQIfj5+QHctG1sbCwff/wxycnJuLq68sgjj9CpUyfOnj3L7Nmz2bNnDyaTiaCgIF5//XUcHHScIZIXd82DVkTuVBaLhUmTJhESEsKsWbNwcHAgLi6OP/7444ZtevTowd69e7n33nttz1XPyMjg3//+N0888QQvv/wyR44cYfz48QQFBREUFATAzz//zNixY6lSpQqzZs3is88+Y+jQobdsO3fuXKKioqhduzZpaWmcPHkSgJUrV+Lp6cn7778PwP79+/U0Q5HboF9/RQzuwIEDpKSk8OSTT+Li4kLJkiWpVatWnrcTGxuLj48Pbdq0wdHRkapVqxIeHs6vv/5qWyc8PJxq1arh6OhIy5YtOXz4cK7aOjo6cuzYMS5cuEDp0qVtj4B1dHTk9OnTJCUl4eTkRO3atRXmIrdBR+YiBpeUlISPjw+Ojo752s6pU6fYv38/ffv2tS3Lysrivvvus70vV66c7bWzszMZGRm5ajt8+HCWLl3KJ598QsWKFenVqxc1atSgU6dOfPHFF/z73/8GICIigsjIyHz9HCJ3I4W5iMF5e3uTlJREVlZWtkB3cXEhMzPT9v706dPZ2l17BOzl5UVwcDDjxo3Lcx9u1bZatWq89NJLmM1mvvvuO6Kjo5kzZw6urq706dOHPn36cPToUd544w3uueceQkND89wHkbuZhtlFDK5atWp4eHiwaNEiMjIyyMzMZM+ePVSuXJndu3eTlJTEhQsXiImJydbO3d2dEydO2N43bNiQ48ePs379esxmM2azmQMHDnDs2LFb9uFmbc1mMz/99BMXLlzAyckJNzc32wS3rVu3kpiYiNVqxdXVFQcHB01+E7kNOjIXMTgHBwdGjRrFBx98wMCBAzGZTLRo0YJ+/frRrFkzRowYQZkyZejcuTO//fabrd3DDz/MrFmzWLVqFffeey/9+vXjlVdeYcGCBSxYsACr1UqlSpV46qmnbtkHV1fXm7Zdv349H3zwARaLBX9/fwYPHgzA8ePH+eCDDzh79iylSpXiwQcfJCQkpGAKJXIH06VpIiIiBqfxLBEREYNTmIuIiBicwlxERMTgFOYiIiIGpzAXERExOIW5iIiIwSnMRUREDE5hLiIiYnAKcxEREYP7f5Wek1ztvaFnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 8))\n",
    "plt.scatter([item[0] for item in animal_lookup.values()],\n",
    "            [item[1] for item in animal_lookup.values()])\n",
    "plt.xlabel('cuteness')\n",
    "plt.ylabel('size')\n",
    "for label, (cute, size) in animal_lookup.items():\n",
    "    plt.text(cute+1, size+1, label, fontsize=12)\n",
    "plt.arrow(\n",
    "    *(animal_lookup['tarantula']),\n",
    "    *(np.array(animal_lookup['hamster']) - np.array(animal_lookup['tarantula'])),\n",
    "    fc=\"b\", ec=\"b\", head_width=1.5, head_length=2, linewidth=1.5)\n",
    "plt.arrow(\n",
    "    *(animal_lookup['chicken']),\n",
    "    *(np.array(animal_lookup['hamster']) - np.array(animal_lookup['tarantula'])),\n",
    "    fc=\"r\", ec=\"r\", head_width=1.5, head_length=2, linewidth=1.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can understand this arrow as being the *relationship* between a tarantula and a hamster, in terms of their size and cuteness (i.e., hamsters and tarantulas are about the same size, but hamsters are much cuter). In the same diagram, I've also transposed this same arrow (this time in red) so that its origin point is \"chicken.\" The arrow ends closest to \"kitten.\" What we've discovered is that the animal that is about the same size as a chicken but much cuter is... a kitten. To put it in terms of an analogy:\n",
    "\n",
    "    Tarantulas are to hamsters as chickens are to kittens.\n",
    "    \n",
    "A sequence of numbers used to identify a point is called a *vector*, and the kind of math we've been doing so far is called *linear algebra.* (Linear algebra is surprisingly useful across many domains: It's the same kind of math you might do to, e.g., simulate the velocity and acceleration of a sprite in a video game.)\n",
    "\n",
    "A set of vectors that are all part of the same data set is often called a *vector space*. The vector space of animals in this section has two *dimensions*, by which I mean that each vector in the space has two numbers associated with it (i.e., two columns in the spreadsheet). The fact that this space has two dimensions just happens to make it easy to *visualize* the space by drawing a 2D plot. But most vector spaces you'll work with will have more than two dimensions—sometimes many hundreds. In those cases, it's more difficult to visualize the \"space,\" but the math works pretty much the same."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Language with vectors: colors\n",
    "\n",
    "So far, so good. We have a system in place—albeit highly subjective—for talking about animals and the words used to name them. I want to talk about another vector space that has to do with language: the vector space of colors.\n",
    "\n",
    "Colors are often represented in computers as vectors with three dimensions: red, green, and blue. Just as with the animals in the previous section, we can use these vectors to answer questions like: which colors are similar? What's the most likely color name for an arbitrarily chosen set of values for red, green and blue? Given the names of two colors, what's the name of those colors' \"average\"?\n",
    "\n",
    "We'll be working with this [color data](https://github.com/dariusk/corpora/blob/master/data/colors/xkcd.json) from the [xkcd color survey](https://blog.xkcd.com/2010/05/03/color-survey-results/). The data relates a color name to the RGB value associated with that color. [Here's a page that shows what the colors look like](https://xkcd.com/color/rgb/). Download the color data and put it in the same directory as this notebook.\n",
    "\n",
    "A few notes before we proceed:\n",
    "\n",
    "* We're using `numpy` arrays below to implement vector arithmetic. You'll need to install `numpy` to get the examples to work. (If you used Anaconda to install Python, you have `numpy` already.) [Here's a quick introduction on how to use numpy for vector arithmetic.](https://docs.scipy.org/doc/numpy/user/quickstart.html)\n",
    "* If you're interested in perceptually accurate color math in Python, consider using the [colormath library](http://python-colormath.readthedocs.io/en/latest/).\n",
    "\n",
    "Now, import the `json` library and load the color data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "color_data = json.loads(open(\"xkcd.json\").read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following function converts colors from hex format (`#1a2b3c`) to a tuple of integers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def hex_to_int(s):\n",
    "    s = s.lstrip(\"#\")\n",
    "    return np.array([int(s[:2], 16), int(s[2:4], 16), int(s[4:6], 16)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the following cell creates a dictionary and populates it with mappings from color names to RGB vectors for each color in the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "colors = dict()\n",
    "for item in color_data['colors']:\n",
    "    colors[item[\"color\"]] = hex_to_int(item[\"hex\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Testing it out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([110, 117,  14])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colors['olive']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([229,   0,   0])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colors['red']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colors['black']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0, 255, 255])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colors['cyan']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vector math\n",
    "\n",
    "Before we keep going, we'll need some functions for performing basic vector \"arithmetic.\" These functions will work with vectors in spaces of any number of dimensions.\n",
    "\n",
    "The first function returns the Euclidean distance between two points:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy.linalg import norm\n",
    "def distance(a, b):\n",
    "    return norm(a - b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "190.7275543805876"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distance(colors['cyan'], colors['blue'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0990195135927845"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distance(np.array([10, 1]), np.array([5, 2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subtracting vectors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ -3, 188,  32])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colors['cyan'] - colors['blue']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding vectors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  3, 322, 478])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colors['cyan'] + colors['blue']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can find the average of two vectors using the expected formula:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1.5, 161. , 239. ])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(colors['cyan'] + colors['blue']) / 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or use the following function, which finds the mean of any number of vectors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def meanv(vecs):\n",
    "    total = np.sum(vecs, axis=0)\n",
    "    return total / len(vecs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([195.,  43.,  75.])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "meanv([colors['red'], colors['pink'], colors['maroon']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just as a test, the following cell shows that the distance from \"red\" to \"green\" is greater than the distance from \"red\" to \"pink\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distance(colors['red'], colors['green']) > distance(colors['red'], colors['pink'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Finding the closest item\n",
    "\n",
    "Just as we wanted to find the animal that most closely matched an arbitrary point in cuteness/size space, we'll want to find the closest color name to an arbitrary point in RGB space. The easiest way to find the closest item to an arbitrary vector is simply to find the distance between the target vector and each item in the space, in turn, then sort the list from closest to most distant.\n",
    "\n",
    "Calculating the distance between two points, however, is computationally expensive, especially when you're working with data that has many dimensions. To solve this problem, computer scientists and mathematicians came up with the idea of [approximate nearest neighbor search](https://en.wikipedia.org/wiki/Nearest_neighbor_search#Approximation_methods), a technique for finding similar points in high-dimensional spaces that make use of various tricks to speed up the process (potentially at the cost of accuracy).\n",
    "\n",
    "We're going to use a library I made called [Simple Neighbors](https://github.com/aparrish/simpleneighbors) that builds such an approximate nearest neighbors index to quickly return the closest items for any given vector. (Simple Neighbors is based on [Annoy](https://pypi.python.org/pypi/annoy).)\n",
    "\n",
    "Install Simple Neighbors like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting simpleneighbors\n",
      "  Using cached simpleneighbors-0.1.0-py2.py3-none-any.whl (12 kB)\n",
      "Installing collected packages: simpleneighbors\n",
      "Successfully installed simpleneighbors-0.1.0\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "!{sys.executable} -m pip install simpleneighbors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll want to install Annoy as well, to speed up the nearest neighbor search. As of this writing, I recommend Annoy 1.16.3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting annoy==1.16.3\n",
      "  Using cached annoy-1.16.3.tar.gz (644 kB)\n",
      "Building wheels for collected packages: annoy\n",
      "  Building wheel for annoy (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for annoy: filename=annoy-1.16.3-cp38-cp38-macosx_10_9_x86_64.whl size=67466 sha256=a58d081c546e5df896ba76ec57b4aa8c86617b99063c044fb9ddc3bb44ef9116\n",
      "  Stored in directory: /Users/allison/Library/Caches/pip/wheels/93/66/00/3527630e17462dcb505b4688f787b40bc020268237d54e5e79\n",
      "Successfully built annoy\n",
      "Installing collected packages: annoy\n",
      "Successfully installed annoy-1.16.3\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "!{sys.executable} -m pip install annoy==1.16.3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you get an error from the above, and you're using Anaconda, you can try installing the Anaconda package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting package metadata (current_repodata.json): done\n",
      "Solving environment: done\n",
      "\n",
      "\n",
      "==> WARNING: A newer version of conda exists. <==\n",
      "  current version: 4.10.1\n",
      "  latest version: 4.10.3\n",
      "\n",
      "Please update conda by running\n",
      "\n",
      "    $ conda update -n base -c defaults conda\n",
      "\n",
      "\n",
      "\n",
      "## Package Plan ##\n",
      "\n",
      "  environment location: /Users/allison/opt/miniconda3/envs/rwet-2022\n",
      "\n",
      "  added / updated specs:\n",
      "    - python-annoy\n",
      "\n",
      "\n",
      "The following packages will be downloaded:\n",
      "\n",
      "    package                    |            build\n",
      "    ---------------------------|-----------------\n",
      "    libcxx-11.1.0              |       habf9029_0         1.0 MB  conda-forge\n",
      "    python-annoy-1.17.0        |   py38h0a5c65b_2          61 KB  conda-forge\n",
      "    ------------------------------------------------------------\n",
      "                                           Total:         1.1 MB\n",
      "\n",
      "The following NEW packages will be INSTALLED:\n",
      "\n",
      "  python-annoy       conda-forge/osx-64::python-annoy-1.17.0-py38h0a5c65b_2\n",
      "\n",
      "The following packages will be UPDATED:\n",
      "\n",
      "  ca-certificates    pkgs/main::ca-certificates-2021.5.25-~ --> conda-forge::ca-certificates-2021.5.30-h033912b_0\n",
      "  libcxx                         pkgs/main::libcxx-10.0.0-1 --> conda-forge::libcxx-11.1.0-habf9029_0\n",
      "\n",
      "The following packages will be SUPERSEDED by a higher-priority channel:\n",
      "\n",
      "  certifi            pkgs/main::certifi-2021.5.30-py38hecd~ --> conda-forge::certifi-2021.5.30-py38h50d1736_0\n",
      "  openssl              pkgs/main::openssl-1.1.1k-h9ed2024_0 --> conda-forge::openssl-1.1.1k-h0d85af4_0\n",
      "\n",
      "\n",
      "\n",
      "Downloading and Extracting Packages\n",
      "python-annoy-1.17.0  | 61 KB     | ##################################### | 100% \n",
      "libcxx-11.1.0        | 1.0 MB    | ##################################### | 100% \n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: done\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "!conda install -y --prefix {sys.prefix} -c conda-forge python-annoy "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If neither of those works—especially if you're using Windows—you may need to install a C++ compiler, or you can use this notebook on Binder. (If you're a student, come see me for more info.)\n",
    "\n",
    "Once you have the library installed, import it like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "from simpleneighbors import SimpleNeighbors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first parameter to `SimpleNeighbors()` is the number of dimensions in the data; the second is the distance metric to use. (This defaults to cosine distance, but in this case we want to use Euclidean distance.)\n",
    "\n",
    "The `.add_one()` method adds an item and a vector to the index. Once all the items are added, `.build()` actually builds the index. This should go quick, since we don't have that much data!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "color_lookup = SimpleNeighbors(3, 'euclidean')\n",
    "for name, vec in colors.items():\n",
    "    color_lookup.add_one(name, vec)\n",
    "color_lookup.build()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To find the nearest item to a specified vector, pass the vector to the `.nearest()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['red',\n",
       " 'fire engine red',\n",
       " 'bright red',\n",
       " 'tomato red',\n",
       " 'cherry red',\n",
       " 'scarlet',\n",
       " 'vermillion',\n",
       " 'orangish red',\n",
       " 'cherry',\n",
       " 'lipstick red',\n",
       " 'darkish red',\n",
       " 'neon red']"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.nearest(colors['red'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Limit the number of results returned with the second parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['red', 'fire engine red', 'bright red']"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.nearest(colors['red'], 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are the colors closest to (150, 60, 150):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['warm purple',\n",
       " 'medium purple',\n",
       " 'ugly purple',\n",
       " 'light eggplant',\n",
       " 'purpleish',\n",
       " 'purplish',\n",
       " 'purply',\n",
       " 'light plum',\n",
       " 'purple',\n",
       " 'muted purple',\n",
       " 'dull purple',\n",
       " 'dusty purple']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.nearest([150, 60, 150])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `.dist()` method gives the distance between two items in the index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "94.28679656982422"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.dist('rose', 'pink')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "221.90313720703125"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.dist('green', 'purple')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And you can check the `.corpus` attribute to see if an item is even in the index to begin with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'orange' in color_lookup.corpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'kitten' in color_lookup.corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Color magic\n",
    "\n",
    "The magical part of representing words as vectors is that the vector operations we defined earlier appear to operate on language the same way they operate on numbers. For example, if we find the word closest to the vector resulting from subtracting \"red\" from \"purple,\" we get a series of \"blue\" colors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['cobalt blue',\n",
       " 'royal blue',\n",
       " 'darkish blue',\n",
       " 'true blue',\n",
       " 'royal',\n",
       " 'prussian blue',\n",
       " 'dark royal blue',\n",
       " 'deep blue',\n",
       " 'marine blue',\n",
       " 'deep sea blue',\n",
       " 'darkblue',\n",
       " 'twilight blue']"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.nearest(colors['purple'] - colors['red'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This matches our intuition about RGB colors, which is that purple is a combination of red and blue. Take away the red, and blue is all you have left.\n",
    "\n",
    "You can do something similar with addition. What's blue plus green?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['bright turquoise',\n",
       " 'bright light blue',\n",
       " 'bright aqua',\n",
       " 'cyan',\n",
       " 'neon blue',\n",
       " 'aqua blue',\n",
       " 'bright cyan',\n",
       " 'bright sky blue',\n",
       " 'aqua',\n",
       " 'bright teal',\n",
       " 'aqua marine',\n",
       " 'greenish cyan']"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.nearest(colors['blue'] + colors['green'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's right, it's something like turquoise or cyan! What if we find the average of black and white? Predictably, we get gray:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['medium grey',\n",
       " 'purple grey',\n",
       " 'steel grey',\n",
       " 'battleship grey',\n",
       " 'grey purple',\n",
       " 'purplish grey',\n",
       " 'greyish purple',\n",
       " 'steel',\n",
       " 'warm grey',\n",
       " 'green grey',\n",
       " 'brown grey',\n",
       " 'bluish grey']"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the average of black and white: medium grey\n",
    "color_lookup.nearest(meanv([colors['white'], colors['black']]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just as with the tarantula/hamster example from the previous section, we can use color vectors to reason about relationships between colors. In the cell below, finding the difference between \"pink\" and \"red\" then adding it to \"blue\" seems to give us a list of colors that are to blue what pink is to red (i.e., a slightly lighter, less saturated shade):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['neon blue',\n",
       " 'bright sky blue',\n",
       " 'bright light blue',\n",
       " 'cyan',\n",
       " 'bright cyan',\n",
       " 'bright turquoise',\n",
       " 'clear blue',\n",
       " 'azure',\n",
       " 'dodger blue',\n",
       " 'lightish blue',\n",
       " 'sky blue',\n",
       " 'aqua blue']"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# an analogy: pink is to red as X is to blue\n",
    "pink_to_red = colors['pink'] - colors['red']\n",
    "color_lookup.nearest(pink_to_red + colors['blue'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another example of color analogies: Navy is to blue as true green/dark grass green is to green:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['true green',\n",
       " 'dark grass green',\n",
       " 'grassy green',\n",
       " 'racing green',\n",
       " 'bottle green',\n",
       " 'dark olive green',\n",
       " 'darkgreen',\n",
       " 'forrest green',\n",
       " 'grass green',\n",
       " 'navy green',\n",
       " 'dark olive',\n",
       " 'hunter green']"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# another example: \n",
    "navy_to_blue = colors['navy'] - colors['blue']\n",
    "color_lookup.nearest(navy_to_blue + colors['green'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The examples above are fairly simple from a mathematical perspective but nevertheless *feel* magical: they're demonstrating that it's possible to use math to reason about how people use language."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interlude: A Love Poem That Loses Its Way"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Roses are red, violets are blue\n",
      "Roses are vermillion, violets are deep sky blue\n",
      "Roses are bright orange, violets are water blue\n",
      "Roses are deep orange, violets are windows blue\n",
      "Roses are dark orange, violets are cornflower blue\n",
      "Roses are brick orange, violets are faded blue\n",
      "Roses are orange brown, violets are cool blue\n",
      "Roses are dirty orange, violets are off blue\n",
      "Roses are pumpkin, violets are steel blue\n",
      "Roses are rusty orange, violets are grey blue\n",
      "Roses are brick orange, violets are greyish blue\n",
      "Roses are rust orange, violets are grey/blue\n",
      "Roses are dark orange, violets are bluish grey\n",
      "Roses are burnt sienna, violets are grey/blue\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "red = colors['red']\n",
    "blue = colors['blue']\n",
    "for i in range(14):\n",
    "    rednames = color_lookup.nearest(red)\n",
    "    bluenames = color_lookup.nearest(blue)\n",
    "    print(\"Roses are \" + rednames[0] + \", violets are \" + bluenames[0])\n",
    "    red = colors[random.choice(rednames[1:])]\n",
    "    blue = colors[random.choice(bluenames[1:])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Doing bad digital humanities with color vectors\n",
    "\n",
    "With the tools above in hand, we can start using our vectorized knowledge of language toward academic ends. In the following example, I'm going to calculate the average color of Mary Shelley's *Frankenstein*.\n",
    "\n",
    "(Before you proceed, make sure to [download the text file from Project Gutenberg](http://www.gutenberg.org/files/84/84-0.txt) and place it in the same directory as this notebook.)\n",
    "\n",
    "First, we'll load [spaCy](https://spacy.io/). Note: For the rest of this tutorial to work, you'll want to download at least the medium model for English. The default \"small\" model doesn't include word vectors. I've also written an [introduction to spaCy](nlp-concepts-with-spacy.ipynb) that includes installation instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "nlp = spacy.load('en_core_web_md')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To calculate the average color, we'll follow these steps:\n",
    "\n",
    "1. Parse the text into words\n",
    "2. Check every word to see if it names a color in our vector space. If it does, add it to a list of vectors.\n",
    "3. Find the average of that list of vectors.\n",
    "4. Find the color(s) closest to that average vector.\n",
    "\n",
    "The following cell performs steps 1-3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[125.52050473 134.0851735  121.63722397]\n"
     ]
    }
   ],
   "source": [
    "doc = nlp(open(\"84-0.txt\").read())\n",
    "# use word.lower_ to normalize case\n",
    "drac_colors = [colors[word.lower_] for word in doc if word.lower_ in colors]\n",
    "avg_color = meanv(drac_colors)\n",
    "print(avg_color)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we'll pass the averaged color vector to the `closest()` function, yielding... well, it's just a grey mush, which is kinda what you'd expect from adding a bunch of colors together willy-nilly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['medium grey',\n",
       " 'green grey',\n",
       " 'steel grey',\n",
       " 'grey green',\n",
       " 'brown grey',\n",
       " 'battleship grey',\n",
       " 'greeny grey',\n",
       " 'purple grey',\n",
       " 'warm grey',\n",
       " 'grey/green',\n",
       " 'slate green',\n",
       " 'steel']"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "color_lookup.nearest(avg_color)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the other hand, here's what we get when we average the colors of Charlotte Perkins Gilman's classic *The Yellow Wallpaper*. ([Download from here](http://www.gutenberg.org/cache/epub/1952/pg1952.txt) and save in the same directory as this notebook if you want to follow along.) The result definitely reflects the content of the story, so maybe we're on to something here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['pea',\n",
       " 'puke yellow',\n",
       " 'sick green',\n",
       " 'vomit yellow',\n",
       " 'booger',\n",
       " 'olive yellow',\n",
       " 'snot',\n",
       " 'gross green',\n",
       " 'dirty yellow',\n",
       " 'mustard yellow',\n",
       " 'dark yellow',\n",
       " 'baby puke green']"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc = nlp(open(\"1952-0.txt\").read())\n",
    "wallpaper_colors = [colors[word.lower_] for word in doc if word.lower_ in colors]\n",
    "avg_color = meanv(wallpaper_colors)\n",
    "color_lookup.nearest(avg_color)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Exercise for the reader: Use the vector arithmetic functions to rewrite a text, making it...\n",
    "\n",
    "* more blue (i.e., add `colors['blue']` to each occurrence of a color word); or\n",
    "* more light (i.e., add `colors['white']` to each occurrence of a color word); or\n",
    "* darker (i.e., attenuate each color. You might need to write a vector multiplication function to do this one right.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Distributional semantics\n",
    "\n",
    "In the previous section, the examples are interesting because of a simple fact: colors that we think of as similar are \"closer\" to each other in RGB vector space. In our color vector space, or in our animal cuteness/size space, you can think of the words identified by vectors close to each other as being *synonyms*, in a sense: they sort of \"mean\" the same thing. They're also, for many purposes, *functionally identical*. Think of this in terms of writing, say, a search engine. If someone searches for \"mauve trousers,\" then it's probably also okay to show them results for, say,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mauve trousers\n",
      "dusty rose trousers\n",
      "dusky rose trousers\n",
      "brownish pink trousers\n",
      "old pink trousers\n",
      "reddish grey trousers\n",
      "dirty pink trousers\n",
      "old rose trousers\n",
      "light plum trousers\n",
      "ugly pink trousers\n",
      "pinkish brown trousers\n",
      "dusky pink trousers\n"
     ]
    }
   ],
   "source": [
    "for cname in color_lookup.nearest(colors['mauve']):\n",
    "    print(cname + \" trousers\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's all well and good for color words, which intuitively seem to exist in a multidimensional continuum of perception, and for our animal space, where we've written out the vectors ahead of time. But what about... arbitrary words? Is it possible to create a vector space for all English words that has this same \"closer in space is closer in meaning\" property?\n",
    "\n",
    "To answer that, we have to back up a bit and ask the question: what does *meaning* mean? No one really knows, but one theory popular among computational linguists, computer scientists and other people who make search engines is the [Distributional Hypothesis](https://en.wikipedia.org/wiki/Distributional_semantics), which states that:\n",
    "\n",
    "    Linguistic items with similar distributions have similar meanings.\n",
    "    \n",
    "What's meant by \"similar distributions\" is *similar contexts*. Take for example the following sentences:\n",
    "\n",
    "    It was really cold yesterday.\n",
    "    It will be really warm today, though.\n",
    "    It'll be really hot tomorrow!\n",
    "    Will it be really cool Tuesday?\n",
    "    \n",
    "According to the Distributional Hypothesis, the words `cold`, `warm`, `hot` and `cool` must be related in some way (i.e., be close in meaning) because they occur in a similar context, i.e., between the word \"really\" and a word indicating a particular day. (Likewise, the words `yesterday`, `today`, `tomorrow` and `Tuesday` must be related, since they occur in the context of a word indicating a temperature.)\n",
    "\n",
    "In other words, according to the Distributional Hypothesis, a word's meaning is just a big list of all the contexts it occurs in. Two words are closer in meaning if they share contexts."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Word vectors by counting contexts\n",
    "\n",
    "So how do we turn this insight from the Distributional Hypothesis into a system for creating general-purpose vectors that capture the meaning of words? Maybe you can see where I'm going with this. What if we made a *really big* spreadsheet that had one column for every context for every word in a given source text. Let's use a small source text to begin with, such as this excerpt from Dickens:\n",
    "\n",
    "    It was the best of times, it was the worst of times.\n",
    "\n",
    "The code in the following cell builds a table of what such a spreadsheet might look like. (Don't worry about understanding this code! But feel free to play around with it.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>START ___ was</th>\n",
       "      <th>it ___ the</th>\n",
       "      <th>was ___ best</th>\n",
       "      <th>the ___ of</th>\n",
       "      <th>best ___ times</th>\n",
       "      <th>of ___ it</th>\n",
       "      <th>times ___ was</th>\n",
       "      <th>was ___ worst</th>\n",
       "      <th>worst ___ times</th>\n",
       "      <th>of ___ END</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>it</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>was</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>the</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>best</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>of</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>times</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>worst</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       START ___ was  it ___ the  was ___ best  the ___ of  best ___ times  \\\n",
       "it                 1           0             0           0               0   \n",
       "was                0           2             0           0               0   \n",
       "the                0           0             1           0               0   \n",
       "best               0           0             0           1               0   \n",
       "of                 0           0             0           0               1   \n",
       "times              0           0             0           0               0   \n",
       "worst              0           0             0           1               0   \n",
       "\n",
       "       of ___ it  times ___ was  was ___ worst  worst ___ times  of ___ END  \n",
       "it             0              1              0                0           0  \n",
       "was            0              0              0                0           0  \n",
       "the            0              0              1                0           0  \n",
       "best           0              0              0                0           0  \n",
       "of             0              0              0                1           0  \n",
       "times          1              0              0                0           1  \n",
       "worst          0              0              0                0           0  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from collections import defaultdict\n",
    "src = \"it was the best of times it was the worst of times\"\n",
    "tokens = ['START'] + src.split() + ['END']\n",
    "contexts = defaultdict(lambda: defaultdict(int))\n",
    "tok_types = {}\n",
    "for i in range(len(tokens)-2):\n",
    "    tok_types[tokens[i+1]] = 1\n",
    "    contexts[(tokens[i], tokens[i+2])][tokens[i+1]] += 1\n",
    "data = {\" ___ \".join(k): [v[tok] for tok in tok_types.keys()] for k, v in contexts.items()}\n",
    "df = pd.DataFrame(data=data, index=tok_types)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The spreadsheet has one column for every possible context, and one row for every word. The values in each cell correspond with how many times the word occurs in the given context. The numbers in the columns constitute that word's vector, i.e., the vector for the word `of` is\n",
    "\n",
    "    [0, 0, 0, 0, 1, 0, 0, 0, 1, 0]\n",
    "    \n",
    "Because there are ten possible contexts, this is a ten dimensional space! It might be strange to think of it, but you can do vector arithmetic on vectors with ten dimensions just as easily as you can on vectors with two or three dimensions, and you could use the same distance formula that we defined earlier to get useful information about which vectors in this space are similar to each other. In particular, the vectors for `best` and `worst` are actually the same (a distance of zero), since they occur only in the same context (`the ___ of`):\n",
    "\n",
    "    [0, 0, 0, 1, 0, 0, 0, 0, 0, 0]\n",
    "    \n",
    "Of course, the conventional way of thinking about \"best\" and \"worst\" is that they're *antonyms*, not *synonyms*. But they're also clearly two words of the same kind, with related meanings (through opposition), a fact that is captured by this distributional model.\n",
    "\n",
    "### Contexts and dimensionality\n",
    "\n",
    "Of course, in a corpus of any reasonable size, there will be many thousands if not many millions of possible contexts. It's difficult enough working with a vector space of ten dimensions, let alone a vector space of a million dimensions! It turns out, though, that many of the dimensions end up being superfluous and can either be eliminated or combined with other dimensions without significantly affecting the predictive power of the resulting vectors. The process of getting rid of superfluous dimensions in a vector space is called [dimensionality reduction](https://en.wikipedia.org/wiki/Dimensionality_reduction), and most implementations of count-based word vectors make use of dimensionality reduction so that the resulting vector space has a reasonable number of dimensions (say, 100—300, depending on the corpus and application).\n",
    "\n",
    "The question of how to identify a \"context\" is itself very difficult to answer. In the toy example above, we've said that a \"context\" is just the word that precedes and the word that follows. Depending on your implementation of this procedure, though, you might want a context with a bigger \"window\" (e.g., two words before and after), or a non-contiguous window (skip a word before and after the given word). You might exclude certain \"function\" words like \"the\" and \"of\" when determining a word's context, or you might [lemmatize](https://en.wikipedia.org/wiki/Lemmatisation) the words before you begin your analysis, so two occurrences with different \"forms\" of the same word count as the same context. These are all questions open to research and debate, and different implementations of procedures for creating count-based word vectors make different decisions on this issue.\n",
    "\n",
    "### GloVe vectors\n",
    "\n",
    "But you don't have to create your own word vectors from scratch! Many researchers have made downloadable databases of pre-trained vectors. One such project is Stanford's [Global Vectors for Word Representation (GloVe)](https://nlp.stanford.edu/projects/glove/). These 300-dimensional vectors are included with spaCy, and they're the vectors we'll be using for the rest of this tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Word vectors in spaCy\n",
    "\n",
    "Okay, let's have some fun with real word vectors. We're going to use the GloVe vectors that come with spaCy to creatively analyze and manipulate the text of *Frankenstein*. First, make sure you've got `spacy` imported:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following cell loads the language model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp = spacy.load('en_core_web_md')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the vector of any word in spaCy's vocabulary using the `vocab` attribute, like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-2.2743e-01, -5.1464e-02, -4.3421e-02, -1.0523e-01, -3.3389e-01,\n",
       "       -4.9611e-01, -6.4342e-01, -4.6994e-01,  3.9693e-01,  8.4902e-01,\n",
       "       -4.0845e-01, -1.8312e-01, -6.2564e-01, -1.0160e-01, -3.6914e-01,\n",
       "        6.8634e-01,  2.6607e-01,  3.2985e-01, -9.8570e-02, -1.3281e-01,\n",
       "       -4.7505e-01, -1.9249e-01, -3.1917e-01, -1.8536e-01,  1.1118e-01,\n",
       "        7.3772e-02, -3.0407e-01, -2.7552e-01,  6.1108e-01, -3.6344e-01,\n",
       "       -4.5849e-01, -1.2872e-01,  1.5175e-01,  3.3248e-01,  3.0900e-01,\n",
       "       -2.8488e-01,  2.5544e-01, -9.4332e-01, -5.5746e-01,  5.8764e-02,\n",
       "        1.1174e-01,  2.0032e-01, -4.1090e-01, -5.4444e-01, -4.3831e-02,\n",
       "        1.6265e-01, -6.8028e-01,  2.8266e-01,  1.8177e-01, -5.6184e-01,\n",
       "        7.0911e-02, -3.4996e-01, -3.1639e-01,  1.7666e-01, -9.4568e-03,\n",
       "        4.4389e-01,  7.6684e-02, -2.1797e-01,  1.3728e-03,  2.3474e-01,\n",
       "       -1.8564e-01, -4.2277e-01,  2.5585e-01, -6.2553e-01, -1.4335e-01,\n",
       "       -1.8835e-01,  3.5240e-01,  2.0764e-01,  8.8644e-02, -2.0873e-01,\n",
       "       -3.9081e-01, -1.5079e-01, -3.4469e-01, -3.2128e-01, -1.2094e-01,\n",
       "       -6.6444e-03, -1.6742e-01, -3.5412e-01,  3.5457e-01, -6.8729e-01,\n",
       "        4.2718e-01,  3.2739e-01, -5.2189e-01,  1.9016e-01, -2.0203e-01,\n",
       "       -2.5103e-02,  1.4170e+00,  5.4864e-01,  6.3232e-01,  9.1078e-02,\n",
       "        1.6614e-01,  6.4225e-01,  9.4285e-02, -5.8877e-01, -5.8017e-01,\n",
       "        2.4200e-02, -5.9718e-02, -2.0356e-01,  3.6787e-01, -2.1599e-04,\n",
       "        1.2642e-01,  2.1863e-01,  1.4783e-01,  1.1456e-01,  5.1725e-01,\n",
       "       -5.2379e-01,  1.9920e-01, -3.4157e-01, -4.5679e-01, -3.6249e-01,\n",
       "       -1.5894e-01,  4.1326e-01,  3.3038e-01, -6.0792e-01,  3.3837e-02,\n",
       "       -1.3185e-01, -2.3943e-01,  2.0958e-01,  5.4647e-01, -1.6166e-01,\n",
       "       -8.4986e-02, -3.7066e-03,  4.1813e-01, -6.2813e-01, -1.5596e-01,\n",
       "        2.7174e-01,  2.6749e-01, -3.1466e-01, -3.0005e-01, -1.1754e-03,\n",
       "       -7.6564e-02, -4.1596e-01, -5.1201e-01,  3.3995e-01,  3.3515e-01,\n",
       "        2.3181e-01, -2.3126e-01,  6.4867e-01, -1.7327e-01,  2.1906e-01,\n",
       "       -2.4825e+00, -3.8253e-01, -2.7501e-01,  3.8250e-01,  3.9909e-01,\n",
       "       -3.4766e-01, -3.4750e-01,  1.1423e-01,  3.8426e-01, -4.5397e-02,\n",
       "        3.3079e-01,  3.0611e-01, -1.6365e-01, -2.1840e-01, -4.9622e-01,\n",
       "        3.2069e-01, -1.0056e-01,  4.7965e-01, -6.1692e-01, -7.1039e-01,\n",
       "        6.7294e-03, -1.3760e-01, -2.4534e-01, -6.3200e-01, -1.3581e-01,\n",
       "        5.0657e-02, -2.6497e-01,  4.7350e-02, -2.1670e-01,  1.1975e-01,\n",
       "       -1.5930e-01,  6.5097e-02,  6.2833e-01, -4.1485e-01, -4.9077e-01,\n",
       "        5.3359e-01, -2.8512e-01, -6.8738e-02, -1.5921e-01,  7.1961e-01,\n",
       "       -9.9519e-02, -4.4382e-01,  1.2937e-01, -3.0042e-01, -4.5345e-01,\n",
       "       -1.2133e-01, -5.7589e-02,  1.7615e-03, -1.4016e-01, -5.4348e-01,\n",
       "       -2.7313e-01,  7.9147e-02,  9.2326e-02, -3.2881e-01,  4.2704e-01,\n",
       "        2.2025e-01, -2.8184e-01, -1.0668e-01,  7.8995e-01,  1.0433e-01,\n",
       "       -7.1160e-01,  5.0875e-01, -4.4899e-01, -1.5814e-01,  8.9031e-02,\n",
       "        1.9779e-01,  1.7832e-01, -1.5837e-01,  4.1920e-01,  1.1525e-01,\n",
       "       -5.6151e-01,  1.5977e-03, -6.3471e-01,  1.7369e-01, -1.6660e-01,\n",
       "        1.1221e-01, -6.8455e-02,  2.5285e-01,  3.2207e-01, -9.7379e-02,\n",
       "       -6.0705e-02,  2.1614e-03,  4.7962e-01, -6.2126e-01, -3.8508e-01,\n",
       "       -3.3929e-02,  1.6450e-01,  8.4856e-01,  1.6799e-01,  7.0092e-02,\n",
       "       -3.7176e-01,  3.3799e-01, -4.4381e-01, -4.1767e-01,  4.5403e-01,\n",
       "       -1.2410e-01, -1.2079e-01, -1.7261e-01,  6.0124e-01, -4.0454e-01,\n",
       "       -7.5649e-01, -8.0093e-02, -4.0163e-01,  1.4112e-01, -1.0350e-01,\n",
       "       -3.3089e-02,  1.1493e-01,  5.8734e-01,  3.3808e-01,  2.3712e-01,\n",
       "       -1.8119e-01, -9.0462e-02, -7.5090e-02,  3.0095e-01,  2.4288e-01,\n",
       "       -1.1910e-01, -7.9882e-01,  1.8590e-01,  1.8869e-01, -3.0644e-01,\n",
       "       -1.1699e-01, -1.9925e-01, -2.5868e-02, -1.9041e-01, -2.1018e-01,\n",
       "        3.7344e-01, -4.6269e-01, -2.2280e-01, -3.7530e-01, -2.1085e-02,\n",
       "       -4.1236e-01,  7.1003e-01, -1.3208e-01,  1.5749e-01, -2.3107e-01,\n",
       "        1.0245e-02,  1.2600e-01,  2.5397e-01,  4.6801e-02,  8.9900e-03,\n",
       "        2.9246e-01,  1.7141e-01,  1.7920e-01, -4.3189e-01,  5.1214e-01,\n",
       "        4.9256e-01,  3.1945e-01, -3.5943e-01,  1.0088e-01, -3.7515e-01,\n",
       "        2.1281e-02,  2.0247e-01, -3.4238e-02,  3.3511e-01, -4.2808e-01,\n",
       "        7.4968e-01, -1.2136e-01,  4.3539e-01,  1.5098e-01, -1.5952e-01],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.vocab['kitten'].vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "spaCy's English model uses 300-dimensional pre-trained GloVe vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the sake of convenience, the following function gets the vector of a given string from spaCy's vocabulary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "def vec(s):\n",
    "    return nlp.vocab[s].vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filter by word frequency\n",
    "\n",
    "In just a second, I'm going to show you how to make a nearest-neighbors lookup to find words that are semantically similar. We'll use the vectors from spaCy, but there's a problem—spaCy's database of words is very large. You can find out how many words are in spaCy's database by accessing the `.meta` attribute of the language object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'width': 300,\n",
       " 'vectors': 20000,\n",
       " 'keys': 684830,\n",
       " 'name': 'en_core_web_md.vectors'}"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.meta['vectors']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As of this writing, there are 20k vectors in the database, but over 680k individual tokens. (Many tokens don't have vectors; some tokens are mapped to the same vector.) That's too many tokens for our nearest-neighbor lookup to be reasonably fast. Also, because the tokens are drawn from the corpus that spaCy's model is trained on, many of the tokens are very uncommon words or misspellings of other words. We probably don't want to use this for text replacement tasks, which is what we've got an eye on doing in the examples below.\n",
    "\n",
    "The best way to fix this problem is to filter the words before we put them into the nearest-neighbor lookup. We're going to do this by *token frequency*, i.e., how frequently that word appears in English. A word's frequency is calculated as the number of times that word occurs in a corpus, divided by the total number of words in the corpus. The [wordfreq](https://github.com/LuminosoInsight/wordfreq) Python package has a good database of word frequencies in many different languages. To simplify things for this notebook, I've prepared [a JSON file with the top 25k English words, along with their probabilities](https://raw.githubusercontent.com/aparrish/wordfreq-en-25000/main/wordfreq-en-25000-log.json). Before you continue, download that file into the same directory as this notebook, or with `curl` like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100  898k  100  898k    0     0   511k      0  0:00:01  0:00:01 --:--:--  511k\n"
     ]
    }
   ],
   "source": [
    "!curl -L -O https://raw.githubusercontent.com/aparrish/wordfreq-en-25000/main/wordfreq-en-25000-log.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code in the following cell loads this data into a dictionary that we can use to look up the probability of a given word:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "prob_lookup = dict(json.load(open(\"./wordfreq-en-25000-log.json\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can look up a word's probability with the `prob_lookup` dictionary. As you might imagine, the frequency of most words is very small, meaning that if we stored the frequency as a decimal fraction (e.g., 0.00006), the data on disk would mostly consist of the digit `0`. To make the numbers easier to store and work with, the log of the word probabilities, rather than the probability itself:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-5.7108"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob_lookup['me']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Log probabilities are always negative; the closer to zero, the more frequent the token. To get the original probability number, use `math.exp()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0033100234666365628"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import math\n",
    "math.exp(prob_lookup['me'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(You can interpret the result of the above expression to mean that the word `me` occurs about 33 times in every ten thousand words.)\n",
    "\n",
    "Note that all of the tokens in this database are stored in lower case, and you'll get a `KeyError` for words that are not present in the database! Some other examples of using the lookup:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-12.0892"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob_lookup['allison']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cats aren't more frequent than dogs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob_lookup['cats'] > prob_lookup['dogs']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unknown tokens raise `KeyError`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'asdfasdf'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-62-6121b64df678>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mprob_lookup\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'asdfasdf'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m: 'asdfasdf'"
     ]
    }
   ],
   "source": [
    "prob_lookup['asdfasdf']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But you can use the `dict` object's `.get()` method to return a somewhat reasonable estimate of a word's probability, even if it's absent:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-20.0"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob_lookup.get('asdfasdf', -20.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking up synonyms\n",
    "\n",
    "So now we can finally make our synonym lookup! Here I make a `SimpleNeighbors` index that loads in all words from spaCy's vocab that (a) have an associated vector and (b) are in our database of the top 25k words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "lookup = SimpleNeighbors(300)\n",
    "for word in prob_lookup.keys():\n",
    "    if nlp.vocab[word].has_vector:\n",
    "        lookup.add_one(word, vec(word))\n",
    "lookup.build()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This leaves us with nearly 25k words in the lookup (a few of the words from the word frequency list don't have vectors in spaCy, apparently):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "24575"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(lookup)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can get synonyms for words by looking up the word's vector and finding the nearest word in the index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['basketball',\n",
       " 'volleyball',\n",
       " 'lacrosse',\n",
       " 'football',\n",
       " 'soccer',\n",
       " 'baseball',\n",
       " 'softball',\n",
       " 'hockey',\n",
       " 'tennis',\n",
       " 'racket',\n",
       " 'badminton',\n",
       " 'gymnastics']"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lookup.nearest(vec('basketball'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Exercise: Limit the synonym lookup to words with a frequency greater than a particular threshold."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fun with spaCy and vector arithmetic\n",
    "\n",
    "Now we can start doing vector arithmetic and finding the closest words to the resulting vectors. For example, what word is closest to the halfway point between day and night?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['night',\n",
       " 'day',\n",
       " 'evening',\n",
       " 'morning',\n",
       " 'afternoon',\n",
       " 'midday',\n",
       " 'nights',\n",
       " 'weekend',\n",
       " 'outing',\n",
       " 'tomorrow',\n",
       " 'every',\n",
       " 'next']"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# halfway between day and night\n",
    "lookup.nearest(meanv([vec(\"day\"), vec(\"night\")]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Variations of `night` and `day` are still closest, but after that we get words like `evening` and `morning`, which are indeed halfway between day and night!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are the closest words in *Frankenstein* to \"wine\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['wine',\n",
       " 'wines',\n",
       " 'winery',\n",
       " 'tasting',\n",
       " 'beer',\n",
       " 'lager',\n",
       " 'drink',\n",
       " 'drinks',\n",
       " 'beverages',\n",
       " 'fruit',\n",
       " 'fig',\n",
       " 'cocktail']"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lookup.nearest(vec(\"wine\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you subtract \"alcohol\" from \"wine\" and find the closest words to the resulting vector, you're left with the superlatives you might use to describe wine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['elegant',\n",
       " 'exquisite',\n",
       " 'elegance',\n",
       " 'graceful',\n",
       " 'fabulous',\n",
       " 'fab',\n",
       " 'magnificent',\n",
       " 'splendid',\n",
       " 'marvellous',\n",
       " 'delightful',\n",
       " 'lovely',\n",
       " 'charming']"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lookup.nearest(vec(\"wine\") - vec(\"alcohol\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The closest words to \"water\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['water',\n",
       " 'seawater',\n",
       " 'salt',\n",
       " 'brine',\n",
       " 'dry',\n",
       " 'boiling',\n",
       " 'bubbling',\n",
       " 'heat',\n",
       " 'heats',\n",
       " 'cubic',\n",
       " 'gallons',\n",
       " 'litres']"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lookup.nearest(vec(\"water\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But if you add \"boil\" to \"water,\" you get \"simmer\" and \"steamed\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['water',\n",
       " 'seawater',\n",
       " 'boil',\n",
       " 'simmer',\n",
       " 'boiling',\n",
       " 'bubbling',\n",
       " 'salt',\n",
       " 'brine',\n",
       " 'boiled',\n",
       " 'salted',\n",
       " 'steamed',\n",
       " 'heat']"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lookup.nearest(vec(\"water\") + vec(\"boil\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Replace with synonym\n",
    "\n",
    "The following example replaces all nouns, verbs and adjectives with a closely-related word from the synonym lookup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "frost_doc = nlp(open(\"frost.txt\").read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Two roads evolved in a orange wood,\n",
      "And sorry I might not trips both\n",
      "And be one traveller, long I stood\n",
      "And turned down one as far as I could\n",
      "To where it bends in the bushes;\n",
      "\n",
      "Then went the these, as just as fair,\n",
      "And being perhaps the way claimants,\n",
      "Because it was meadow and did clothes;\n",
      "Though as for that the passes there\n",
      "Had worn them really about the one,\n",
      "\n",
      "And both that afternoon equally sit\n",
      "In leaves no step had tread black.\n",
      "Oh, I kept the fifth for another week!\n",
      "Yet realize how going leads on to going,\n",
      "I doubting if I not ever come back.\n",
      "\n",
      "I hereby be talking this with a chuckle\n",
      "Somewhere ages and children hence:\n",
      "Two townships evolved in a wood, and I—\n",
      "I took the that less traveled by,\n",
      "And that has make all the difference.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "output = []\n",
    "for word in frost_doc:\n",
    "    if word.is_alpha and word.pos_ in ('NOUN', 'VERB', 'ADJ'):\n",
    "        new_word = random.choice(lookup.nearest(word.vector, 3))\n",
    "        output.append(new_word)\n",
    "    else:\n",
    "        output.append(word.text)\n",
    "    output.append(word.whitespace_)\n",
    "print(''.join(output))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tinting meaning\n",
    "\n",
    "You can \"tint\" meaning with word vectors as well. The methodology demonstrated below is to take the word vector for every noun, verb and adjective in the text, and then look up the word closest to a weighted average between the original word and a target word. (You can control the weight of the average using the `factor` variable below.) On balance, this leads to word replacements that still at least somewhat \"make sense\" in the original context, but have a bit of meaning from the target word."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "target_word = 'spaceship'\n",
    "factor = 0.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Two southbound planet in a blue wood,\n",
      "And sorry I could not airbus both\n",
      "And be one traveller, long I stared\n",
      "And looked down something as far as I they\n",
      "To where it robot in the shrub;\n",
      "\n",
      "Then aboard the airplane, as just as would,\n",
      "And could perhaps the better claims,\n",
      "Because it was sod and never armor;\n",
      "Though as for that the plane there\n",
      "Had resemble them really about the weird,\n",
      "\n",
      "And both that arrived equally suddenly\n",
      "In dark no turn had michelin purple.\n",
      "Oh, I abruptly the finally for another days!\n",
      "Yet thing how it plane on to somehow,\n",
      "I universe if I might ever they back.\n",
      "\n",
      "I humankind be tells this with a wink\n",
      "Somewhere universe and planets hence:\n",
      "Two crossing planetary in a hardwood, and I—\n",
      "I aboard the another somewhere aboard by,\n",
      "And that has ever all the cosmos.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "output = []\n",
    "for word in frost_doc:\n",
    "    if word.is_alpha and word.pos_ in ('NOUN', 'VERB', 'ADJ'):\n",
    "        new_word = random.choice(\n",
    "            lookup.nearest((word.vector*(1-factor)) + (vec(target_word)*factor), 5))\n",
    "        output.append(new_word)\n",
    "    else:\n",
    "        output.append(word.text)\n",
    "    output.append(word.whitespace_)\n",
    "print(''.join(output))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Semantic similarity search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every spaCy span (e.g., a document or a sentence) has a `.vector` attribute that gives a \"summary\" vector for the span in question. (By default, this is calculated by averaging together all of the vectors.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "sent = nlp(\"Programming computers is fun!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.09078191e-01,  1.51335195e-01,  2.12372467e-01, -1.53214008e-01,\n",
       "        1.09147593e-01,  1.73588041e-02,  8.59839991e-02, -2.55478203e-01,\n",
       "        2.09430605e-01,  1.65583992e+00, -9.05231461e-02, -1.32969588e-01,\n",
       "        3.42764035e-02,  1.50568604e-01, -6.61468059e-02, -1.55275792e-01,\n",
       "       -1.31082803e-01,  1.50442791e+00, -2.02654406e-01, -8.86391997e-02,\n",
       "        4.99612018e-02, -1.54533401e-01, -1.00691400e-01,  5.44592626e-02,\n",
       "        2.60579765e-01, -7.77239352e-03,  8.82299989e-02, -2.72476021e-02,\n",
       "        1.29189998e-01, -1.21251188e-01, -2.45104164e-01, -2.30066583e-01,\n",
       "        9.57625657e-02,  2.77934074e-02,  1.57128006e-01,  1.09506592e-01,\n",
       "       -5.01117930e-02,  3.19651991e-01, -1.52970748e-02,  5.11708446e-02,\n",
       "        4.09313962e-02,  1.67218000e-01,  3.71664017e-02,  1.76439993e-02,\n",
       "        6.30135983e-02,  2.77248025e-01, -2.26011992e-01, -9.00347978e-02,\n",
       "        2.31373996e-01,  1.76831819e-02, -4.28502038e-02,  1.16572000e-01,\n",
       "        1.63890302e-01, -5.13000181e-03, -6.84387982e-02, -1.40187204e-01,\n",
       "        4.90408018e-02, -1.48198009e-02,  4.10723984e-02,  4.03099991e-02,\n",
       "        3.63446400e-02,  6.02504015e-02, -2.44283993e-02, -6.33216053e-02,\n",
       "        3.33246201e-01, -1.65840611e-01, -1.42393991e-01,  2.63868392e-01,\n",
       "       -7.06279352e-02, -1.15957949e-02,  2.43643999e-01, -7.76533633e-02,\n",
       "        2.68735200e-01, -2.65026003e-01,  6.98260069e-02,  5.55027649e-02,\n",
       "        6.69239983e-02,  1.02014199e-01, -2.83342004e-02,  9.05580044e-01,\n",
       "       -1.48870006e-01, -2.06700023e-02, -9.18637961e-02,  1.17958404e-01,\n",
       "        3.48474011e-02,  3.01625971e-02, -2.66392171e-01,  6.03520051e-02,\n",
       "        4.36886013e-01,  3.55404627e-04,  9.28714313e-03,  4.18313980e-01,\n",
       "       -8.68320018e-02,  9.79305953e-02,  1.07736051e-01, -1.05597004e-01,\n",
       "       -1.53806001e-01, -1.11872196e-01,  6.13286011e-02,  8.13250020e-02,\n",
       "        1.34859979e-02, -1.45868644e-01, -3.23198020e-01, -3.61048020e-02,\n",
       "        1.01260401e-01, -5.27896583e-01,  2.11459801e-01,  6.45498037e-02,\n",
       "        5.44675998e-02,  4.04881984e-02,  5.71960025e-02, -1.76566601e-01,\n",
       "        1.55324399e-01, -1.41639993e-01, -1.76177889e-01, -2.53822207e-01,\n",
       "       -1.48269609e-01,  1.15075789e-01, -7.15504065e-02,  1.12987004e-01,\n",
       "        6.85439929e-02, -1.00209795e-01,  1.24472596e-01, -1.65226191e-01,\n",
       "        2.14665998e-02,  1.31642014e-01,  1.09824404e-01,  1.41431212e-01,\n",
       "        2.18189210e-01, -3.42240818e-02,  1.91150591e-01, -6.14733994e-02,\n",
       "       -1.36373192e-01, -2.20621955e-02, -1.83210019e-02, -5.04960045e-02,\n",
       "       -6.72888011e-02,  1.17635809e-01,  1.11440256e-01,  2.77000397e-01,\n",
       "       -1.40339601e+00,  1.59925997e-01,  4.14776236e-01, -5.32246009e-02,\n",
       "        9.94336009e-02, -1.21174000e-01,  2.33742930e-02, -1.68137982e-01,\n",
       "       -5.25300018e-02, -1.62744999e-01, -1.84371993e-02,  1.27291799e-01,\n",
       "        3.46560031e-02, -3.79859796e-03,  3.25425640e-02, -2.06223205e-01,\n",
       "        6.51135966e-02, -2.54101604e-01,  2.12639980e-02, -2.53134608e-01,\n",
       "       -3.12441196e-02, -7.77560333e-03,  1.54054984e-01, -1.85033411e-01,\n",
       "       -9.07502919e-02,  1.48299960e-02, -1.31761596e-01, -9.24229994e-02,\n",
       "        1.02589414e-01,  1.28178000e-01, -2.28297599e-02,  2.08275229e-01,\n",
       "        1.61991984e-01, -2.20977981e-02,  1.49040073e-02, -1.04619991e-02,\n",
       "       -1.39642403e-01, -9.92640294e-03, -1.11510001e-01,  8.21727961e-02,\n",
       "        8.41722041e-02, -2.92574227e-01, -4.99619991e-02, -1.13490000e-01,\n",
       "       -6.10653982e-02, -1.06445970e-02, -3.00779995e-02, -2.66663972e-02,\n",
       "        9.19132009e-02, -1.09540019e-02, -4.19231988e-02, -9.31339711e-02,\n",
       "       -1.30866006e-01,  2.00029343e-01, -3.60463932e-02,  1.66117996e-01,\n",
       "        2.14560404e-02, -2.79834002e-01,  2.95516193e-01,  1.95245996e-01,\n",
       "       -7.03940019e-02,  1.68732200e-02,  2.67192185e-01,  1.27668027e-02,\n",
       "        7.61620095e-03, -7.91379958e-02, -1.82687804e-01,  2.69200020e-02,\n",
       "        2.37371996e-01, -2.33727008e-01, -3.09768796e-01, -2.82416195e-01,\n",
       "       -4.95892018e-02, -1.61307290e-01, -8.55911747e-02,  2.08437994e-01,\n",
       "        8.23039934e-02, -1.89800188e-03, -4.33899969e-01,  7.30374008e-02,\n",
       "        1.23745993e-01,  1.02426387e-01, -1.32522792e-01,  1.08762406e-01,\n",
       "       -3.76218036e-02, -1.29930809e-01,  1.45559996e-01,  2.57495999e-01,\n",
       "        1.42793730e-01,  2.29041986e-02, -1.84964210e-01,  4.56799287e-03,\n",
       "        7.56099969e-02, -4.75694016e-02,  3.27790007e-02,  9.00008008e-02,\n",
       "        7.23747984e-02, -5.11239991e-02, -2.02823997e-01,  1.57281995e-01,\n",
       "        2.39320993e-01, -9.58018005e-02,  2.24139988e-02,  7.19779953e-02,\n",
       "        8.29880014e-02, -3.00521433e-01, -2.94486851e-01, -7.89007992e-02,\n",
       "       -3.31512019e-02,  3.41201603e-01,  9.71827134e-02,  3.63738015e-02,\n",
       "       -6.48769960e-02, -3.42620024e-03,  5.39860129e-03,  1.74937956e-02,\n",
       "        1.37041003e-01, -2.90899929e-02, -2.22800067e-03, -9.40222591e-02,\n",
       "        2.38341004e-01,  1.49065211e-01, -2.84128994e-01,  2.21151989e-02,\n",
       "       -2.50313818e-01, -8.55891854e-02, -2.11878985e-01, -5.92008010e-02,\n",
       "        8.13369930e-01,  1.47094816e-01, -6.53312624e-01, -9.30460021e-02,\n",
       "       -3.83621067e-01, -2.49031395e-01, -3.88560802e-01, -2.20031217e-01,\n",
       "        2.55477987e-02, -2.68832035e-02, -3.33619975e-02,  6.28920048e-02,\n",
       "        3.20904583e-01, -2.24779993e-01,  1.38928024e-02,  1.64185375e-01,\n",
       "       -2.99731940e-02, -2.52801239e-01,  1.64113805e-01, -2.02426016e-02,\n",
       "       -7.10171908e-02, -1.34364396e-01, -3.14302206e-01, -2.45522186e-01,\n",
       "       -1.44386008e-01,  1.45213991e-01,  1.21419206e-01, -1.19551197e-01,\n",
       "        1.31567001e-01, -1.18598007e-01, -3.15375596e-01,  1.99001342e-01],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sent.vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly, we can find the single word closest in meaning to this sequence of words:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['so',\n",
       " 'kind',\n",
       " 'well',\n",
       " 'it',\n",
       " 'everything',\n",
       " 'but',\n",
       " 'even',\n",
       " 'that',\n",
       " 'always',\n",
       " 'time',\n",
       " 'one',\n",
       " 'is']"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lookup.nearest(sent.vector)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, we can use this as a sort of rudimentary search engine, making it possible to find sentences in a text that are close in meaning to any arbitrary sentence we type in. Here's how to do that! First, we'll get the list of sentences:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "doc = nlp(open(\"./84-0.txt\").read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we'll create a nearest neighbors lookup with the vectors for each sentence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "sentence_lookup = SimpleNeighbors(300)\n",
    "for sent in doc.sents:\n",
    "    # replace linebreaks to make the output a bit more neat\n",
    "    sentence_lookup.add_one(sent.text.replace(\"\\n\", \" \"), sent.vector)\n",
    "sentence_lookup.build()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can find similar sentences like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['I greedily devoured the remnants of the shepherd’s breakfast, which consisted of bread, cheese, milk, and wine; the latter, however, I did not like.  ',\n",
       " 'We shall make our bed of dried leaves; the sun will shine on us as on man and will ripen our food.  ',\n",
       " 'Nature decayed around me, and the sun became heatless; rain and snow poured around me; mighty rivers were frozen; the surface of the earth was hard and chill, and bare, and I found no shelter.  ',\n",
       " 'I am already far north of London, and as I walk in the streets of Petersburgh, I feel a cold northern breeze play upon my cheeks, which braces my nerves and fills me with delight.  ',\n",
       " 'By slow degrees he recovered and ate a little soup, which restored him wonderfully.  ',\n",
       " 'The wind, which had hitherto carried us along with amazing rapidity, sank at sunset to a light breeze; the soft air just ruffled the water and caused a pleasant motion among the trees as we approached the shore, from which it wafted the most delightful scent of flowers and hay.  ',\n",
       " 'Vegetables and bread, when they indulged in such luxuries, and even fresh water, was to be procured from the mainland, which was about five miles distant.  ',\n",
       " 'I lay at the bottom of the boat, and as I gazed on the cloudless blue sky, I seemed to drink in a tranquillity to which I had long been a stranger.',\n",
       " 'Winter, spring, and summer passed away during my labours; but I did not watch the blossom or the expanding leaves—sights which before always yielded me supreme delight—',\n",
       " 'I lighted the dry branch of a tree and danced with fury around the devoted cottage, my eyes still fixed on the western horizon, the edge of which the moon nearly touched.  ',\n",
       " '“As the sun became warmer and the light of day longer, the snow vanished, and I beheld the bare trees and the black earth.  ',\n",
       " 'Immense and rugged mountains of ice often barred up my passage, and I often heard the thunder of the ground sea, which threatened my destruction.  ']"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence_lookup.nearest(nlp(\"My favorite food is strawberry ice cream.\").vector)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could do something similar with spaCy noun chunks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "chunk_lookup = SimpleNeighbors(300)\n",
    "for chunk in doc.noun_chunks:\n",
    "    chunk_text = chunk.text.replace(\"\\n\", \" \")\n",
    "    if chunk_text not in chunk_lookup.corpus:\n",
    "        chunk_lookup.add_one(chunk_text, chunk.vector)\n",
    "chunk_lookup.build()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['the birds',\n",
       " 'The birds',\n",
       " 'the wild animals',\n",
       " 'the frogs',\n",
       " 'the wild sea',\n",
       " 'no albatross',\n",
       " 'The wounded deer',\n",
       " 'such lovely creatures',\n",
       " 'the prey',\n",
       " 'a shrill and dreadful scream',\n",
       " 'a devouring blackness',\n",
       " 'this little creature']"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chunk_lookup.nearest(nlp(\"angry birds\").vector)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or even lookups for each part of speech:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "adj_lookup = SimpleNeighbors(300)\n",
    "for word in doc:\n",
    "    # .corpus of the lookup lets us determine if the word has already been added\n",
    "    if word.tag_ == 'JJ' and word.text.lower() not in adj_lookup.corpus:\n",
    "        adj_lookup.add_one(word.text.lower(), word.vector)\n",
    "adj_lookup.build()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['happy',\n",
       " 'glad',\n",
       " 'excited',\n",
       " 'grateful',\n",
       " 'adored',\n",
       " 'loved',\n",
       " 'overjoyed',\n",
       " 'contented',\n",
       " 'sad',\n",
       " 'surprised',\n",
       " 'wonderful',\n",
       " 'welcome']"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adj_lookup.nearest(vec(\"happy\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rewriting with parts of speech from another text\n",
    "\n",
    "One last example and we'll call it a day. The following code uses the noun chunks and adjective lookups that I made above to rewrite one text (Frost's *The Road Not Taken*) with words and phrases from Frankenstein."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "frost_doc = nlp(open(\"frost.txt\").read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Two the interchange diverged in a green woods,\n",
      "And sorry I could not travel both\n",
      "And be one a merchant, long I stood\n",
      "And looked down one place as far as I could\n",
      "To where it bent in the herbage;\n",
      "\n",
      "Then took the certain, as just as honest,\n",
      "And having perhaps the better not such proof,\n",
      "Because it was dry and wanted dress;\n",
      "Though as for that the the farther end there\n",
      "Had worn them really about the only,\n",
      "\n",
      "And both that a Sunday afternoon equally lay\n",
      "In a leaf no a second step had trodden white.\n",
      "Oh, I kept the last for another one day!\n",
      "Yet knowing how how all the life leads on to the good people,\n",
      "I doubted if I should ever come back.\n",
      "\n",
      "I shall be telling this with a deep groans\n",
      "Somewhere past ages and past ages hence:\n",
      "Two towns diverged in a furniture, and I—\n",
      "I took the that one less travelled by,\n",
      "And that has made all the a greater proportion.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "output = []\n",
    "for word in frost_doc:\n",
    "    if word.is_alpha and word.pos_ == 'NOUN':\n",
    "        new_word = random.choice(chunk_lookup.nearest(word.vector, 5))\n",
    "        output.append(new_word)\n",
    "    elif word.is_alpha and word.tag_ == 'JJ':\n",
    "        new_word = random.choice(adj_lookup.nearest(word.vector, 5))\n",
    "        output.append(new_word)\n",
    "    else:\n",
    "        output.append(word.text)\n",
    "    output.append(word.whitespace_)\n",
    "print(''.join(output))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Beyond word vectors\n",
    "\n",
    "The core assumptions of word vectors as a technology are that (A) words are a meaningful unit of language, and (b) words can be said to have stable \"meanings\" across texts. Neither of these assertions are true, but (for better or worse) they reflect beliefs that many people hold about how language works. The fact that word vectors reflect these beliefs, in my opinion, makes text analysis and text generation based on word vectors more intuitive than  the alternatives.\n",
    "\n",
    "More recently, the task of finding semantic similarity between passages of text is accomplished using systems that operate not on words as units, but on units determined by unsupervised tokenization algorithms like [SentencePiece](https://github.com/google/sentencepiece). These systems often use neural network machine learning models (like [transformers](https://en.wikipedia.org/wiki/Transformer_(machine_learning_model))) to calculate vectors for words and longer stretches of text that take the surrounding textual context into account. These embeddings are often more accurate, at the cost of explainability and increased use of computational resources. Here are a few resources for learning more about this method:\n",
    "\n",
    "* Google's [Universal Sentence Encoder](https://www.tensorflow.org/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder)\n",
    "* The [sentence-transformers](https://github.com/UKPLab/sentence-transformers) library for Python\n",
    "* [Awesome Sentence Embedding](https://github.com/Separius/awesome-sentence-embedding), a big list of sentence embedding resources"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}