word-document-information.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/archivesunleashed/notebooks/blob/main/word-document-information.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Word Documents (e.g., .doc, .odt, .rtf, .wpd) Information Dataset Exploration\n",
        "\n",
        "We're going to take a look at a few examples of how we can explore the Word Documents (e.g., .doc, .odt, .rtf, .wpd) Information dataset. \n",
        "\n",
        "The first thing we need to do is enter the URL for our Word Documents (e.g., .doc, .odt, .rtf, .wpd) Information dataset in the cell below. You can get this by right clicking the Download icon, and selecting \"Copy Link\"."
      ],
      "metadata": {
        "id": "vAyuRQ2PJIdc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dataset = 'https://webdata.archive-it.org/ait/files/download/ARCHIVEIT-14489/WordProcessorInformationExtraction/word-document-information.csv.gz?access=DQP7JACJV7LYRHFJFQ74E7FING7HHYEV' #@param {type:\"string\"}\n",
        "print(dataset)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "form",
        "id": "RfhJiesWVpAf",
        "outputId": "03b3233c-ba29-4332-d616-b469938a15d1"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "https://webdata.archive-it.org/ait/files/download/ARCHIVEIT-14489/WordProcessorInformationExtraction/word-document-information.csv.gz?access=DQP7JACJV7LYRHFJFQ74E7FING7HHYEV\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## pandas\n",
        "\n",
        "Next, we'll setup our environment so we can load our Word Documents (e.g., .doc, .odt, .rtf, .wpd) Information dataset into [pandas](https://pandas.pydata.org) DataFrames. If you're unfamiliar with DataFrames, but you're worked with word documentss before, you should feel comfortable pretty quick."
      ],
      "metadata": {
        "id": "Z14F2cIWJVW0"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "Chh6tt3HHF1s"
      },
      "outputs": [],
      "source": [
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Data Table Display\n",
        "\n",
        "Colab includes an extension that renders pandas DataFrames into interactive displays that can be filtered, sorted, and explored dynamically. This can be very useful for taking a look at what each DataFrame provides, and doing some intital filtering!\n",
        "\n",
        "Data table display for pandas dataframes can be enabled by running:\n",
        "```python\n",
        "%load_ext google.colab.data_table\n",
        "```\n",
        "and disabled by running\n",
        "```python\n",
        "%unload_ext google.colab.data_table\n",
        "```"
      ],
      "metadata": {
        "id": "sH81XCf3I3xY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%load_ext google.colab.data_table"
      ],
      "metadata": {
        "id": "-qyCnbvBI7n6"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Loading our ARCH Dataset as a DataFrame\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "Next, we'll create pandas DataFrame from our dataset, and show a preview of it using the Data Table Display."
      ],
      "metadata": {
        "id": "6prR7j1zI_D5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "word_documents = pd.read_csv(dataset, compression='gzip')\n",
        "word_documents"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 667
        },
        "id": "YL0LQaUNHRKx",
        "outputId": "831f8947-4d53-4ac2-fbd3-e2920e101acb"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        crawl_date                                                url  \\\n",
              "0   20200703084330  http://kremer.cpsc.ucalgary.ca/courses/cpsc433...   \n",
              "1   20200708003621  http://kremer.cpsc.ucalgary.ca/courses/cpsc433...   \n",
              "2   20200708015505  http://kremer.cpsc.ucalgary.ca/courses/cpsc433...   \n",
              "3   20200708053559  http://kremer.cpsc.ucalgary.ca/courses/cpsc433...   \n",
              "4   20200710021203  https://www.pjl.ucalgary.ca/data/safety/placar...   \n",
              "5   20200710021345  https://www.pjl.ucalgary.ca/data/safety/signag...   \n",
              "6   20200710144656  https://www.pjl.ucalgary.ca/data/safety/traini...   \n",
              "7   20200714180727  https://www.mcmaster.ca/academicintegrity/inst...   \n",
              "8   20200714180809  https://www.mcmaster.ca/academicintegrity/inst...   \n",
              "9   20200714180818  https://www.mcmaster.ca/academicintegrity/inst...   \n",
              "10  20200702145628  http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...   \n",
              "11  20200703100729  https://csclub.uwaterloo.ca/misc/usage_agreeme...   \n",
              "12  20200703104915  http://rspace.uwaterloo.ca/xmlui/bitstream/han...   \n",
              "13  20200703113308       http://wsic.uwaterloo.ca/c61/c61-summary.odt   \n",
              "14  20200703113328       http://wsic.uwaterloo.ca/c61/petition_en.odt   \n",
              "15  20200704092550  http://cacr.uwaterloo.ca/conferences/1999/isw-...   \n",
              "16  20200705165409  http://www.cgl.uwaterloo.ca/zmeraj/cae_bibtex.rtf   \n",
              "17  20200707030441  https://ece.uwaterloo.ca/~cgebotys/NEW/TEMPLAT...   \n",
              "18  20200716123414  https://ece.uwaterloo.ca/~cgebotys/NEW/ece602/...   \n",
              "19  20200718115158  https://openjournals.uwaterloo.ca/index.php/ro...   \n",
              "20  20210102104833  http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...   \n",
              "21  20211230120639  https://lcuploads-ca.s3.ca-central-1.amazonaws...   \n",
              "\n",
              "                                             filename extension  \\\n",
              "0                      assignmentProposalTemplate.odt       odt   \n",
              "1                                 softConstraints.rtf       rtf   \n",
              "2                                 softConstraints.rtf       rtf   \n",
              "3                                 softConstraints.rtf       rtf   \n",
              "4                                      Board-Sign.odt       odt   \n",
              "5                               Safey-Board-Title.odt       odt   \n",
              "6                                       TA-record.odt       odt   \n",
              "7                                 suspectcheating.doc       doc   \n",
              "8                                  meetingrequest.doc       doc   \n",
              "9                                informingstudent.doc       doc   \n",
              "10                                   ClassListF04.rtf       rtf   \n",
              "11                                usage_agreement.odt       odt   \n",
              "12  Anita%20Tiphagne%20-%20Scheduled%20Tribes.rtf;...       rtf   \n",
              "13                                    c61-summary.odt       odt   \n",
              "14                                    petition_en.odt       odt   \n",
              "15                                          edgar.ppt       doc   \n",
              "16                                     cae_bibtex.rtf       rtf   \n",
              "17                       TEMPLATE_PAPER_ECE700T02.rtf       rtf   \n",
              "18                             Transmag-ieeeTvlsi.rtf       rtf   \n",
              "19                                               1855       doc   \n",
              "20                                   ClassListF04.rtf       rtf   \n",
              "21  20210927_Digital_Humanities_Python_Lesson_Note...       rtf   \n",
              "\n",
              "                       mime_type_web_server  \\\n",
              "0   application/vnd.oasis.opendocument.text   \n",
              "1                           application/rtf   \n",
              "2                           application/rtf   \n",
              "3                           application/rtf   \n",
              "4   application/vnd.oasis.opendocument.text   \n",
              "5   application/vnd.oasis.opendocument.text   \n",
              "6   application/vnd.oasis.opendocument.text   \n",
              "7                        application/msword   \n",
              "8                        application/msword   \n",
              "9                        application/msword   \n",
              "10                          application/rtf   \n",
              "11  application/vnd.oasis.opendocument.text   \n",
              "12                            text/richtext   \n",
              "13  application/vnd.oasis.opendocument.text   \n",
              "14  application/vnd.oasis.opendocument.text   \n",
              "15            application/vnd.ms-powerpoint   \n",
              "16                 application/octet-stream   \n",
              "17                          application/rtf   \n",
              "18                          application/rtf   \n",
              "19                       application/msword   \n",
              "20                          application/rtf   \n",
              "21                 application/octet-stream   \n",
              "\n",
              "                             mime_type_tika                               md5  \\\n",
              "0   application/vnd.oasis.opendocument.text  2275849e3be59e550a89672aa5e78bc8   \n",
              "1                           application/rtf  99226fd2c2d479ce7932eaad5251deba   \n",
              "2                           application/rtf  90230f636f0dc988e0e9d597ef049788   \n",
              "3                           application/rtf  43737ec65c00b432c9a5543a1be8b9a4   \n",
              "4   application/vnd.oasis.opendocument.text  573c1418d06c55f3611c63bbd7c8222a   \n",
              "5   application/vnd.oasis.opendocument.text  47101dec746918a9d0e8e7c42bdf2961   \n",
              "6   application/vnd.oasis.opendocument.text  ccf92c810a413d9b02392df15424a1df   \n",
              "7                        application/msword  3b730a176c9c52df0ce0a6595bd4b6ae   \n",
              "8                        application/msword  4d388255bdfbed217268189bd49f91c3   \n",
              "9                        application/msword  cb50283bbc09e56a7c46ed40cd878066   \n",
              "10                          application/rtf  022298f038537bd30aefd3e7d4b8e9ff   \n",
              "11  application/vnd.oasis.opendocument.text  fe0562c790e1e5c3e2a07840a19885b1   \n",
              "12                          application/rtf  81ca5af0d11d33f3980dea62d73c58e3   \n",
              "13  application/vnd.oasis.opendocument.text  935bba178bb3edea20e01b6b27d1c679   \n",
              "14  application/vnd.oasis.opendocument.text  dbc1557aac1f7b553bd80289286ea41a   \n",
              "15                       application/msword  6625b64573c05f16f8e5ef29d37cd6bc   \n",
              "16                          application/rtf  666a6dca42e04e1cfdbe0ff2c26a0715   \n",
              "17                          application/rtf  2bd445b1e94d1257c86ebae6652909c6   \n",
              "18                          application/rtf  8cce3a728d62e1c19250165750e43896   \n",
              "19                       application/msword  ec603e70f3882eec35c08e40d79856e3   \n",
              "20                          application/rtf  022298f038537bd30aefd3e7d4b8e9ff   \n",
              "21                          application/rtf  3f8c2e7f4ebc656ee826a4a22b89943b   \n",
              "\n",
              "                                        sha1  \n",
              "0   9200fae6eddc8b2ebab3f5677bebcddbf3358c30  \n",
              "1   394883aa96d96b6fc7bd2286552a60cabc804133  \n",
              "2   134d96ada8d0975f3d738c09e06d9ee88afb7da1  \n",
              "3   51f4e3698bfa141e3ae60cfd8a85381c6cb3bbbc  \n",
              "4   25defc402ea989d52298f0fa10a1663ce89c3137  \n",
              "5   0563b660035365b4cb693ed8894be7d9f85e6a65  \n",
              "6   3cdbccf14e71b5d633f69e223916c76111e7538b  \n",
              "7   d470119ee7c91dc3eb95dc1f29fde9056362ad89  \n",
              "8   91ccff7150116f8d916e22bb7a84c03f5515c441  \n",
              "9   4ccb6aba7781fda57a161ddc2fa07ecd74995a1e  \n",
              "10  3d58dfce44ed66828f290c79cd4fac3af0cee6a3  \n",
              "11  dc9980a46c2952c67ad637803bef1b24e7a19a0f  \n",
              "12  eb6ed0c7c295ea1bf8b1aee7cf2b25f12687de83  \n",
              "13  b5a0dd8a1a3ea6465660443adcf2d993a6df8d71  \n",
              "14  ea0140f16c930691ec183a8c6908e89d95499bbe  \n",
              "15  88ea28f21b55e06daa736f43285f321a13a91da9  \n",
              "16  c76bee3ab808cdd68f3aef1c3ad4aadc868eb088  \n",
              "17  34ef7aedaeb0e75f4934fa4dfdf82001da1c1e22  \n",
              "18  912bbd256c88e1c077adc05ed8bdf1ed07bb5bed  \n",
              "19  6fa1587b8340b064ac0b50d6efa337d7d42a0421  \n",
              "20  3d58dfce44ed66828f290c79cd4fac3af0cee6a3  \n",
              "21  913a2a3e73f91cd0da3c0e50e1764c8229c3aa95  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c6d0787c-e80e-400a-b8fe-ab43d3b59681\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>crawl_date</th>\n",
              "      <th>url</th>\n",
              "      <th>filename</th>\n",
              "      <th>extension</th>\n",
              "      <th>mime_type_web_server</th>\n",
              "      <th>mime_type_tika</th>\n",
              "      <th>md5</th>\n",
              "      <th>sha1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>20200703084330</td>\n",
              "      <td>http://kremer.cpsc.ucalgary.ca/courses/cpsc433...</td>\n",
              "      <td>assignmentProposalTemplate.odt</td>\n",
              "      <td>odt</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>2275849e3be59e550a89672aa5e78bc8</td>\n",
              "      <td>9200fae6eddc8b2ebab3f5677bebcddbf3358c30</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>20200708003621</td>\n",
              "      <td>http://kremer.cpsc.ucalgary.ca/courses/cpsc433...</td>\n",
              "      <td>softConstraints.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>99226fd2c2d479ce7932eaad5251deba</td>\n",
              "      <td>394883aa96d96b6fc7bd2286552a60cabc804133</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>20200708015505</td>\n",
              "      <td>http://kremer.cpsc.ucalgary.ca/courses/cpsc433...</td>\n",
              "      <td>softConstraints.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>90230f636f0dc988e0e9d597ef049788</td>\n",
              "      <td>134d96ada8d0975f3d738c09e06d9ee88afb7da1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>20200708053559</td>\n",
              "      <td>http://kremer.cpsc.ucalgary.ca/courses/cpsc433...</td>\n",
              "      <td>softConstraints.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>43737ec65c00b432c9a5543a1be8b9a4</td>\n",
              "      <td>51f4e3698bfa141e3ae60cfd8a85381c6cb3bbbc</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>20200710021203</td>\n",
              "      <td>https://www.pjl.ucalgary.ca/data/safety/placar...</td>\n",
              "      <td>Board-Sign.odt</td>\n",
              "      <td>odt</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>573c1418d06c55f3611c63bbd7c8222a</td>\n",
              "      <td>25defc402ea989d52298f0fa10a1663ce89c3137</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>20200710021345</td>\n",
              "      <td>https://www.pjl.ucalgary.ca/data/safety/signag...</td>\n",
              "      <td>Safey-Board-Title.odt</td>\n",
              "      <td>odt</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>47101dec746918a9d0e8e7c42bdf2961</td>\n",
              "      <td>0563b660035365b4cb693ed8894be7d9f85e6a65</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>20200710144656</td>\n",
              "      <td>https://www.pjl.ucalgary.ca/data/safety/traini...</td>\n",
              "      <td>TA-record.odt</td>\n",
              "      <td>odt</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>ccf92c810a413d9b02392df15424a1df</td>\n",
              "      <td>3cdbccf14e71b5d633f69e223916c76111e7538b</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>20200714180727</td>\n",
              "      <td>https://www.mcmaster.ca/academicintegrity/inst...</td>\n",
              "      <td>suspectcheating.doc</td>\n",
              "      <td>doc</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>3b730a176c9c52df0ce0a6595bd4b6ae</td>\n",
              "      <td>d470119ee7c91dc3eb95dc1f29fde9056362ad89</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>20200714180809</td>\n",
              "      <td>https://www.mcmaster.ca/academicintegrity/inst...</td>\n",
              "      <td>meetingrequest.doc</td>\n",
              "      <td>doc</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>4d388255bdfbed217268189bd49f91c3</td>\n",
              "      <td>91ccff7150116f8d916e22bb7a84c03f5515c441</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>20200714180818</td>\n",
              "      <td>https://www.mcmaster.ca/academicintegrity/inst...</td>\n",
              "      <td>informingstudent.doc</td>\n",
              "      <td>doc</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>cb50283bbc09e56a7c46ed40cd878066</td>\n",
              "      <td>4ccb6aba7781fda57a161ddc2fa07ecd74995a1e</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>20200702145628</td>\n",
              "      <td>http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...</td>\n",
              "      <td>ClassListF04.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>022298f038537bd30aefd3e7d4b8e9ff</td>\n",
              "      <td>3d58dfce44ed66828f290c79cd4fac3af0cee6a3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>20200703100729</td>\n",
              "      <td>https://csclub.uwaterloo.ca/misc/usage_agreeme...</td>\n",
              "      <td>usage_agreement.odt</td>\n",
              "      <td>odt</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>fe0562c790e1e5c3e2a07840a19885b1</td>\n",
              "      <td>dc9980a46c2952c67ad637803bef1b24e7a19a0f</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>20200703104915</td>\n",
              "      <td>http://rspace.uwaterloo.ca/xmlui/bitstream/han...</td>\n",
              "      <td>Anita%20Tiphagne%20-%20Scheduled%20Tribes.rtf;...</td>\n",
              "      <td>rtf</td>\n",
              "      <td>text/richtext</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>81ca5af0d11d33f3980dea62d73c58e3</td>\n",
              "      <td>eb6ed0c7c295ea1bf8b1aee7cf2b25f12687de83</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>20200703113308</td>\n",
              "      <td>http://wsic.uwaterloo.ca/c61/c61-summary.odt</td>\n",
              "      <td>c61-summary.odt</td>\n",
              "      <td>odt</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>935bba178bb3edea20e01b6b27d1c679</td>\n",
              "      <td>b5a0dd8a1a3ea6465660443adcf2d993a6df8d71</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>20200703113328</td>\n",
              "      <td>http://wsic.uwaterloo.ca/c61/petition_en.odt</td>\n",
              "      <td>petition_en.odt</td>\n",
              "      <td>odt</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>application/vnd.oasis.opendocument.text</td>\n",
              "      <td>dbc1557aac1f7b553bd80289286ea41a</td>\n",
              "      <td>ea0140f16c930691ec183a8c6908e89d95499bbe</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>20200704092550</td>\n",
              "      <td>http://cacr.uwaterloo.ca/conferences/1999/isw-...</td>\n",
              "      <td>edgar.ppt</td>\n",
              "      <td>doc</td>\n",
              "      <td>application/vnd.ms-powerpoint</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>6625b64573c05f16f8e5ef29d37cd6bc</td>\n",
              "      <td>88ea28f21b55e06daa736f43285f321a13a91da9</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>20200705165409</td>\n",
              "      <td>http://www.cgl.uwaterloo.ca/zmeraj/cae_bibtex.rtf</td>\n",
              "      <td>cae_bibtex.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/octet-stream</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>666a6dca42e04e1cfdbe0ff2c26a0715</td>\n",
              "      <td>c76bee3ab808cdd68f3aef1c3ad4aadc868eb088</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>20200707030441</td>\n",
              "      <td>https://ece.uwaterloo.ca/~cgebotys/NEW/TEMPLAT...</td>\n",
              "      <td>TEMPLATE_PAPER_ECE700T02.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>2bd445b1e94d1257c86ebae6652909c6</td>\n",
              "      <td>34ef7aedaeb0e75f4934fa4dfdf82001da1c1e22</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>20200716123414</td>\n",
              "      <td>https://ece.uwaterloo.ca/~cgebotys/NEW/ece602/...</td>\n",
              "      <td>Transmag-ieeeTvlsi.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>8cce3a728d62e1c19250165750e43896</td>\n",
              "      <td>912bbd256c88e1c077adc05ed8bdf1ed07bb5bed</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>20200718115158</td>\n",
              "      <td>https://openjournals.uwaterloo.ca/index.php/ro...</td>\n",
              "      <td>1855</td>\n",
              "      <td>doc</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>application/msword</td>\n",
              "      <td>ec603e70f3882eec35c08e40d79856e3</td>\n",
              "      <td>6fa1587b8340b064ac0b50d6efa337d7d42a0421</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>20210102104833</td>\n",
              "      <td>http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...</td>\n",
              "      <td>ClassListF04.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>022298f038537bd30aefd3e7d4b8e9ff</td>\n",
              "      <td>3d58dfce44ed66828f290c79cd4fac3af0cee6a3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>20211230120639</td>\n",
              "      <td>https://lcuploads-ca.s3.ca-central-1.amazonaws...</td>\n",
              "      <td>20210927_Digital_Humanities_Python_Lesson_Note...</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/octet-stream</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>3f8c2e7f4ebc656ee826a4a22b89943b</td>\n",
              "      <td>913a2a3e73f91cd0da3c0e50e1764c8229c3aa95</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c6d0787c-e80e-400a-b8fe-ab43d3b59681')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-c6d0787c-e80e-400a-b8fe-ab43d3b59681 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-c6d0787c-e80e-400a-b8fe-ab43d3b59681');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a8bd4d5e58f96183/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 20200703084330,\n            'f': \"20200703084330\",\n        },\n\"http://kremer.cpsc.ucalgary.ca/courses/cpsc433/F2014/assignment/assignmentProposalTemplate.odt\",\n\"assignmentProposalTemplate.odt\",\n\"odt\",\n\"application/vnd.oasis.opendocument.text\",\n\"application/vnd.oasis.opendocument.text\",\n\"2275849e3be59e550a89672aa5e78bc8\",\n\"9200fae6eddc8b2ebab3f5677bebcddbf3358c30\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 20200708003621,\n            'f': \"20200708003621\",\n        },\n\"http://kremer.cpsc.ucalgary.ca/courses/cpsc433/W2006/softConstraints.rtf\",\n\"softConstraints.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"99226fd2c2d479ce7932eaad5251deba\",\n\"394883aa96d96b6fc7bd2286552a60cabc804133\"],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 20200708015505,\n            'f': \"20200708015505\",\n        },\n\"http://kremer.cpsc.ucalgary.ca/courses/cpsc433/F2006/softConstraints.rtf\",\n\"softConstraints.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"90230f636f0dc988e0e9d597ef049788\",\n\"134d96ada8d0975f3d738c09e06d9ee88afb7da1\"],\n [{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 20200708053559,\n            'f': \"20200708053559\",\n        },\n\"http://kremer.cpsc.ucalgary.ca/courses/cpsc433/F2013/softConstraints.rtf\",\n\"softConstraints.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"43737ec65c00b432c9a5543a1be8b9a4\",\n\"51f4e3698bfa141e3ae60cfd8a85381c6cb3bbbc\"],\n [{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 20200710021203,\n            'f': \"20200710021203\",\n        },\n\"https://www.pjl.ucalgary.ca/data/safety/placards/Board-Sign.odt\",\n\"Board-Sign.odt\",\n\"odt\",\n\"application/vnd.oasis.opendocument.text\",\n\"application/vnd.oasis.opendocument.text\",\n\"573c1418d06c55f3611c63bbd7c8222a\",\n\"25defc402ea989d52298f0fa10a1663ce89c3137\"],\n [{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 20200710021345,\n            'f': \"20200710021345\",\n        },\n\"https://www.pjl.ucalgary.ca/data/safety/signage/Safey-Board-Title.odt\",\n\"Safey-Board-Title.odt\",\n\"odt\",\n\"application/vnd.oasis.opendocument.text\",\n\"application/vnd.oasis.opendocument.text\",\n\"47101dec746918a9d0e8e7c42bdf2961\",\n\"0563b660035365b4cb693ed8894be7d9f85e6a65\"],\n [{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 20200710144656,\n            'f': \"20200710144656\",\n        },\n\"https://www.pjl.ucalgary.ca/data/safety/training/training-record-templates/previous-version/TA-record.odt\",\n\"TA-record.odt\",\n\"odt\",\n\"application/vnd.oasis.opendocument.text\",\n\"application/vnd.oasis.opendocument.text\",\n\"ccf92c810a413d9b02392df15424a1df\",\n\"3cdbccf14e71b5d633f69e223916c76111e7538b\"],\n [{\n            'v': 7,\n            'f': \"7\",\n        },\n{\n            'v': 20200714180727,\n            'f': \"20200714180727\",\n        },\n\"https://www.mcmaster.ca/academicintegrity/instructors/proc_forms/suspectcheating.doc\",\n\"suspectcheating.doc\",\n\"doc\",\n\"application/msword\",\n\"application/msword\",\n\"3b730a176c9c52df0ce0a6595bd4b6ae\",\n\"d470119ee7c91dc3eb95dc1f29fde9056362ad89\"],\n [{\n            'v': 8,\n            'f': \"8\",\n        },\n{\n            'v': 20200714180809,\n            'f': \"20200714180809\",\n        },\n\"https://www.mcmaster.ca/academicintegrity/instructors/proc_forms/meetingrequest.doc\",\n\"meetingrequest.doc\",\n\"doc\",\n\"application/msword\",\n\"application/msword\",\n\"4d388255bdfbed217268189bd49f91c3\",\n\"91ccff7150116f8d916e22bb7a84c03f5515c441\"],\n [{\n            'v': 9,\n            'f': \"9\",\n        },\n{\n            'v': 20200714180818,\n            'f': \"20200714180818\",\n        },\n\"https://www.mcmaster.ca/academicintegrity/instructors/proc_forms/informingstudent.doc\",\n\"informingstudent.doc\",\n\"doc\",\n\"application/msword\",\n\"application/msword\",\n\"cb50283bbc09e56a7c46ed40cd878066\",\n\"4ccb6aba7781fda57a161ddc2fa07ecd74995a1e\"],\n [{\n            'v': 10,\n            'f': \"10\",\n        },\n{\n            'v': 20200702145628,\n            'f': \"20200702145628\",\n        },\n\"http://www.arts.uwaterloo.ca/%7eraha/392AF04_web/ClassListF04.rtf\",\n\"ClassListF04.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"022298f038537bd30aefd3e7d4b8e9ff\",\n\"3d58dfce44ed66828f290c79cd4fac3af0cee6a3\"],\n [{\n            'v': 11,\n            'f': \"11\",\n        },\n{\n            'v': 20200703100729,\n            'f': \"20200703100729\",\n        },\n\"https://csclub.uwaterloo.ca/misc/usage_agreement.odt\",\n\"usage_agreement.odt\",\n\"odt\",\n\"application/vnd.oasis.opendocument.text\",\n\"application/vnd.oasis.opendocument.text\",\n\"fe0562c790e1e5c3e2a07840a19885b1\",\n\"dc9980a46c2952c67ad637803bef1b24e7a19a0f\"],\n [{\n            'v': 12,\n            'f': \"12\",\n        },\n{\n            'v': 20200703104915,\n            'f': \"20200703104915\",\n        },\n\"http://rspace.uwaterloo.ca/xmlui/bitstream/handle/20.500.11895/470/Anita%20Tiphagne%20-%20Scheduled%20Tribes.rtf;jsessionid=9235B9AED183136CA713FBE07AA4688E?sequence=1\",\n\"Anita%20Tiphagne%20-%20Scheduled%20Tribes.rtf;jsessionid=9235B9AED183136CA713FBE07AA4688E\",\n\"rtf\",\n\"text/richtext\",\n\"application/rtf\",\n\"81ca5af0d11d33f3980dea62d73c58e3\",\n\"eb6ed0c7c295ea1bf8b1aee7cf2b25f12687de83\"],\n [{\n            'v': 13,\n            'f': \"13\",\n        },\n{\n            'v': 20200703113308,\n            'f': \"20200703113308\",\n        },\n\"http://wsic.uwaterloo.ca/c61/c61-summary.odt\",\n\"c61-summary.odt\",\n\"odt\",\n\"application/vnd.oasis.opendocument.text\",\n\"application/vnd.oasis.opendocument.text\",\n\"935bba178bb3edea20e01b6b27d1c679\",\n\"b5a0dd8a1a3ea6465660443adcf2d993a6df8d71\"],\n [{\n            'v': 14,\n            'f': \"14\",\n        },\n{\n            'v': 20200703113328,\n            'f': \"20200703113328\",\n        },\n\"http://wsic.uwaterloo.ca/c61/petition_en.odt\",\n\"petition_en.odt\",\n\"odt\",\n\"application/vnd.oasis.opendocument.text\",\n\"application/vnd.oasis.opendocument.text\",\n\"dbc1557aac1f7b553bd80289286ea41a\",\n\"ea0140f16c930691ec183a8c6908e89d95499bbe\"],\n [{\n            'v': 15,\n            'f': \"15\",\n        },\n{\n            'v': 20200704092550,\n            'f': \"20200704092550\",\n        },\n\"http://cacr.uwaterloo.ca/conferences/1999/isw-march/edgar.ppt\",\n\"edgar.ppt\",\n\"doc\",\n\"application/vnd.ms-powerpoint\",\n\"application/msword\",\n\"6625b64573c05f16f8e5ef29d37cd6bc\",\n\"88ea28f21b55e06daa736f43285f321a13a91da9\"],\n [{\n            'v': 16,\n            'f': \"16\",\n        },\n{\n            'v': 20200705165409,\n            'f': \"20200705165409\",\n        },\n\"http://www.cgl.uwaterloo.ca/zmeraj/cae_bibtex.rtf\",\n\"cae_bibtex.rtf\",\n\"rtf\",\n\"application/octet-stream\",\n\"application/rtf\",\n\"666a6dca42e04e1cfdbe0ff2c26a0715\",\n\"c76bee3ab808cdd68f3aef1c3ad4aadc868eb088\"],\n [{\n            'v': 17,\n            'f': \"17\",\n        },\n{\n            'v': 20200707030441,\n            'f': \"20200707030441\",\n        },\n\"https://ece.uwaterloo.ca/~cgebotys/NEW/TEMPLATE_PAPER_ECE700T02.rtf\",\n\"TEMPLATE_PAPER_ECE700T02.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"2bd445b1e94d1257c86ebae6652909c6\",\n\"34ef7aedaeb0e75f4934fa4dfdf82001da1c1e22\"],\n [{\n            'v': 18,\n            'f': \"18\",\n        },\n{\n            'v': 20200716123414,\n            'f': \"20200716123414\",\n        },\n\"https://ece.uwaterloo.ca/~cgebotys/NEW/ece602/notes/Transmag-ieeeTvlsi.rtf\",\n\"Transmag-ieeeTvlsi.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"8cce3a728d62e1c19250165750e43896\",\n\"912bbd256c88e1c077adc05ed8bdf1ed07bb5bed\"],\n [{\n            'v': 19,\n            'f': \"19\",\n        },\n{\n            'v': 20200718115158,\n            'f': \"20200718115158\",\n        },\n\"https://openjournals.uwaterloo.ca/index.php/rofea/article/download/1439/1855\",\n\"1855\",\n\"doc\",\n\"application/msword\",\n\"application/msword\",\n\"ec603e70f3882eec35c08e40d79856e3\",\n\"6fa1587b8340b064ac0b50d6efa337d7d42a0421\"],\n [{\n            'v': 20,\n            'f': \"20\",\n        },\n{\n            'v': 20210102104833,\n            'f': \"20210102104833\",\n        },\n\"http://www.arts.uwaterloo.ca/%7eraha/392AF04_web/ClassListF04.rtf\",\n\"ClassListF04.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"022298f038537bd30aefd3e7d4b8e9ff\",\n\"3d58dfce44ed66828f290c79cd4fac3af0cee6a3\"],\n [{\n            'v': 21,\n            'f': \"21\",\n        },\n{\n            'v': 20211230120639,\n            'f': \"20211230120639\",\n        },\n\"https://lcuploads-ca.s3.ca-central-1.amazonaws.com/273/14944/20211130122126/20210927_Digital_Humanities_Python_Lesson_Notes.rtf?response-content-disposition=attachment&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=0ED81C7CPQQADHEZG902%2F20211230%2Fca-central-1%2Fs3%2Faws4_request&X-Amz-Date=20211230T112543Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Signature=739aeeb001c51ebab3960b5c3eeec5b62227b11dc2db1aafd209dc9873175028\",\n\"20210927_Digital_Humanities_Python_Lesson_Notes.rtf\",\n\"rtf\",\n\"application/octet-stream\",\n\"application/rtf\",\n\"3f8c2e7f4ebc656ee826a4a22b89943b\",\n\"913a2a3e73f91cd0da3c0e50e1764c8229c3aa95\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"crawl_date\"], [\"string\", \"url\"], [\"string\", \"filename\"], [\"string\", \"extension\"], [\"string\", \"mime_type_web_server\"], [\"string\", \"mime_type_tika\"], [\"string\", \"md5\"], [\"string\", \"sha1\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    "
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3HPwOCNAvqMe"
      },
      "source": [
        "# Data Analysis\n",
        "\n",
        "Now that we have all of our datasets loaded up, we can begin to work with them!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J6Pkg0prv3BE"
      },
      "source": [
        "## Counting total files, and unique files\n",
        "\n",
        "Let's take a quick look at how to count items in DataFrames, and use total and unique files as an example to work with.\n",
        "\n",
        "It's definitely work checking out the [pandas documentation](https://pandas.pydata.org/docs/index.html). There are a lot of good examples available, along with a robust [API reference](https://pandas.pydata.org/docs/reference/index.html#api)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DFX4Gl3wv7bi"
      },
      "source": [
        "\n",
        "### How many word documents files are in this collection?\n",
        "\n",
        "We can take our `word documents` variable try a couple of functions to get the same answer.\n",
        "\n",
        "1.   `len(word_documents.index)`\n",
        "  * Get the length of the DataFrame's index.\n",
        "2.   `word_documents.shape[0]`\n",
        "  * Get the shape or dimensionality of the DataFrame, and take the first item in the tuple.\n",
        "3.  `word_documents.count()`\n",
        "  * Count the number of rows for each column.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HTv8Oet3jiTH",
        "outputId": "5b6894d0-9465-4f8d-a87d-52d0295dc4f0"
      },
      "source": [
        "len(word_documents.index)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "22"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6rYEERnTjifk",
        "outputId": "7f847c08-85d5-4628-f424-d9a9cb747218"
      },
      "source": [
        "word_documents.shape[0]"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "22"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bn-1v127aKIG",
        "outputId": "2eeb1d4e-2758-495d-c85e-d940f3628c1d"
      },
      "source": [
        "word_documents.count()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "crawl_date              22\n",
              "url                     22\n",
              "filename                22\n",
              "extension               22\n",
              "mime_type_web_server    22\n",
              "mime_type_tika          22\n",
              "md5                     22\n",
              "sha1                    22\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "38veKiPhwKo4"
      },
      "source": [
        "### How many unique word documents files are in the collection?\n",
        "\n",
        " We can see if an word documents files is unique or not by computing an [MD5 hash](https://en.wikipedia.org/wiki/MD5#MD5_hashes) of it, and comparing them. The exact same word documents file might have a filename of `example.jpg` or `foo.jpg`. If the hash is computed for each, we can see that even with different file names, they are actually the same word documents file. So, since we have both a `MD5` and `SHA1` hash column available in our DataFrame, we can just find the unique values, and count them!\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WesM3kQowM5B",
        "outputId": "1b8d2c6f-6f25-4669-8948-2eddb51d7b49"
      },
      "source": [
        "len(word_documents.md5.unique())"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "21"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZIXkI0-1wWQf"
      },
      "source": [
        "### What are the top 10 most occurring word documents files in the collection?\n",
        "\n",
        "Here we can take advantage of [`value_counts()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.value_counts.html) to provide us with a list of MD5 hashes, and their respective counts."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8Ts03OFyjPIM",
        "outputId": "161426a0-5800-46de-e510-5f5a2e74c42a"
      },
      "source": [
        "word_documents[\"md5\"].value_counts().head(10)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "022298f038537bd30aefd3e7d4b8e9ff    2\n",
              "fe0562c790e1e5c3e2a07840a19885b1    1\n",
              "ec603e70f3882eec35c08e40d79856e3    1\n",
              "8cce3a728d62e1c19250165750e43896    1\n",
              "2bd445b1e94d1257c86ebae6652909c6    1\n",
              "666a6dca42e04e1cfdbe0ff2c26a0715    1\n",
              "6625b64573c05f16f8e5ef29d37cd6bc    1\n",
              "dbc1557aac1f7b553bd80289286ea41a    1\n",
              "935bba178bb3edea20e01b6b27d1c679    1\n",
              "81ca5af0d11d33f3980dea62d73c58e3    1\n",
              "Name: md5, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FG7pGZUEwlaI"
      },
      "source": [
        "\n",
        "### What's the information around all of the occurances of `022298f038537bd30aefd3e7d4b8e9ff`?\n",
        "\n",
        "What, you mean you don't know what `022298f038537bd30aefd3e7d4b8e9ff` means? \n",
        "\n",
        "Let's find those word documents files in the DataFrame. We can here see some of the filenames used, it's dimensions, and it's URL.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 137
        },
        "id": "msmmm65lkSIK",
        "outputId": "5658346a-5189-4153-8b3f-7cd2c5d75fa0"
      },
      "source": [
        "word_documents.loc[word_documents[\"md5\"] == \"022298f038537bd30aefd3e7d4b8e9ff\"]"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        crawl_date                                                url  \\\n",
              "10  20200702145628  http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...   \n",
              "20  20210102104833  http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...   \n",
              "\n",
              "            filename extension mime_type_web_server   mime_type_tika  \\\n",
              "10  ClassListF04.rtf       rtf      application/rtf  application/rtf   \n",
              "20  ClassListF04.rtf       rtf      application/rtf  application/rtf   \n",
              "\n",
              "                                 md5                                      sha1  \n",
              "10  022298f038537bd30aefd3e7d4b8e9ff  3d58dfce44ed66828f290c79cd4fac3af0cee6a3  \n",
              "20  022298f038537bd30aefd3e7d4b8e9ff  3d58dfce44ed66828f290c79cd4fac3af0cee6a3  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-2cef030f-c6b0-430b-9b88-00940038777c\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>crawl_date</th>\n",
              "      <th>url</th>\n",
              "      <th>filename</th>\n",
              "      <th>extension</th>\n",
              "      <th>mime_type_web_server</th>\n",
              "      <th>mime_type_tika</th>\n",
              "      <th>md5</th>\n",
              "      <th>sha1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>20200702145628</td>\n",
              "      <td>http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...</td>\n",
              "      <td>ClassListF04.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>022298f038537bd30aefd3e7d4b8e9ff</td>\n",
              "      <td>3d58dfce44ed66828f290c79cd4fac3af0cee6a3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>20210102104833</td>\n",
              "      <td>http://www.arts.uwaterloo.ca/%7eraha/392AF04_w...</td>\n",
              "      <td>ClassListF04.rtf</td>\n",
              "      <td>rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>application/rtf</td>\n",
              "      <td>022298f038537bd30aefd3e7d4b8e9ff</td>\n",
              "      <td>3d58dfce44ed66828f290c79cd4fac3af0cee6a3</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2cef030f-c6b0-430b-9b88-00940038777c')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-2cef030f-c6b0-430b-9b88-00940038777c button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-2cef030f-c6b0-430b-9b88-00940038777c');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/a8bd4d5e58f96183/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 10,\n            'f': \"10\",\n        },\n{\n            'v': 20200702145628,\n            'f': \"20200702145628\",\n        },\n\"http://www.arts.uwaterloo.ca/%7eraha/392AF04_web/ClassListF04.rtf\",\n\"ClassListF04.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"022298f038537bd30aefd3e7d4b8e9ff\",\n\"3d58dfce44ed66828f290c79cd4fac3af0cee6a3\"],\n [{\n            'v': 20,\n            'f': \"20\",\n        },\n{\n            'v': 20210102104833,\n            'f': \"20210102104833\",\n        },\n\"http://www.arts.uwaterloo.ca/%7eraha/392AF04_web/ClassListF04.rtf\",\n\"ClassListF04.rtf\",\n\"rtf\",\n\"application/rtf\",\n\"application/rtf\",\n\"022298f038537bd30aefd3e7d4b8e9ff\",\n\"3d58dfce44ed66828f290c79cd4fac3af0cee6a3\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"crawl_date\"], [\"string\", \"url\"], [\"string\", \"filename\"], [\"string\", \"extension\"], [\"string\", \"mime_type_web_server\"], [\"string\", \"mime_type_tika\"], [\"string\", \"md5\"], [\"string\", \"sha1\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    "
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GbLLZW2awzCv"
      },
      "source": [
        "### What are the top 10 most occuring filenames in the collection?\n",
        "\n",
        "Note that this is of course different than the MD5 results up above. Here we are focusing _just_ on filename. So `ClassListF04.rtf` for example, might actually be referring to different word documents files who happen to have the same name.\n",
        "\n",
        "Here we can use `value_counts()` again, but this time we'll create a variable for the top filenames so we can use it later.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pQaw54ACkwdZ",
        "outputId": "aa53d7d4-8526-463f-d54e-1bce34466c9f"
      },
      "source": [
        "top_filenames = word_documents[\"filename\"].value_counts().head(10)\n",
        "top_filenames"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "softConstraints.rtf               3\n",
              "ClassListF04.rtf                  2\n",
              "assignmentProposalTemplate.odt    1\n",
              "c61-summary.odt                   1\n",
              "1855                              1\n",
              "Transmag-ieeeTvlsi.rtf            1\n",
              "TEMPLATE_PAPER_ECE700T02.rtf      1\n",
              "cae_bibtex.rtf                    1\n",
              "edgar.ppt                         1\n",
              "petition_en.odt                   1\n",
              "Name: filename, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z7F3re20BQRI"
      },
      "source": [
        "### Let's create our first graph!\n",
        "\n",
        "We'll plot the data first with pandas [plot](https://pandas.pydata.org/docs/reference/api/pandas.Series.plot.html) functionality, and then plot the data with [Altair](https://altair-viz.github.io/)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 809
        },
        "id": "sRvlstfsBWEZ",
        "outputId": "af1f27ec-0aa7-42b1-b6de-e96edeb781a7"
      },
      "source": [
        "top_filenames_chart = top_filenames.plot.bar(figsize=(25, 10))\n",
        "\n",
        "top_filenames_chart.set_title(\"Top Filenames\", fontsize=22)\n",
        "top_filenames_chart.set_xlabel(\"Filename\", fontsize=20)\n",
        "top_filenames_chart.set_ylabel(\"Count\", fontsize=20)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'Count')"
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1800x720 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABa8AAAMHCAYAAAA+YqspAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde7ztdV3n8fdHQDO1SDmPVOR4TC0brdDOeMkuVFYqM9oYlmYKZPHINC2bSbQSxWZinLJyqHzQyCBdzFsRCt4yL2GBAiEo6IhJguIVhchLoJ/5Y/2O7TZrn7PXYe+9vofzfD4e+7H2Wr/v77c++5z9B7zO7/Fd1d0BAAAAAICR3GrZAwAAAAAAwGriNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgCALVdVvRdfpw0w9/PWMecZq9Y+b9U1jphef9syfgYAANhXHLjsAQAA2C+9bM5rd07yI0n+Jcmr5xw/Z1MnWsyHsvY8F27lIAAAcEslXgMAsOW6+5jVr1XVEZnF60/POz6Yc9Yx48lJ/jzJpzd/HAAAuOURrwEAYBN096cjXAMAwF6z5zUAAPuEqrpvVZ1eVVdW1Zeq6tNVdXZVPWKN9adNe0sfU1WHV9UZ0zlfqKoLqurYTZ537p7X6zjvTlX1G1V1SVVdX1X/UlUXVtUvVdVBc9av/DnvVVV/VlWfmP6M3l9Vz6qqm/x3f1Vtq6pnVNUbqurDVfXFqrq2qs6tqqdW1QFzztkxvdcVVXWrqnpmVb1v+jO9qqpeVFVfO639hqr63Wntl6rqg1X1zN383FVVj6uqN01/T1+qqo9U1R9V1Y41zvnhqjqrqj5ZVTdU1TXTz3xqVT1gkT93AADGI14DADC8qnpUkguSPDHJtUlek+TSzLYZObuqXrCb0x+U5O+T3C/Jm5P8XZLvSHJqVb14M+deVFV9W5KLk/xqkoOTvC3J25PcPcmLkry+qm69xumHZ/Zn9KAkb03yziT3THJSkt+bs/5Hkvxukvsm+XCSv8xsv+7DM9vy5DVVVbsZ98+SnDid+6Ykt0vyS9N5d0xyXpKfSPLuJH+bZEeS366q58z5uQ/KbJ/zlyf57sz+bs/MbP/zn0lyYVXtXHXOMUnemOThSS6fzn9nki8mOSbJD+9mdgAA9gG2DQEAYGhVdeckf5zkNkl+ubtftOLYEUnOSvJrVXVOd79xziV+LsmLkzyzu788nfegzEL2L1TVG7r77E3+Mfaoqm6b5K+S3DXJs5P8VnffOB27Y5JXJHlYkucked6cSzwjyfOTnNjdX5nO+97MQvbPV9ULu/vKFesvSPLg7j5v1Rx3SXJ2kkcn+fHpfVe7e2aR+Ju7+2PTeYcl+YfMYvLbk7wnyRO7+4vT8SOTvC7J8VX1u939+RXXe0GSxyR5R5IndPdVK+Z5WpL/neTPq+o+u/5Mkjx3evye7v67VT/D3ZJ83Zy5AQDYh7jzGgCA0f1sZiHynSvDdZJ099syC5tJ8l/XOP+jSX5lV7iezjsvye9MT39pL2Y6eto+4yZfe3GtXY5Jco8kr+zuk1ZE2nT3NUmOTnJDkqeucUf0u5M8f1e4ns57R2Z3J98qyfevXNzdl60O19PrVyf5lenpUbuZ9+m7wvV03pVJ/mR6evckT9kVrqfjZ2V2V/kdknz1LuopzD89yfVJHrsyXE/nnZzZP1DcM8nKLWK+McnnVofr6ZyruvvS3cwOAMA+wJ3XAACM7vumx5etcfzUJM9K8t1VdcDKSD15dXd/ac55f5zZ3bvfXVUHrozF6/ChJOcssH49Hjk9vmrewe7+WFV9MMl/SHLvJP9v1ZKzu3tePH9/ZtH3rqsPVNWBSX4gyUOS3DnJ1ySpzAJzknzzGrPekOQtc16/fHo8f/rAytU+mOTbV83y/Ulum+Ss7v7kGu/39iRHTnO+dnrtXUmOqKrTM/uHiIvW+PkBANhHidcAAIzu0Onxw2scvyLJVzILr3dKsjqArnXeR1ad94kFZjqnu49ZYP16fNP0+KrdbzWdJNmWm8brj6yx9rrp8WtWvlhV35zkjCTfupv3WWvrjY/P+UeCZHb3dJJcNefYyuMrZ9n1cx+5jjvXt634/ucz24bkidPXtVX1riR/neT07v74Hq4FAMDgxGsAAPYVt/S7ag+YHs9KMu+u5ZU+M+e1r8x5bXdenVm4PjPJC5NcluTa7v7yFLY/kNld2PPs6b0WmWXXz/2BJOfuYe1Xtznp7suq6j6ZffDkDyR5aGZ3cf9QkhOq6se6+w0LzAEAwGDEawAARvfRJPfJ7A7deVtV7MhsT+cvJrlmjePzbF9x3rwYvNWuTPItSf5w2h9600zR99syu0v9MXPuor7XZr7/Krs+RPKSRe9m7+4bMrv7+nVJUlXfkOSEzD688qX5t7v2AQDYB/nARgAARvf26fFJaxw/dno8Z419q4+qqlvPef0J0+M7F9zverO8fnp87Ba81x2nx4+tsf3HE+a8tln+OrM9tB9WVQffnAt192eT/LfM7vy+a1Vt28MpAAAMTLwGAGB0f5TknzP7YMWnrzxQVd+b5Bemp7+9xvl3S3JSVX31v32r6j8meeb09Pc2dty9dkpmdyEfXVXPq6qvXb2gqu5RVT+1Ae/1wcwC7/2mP8OV73FsksdvwHusS3d/IsnvJzk4yZnTXeH/TlXdrqp+sqq+cXr+tVX1zDXi9JGZ/X/OdUk+t4mjAwCwyWwbAgDA0Lr741X1xCSvSPJ7VfUzSd6b5K5JviezUPkbu9nf+CWZfbjff66q8zP70L/vy+y/hf+gu1+72T/DenT39VV1ZGZbYJyQ5Beq6uIkH0tyh8z2p75XZvs+/8nNfK9PVdUfJHlakrdW1duTfDyzrUTul+Q3kzz75rzHgn4ls7/PH0/y3qq6KMk/ZrbP+Y4k35HkNpn9GXwiya0z+8eKF1bVJfm3GH/PJDun8541bSsCAMA+yp3XAAAMr7v/KrMo+SdJ7pTkqMxC65uSHNndv76b089L8l1J3p/Zh/s9NMklSX42s3g7jO6+JMm3J3lOZkH2AZn9rA/I7EMcX5DkuA16u2dM13pPkgcmeURmYfgRmd0FvmW6+4bu/okkj8os3t81yY8meViS2yV5eZL/kuRD0ynXJ3lKZh86edvM/l4fldnd23+W5CHd/ZKt/BkAANh41X1L/9B2AAD2R1V1WpKjkxzb3actdxoAAGBR7rwGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI49rwEAAAAAGI47rwEAAAAAGM6Byx5gMxxyyCG9Y8eOZY8BAAAAAMBuXHDBBZ/u7m3zjt0i4/WOHTty/vnnL3sMAAAAAAB2o6r+aa1jtg0BAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwlhqvq+prqupdVfWeqnpfVT1/zprbVNUrquryqjqvqnZs/aQAAAAAAGylZd95/aUkP9Dd35Hk8CQPr6oHr1rz5CSf7e57JfmdJP9zi2cEAAAAAGCLLTVe98z109ODpq9etezRSV42ff/qJD9YVbVFIwIAAAAAsATLvvM6VXVAVV2U5JNJ3tzd561acmiSK5Oku29Mcm2SO23tlAAAAAAAbKUDlz1Ad385yeFVdXCSv6yq+3X3exe9TlUdl+S4JNm+ffsGT7n3dhx/1rJHGNIVJx257BEAAAAAgIEt/c7rXbr7c0nemuThqw59NMlhSVJVByb5+iSfmXP+Kd29s7t3btu2bbPHBQAAAABgEy01XlfVtumO61TVbZP8UJL3r1p2ZpKjp++PSvI33b16X2wAAAAAAG5Blr1tyF2SvKyqDsgspL+yu19XVScmOb+7z0zy0iR/XFWXJ7kmyeOWNy4AAAAAAFthqfG6uy9Ocv85rz93xfdfTPLYrZwLAAAAAIDlGmbPawAAAAAA2EW8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHCWGq+r6rCqemtVXVpV76uqZ8xZc0RVXVtVF01fz13GrAAAAAAAbJ0Dl/z+Nyb55e6+sKrukOSCqnpzd1+6at3fdvd/WsJ8AAAAAAAswVLvvO7uq7v7wun7f05yWZJDlzkTAAAAAADLN8ye11W1I8n9k5w35/BDquo9VfX6qrrvlg4GAAAAAMCWW/a2IUmSqrp9ktck+cXuvm7V4QuT3L27r6+qRyY5I8m951zjuCTHJcn27ds3eWIAAAAAADbT0u+8rqqDMgvXf9rdf7H6eHdf193XT9+fneSgqjpkzrpTuntnd+/ctm3bps8NAAAAAMDmWWq8rqpK8tIkl3X3i9ZYc+dpXarqgZnN/JmtmxIAAAAAgK227G1DHprkiUkuqaqLpteek2R7knT3S5IcleQpVXVjki8keVx39zKGBQAAAABgayw1Xnf3OUlqD2tOTnLy1kwEAAAAAMAIlr7nNQAAAAAArCZeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAw1lqvK6qw6rqrVV1aVW9r6qeMWdNVdWLq+ryqrq4qh6wjFkBAAAAANg6By75/W9M8svdfWFV3SHJBVX15u6+dMWaRyS59/T1oCR/OD0CAAAAAHALtdQ7r7v76u6+cPr+n5NcluTQVcseneT0njk3ycFVdZctHhUAAAAAgC207Duvv6qqdiS5f5LzVh06NMmVK55fNb129arzj0tyXJJs3759s8aETbXj+LOWPcJwrjjpyGWPAAAAAMASDPGBjVV1+ySvSfKL3X3d3lyju0/p7p3dvXPbtm0bOyAAAAAAAFtq6fG6qg7KLFz/aXf/xZwlH01y2Irnd5teAwAAAADgFmqp8bqqKslLk1zW3S9aY9mZSZ5UMw9Ocm13X73GWgAAAAAAbgGWvef1Q5M8McklVXXR9NpzkmxPku5+SZKzkzwyyeVJPp/k2CXMCQAAAADAFlpqvO7uc5LUHtZ0kqduzUQAAAAAAIxg6XteAwAAAADAauI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA4C8XrqtpeVV+3hzV3qKrtN28sAAAAAAD2Z4veef3hJM/Yw5qnT+sAAAAAAGCvLBqva/oCAAAAAIBNsxl7Xt85yb9swnUBAAAAANhPHLinBVX1pFUvHT7ntSQ5IMn2JD+V5JINmA0AAAAAgP3UHuN1ktOS9PR9J3n09LXaru1EPp/k+Td7MgAAAAAA9lvridfHTo+V5NQkZyT5qznrvpzkM0n+vrs/tzHjAQAAAACwP9pjvO7ul+36vqqOTnJGd5++qVMBAAAAALBfW8+d11/V3d+/WYMAAAAAAMAut1r2AAAAAAAAsNrC8bqqvq+qXldVn6yqG6rqy3O+btyMYQEAAAAA2D8stG1IVR2Z2Qc2HpDkI0k+kESoBgAAAABgQy0Ur5M8L8kNSY7s7jdt/DgAAAAAALD4tiH3S/IK4RoAAAAAgM20aLy+Psk1mzEIAAAAAADssmi8fkuSh2zGIAAAAAAAsMui8fpZSe5ZVb9WVbUZAwEAAAAAwKIf2HhCkvcleX6Sn66qi5J8bs667u4n39zhAAAAAADYPy0ar49Z8f2O6WueTiJeAwAAAACwVxaN1/fYlCkAAAAAAGCFheJ1d//TZg0CAAAAAAC7LPqBjQAAAAAAsOkWuvO6qravd213f2TxcQAAAAAAYPE9r6/I7MMY96T34toAAAAAAJBk8cB8eubH64OTHJ7k7knelsTe2AAAAAAA7LVFP7DxmLWOVdWtkvx6kp9LcvTNGwsAAAAAgP3Zhn1gY3d/pbufn9nWIidt1HUBAAAAANj/bFi8XuHvkvzwJlwXAAAAAID9xGbE6zsmud0mXBcAAAAAgP3EhsbrqnpYkp9I8t6NvC4AAAAAAPuXhT6wsar+ZjfXOSzJ9un5iTdnKAAAAAAA9m8LxeskR6zxeif5bJI3Jvmt7l4rcgMAAAAAwB4tFK+7ezP2yAYAAAAAgH9HjAYAAAAAYDiLbhvy71TVHZIcnOTa7r5uY0YCAAAAAGB/t/Cd11V1YFUdX1WXJ/lckiuSfLaqLp9ev1lBHAAAAAAAForXVXXrJG9K8t+T7EhyZZJ3TY87ptf/elq3nuudWlWfrKr3rnH8iKq6tqoumr6eu8i8AAAAAADsmxa98/qZSY5IclaSb+3uHd39kO7ekeRbkrw2yfdM69bjtCQP38Oav+3uw6evExecFwAAAACAfdCi8fonk7w3yY929wdXHujuDyV5TJL3JXnCei7W3e9Ics2CMwAAAAAAcAu3aLy+V5LXd/dX5h2cXn99knve3MFWeEhVvaeqXl9V993A6wIAAAAAMKhFP1zxX5Pcfg9rbpfkhr0b5yYuTHL37r6+qh6Z5Iwk9563sKqOS3Jckmzfvn2D3h4AAAAAgGVY9M7ri5McVVXb5h2sqkOSHJXkPTd3sCTp7uu6+/rp+7OTHDS9x7y1p3T3zu7euW3b3PEAAAAAANhHLBqvT06yLcm7qurJVfVNVXXbqrpHVR2b5Lzp+MkbMVxV3bmqavr+gdO8n9mIawMAAAAAMK6Ftg3p7ldW1eFJjk9yypwlleSF3f3K9Vyvql6e5Igkh1TVVUlOSHLQ9F4vyewu7qdU1Y1JvpDkcd3di8wMAAAAAMC+Z9E9r9Pdz6mqM5M8Ocn9k3x9kmuT/EOSU7v77xe41uP3cPzkbNBd3AAAAAAA7DsWjtdJ0t3nJjl3g2cBAAAAAIAk69jzuqpuXVXvqqq3VNVBe1j3lqo6d3frAAAAAABgT9bzgY0/leQ7k/x2d9+w1qLu/tck/yvJA5M8YWPGAwAAAABgf7SeeP2YJP/Y3WfvaWF3vyHJB5M89uYOBgAAAADA/ms98fr+Sd62wDXfkeTwvZoGAAAAAACyvnh9SJJPLHDNTyS5096NAwAAAAAA64vXX0hy+wWuefskX9y7cQAAAAAAYH3x+sokOxe45s4kH9m7cQAAAAAAYH3x+m1JHlJVewzYVfWdSb4ryVtv5lwAAAAAAOzH1hOvT07SSV5VVd+61qKquk+SVyX5cpI/2JjxAAAAAADYHx24pwXd/YGqOjHJ85L8Q1W9OsnfJLlqWnJokh9M8mNJbpPkud39gc0ZFwAAAACA/cEe43WSdPeJVXVjkhOS/GSSx69aUkluSPKr3f2bGzsiAAAAAAD7m3XF6yTp7v9RVX+a5KeTPDTJXaZDVyc5J8n/7e5/2vgRAQAAAADY36w7XifJFKdP2KRZAAAAAAAgyfo+sBEAAAAAALaUeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA4S43XVXVqVX2yqt67xvGqqhdX1eVVdXFVPWCrZwQAAAAAYOst+87r05I8fDfHH5Hk3tPXcUn+cAtmAgAAAABgyZYar7v7HUmu2c2SRyc5vWfOTXJwVd1la6YDAAAAAGBZln3n9Z4cmuTKFc+vml4DAAAAAOAW7MBlD7BRquq4zLYWyfbt25c8DcDm2nH8WcseYUhXnHTkskcYkt+Xm/K7Mp/flfn8vszn9+Wm/K7M53dlPr8v8/l9uSm/K/P5XZnP78t8fl9ual/5XRn9zuuPJjlsxfO7Ta/dRHef0t07u3vntm3btmQ4AAAAAAA2x+jx+swkT6qZBye5truvXvZQAAAAAABsrqVuG1JVL09yRJJDquqqJCckOShJuvslSc5O8sgklyf5fJJjlzMpAAAAAABbaanxursfv4fjneSpWzQOAAAAAACDGH3bEAAAAAAA9kPiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAOs6u/0AACAASURBVAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAAAAADEe8BgAAAABgOOI1AAAAAADDEa8BAAAAABiOeA0AAAAAwHDEawAAAAAAhiNeAwAAAAAwHPEaAAAAAIDhiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMJylx+uqenhVfaCqLq+q4+ccP6aqPlVVF01fP7OMOQEAAAAA2DoHLvPNq+qAJL+f5IeSXJXk3VV1ZndfumrpK7r7aVs+IAAAAAAAS7HsO68fmOTy7v7H7v7XJH+e5NFLngkAAAAAgCVbdrw+NMmVK55fNb222o9V1cVV9eqqOmzeharquKo6v6rO/9SnPrUZswIAAAAAsEWWHa/X47VJdnT3tyd5c5KXzVvU3ad0987u3rlt27YtHRAAAAAAgI217Hj90SQr76S+2/TaV3X3Z7r7S9PT/5PkO7doNgAAAAAAlmTZ8frdSe5dVfeoqlsneVySM1cuqKq7rHj6qCSXbeF8AAAAAAAswYHLfPPuvrGqnpbkjUkOSHJqd7+vqk5Mcn53n5nk6VX1qCQ3JrkmyTFLGxgAAAAAgC2x1HidJN19dpKzV7323BXfPzvJs7d6LgAAAAAAlmfZ24YAAAAAAMBNiNcAAAAAAAxHvAYAAAAAYDjiNQAAAAAAwxGvAQAAAAAYjngNAAAAAMBwxGsAAAAAAIYjXgMAAAAAMBzxGgAAAACA4YjXAAAAAAAMR7wGAAAAAGA44jUAAAAAAMMRrwEAAAAAGI54DQAAAADAcMRrAAAAAACGI14DAAAAADAc8RoAAAAAgOGI1wAAwP9n787DfTvn+/8/X4khMRyCoKYIggaJDGZtiR9KDRESNbWGqiqls6EDRYUqbfHVmkqirZai0qCoGmvILAmVmtXYajXSkEji9fvj89nJzs4+5+x9Ivten32ej+va1/mse+0dL9e1rr3Xeq/7ft+SJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTY7Fa0mSJEmSJEnS5Fi8liRJkiRJkiRNjsVrSZIkSZIkSdLkWLyWJEmSJEmSJE2OxWtJkiRJkiRJ0uRYvJYkSZIkSZIkTc7w4nWSn05yRpLPJXn6KuevmOTv5uc/keTGG59SkiRJkiRJkrSRhhavk+wK/D/gPsC+wMOS7Lvi2x4HfKftzYA/AV64sSklSZIkSZIkSRtt9Mzr2wOfa/uFtj8A/hZ44IrveSBw1Pzz3wP3SJINzChJkiRJkiRJ2mCji9fXB/5j2fFX52Orfk/b84EzgWtuSDpJkiRJkiRJ0hBpO+5/PHkI8NNtf2F+/CjgDm2fvOx7Tp9/z1fnx5+ff8+3V/y3fhH4xfnhLYAzNuD/wqK5FvDt7X6X5LWi9fF60Vp5rWg9vF60Vl4rWg+vF62V14rWw+tFa+W1srq92u652onLbXSSFb4G3HDZ8Q3mY6t9z1eTXA64GvDfK/9DbV8FvOoyyrkpJDmh7cGjc2j6vFa0Hl4vWiuvFa2H14vWymtF6+H1orXyWtF6eL1orbxW1m9025DjgX2S7J3kCsDPAses+J5jgJ+ff34I8C8dOV1ckiRJkiRJknSZGzrzuu35SZ4MvBvYFfjLtp9K8hzghLbHAK8F3pDkc8D/MCtwS5IkSZIkSZI2sdFtQ2j7TuCdK8Z+f9nnc4DDNzrXJmVbFa2V14rWw+tFa+W1ovXwetFaea1oPbxetFZeK1oPrxetldfKOg3dsFGSJEmSJEmSpNWM7nktSZIkSZIkSdIlWLyWJEmSNHlJ7jL/94qjs2j6vF4kSdocLF5vUkleOP/XfuHapiR7j86gxbPag6APh5Kky9hL5/9+bGgKLQqvF0nSZPgMveMsXm9e900S4Bmjg2jy/h4gyftGB9FCWe1B0IdDXUKSN6xlTJLW4LwkrwKun+SlK79Gh9PkeL1ozZaehZYmgUnbs9q14vWj7fAZegddbnQAXWb+CfgOcJUk3102HqBtt4yJpQnaJckzgZsn+fWVJ9u+ZEAmTVSS6wLXB3ZPcgCz3ykAW4ArDQumKbvV8oMkuwIHDcqiBZHkrsDtgdPbvmd0Hk3G/YD/D7g3cOLgLJo+rxetx48luTPwgCR/y0X3uAC0PWlMLE3YPYGnrRi7zypj2sn5DH3pWbzepNr+FvBbSd7e9oGj82jSfhY4lNnvg6sOzqLpuzfwaOAGwPIXG2cBzxwRSNOU5BnMrondl71EDfAD4FXDgmmSkhzX9vbzz48HngS8DXhWkgPbvmBoQE1C228Df5vkq20/svzcUn9jaYnXi9bp94HfY3aP+2IuXrwucMiIUJqeJE8Efhm4SZJTl526KvCvY1Jp4nyGvpTSdnQGXYaSvLDt07Y3JiX55bavWDG2d9svjsqk6Ury4LZvGZ1D05fkyLa2sNI2JTm57QHzz8cD9237X0muDHy87W3GJtSUJDmp7YHbG5PA60Xrk+T32j53dA5NV5KrAXsARwJPX3bqrLb/MyaVFoHP0DvO4vUmt5WbtVPb7jcqk6ZpK9fKiW1d3q8LrdZaZjnbzGhJkm0WBVx+q+WSfBK4G7P9WN7d9uBl5y4sbGvnluROwJ2BXwX+ZNmpLcCD2u4/JJgmyetFOyLJ+9reY3tjElx4v3tXZrPz/9X7W63GZ+hLz7Yhm9SypSw3dSmLtiXJLZn1pL1aksOWndoC7DYmlSZsqbXMLYDbAcfMj+8PHDckkabqxfN/dwMOBj7JbAnufsAJwJ0G5dI0XY1ZT9oATfJjbb+R5Cqs6DuqndoVgKtwyVZn3wUeMiSRpszrRWuWZDfgysC1kuzBxXvSXn9YME1Wkt8DjgDeOh96XZI3t33ewFiaJp+hLyVnXm9SLmXRWiV5ILOe1w/gol+iMOu/9LdtPzokmCYtyYeAn2l71vz4qsA72v7k2GSamiRvBZ7V9rT58a2BZ7e1cKDtSnIl4Dq2sNKS+aavb2r74NFZNH1eL1qrJE9lNkv/esDXuKh4/V3g1W1fPiqbpinJGcD+bc+ZH+8OnNL2FmOTaap8ht5xzrzepNqemeT/gAPafnl0Hk1X27cnORZ4Wtvnj86jhXEdZhvvLfnBfExa6RZLhWuAtqcn+fGRgbQYklxj/sLdwrUu1PaCJNcbnUOLwetFa9X2z5K8HHimPa+1Rl9ntsLwnPnxFZm9+JC2xmfoHWTxehOb36ydkeRGbb8yOo+ma36tHApYvNZaHQ0cl+RtzGamPBB4/dBEmqpTk7wG+Kv58SOAU7fx/doJJfndpWW2SfYF/gG4fJIAD237iaEBNTWnJDkGeDNw9tJg27du/Ue0E/N60ZrMn4kOAyxeay3OBD6V5L3Mel7fk9nz0UsB2j5lZDhNks/QO8i2IZvcfFnCAcz66Cy/WXvAsFCapCR/Alwe+Dsufq246YRWNd+g5CeY3ax9uO3JgyNpguY9JJ8ILC2H+xDw50tLLCW4+KbBSd4BvLztu5LcHvjTtncem1BTkuR1qwy37WM3PIwmz+tF65Hkj4GPAW+txRJtQ5Kf39b5tkdtVBYtDp+hd4zF600uyU+tNt72gxudRdOW5P2rDLftIRseRgshyf7MCpJLf3g/OTiSJirJFZhtUFLgjLbnDY6kiVlRvD657QHLzl3sWJKky0qSs5ht3Hg+s3YQYfZMtGVoME3S/B73llx0j/uD7fyIdnI+Q+8Yi9eSpHWbb2rzeOAtzG7qHwS8qu3LhgbT5CS5G3AU8CVm18oNgZ9v+6GBsTQxSf6X2az8AHcE9mr7vfm509veemQ+TV+S+7U9dnQOLQavF0mXVpL7Aq8EPs/s/mVv4Alt3zU0mCbLZ+gdZ/F6J5TkVW1/cXQOTV+SA20botUkORW4U9uz58dXBj7Wdr+xyTQ1SU4EHt72jPnxzYE3tj1obDJNySorxU5s+39JrgM8pO3/G5FLiyPJH7R91ugcWgxeL5IurSSfAe7X9nPz45sC72h7y7HJNFU+Q+84N2zcOb1ydAAtjCcyezMorRTggmXHF8zHpJUuv1S4Bmj770kuPzKQpmdr7czafguwcK3tshCp9fB60XrYvkpbcdZS4XruC8BZo8JoIfgMvYMsXu9EkuwCXKXtiaOzaDG0tXCtrXkd8In5TskAhwKvHZhH03VCktcAfzU/fgRwwsA8mqAkBwMvAr4GPAP4S+D2wL8Dj297ysB4mogkh7T9lySHrXa+7Vs3OpOmLckWYM+2n18xvl/bUwfF0oKxcK2tOCHJO4E3MetffDhw/NLfKP8maRU+Q+8g24Zsckn+BvglZm90jge2AH/W9kVDg2nSkjy/7TNH59C0zXdKvuv80J2StaokVwSexLJrBXhF23PHpdLUJDkOeBZwdeCPgF9r+/dJ7gE8r+2dhgbUJCy1ekjyulVOt+1jNzyUJivJEcCfAv8JXB54dNvj5+cu3CRWWi7Jvm0/vWLsbm0/MCiSJmorf4uW+DdJq/IZesdYvN7kkpzS9rZJHgEcCDydWR9Je+oIgCQvXTkEPAo4GqDtUzY8lCRpp7J8SXaSr7S90WrnJGmtkpwC3KftN5Lcntm97TPavs3fK9qaJKcDb2D2InW3+b8H+xJVksbZZXQAXeYuP+8teihwTNvzRgfS5DwIuAazZfwnzv89b/7ZFjNasyTHjs6gxZDk2aMzaHLOSXKvJIcDTXIoXLiR4wXb/lHtbJI8NcmWzLwmyUlJ7jU6lyZn17bfAGh7HHB34HeTPIXZEn9pNXcAbgh8lNnK5a8DdxmaSAsjyf1GZ9Bi8Rl6bSxeb36vBL4EXBn4UJK9gDOHJtLU7At8G/hp4L1tj2K2+cRR88/SWtkjXZeQ5FeS7LFi2BdjWumXgN8AHgvcG7h7kv8FXgG4AkgrPbbtd4F7AddktmLsBWMjaYLOSnLTpYN5IftuwAOBW40Kpck7D/g+sDuzmddfbPvDsZE0NfOXpzdc5dTtNjyMFp3P0Gtg25BNLsnebb+47DjAzdp+dmAsTVCSg4A/Bt4BPLntjccm0tQl2R24UdszRmfRdCV5HvCzwEnMNuF7d7350DokeUzbbfWV1E4myalt90vyZ8AHbAOh1STZH/jeyuee+arUI9r+9ZhkmrIknwTeDjwXuBbwF8AP2h4+NJgmJ8lpbW8zOoe0M7B4vcmtthlJkhPbHjQqk6Zr/nLjl4E7tX3k6DyariT3Z/ay4wpt905yW+A5bR8wOJomaP675V7AY4CDme3K/tq2nx8aTAthZQ9sab5J1vWBvYH9gV2ZFbG9v9UlJLkOs+sF4GttvzUyj6YtycFtT1gx9qi2bxiVSdOU5Cjg5UsbwUrbk+QuwLOBvYDLMdtvrG1vMjLXIrjc6AC6bCS5JbPlcFdLctiyU1uYLX+SLmbZjf3HgL8fHEfT92zg9sAHANqekmTvkYE0XW2b5JvAN4HzgT2Av0/y3ra/PTadpiDJqVs7BVxnI7NoITwOuC3whbbfS3JNZi/HpAvNX6z/BXA14Gvz4RvMWxI9se3Jw8Jpyk5M8kjgJm2fk+RGgKsMtZo7AI9I8mXgbC4qRO43NpYm7LXArzFroeieLutg8XrzugVwP+DqwP2XjZ+FPXW0zHZu7H+57UnDwmnKzmt75mxC7YVcyqNLSPJU4OeY9dZ/DfBbbc9LsgvwWcDitWBWoL438J0V42G2aZZEkgNXDN1kxd8habnXA09o+4nlg0nuOD+3/4BMmr5XAD8EDgGew+z5+S3Yy1iXdO/RAbRwzmz7rtEhFpHF602q7duBtye5U9uPjc6jSXs9W7+xfx3e2Gt1n0rycGDXJPsw21DNApNWswdwWNsvLx9s+0N3ZNcyxwJXaXvKyhNJPrDxcTRRL97GuTIrNklLrrzy/hag7ceTXHlEIC2EO7Q9MMnJAG2/k+QKo0NpepbubZNcG1e3a23en+RFwFuBc5cGnTC4ffa83uSS7MlspvWNWfayou1jR2XStCT5bNt9tnLuc21vttGZNH1JrgT8DrM+xgDvBp7b9tyt/5R2Nkl2BT7V9pajs0iSdi5JXgrcFDga+I/58A2ZrQb6Ytsnj8qm6UryCeDOwPHzIvaewHvcEFYrJXkAs5eq1wP+k1kf439re6uhwTRZSd6/ynDb+vJ9Oyxeb3JJPgp8mBU9ddq+ZVgoTYo39toRSQ5v++btjUlJ3g78StuvjM4iaXOY90h/I/AmN37VtiS5D/BAlm3YCBzT9p3jUmnKkjwCeChwIHAU8BDg99q+aWgwTU6STzJb8fPPbQ9IcnfgkW0fNziatOlYvN7kkpzS9rajc2javLHXeiU5qe2B2xuTknwIOAA4jtlmNgC0fcCwUJIWWpK9mBWXHsqsN+3fMStk+5JMF0ry/LbPHJ1DiyfJLYF7MNt34X1t/21wJE1QkhPaHjwvYh8wb4n3yba23dSqklwHeD5wvbb3SbIvcKe2rx0cbfIsXm9ySZ4HfNQipLYmyeXanj86hxbD/EXHfYEjmBULlmwB9m17+yHBNFlJfmq18bYf3Ogskjaf+b4Lvwc8ou2uo/NoOnyprh2R5A1tH7W9MSnJPwOHAkcC12LWOuR2be88NJgmK8m7mO0r9jtt909yOeDktrcZHG3y3LBx83sq8Mwk5wLnMXt73LZbxsbShBzHbFkcSV7W9lcG59G0fR04AXgAs3ZES84Cfm1IIk2aRWpJl4UVs68vAH57bCJN0K5J9mD2/HMJbf9ng/NoMVysX/F8/46DBmXRtD0QOIfZM9AjgKsBzxmaSFN3rbZvSvIMgLbnJ7lgez8ki9ebXturjs6gyVt+Q3+XYSm0ENp+Evhkkr9pe97oPJq+JHcEXgb8OHAFYFfgbF+iStpR8w3VLg+8CTi87RcGR9I03ZLZi/bVitcFbrKxcTRl82LSM4Hdk3yXi66bHwCvGhZMk9X27GWHRw0LokVydpJrMvsbtPScdObYSIvB4vUmleSWbT+TZNWlcm1P2uhMmix7B2lH3DjJkcC+wG5Lg219ENRKLwd+FngzcDCzzWBvPjSRpIWU5LC2bwV+ru0Zo/No8j7d9oDRIbQY2h4JHJnkyLbPGJ1H05fkLC75LH0ms1Wqv+GLVa3i14FjgJsm+VdgT2abwmo77Hm9SSV5VdtfTPL+VU637SEbHkqTlOR7wOeYzS646fwzXNRiZr9R2TRdST4CPAv4E+D+wGOAXdr+/tBgmpxlm9mcuvT7JMnJFhQkrZc9jLUe/q3RjkgS4EHAXZkVJj/c9h/GptIUJXku8FXgb5g9O/8ss+fpk4Antr3buHSaqnmf61swu2bOWL6aOck92753WLgJs3gt7eTmPSO3qu2XNyqLFkeSE9selOS0pQ0mlsZGZ9O0JPkQ8P8BrwG+CXwDeLQ7sUtaL4vXWo8kj277+tE5tFiSvAK4GfDG+dBDgc+3fdK4VJqiJJ9ceT+b5JS2t13tnLQ93udsnW1DdgJJbs0ll/YfPS6RpmSpOJ3khW2ftvxckhcCT1v1B7WzOzfJLsBnkzwZ+BpwlcGZNE2PYtbn+snMNrS5IfDgoYkkLapbJjl1lXFXi2k1/5DkBcChwLWZzaL9T+DtwAva/u/IcJqsQ4Af73yWX5KjgE+NjaSJ+l6SI4C/nx8/hNkGjmBrTu2YVTcYljOvN70kzwLuxqx4/U7gPsBH2tpXRxez2lu+5cv8peWS3A74N+DqwHOZ7a79R20/PjSYJGnTSvIp4L5bO+9qMS2X5N3AvwBHtf3mfOy6wM8D92h7r5H5NE1JjgWetGyCz17Ay9vef2wyTU2SmwB/BtyJWbH648wmanwNOKjtRwbG0wJy5vXWWbze5JKcBuwPnNx2/yTXAf6q7T0HR9NEJHki8MtcvN81wFWBf237yCHBJG0KSe7H7AXHXsxWfC3NkNwyNJikhWMPY61Hkn9vu+oGwUnOaHuLjc6k6Uryj8wKkFcDbgccNz++A3Cc/YslXdYsXm+dbUM2v++3/WGS85NsYbZU7oajQ2lS/gZ4F3Ak8PRl42e1/Z8xkTRVy27sV9X2ARsYR4vhT4HDgNPqG3NJl86/jg6ghfKlJL/NbOb1twDmE3keDfzHyGCapD8eHUCLIcnL2Pbz0FM2MI42ly+NDjBVFq83vxOSXB14NXAi8H/Ax8ZG0pS0PRM4M8nvAt9se26SuwH7JTnafoBawRt7rdd/AKdbuJZ0abV9MlxYgHw+cL2290myL3Cntq8dGlBT81BmEzM+OL9mYLZx8DHAEcNSaZLafnB0Bi2ME+b/3oVZe9a/mx8fDnx6SCItjCR3Bm7Msnrs0p50bQ8bFGvybBuyiSUJcIO2/zE/vjGwpe1qG91oJ5fkFOBgZr9I38lsM5tbtd1qb0nt3JJcAbgls5kHZ7T9weBImqB5f/TnAh8Ezl0ab/uSYaEkLbQk7wJeB/zOvC3e5Zi1yLvN4GiSFlSSj7S9a5KzuPisWtudaVVJPg7cte358+PLAx9ue8exyTRVSd7ArF3rKcAF8+E6W3/7nHm9ibVtkncCt5kff2lsIk3cD9uen+Qw4GVtX5bk5NGhNE1Jfgb4C+DzzG7q907yhLbvGptME/SHzFb97AZcYXAWSZvDtdq+KckzAOb3Lxds74e080lyb+BQ4Przoa8Bb2/7T+NSaYra3nX+71VHZ9HC2APYAiy12rzKfEzamoOBfV2Run4Wrze/k5Lcru3xo4No8s5L8jDg54Cl3bQvPzCPpu3FwN3bfg4gyU2BdzDrny4td722tx4dQtKmcnaSazKfHZnkjsCZYyNpapL8KXBz4Gjgq/PhGwBPSXKftk8dFk6TluRA4K7Mfsd8pK0TerSaFzCrt3yA2WSenwSePTKQJu904LrAN0YHWTS2DdnkknwGuBnwZeBsLlr2tN/QYJqceb/IXwI+1vaNSfYGjmj7wsHRNEFJjm97u2XHYbYT++228WPaCSX5I+Cf275ndBZJm8O8sPQy4NbMHgT3BB5iazwtl+Tf2958lfEA/952nwGxNHFJfp9Z7+K3zocOBd7c9nnjUmmK5r9LHgX8KrOi9SnAddseNzKXpivJ+4HbAsdx8XaKDxgWakFYvN7kkuy12njbL290FkmbR5I/B/YC3sRsVsrhwFeAfwZo+9at/7R2JvPekVdmdoN2HvaOlPQjMO9zfQtmv1POaHve4EiamCSnAo9buQI1ye2B19ojXatJcgawf9tz5se7A6e0vcXYZJqa+fPQD4FD2v54kj2A9ziZR1uT5KdWG3fD2O2zbcjm97y2j1o+MG8S/6itfL92Mkne1PaIJKdx8c1JAHCWvrZiN+BbwNIf4P8CdmfWcqZcNFtFOzl7R0r6UUtyJeDXgb3aPj7JPklu0fbY0dk0KY8BXpHkqlzUNuSGzFrMPHpUKE3e15nd554zP74is17p0kp3aHvg0j5Rbb8z39BeWlXbDya5DrD0guO4tv85MtOisHi9+d1q+UGSXYGDBmXRNC31+7vf0BRaKG0fMzqDFkeS/YAbs+y+w9n5ki6F1wEnAneaH38NeDNg8VoXansicIck12XZho1tvzkwliYqycuYTcA4E/hUkvfOj+/JbIm/tNJ58/rK0v4LezKbiS2tKskRwIuADzBbOfayJL/V9u+HBlsAFq83qfnu688Edk/y3aVh4AfAq4YF0+S0/cb830u0kknyr8BdNjyUJm/eE/1XuGRB0n5dupgkfwnsB3yKi27onZ0v6dK4aduHzjeapu335r1HpQsl2a/tqfNitQVrbc8J839PBN62bPwDGx9FC+KlzK6Vayf5Q+AhwO+OjaSJ+x3gdkuzrecvPP4ZsHi9HRavN6m2RwJHJjmy7TNG59HCutHoAJqsfwBeC/wjzjDQtt2x7b6jQ0jaVH4w70O7NNvtpizb+EiaOznJF4C/Bd7Y9tOjA2m62h61lu9L8pa2D76s82j62v51khOBezCbKHho238bGSgebgAAIABJREFUHEvTtsuKNiH/DewyKswisXi9+R2b5Mptz07ySOBA4M/csFFr5I6u2ppz2r50dAgthI8l2deigaQfoWcB/wTcMMlfM1sl9uihiTRFpzLb5+dhwDFJzgbeCPxt2y+NDKaFdpPRATQdbT8DfGZ0Di2Mf0rybmZ/iwAeCrxzYJ6Fkdba1GY232V7f2ZLtl8PvAY4ou2qu5xq55PksK2dAv6i7Z4bmUeLIcnDgX2A97Bstlvbk4aF0iTNd9U+htmS7XOZ/W6pm8FKujSSXBO4I7PfKR9v++3BkTQxSU5qe+Cy49sDPwscAXyl7Z2HhdPCWnldSdJ6JHkwF7Vm/XDbt23r+zXjzOvN7/y2TfJA4OVtX5vkcaNDaVLuv41zbnykrbkNs9lMh3DxPsaHDEukqXots2vlNGwxI+lHYN7f+j7ATdo+J8mNkty+rZuqabmL9UGfXx/HJfkN4CfHRJIk7czavgV4y+gci8bi9eZ31nzzxkcCP5lkF+DygzNpQto+ZnQGLaTDmRUNfjA6iCbvv9oeMzqEpE3lFcxehh0CPAc4i9mD4O1GhtLkvGi1wc6WHn9wg7No83BzWEnrkuQjbe+a5Cwu3pp1aUXqlkHRFoaNwTe/hzJbpv24+U7bN2ArN3LauSV5apItmXlNkpOS3Gt0Lk3W6cDVR4fQQjg5yd8keViSw5a+RoeStNDu0PZJwDkAbb8DXGFsJE3QVlcQJnFTcm1Vkt2T3GIrp5+2oWEkLby2d53/e9W2W5Z9XdXC9dpYvN7k2n6z7Uvafnh+/JW2R4/OpUl6bNvvAvcCrslsmf8LxkbShF0d+EySdyc5ZulrdChN0u7MXqLei1mbovsD9xuaSNKiOy/JrsxnLyXZE9sS6ZI+sPQhyftWnPuHjY2iRZHk/sApzDaFJcltl9/jtn3PqGySFluSN6xlTJdk25BNbj677YXAtZktSXBZgrZmaQncfYGj235q3lNSWs2zRgfQYrA1kaTLwEuBtwHXTvKHwEOA3x0bSRO0/D72Gts4Jy33bOD2zF9+tD0lyd4jA0naNG61/CDJ5YCDBmVZKBavN78/Au7f9t9GB9HknZjkPcDewDOSXBVnMWkr2n4wyV7APm3/OcmVgF1H59L0JHkdF+/tBkDbxw6II2kTaPvXSU4E7sGsCHmo97paRbfyebVjacl5bc9cMYfH60XSDpvvQ/dMYPck310aBn4AvGpYsAVi8Xrz+5Y381qjxwG3Bb7Q9ntJrgE4Y1KrSvJ44BeZzWS6KXB94C+YFRKk5Zb3HN0NeBDw9UFZJC2wJFvafnd+j/KfwBuXnbtG2/8Zl04TdO0kv86sQLD0mfnxnuNiaeI+leThwK5J9gGeAnx0cCZJC6ztkcCRSY5s+4zReRZRZpsta7NK8mfAdZn1dTt3abztW4eF0iQluQtwStuzkzwSOBD4s7ZfHhxNE5TkFGZLKj/R9oD52GltbzM2maYuyS7AR9reeXQWSYslybFt75fki1w0E3JpemTb3mRQNE1Qkm22OGv7BxuVRYtjvprwd5jt1RHg3cBz254zNJikhZXklm0/k+TA1c63PWmjMy0ai9eb3Hy59kp1ubZWSnIqsD+wH/B64DXAEW1/amQuTVOST7S9Q5KT2x4w79d1Utv9RmfTtCW5BfCOtjcbnUWSJEmSLktJXtX2F5O8f5XTbXvIhodaMLYN2eTcKEvrcH7bJnkg8PK2r03yuNGhNC1Jntz25cAHkyz17bon8MvAP45NpylKchYX7xX5TeBpg+JI2mSSPLvts0fn0PQkeRHwubavXDH+BGDvtk8fk0xTlmRP4LeZbay229K4xSVJO6rtL84/3mflKo4ku63yI1phl9EBdNlKcoMkb0vyn/OvtyS5wehcmqSz5hsJPBJ4x3xp/+UHZ9L0LK3aeDrwX8BpwBOAdwK/OyqUpqvtVdtuWfZ187ZvGZ1L0qbxgNEBNFmHsPpGWK8G7rfBWbQ4/hr4DLNN7P8A+BJw/MhAkjaN1frn21N/DSxeb36vA44Brjf/+sf5mLTSQ5n1RX9c228CNwBeNDaSpqrtD9u+uu3hbR8y/2wfKl1CkrskufL88yOTvCTJXqNzSdo0sv1v0U7qiqvdm7T9IV432rprtn0tcF7bD87bbTrrWtIOS3LdJAcxW7V8QJID5193A640ON5CsG3I5rdn2+XF6tcn+dVhaTRZ84L1S5YdfwU4elwiTdR+Sb67yniY9evastGBNHl/DuyfZH/gN5j10z8asJ++pB+Fg0YH0GR9P8k+bT+7fDDJPsD3B2XS9J03//cbSX4G+DpwjYF5JC2+ewOPZjZB8CXLxr8LPHNEoEVj8Xrz++8kjwTeOD9+GPDfA/NoopLcEXgZ8OPAFYBdgf9re7WhwTQ1p7U9YHQILRT76Uv6kUry0hXHAGcCJ7R9+5BQmqLfB96V5HnAifOxg4FnAE7m0dY8L8nVmL1wfxmwBa8XSZdC26OAo5I82PaJO8a2IZvfY4EjmG2Q9Q3gIYCbOGo1L2f2cuOzwO7ALwCvGJpI0mZgP31JP2q7Abdlds/yWWA/ZrOZHpfkT0cG03S0fRdwKHB34PXzr7sBD277zmHBNHWHA2l7etu7A/cEHjQ4k6TN4V+TvDbJuwCS7OuknrWJLUolASQ5oe3BSU5tu9987GRn2Wq5JM9s+/zRObQ4klwXeDhwfNsPJ7kRcLe2tiWStEOSfBy4S9sL5seXAz4M3JXZCqF9R+bT9CW5XNvzR+fQ9Kz2/OMzkaQfhXnR+nXA77Tdf37/cnLb2wyONnm2DdmkkrwI+FzbV64YfwKwd9unj0mmCftekisApyT5I2Yz9V2doZV+bOVy7eXaPmUjw2j6lvfTT3K/tsdiP31Jl84ewFWYtQoBuDJwjbYXJDl3XCxNSZKPtL3r/PMb2j5q2enjgAPHJNPE7ZJkj7bfAUhyDaybSPrRuFbbN81XpdL2/CQXjA61CPwlvHkdAvz2KuOvBk4FLF5rpUcx63P9ZODXgBsCDx6aSFN0wugAWmjPAY4dHULSwvsjZi/bP8Bsw+CfBJ6f5MrAP48Mpkm58rLPt15xLhsZRAvlxcDHkrx5fnw48IcD80jaPM5Ock2gcOG+Y2du+0cEtg3ZtJKc3nblTdrSuU+1vdVGZ5Ik7dxcdivpRyXJjwG3nx8e3/brI/NoepKc1PbAlZ9XO5aWS7Ivs8lgAP/S9tMj80jaHJIcyGwj2FsBnwL2BB7S9tShwRaAM683r+8n2aftZ5cPJtkH+P6gTJqgJKcxf/O3mqX+19JySfYEngbsy2zjLADaHrLVH5LgCaMDSNo0zmHW4mw34GZJbtb2Q4MzaVqunuRBzNrgXT3JYfPxAFcbF0tTNy9WW7CW9KP2aeBtwPeAs4B/AP59aKIF4czrTSrJfZi90XkecOJ8+GDgGcCvusO2lsxfaFwH+I8Vp24IfLPt5zY+laYuyXuAvwN+E/gl4OeB/2r7tKHBtBCS3LPte0fnkLSYkvwC8FTgBsApwB2Bj/kCVcsled22zrd9zEZlkSQpyZuA7wJ/PR96OHD1toePS7UYLF5vYkluDfwWF/V4Ox3447anjUulqUlyLPCMlddFktsAz297/zHJNGVJTmx7UJJTl2bnJzm+7e1GZ9P0JflK2xuNziFpMc1Xjd0O+Hjb2ya5JbN7lsO286PSJST5+bZHjc4hSdrckny67b7bG9Ml2TZkE2t7epJj2/788vEkh7d989Z+Tjud66z2QqPtaUluvPFxtCDOm//7jSQ/A3wduMbAPJqYJMds7RRwzY3MImnTOaftOUlIcsW2n0lyi9GhtLCeCli8liRd1k5Kcse2HwdIcgfghMGZFoLF683vGcDKQvVqY9p5XX0b53bfsBRaNM9LcjXgN5i1KNoC/NrYSJqYnwAeCfzfivFw0SZrkrQjvprk6sx6Rb43yXeALw/OpMWV0QEkSTuFg4CPJvnK/PhGwBlL+5C539jWWbzepOY9r+8LXD/JS5ed2gKcPyaVJuqEJI9v++rlg/N+kidu5We0k2t77PzjmcDdR2bRZH0c+F7bD648keSMAXkkbRJtHzT/+Owk72e2+d4/DYykxWYfTUnSRvjp0QEWlT2vN6n58oNbAs8Bfn/ZqbOA97f9zpBgmpwk12G24+0PuPjmnlcAHtT2m6OyabqS/BGzDWG/z6xgsB/wa23/amgwSdJOIckezDaXvnAyTtuTxiXSokpyctsDRueQJEmrs3i9SSU5qe2BSf6m7cNH59H0Jbk7F23u+am2/zIyj6YtySnzTbIeBNwP+HXgQ233HxxNE5VkC7AP8AVfoEq6NJI8F3g08AXgh/Phtj1kWCgtrCQvb/vk0TkkSdLqbBuyeV0hycOBOyS5xM7rbd86IJMmrO37gfePzqGFsfT342eAN7c9M7FlpC6S5K+AX2377ST3Bl4N/DuwT5LfdONgSZfCEcBN2/5gdBBNW5JdgT3afnt+fAVmLz5+re2PA1i4liRp2ixeb16/BDyC2WZ8919xroDFa0mXxrFJPsOsbcgTk+wJnDM4k6Zl/6ViAfAs4CfbfinJtYD34cbBknbc6czucf9zdBBNV5KfBV4JnJ3ks8AfAn8JHM/sOUmSJC0A24Zsckke1/a1o3NI2nySXAM4s+0FSa4EbLFHupYk+RRwp7bfTfIRZsXrHy6da3ursQklLaokBwNvZ1bEPndpvO0DhoXS5CQ5HTi07eeSHAh8DHhI238cHE2SJK2DxetNbr407peAn5wPfRD4i7bnjUsladEluTzwRPzdoq1IcgTwNOD/AbcAbgYcA9wd+O+2vzEwnqQFNn859krgNC7qeU3bDw4LpclZ2gNo2fHpbW+9rZ+RJEnTY/F6k0vyGuDywFHzoUcBF7T9hXGpJC06f7doLZLcDHg8cHNmrcq+CvxD23cPDSZpoSU5vu3tRufQtCX5KvCSZUO/vvy47Usu8UOSJGlyLF5vckk+2Xb/7Y1J0nr4u0VrleQoZhs3fmd+vAfw4raPHZtM0qJK8hJm7UKO4eJtQ04aFkqTk+RZ2zrf9g82KoskSdpxbti4+V2Q5KZtPw+Q5CbABYMzSVp8/m7RWu23VLgGaPudJAeMDCRp4S39DrnjsrEChwzIoomyOC1J0uZg8Xrz+03g/Um+MD++MfCYcXEkbRK/xUW/WwLshb9btLpdkuyxbOb1NfD+Q9Kl0PbuozNo+pK8qe0R888vbPu0Zefe0/Ze49JJkqS18uFx87smcGtmRetDgTsBZ44MJGnxtX1fkn2YbcQHcEbbc7f1M9ppvRj4WJI3z48PB/5wYB5JCy7JU4HXAWcBrwYOBJ7e9j1Dg2lq9ln2+Z7MNhFesucGZ5EkSTtol9EBdJn7vbbfBbYAdwdeDvz52EiSFl2S3YAnAc8GngU8cT4mXUzbo4HDgG/Nvw5r+4axqSQtuMfO72/vxWyixqOAF4yNpAna1uZObvwkSdKCcOb15rfUg/ZngFe3fUeS540MJGlTOJrZjLeXzY8fDryB2axa6WLafhr49OgckjaNzP+9L3B0208lybZ+QDulK833WNgF2H3+OfOv3YcmkyRJa5bWl86bWZJjga8xWyp3IPB94Li2+w8NJmmhJfl02323NyZJ0o9aktcB1wf2BvYHdgU+0PagocE0KUk+wDZmWNs7XZKkxWDxepNLciXgp4HT2n42yY8Bt7EnoKRLI8lfAS9v+/H58R2AJ7X9ubHJJEmbXZJdgNsCX2j7v0muCVy/7amDo0mSJOlHzOK1JGndkvwbs80avzIfuhFwBnA+0Lb7jcomSdr8klwf2ItlbRDbfmhcIk3NfGPpFwE3A04DfrPt18amkiRJ62XxWpK0bkn22tb5tl/eqCySpJ1LkhcCD2XWS39pf5e2fcC4VJqaJB9mtkfHh4AHAHdqe9jYVJIkab0sXkuSdkiS/YGfmB9+uO0nR+aRJO0ckpwB7Nf23NFZNF1JTml722XHJ7U9cGQmSZK0fruMDiBJWjxJngr8NXDt+ddfJfmVsakkSTuJLwCXHx1Ck7dbkgOSHJjkQGD3FceSJGkBOPNakrRuSU5ltvz27PnxlYGP2etaknRZS/IWYH/gfcCFs6/bPmVYKE1Okvdv43TbHrJhYSRJ0g673Pa/RZKkSwgX9Rll/jmDskiSdi7HzL+krWp7962dS+LMfUmSFoTFa0nSjngd8Ikkb5sfHwq8dmAeSdJOou1RozNo8SQJcAjwcOB+wHXGJpIkSWth2xBJ0rok2QW4I3AOcNf58IfbnjwulSRpZ5FkH+BIYF9gt6XxtjcZFkqTleSOzArWhwLXAJ4EHNP2O0ODSZKkNbF4LUlatyQntz1gdA5J0s4nyUeAZwF/AtwfeAywS9vfHxpMk5Lk+cDhwFeANwJvA05ou/fQYJIkaV12GR1AkrSQ3pfkwfMluJIkbaTd276P2UScL7d9NvAzgzNpen4B+Bbw58Ab2v434MwtSZIWjD2vJUk74gnArwPnJzmH2WaNbbtlbCxJ0k7g3HkLq88meTLwNeAqgzNpen4MuCfwMOBPk7wf2D3J5dqePzaaJElaK9uGSJIkSVoYSW4H/BtwdeC5wBbgRW0/PjSYJivJFZlt0vgw4CeA97V9+NhUkiRpLSxeS5LWbL5J1h8DNwVOBX6r7dfGppIk7SyS7Aq8sO1vjs6iaUuyG/BLwM2Y3bP8Zdvzk2wBDm179NCAkiRpTSxeS5LWLMmHgaOBDwEPAO7U9rCxqSRJO4Oldg9JPt72jqPzaNqS/B1wHvBh4D7Al9r+6thUkiRpvSxeS5LWLMkpbW+77PiktgeOzCRJ2jks/c1J8ufA9YE3A2cvnW/71mHhNDlJTmt7m/nnywHHec8iSdLiccNGSdJ67JbkAGYbNMJs46MLj9ueNCyZJGlnsRvw38AhQJlvGgxYvNZy5y19mM/YH5lFkiTtIGdeS5LWLMn7t3G6bQ/ZsDCSpJ1Kkq8CL+GiYvXyamTbvmRIME1Skgu4aGZ+gN2B780/t+2WUdkkSdLaOfNakrRmbe8Os02Q2p6z/Nx8YyRJki4ruwJX4eJF6yXOyNHFtN11Ld+XZI+237ms80iSpB3jzGtJ0rqt1uva/teSpMuSf2d0WfC6kiRp2px5LUlasyTXZbZJ1u4rel9vAa40LJgkaWdg02JdFryuJEmaMIvXkqT1uDfwaOAGzPqOLjkLeOaIQJKkncY9RgfQpuRSZEmSJsy2IZKkdUvy4LZvGZ1DkiTp0rBtiCRJ0+bMa0nSjjg2ycOBG7Psb0nb5wxLJEmStH62DZEkacJ2GR1AkrSQ3g48EDgfOHvZlyRJ0nBJDln2ee8V5w5bdmg7GkmSJsy2IZKkdUtyettbj84hSZK0muXtQFa2BrFViCRJi8OZ15KkHfHRJLcZHUKSJGkrspXPqx1LkqSJsue1JGlH3BV4dJIvAucyewhs2/3GxpIkSQKgW/m82rEkSZooi9eSpB1xn9EBJEmStuEmSY5h9oJ96TPz4723/mOSJGlK7HktSdohSe4K7NP2dUn2BK7S9oujc0mSJCX5qW2db/vBjcoiSZJ2nMVrSdK6JXkWcDBwi7Y3T3I94M1t7zI4miRJEkle3/bRo3NIkqRLxw0bJUk74kHAA4CzAdp+Hbjq0ESSJEkXcR8OSZI2AXteS5J2xA/aNkkBklx5dCBJkqRlrpTkAGY9ri+h7UkbnEeSJO0Ai9eSpB3xpiSvBK6e5PHAY4FXD84kSZK05PrAi1m9eF3gkI2NI0mSdoQ9ryVJOyTJPYF7MXsofHfb9w6OJEmSBECSk9seMDqHJEm6dCxeS5J2WJItLFvF0/Z/BsaRJEkCtl28TnK7tsdvdCZJkrR+tg2RJK1bkicAfwCcA/yQ2ezrAjcZmUuSJGnuacsPkuwLPGz+9b/AwSNCSZKk9XHmtSRp3ZJ8FrhT22+PziJJkrSaJDfmooL1ecBewMFtvzQulSRJWo9dRgeQJC2kzwPfGx1CkiRpNUk+BryD2WrjB7c9CDjLwrUkSYvFtiGSpB3xDOCjST4BnLs02PYp4yJJkiRd6FvA9YHrAHsCn2XW4kySJC0Q24ZIktYtyXHAR4DTmPW8BqDtUcNCSZIkLZPkasBhzNqG7ANcHbh32+OGBpMkSWtm8VqStG5JTm57wOgckiRJa5Hk2sARzArZN2p7w8GRJEnSGli8liStW5LnA18C/pGLtw35n1GZJEmS1iLJb7b949E5JEnS9lm8liStW5IvrjLctjfZ8DCSJEnrkOQrbW80OockSdo+N2yUJK1b271HZ5AkSdpBGR1AkiStjcVrSdIOSXJn4MYs+1vS9uhhgSRJktbG5ceSJC0Ii9eSpHVL8gbgpsApwAXz4QIWryVJ0nBJTuP/b+9OY+UsyzCO/69Cy74FgkRA9lVDWQRBxYIEISgECCjGiEgwihqIQVBMDBAiwSXIFg0aAREJJIgEPwgoBIVo2SqVsoTNlgIixgooDfvth3knHIc57Zk57XmH9v9LTt6Z53mX650vZ859nrmnf5E6wLumOI4kSRqSPa8lSQNL8hCwc/lLRJIkjaAkWyxpvqoWTFUWSZI0PFdeS5KGMQ/YBPh720EkSZJ6VdWCJIcD2wL3V9VNbWeSJEmDs3gtSRrGRsCDSe4CXukOVtVh7UWSJEnqSPIj4L3An4Czk+xVVWe3HEuSJA3ItiGSpIElmdVvvKr+MNVZJEmSeiWZB8ysqjeSrAncXlV7tJ1LkiQNxpXXkqSBWaSWJEkj7tWqegOgqhYnSduBJEnS4Fx5LUkaWJL/AL2/QF4A7gFOqaonpj6VJElSR5LFwGPdp8A2zfMAVVW7tJVNkiRNnCuvJUnDOB94CriKzh+Bx9D5o3AOcCmwX2vJJEmSYKe2A0iSpMlz5bUkaWBJ5lbVzJ6x+6pq135zkiRJkiRJg3LltSRpGIuTfBK4tnl+FPBy89j/ikqSpFb1aXGW5nm3bci6rQSTJEkDceW1JGlgSbYGLgD2ofOH4Gzga8DTwB5VdUeL8SRJ0kouyfXAJsB1wNVV9WTLkSRJ0hAsXkuSJEmSVjhJ1gOOpPPdHKsD19ApZC9qNZgkSZowi9eSpAlLclpVfS/JRfRpD1JVJ7UQS5IkaVxJptEpYF8InFNV57UcSZIkTZA9ryVJg3io2d7TagpJkqSlSPJB4NPAvsAdwBFVdXu7qSRJ0iBceS1JmpRmNdPaVfVi21kkSZIAkswHngeuBm4FXh87X1VzWoglSZIGZPFakjSwJFcBXwLeAO4G1gUuqKrvtxpMkiQJSHIbfVqcNaqqPjqFcSRJ0pAsXkuSBpbkvqraNclngN2BbwL3VtUuLUeTJEmSJEkriGltB5AkvSNNTzIdOBy4oapeY/zVTZIkSVMqyWljHh/dM3fO1CeSJEnDsHgtSRrGJcB8YC3gj0m2AOx5LUmSRsUxYx6f3jN38FQGkSRJw1u17QCSpHeeqroQuHDM0IIk+7eVR5IkqUfGedzvuSRJGlGuvJYkDSzJyUnWTcfPkswB/OIjSZI0Kmqcx/2eS5KkEeUXNkqSBpZkblXNTHIQ8EXg28Avqmr3lqNJkiSR5E3gv3RWWa8BLO5OAatX1fS2skmSpImzbYgkaRjdj9seQqdo/UASP4IrSZJGxdyq2q3tEJIkaXJsGyJJGsa9SW6mU7y+Kck6wJstZ5IkSeryI8aSJK0AbBsiSRpYkmnArsB0YDVgI2DTqrqo1WCSJElAkqeA88abr6px5yRJ0uiwbYgkaRjHAycDmwH3AXsDfwYsXkuSpFGwCrA2b7U6kyRJ70CuvJYkDSzJ/cCewOyq2jXJjsA5VXVky9EkSZJIMscvkpYk6Z3PnteSpGG8XFUvAyRZraoeBnZoOZMkSVKXK64lSVoB2DZEkjSMp5KsD1wP/C7Jv4EFLWeSJEnqOqDtAJIkafJsGyJJmpQks4D1gBur6tW280iSJEmSpBWDxWtJkiRJkiRJ0six57UkSZIkSZIkaeRYvJYkSZIkSZIkjRyL15IkSdISJNkySSW5vGf88mZ8y1aCSZIkSSs4i9eSJElaaTXF5yX9HNd2RkmSJGlltWrbASRJkqQRcNY44/cBTwM7AS9MXRxJkiRJFq8lSZK00quqM5eyy8NTkUOSJEnSW2wbIkmSJC3BeD2vl3LMB5Jcm+TZJK8mWZjkkiTv7rPvbc35V03yrSSPJnmlOea7SWb0OebwJFcmeSTJS83PvUlOSvK29/hj+nNvleSrSR5M8nKS+c010+x3dJK7mvM9l+TiJGuMc487Nudd2NzjP5JclWSHib5OkiRJ0pK48lqSJElahpIcD/wEeAW4AVgIbAecAByaZO+qerLPoVcB+wK/BV4EDgFOAzYGPt+z77nAm8CddNqarAd8FLgA2BP47DjxfgDsB/wGuBk4DPgOMCPJoua81wO3AwcCXwFWAU7suceDgeuA6c25HgM2A44EPp5k/6qaM/6rJEmSJC1dqqrtDJIkSVIrknTfDPfreT2/qi5PsiXwN+DnVXXcmGMvBz4HbFVV85ux7YF5wJPArKp6esz+B9ApGN9QVUeMGb8NmAXMAQ6sqkXN+FrAXGArYNOqenbMMdtU1eM99zINuAw4Fti7qu7sk3UB8KFuriTr0yk8rwEsBj5SVQ81c6sBfwG2ATavquea8Q2AJ4A3mv0fHHOd9wGzgUeqavc+r6kkSZI0YbYNkSRJkuCMPj/HDXGeE+msRj55bOEaoKpuobMS+9Ak6/Q59hvdwnWz/0vAL+m8Z39/z7ke7zmWqnqTzsprgIPGyXf22FxV9XyTaU3gx93CdTP3CnANMIPOF1Z2HQusD5wxtnDdHDMP+CmwW5Kdx8kgSZIkTYhtQyRJkrTSq6oso1Pt02xnJdmzz/zGdNpwbA/c2zN3T5/9FzbbDcYOJtlYDSURAAACOklEQVQQOJVOa5GtgbV6jtt0nHz9rvFMs+3NA52WJNBpCdLVvceZSc7sc8z2zXYn4ME+85IkSdKEWLyWJEmSlp0Nm+2pS9lv7d6BZhV0r9eb7SrdgabVx9102oncBVwBLGr2XR84GVhtnOu+sIRrLGlu+pix7j1+YZxrdL3tHiVJkqRBWLyWJEmSlp1uAXi9qnpxOV3jBDqF67Oq6syxE0n2oVO8Xp669zizqv66nK8lSZKklZg9ryVJkqRlZ3az3Xc5XmPbZvurPnOzluN1u6biHiVJkiSL15IkSdIydDHwGvDDJNv3TiaZkWSyRd/5zXa/nnPvBpw+yXNPxGXA88AZSfbqnUwyLcl+bztKkiRJGpBtQyRJkqRlpKoeTnI8cCnwQJIbgUfo9Ix+D53Vyv8EdpzEZa6g01P7/CT7A48C2wGfAK4DPjWJcy9VVf0ryVHAr4HZSW4BHgAK2JzOFzpuCKy+PHNIkiRpxWfxWpIkSVqGqurKJHOBU4D9gY8BLwHPANcC10zy/M80q7fPBT4MHAQ8DHwZ+D3LuXjdZLglyS7A15vr7wu8Suceb6V/SxNJkiRpIKmqtjNIkiRJkiRJkvR/7HktSZIkSZIkSRo5Fq8lSZIkSZIkSSPH4rUkSZIkSZIkaeRYvJYkSZIkSZIkjRyL15IkSZIkSZKkkWPxWpIkSZIkSZI0cixeS5IkSZIkSZJGjsVrSZIkSZIkSdLIsXgtSZIkSZIkSRo5Fq8lSZIkSZIkSSPnfxrqj6hOBVdDAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pQgeOObvgLvK"
      },
      "source": [
        "Now let's setup [Altair](https://altair-viz.github.io/), and plot the data with Altair. Altair is useful for creating vizualizations since they can be easily exported as a PNG or SVG."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q7Z4J6qjWaVM"
      },
      "source": [
        "import altair as alt"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 947
        },
        "id": "s0xwvILYWkgg",
        "outputId": "3fdbfacf-3091-46ad-835e-421dac8e19d2"
      },
      "source": [
        "top_filenames_altair = (\n",
        "    word_documents[\"filename\"]\n",
        "    .value_counts()\n",
        "    .head(10)\n",
        "    .rename_axis(\"Filename\")\n",
        "    .reset_index(name=\"Count\")\n",
        ")\n",
        "\n",
        "filenames_bar = (\n",
        "    alt.Chart(top_filenames_altair)\n",
        "    .mark_bar()\n",
        "    .encode(x=alt.X(\"Filename:O\", sort=\"-y\"), y=alt.Y(\"Count:Q\"))\n",
        ")\n",
        "\n",
        "filenames_rule = (\n",
        "    alt.Chart(top_filenames_altair).mark_rule(color=\"red\").encode(y=\"mean(Count):Q\")\n",
        ")\n",
        "\n",
        "\n",
        "filenames_text = filenames_bar.mark_text(align=\"center\", baseline=\"bottom\").encode(\n",
        "    text=\"Count:Q\"\n",
        ")\n",
        "\n",
        "(filenames_bar + filenames_rule + filenames_text).properties(\n",
        "    width=1400, height=700, title=\"Top Filenames\"\n",
        ")"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "<div id=\"altair-viz-2cea6a769db0464cb14a6545fd697f5b\"></div>\n",
              "<script type=\"text/javascript\">\n",
              "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
              "  (function(spec, embedOpt){\n",
              "    let outputDiv = document.currentScript.previousElementSibling;\n",
              "    if (outputDiv.id !== \"altair-viz-2cea6a769db0464cb14a6545fd697f5b\") {\n",
              "      outputDiv = document.getElementById(\"altair-viz-2cea6a769db0464cb14a6545fd697f5b\");\n",
              "    }\n",
              "    const paths = {\n",
              "      \"vega\": \"https://cdn.jsdelivr.net/npm//vega@5?noext\",\n",
              "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm//vega-lib?noext\",\n",
              "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm//vega-lite@4.17.0?noext\",\n",
              "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm//vega-embed@6?noext\",\n",
              "    };\n",
              "\n",
              "    function maybeLoadScript(lib, version) {\n",
              "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
              "      return (VEGA_DEBUG[key] == version) ?\n",
              "        Promise.resolve(paths[lib]) :\n",
              "        new Promise(function(resolve, reject) {\n",
              "          var s = document.createElement('script');\n",
              "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
              "          s.async = true;\n",
              "          s.onload = () => {\n",
              "            VEGA_DEBUG[key] = version;\n",
              "            return resolve(paths[lib]);\n",
              "          };\n",
              "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
              "          s.src = paths[lib];\n",
              "        });\n",
              "    }\n",
              "\n",
              "    function showError(err) {\n",
              "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
              "      throw err;\n",
              "    }\n",
              "\n",
              "    function displayChart(vegaEmbed) {\n",
              "      vegaEmbed(outputDiv, spec, embedOpt)\n",
              "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
              "    }\n",
              "\n",
              "    if(typeof define === \"function\" && define.amd) {\n",
              "      requirejs.config({paths});\n",
              "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
              "    } else {\n",
              "      maybeLoadScript(\"vega\", \"5\")\n",
              "        .then(() => maybeLoadScript(\"vega-lite\", \"4.17.0\"))\n",
              "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
              "        .catch(showError)\n",
              "        .then(() => displayChart(vegaEmbed));\n",
              "    }\n",
              "  })({\"config\": {\"view\": {\"continuousWidth\": 400, \"continuousHeight\": 300}}, \"layer\": [{\"mark\": \"bar\", \"encoding\": {\"x\": {\"field\": \"Filename\", \"sort\": \"-y\", \"type\": \"ordinal\"}, \"y\": {\"field\": \"Count\", \"type\": \"quantitative\"}}}, {\"mark\": {\"type\": \"rule\", \"color\": \"red\"}, \"encoding\": {\"y\": {\"aggregate\": \"mean\", \"field\": \"Count\", \"type\": \"quantitative\"}}}, {\"mark\": {\"type\": \"text\", \"align\": \"center\", \"baseline\": \"bottom\"}, \"encoding\": {\"text\": {\"field\": \"Count\", \"type\": \"quantitative\"}, \"x\": {\"field\": \"Filename\", \"sort\": \"-y\", \"type\": \"ordinal\"}, \"y\": {\"field\": \"Count\", \"type\": \"quantitative\"}}}], \"data\": {\"name\": \"data-8a4b8142a39ce16d719ba0421fc3b2c0\"}, \"height\": 700, \"title\": \"Top Filenames\", \"width\": 1400, \"$schema\": \"https://vega.github.io/schema/vega-lite/v4.17.0.json\", \"datasets\": {\"data-8a4b8142a39ce16d719ba0421fc3b2c0\": [{\"Filename\": \"softConstraints.rtf\", \"Count\": 3}, {\"Filename\": \"ClassListF04.rtf\", \"Count\": 2}, {\"Filename\": \"assignmentProposalTemplate.odt\", \"Count\": 1}, {\"Filename\": \"c61-summary.odt\", \"Count\": 1}, {\"Filename\": \"1855\", \"Count\": 1}, {\"Filename\": \"Transmag-ieeeTvlsi.rtf\", \"Count\": 1}, {\"Filename\": \"TEMPLATE_PAPER_ECE700T02.rtf\", \"Count\": 1}, {\"Filename\": \"cae_bibtex.rtf\", \"Count\": 1}, {\"Filename\": \"edgar.ppt\", \"Count\": 1}, {\"Filename\": \"petition_en.odt\", \"Count\": 1}]}}, {\"mode\": \"vega-lite\"});\n",
              "</script>"
            ],
            "text/plain": [
              "alt.LayerChart(...)"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BneaN9cgGoly"
      },
      "source": [
        "### How about a file format distribution?\n",
        "\n",
        "What _kind_ of word documents files are present? We can discover this by checking their \"media type\", or [MIME type](https://en.wikipedia.org/wiki/Media_type). \n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 959
        },
        "id": "RDd-J8D-GwDk",
        "outputId": "1f9475fe-69fc-4992-da10-bc191e745a25"
      },
      "source": [
        "word_documents_mime_types = (\n",
        "    word_documents[\"mime_type_tika\"]\n",
        "    .value_counts()\n",
        "    .head(5)\n",
        "    .rename_axis(\"MIME Type\")\n",
        "    .reset_index(name=\"Count\")\n",
        ")\n",
        "\n",
        "word_documents_mimes_bar = (\n",
        "    alt.Chart(word_documents_mime_types)\n",
        "    .mark_bar()\n",
        "    .encode(x=alt.X(\"MIME Type:O\", sort=\"-y\"), y=alt.Y(\"Count:Q\"))\n",
        ")\n",
        "\n",
        "word_documents_mime_rule = (\n",
        "    alt.Chart(word_documents_mime_types).mark_rule(color=\"red\").encode(y=\"mean(Count):Q\")\n",
        ")\n",
        "\n",
        "word_documents_mime_text = word_documents_mimes_bar.mark_text(align=\"center\", baseline=\"bottom\").encode(\n",
        "    text=\"Count:Q\"\n",
        ")\n",
        "\n",
        "(word_documents_mimes_bar + word_documents_mime_rule + word_documents_mime_text).properties(\n",
        "    width=1400, height=700, title=\"word_documents (e.g., .doc, .odt, .rtf, .wpd) File Format Distribution\"\n",
        ")"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "<div id=\"altair-viz-b6798c46e36b42e793cf99e66db79f58\"></div>\n",
              "<script type=\"text/javascript\">\n",
              "  var VEGA_DEBUG = (typeof VEGA_DEBUG == \"undefined\") ? {} : VEGA_DEBUG;\n",
              "  (function(spec, embedOpt){\n",
              "    let outputDiv = document.currentScript.previousElementSibling;\n",
              "    if (outputDiv.id !== \"altair-viz-b6798c46e36b42e793cf99e66db79f58\") {\n",
              "      outputDiv = document.getElementById(\"altair-viz-b6798c46e36b42e793cf99e66db79f58\");\n",
              "    }\n",
              "    const paths = {\n",
              "      \"vega\": \"https://cdn.jsdelivr.net/npm//vega@5?noext\",\n",
              "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm//vega-lib?noext\",\n",
              "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm//vega-lite@4.17.0?noext\",\n",
              "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm//vega-embed@6?noext\",\n",
              "    };\n",
              "\n",
              "    function maybeLoadScript(lib, version) {\n",
              "      var key = `${lib.replace(\"-\", \"\")}_version`;\n",
              "      return (VEGA_DEBUG[key] == version) ?\n",
              "        Promise.resolve(paths[lib]) :\n",
              "        new Promise(function(resolve, reject) {\n",
              "          var s = document.createElement('script');\n",
              "          document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
              "          s.async = true;\n",
              "          s.onload = () => {\n",
              "            VEGA_DEBUG[key] = version;\n",
              "            return resolve(paths[lib]);\n",
              "          };\n",
              "          s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
              "          s.src = paths[lib];\n",
              "        });\n",
              "    }\n",
              "\n",
              "    function showError(err) {\n",
              "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
              "      throw err;\n",
              "    }\n",
              "\n",
              "    function displayChart(vegaEmbed) {\n",
              "      vegaEmbed(outputDiv, spec, embedOpt)\n",
              "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
              "    }\n",
              "\n",
              "    if(typeof define === \"function\" && define.amd) {\n",
              "      requirejs.config({paths});\n",
              "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
              "    } else {\n",
              "      maybeLoadScript(\"vega\", \"5\")\n",
              "        .then(() => maybeLoadScript(\"vega-lite\", \"4.17.0\"))\n",
              "        .then(() => maybeLoadScript(\"vega-embed\", \"6\"))\n",
              "        .catch(showError)\n",
              "        .then(() => displayChart(vegaEmbed));\n",
              "    }\n",
              "  })({\"config\": {\"view\": {\"continuousWidth\": 400, \"continuousHeight\": 300}}, \"layer\": [{\"mark\": \"bar\", \"encoding\": {\"x\": {\"field\": \"MIME Type\", \"sort\": \"-y\", \"type\": \"ordinal\"}, \"y\": {\"field\": \"Count\", \"type\": \"quantitative\"}}}, {\"mark\": {\"type\": \"rule\", \"color\": \"red\"}, \"encoding\": {\"y\": {\"aggregate\": \"mean\", \"field\": \"Count\", \"type\": \"quantitative\"}}}, {\"mark\": {\"type\": \"text\", \"align\": \"center\", \"baseline\": \"bottom\"}, \"encoding\": {\"text\": {\"field\": \"Count\", \"type\": \"quantitative\"}, \"x\": {\"field\": \"MIME Type\", \"sort\": \"-y\", \"type\": \"ordinal\"}, \"y\": {\"field\": \"Count\", \"type\": \"quantitative\"}}}], \"data\": {\"name\": \"data-e796ee2a40a2345ece2c452305fa57f3\"}, \"height\": 700, \"title\": \"word_documents (e.g., .doc, .odt, .rtf, .wpd) File Format Distribution\", \"width\": 1400, \"$schema\": \"https://vega.github.io/schema/vega-lite/v4.17.0.json\", \"datasets\": {\"data-e796ee2a40a2345ece2c452305fa57f3\": [{\"MIME Type\": \"application/rtf\", \"Count\": 10}, {\"MIME Type\": \"application/vnd.oasis.opendocument.text\", \"Count\": 7}, {\"MIME Type\": \"application/msword\", \"Count\": 5}]}}, {\"mode\": \"vega-lite\"});\n",
              "</script>"
            ],
            "text/plain": [
              "alt.LayerChart(...)"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QUJR-jjqNxCL"
      },
      "source": [
        "### How do I get the actual word documents?\n",
        "\n",
        "...or, how do I get to the actual binary files described by each file format information derivative?\n",
        "\n",
        "There are a few options!\n",
        "\n",
        "1. `wget` or `curl` from the live URL, or a replay URL\n",
        "  * Live web URL\n",
        "    * `wget` or `curl` the value of the `url` column\n",
        "  * Replay web URL\n",
        "    * `wget` or `curl` the value of the `crawl_date` and `url` column using the following pattern:\n",
        "      * `https://web.archive.org/web/` + `crawl_date` + `/` + `url`\n",
        "        * https://web.archive.org/web/20120119124734/http://www.archive.org/images/glogo.png\n",
        "      * `http://wayback.archive-it.org/14462/` + `crawl_date` + `/` + `url`\n",
        "        * https://wayback.archive-it.org/14462/20210524212740/https://ruebot.net/visualization/elxn42/featured_hu33a17dfb90e2c5ed77f783db14a6e53a_5126291_550x0_resize_q90_box_2.png\n",
        "2. Use a scripting language, such as Python\n",
        "  * Make use of the `url` and `filename` columns (and `crawl_date` if you want to use the replay URL)\n",
        "  * `import requests`\n",
        "  * `requests.get(url, allow_redirects=True)`\n",
        "  * `open('filename', 'wb').write(r.content)`\n",
        "3. Use the [Archives Unleashed Toolkit](https://aut.docs.archivesunleashed.org/docs/extract-binary) (if you have access to the W/ARC files)."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you wanted to download the word documents files using the replay URL, below a method for doing so.\n",
        "\n",
        "First, you'll want to setup a replay url base url. Here we'll use the Archive-It Wayback instance for the collection."
      ],
      "metadata": {
        "id": "8yoFE2xLAlwk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "wayback_url = 'http://wayback.archive-it.org/89/'"
      ],
      "metadata": {
        "id": "RxrOHn_2AicZ"
      },
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next we'll create a new column using a lambda function. If you're familiar with working in word documentss, what we're doing here is basically concatenating some column values together and creating a new column."
      ],
      "metadata": {
        "id": "jCTmrEBGA-Yd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "word_documents['replay_url'] = word_documents.apply(lambda row: str(wayback_url + str(row['crawl_date']) + \"/\" + row['url']), axis=1)"
      ],
      "metadata": {
        "id": "p5HoWxkFA6C8"
      },
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Then we can export that new column we created out to a file, so we can use it with `wget` to download all the word documents files!"
      ],
      "metadata": {
        "id": "q87tSYyIBIiZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "word_documents['replay_url'].head().to_csv('89_word_documents_urls.txt', index=False, header=False)"
      ],
      "metadata": {
        "id": "wyhpsf9wBN1a"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, we can pass the file to `wget` to use as a download list. You can also speed this process up using `xargs` or `parallel`."
      ],
      "metadata": {
        "id": "81a3q7feu5Ji"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget --random-wait -i 89_word_documents_urls.txt"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D_p4qGKoBRaZ",
        "outputId": "f6777ceb-b85b-48c2-efa9-960562881a6b"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-11-24 18:09:29--  http://wayback.archive-it.org/89/20200703084330/http://kremer.cpsc.ucalgary.ca/courses/cpsc433/F2014/assignment/assignmentProposalTemplate.odt\n",
            "Resolving wayback.archive-it.org (wayback.archive-it.org)... 207.241.232.8, 207.241.225.8\n",
            "Connecting to wayback.archive-it.org (wayback.archive-it.org)|207.241.232.8|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [text/html]\n",
            "Saving to: ‘assignmentProposalTemplate.odt.1’\n",
            "\n",
            "\r          assignmen     [<=>                 ]       0  --.-KB/s               \rassignmentProposalT     [ <=>                ]   4.50K  --.-KB/s    in 0s      \n",
            "\n",
            "2022-11-24 18:09:29 (20.3 MB/s) - ‘assignmentProposalTemplate.odt.1’ saved [4613]\n",
            "\n",
            "--2022-11-24 18:09:29--  http://wayback.archive-it.org/89/20200708003621/http://kremer.cpsc.ucalgary.ca/courses/cpsc433/W2006/softConstraints.rtf\n",
            "Reusing existing connection to wayback.archive-it.org:80.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [text/html]\n",
            "Saving to: ‘softConstraints.rtf.3’\n",
            "\n",
            "softConstraints.rtf     [ <=>                ]   4.50K  --.-KB/s    in 0.001s  \n",
            "\n",
            "2022-11-24 18:09:29 (8.29 MB/s) - ‘softConstraints.rtf.3’ saved [4613]\n",
            "\n",
            "--2022-11-24 18:09:29--  http://wayback.archive-it.org/89/20200708015505/http://kremer.cpsc.ucalgary.ca/courses/cpsc433/F2006/softConstraints.rtf\n",
            "Reusing existing connection to wayback.archive-it.org:80.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [text/html]\n",
            "Saving to: ‘softConstraints.rtf.4’\n",
            "\n",
            "softConstraints.rtf     [ <=>                ]   4.50K  --.-KB/s    in 0s      \n",
            "\n",
            "2022-11-24 18:09:29 (24.3 MB/s) - ‘softConstraints.rtf.4’ saved [4613]\n",
            "\n",
            "--2022-11-24 18:09:29--  http://wayback.archive-it.org/89/20200708053559/http://kremer.cpsc.ucalgary.ca/courses/cpsc433/F2013/softConstraints.rtf\n",
            "Reusing existing connection to wayback.archive-it.org:80.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [text/html]\n",
            "Saving to: ‘softConstraints.rtf.5’\n",
            "\n",
            "softConstraints.rtf     [ <=>                ]   4.50K  --.-KB/s    in 0.003s  \n",
            "\n",
            "2022-11-24 18:09:30 (1.34 MB/s) - ‘softConstraints.rtf.5’ saved [4613]\n",
            "\n",
            "--2022-11-24 18:09:30--  http://wayback.archive-it.org/89/20200710021203/https://www.pjl.ucalgary.ca/data/safety/placards/Board-Sign.odt\n",
            "Reusing existing connection to wayback.archive-it.org:80.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [text/html]\n",
            "Saving to: ‘Board-Sign.odt.1’\n",
            "\n",
            "Board-Sign.odt.1        [ <=>                ]   4.50K  --.-KB/s    in 0s      \n",
            "\n",
            "2022-11-24 18:09:30 (37.5 MB/s) - ‘Board-Sign.odt.1’ saved [4613]\n",
            "\n",
            "FINISHED --2022-11-24 18:09:30--\n",
            "Total wall clock time: 0.5s\n",
            "Downloaded: 5 files, 23K in 0.004s (5.08 MB/s)\n"
          ]
        }
      ]
    }
  ]
}