notebooks/extraction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text and Tables Extraction\n",
    "\n",
    "This notebook presents how to use our pipeline to extract text and tables from arXiv papers with available LaTeX source code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "from axcell.helpers.paper_extractor import PaperExtractor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Structure of Directories\n",
    "\n",
    "We cache the artifacts produced by successful execution of the intermediate steps of extraction pipeline. The `root` argument of `PaperExtractor` is a path under which the following directory structue is created:\n",
    "\n",
    "```\n",
    "root\n",
    "├── sources                       # e-print archives\n",
    "├── unpacked_sources              # extracted latex sources (generated automatically)\n",
    "├── htmls                         # converted html files (generated automatically)\n",
    "└── papers                        # extracted text and tables (generated automatically)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "ROOT_PATH = Path('data')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In our case there's a single e-print archive:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[01;36mdata\u001b[00m\r\n",
      "└── \u001b[01;34msources\u001b[00m\r\n",
      "    └── \u001b[01;34m1903\u001b[00m\r\n",
      "        └── 1903.11816v1\r\n",
      "\r\n",
      "2 directories, 1 file\r\n"
     ]
    }
   ],
   "source": [
    "!tree {ROOT_PATH}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "extract = PaperExtractor(ROOT_PATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To extract text and tables from a single paper just pass the path to the archive:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'success'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SOURCES_PATH = ROOT_PATH / 'sources'\n",
    "extract(SOURCES_PATH / '1903' / '1903.11816v1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The subdirectory structure under `sources` directory will be replicated in the other top-level directories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[01;36mdata\u001b[00m\r\n",
      "├── \u001b[01;34mhtmls\u001b[00m\r\n",
      "│   └── \u001b[01;34m1903\u001b[00m\r\n",
      "│       └── \u001b[01;34m1903.11816v1\u001b[00m\r\n",
      "│           └── index.html\r\n",
      "├── \u001b[01;34mpapers\u001b[00m\r\n",
      "│   └── \u001b[01;34m1903\u001b[00m\r\n",
      "│       └── \u001b[01;34m1903.11816v1\u001b[00m\r\n",
      "│           ├── layout_01.csv\r\n",
      "│           ├── layout_02.csv\r\n",
      "│           ├── layout_03.csv\r\n",
      "│           ├── layout_04.csv\r\n",
      "│           ├── layout_05.csv\r\n",
      "│           ├── metadata.json\r\n",
      "│           ├── table_01.csv\r\n",
      "│           ├── table_02.csv\r\n",
      "│           ├── table_03.csv\r\n",
      "│           ├── table_04.csv\r\n",
      "│           ├── table_05.csv\r\n",
      "│           └── text.json\r\n",
      "├── \u001b[01;34msources\u001b[00m\r\n",
      "│   └── \u001b[01;34m1903\u001b[00m\r\n",
      "│       └── 1903.11816v1\r\n",
      "└── \u001b[01;34munpacked_sources\u001b[00m\r\n",
      "    └── \u001b[01;34m1903\u001b[00m\r\n",
      "        └── \u001b[01;34m1903.11816v1\u001b[00m\r\n",
      "            ├── eso-pic.sty\r\n",
      "            ├── iccv.sty\r\n",
      "            ├── iccv_eso.sty\r\n",
      "            ├── ieee.bst\r\n",
      "            ├── \u001b[01;34mimages\u001b[00m\r\n",
      "            ├── submission_465.bbl\r\n",
      "            └── submission_465.tex\r\n",
      "\r\n",
      "12 directories, 20 files\r\n"
     ]
    }
   ],
   "source": [
    "!tree -L 4 {ROOT_PATH}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The extracted data is stored in `papers` directory. We can read it using `PaperCollection` class. `PaperCollection` is a wrapper for `list` of papers with additional functions added for convenience. Due to large number of papers it is recommended to load the dataset in parallel (default uses number of processes equal to number of CPU cores) and store it in a pickle file. Set jobs=1 to disable multiprocessing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from axcell.data.paper_collection import PaperCollection\n",
    "\n",
    "PAPERS_PATH = ROOT_PATH / 'papers'\n",
    "pc = PaperCollection.from_files(PAPERS_PATH)\n",
    "# pc.to_pickle('mypapers.pkl')\n",
    "# pc = PaperCollection.from_pickle('mypapers.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "paper = pc.get_by_id('1903.11816')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper.text.title"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "body{margin:0;padding:0;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen,Ubuntu,Cantarell,Fira Sans,Droid Sans,Helvetica Neue,sans-serif;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}code{font-family:source-code-pro,Menlo,Monaco,Consolas,Courier New,monospace}.tableWrapper{-overflow:auto}.tableWrapper .has-annotations{color:#ff3860}.tableWrapper .model-params{background-color:#209cee;color:rgba(0,0,0,.7)}.tableWrapper .table-meta{background-color:#fff3c5;color:rgba(0,0,0,.7)}.tableWrapper .model-best{background-color:#ff3860;color:rgba(0,0,0,.7)}.tableWrapper .model-ensemble{background-color: #aa38ff;color: #fff;}.tableWrapper .model-competing{background-color:#ffdd57;color:rgba(0,0,0,.7)}.tableWrapper .model-paper{background-color:#ff3860;color:#fff}.tableWrapper .dataset-sub{background-color:#23d160;color:#fff}.tableWrapper .dataset-metric{background-color:#209cee;color:#fff}.tableWrapper .dataset{background-color:#02bd43;color:#fff}.tableWrapper .trash{background-color:#363636;color:#f5f5f5}.tableWrapper .wtf{background-color:#f0f;color:#f5f5f5}.tableWrapper .dataset-task{background-color:#77ecdd;color:rgba(0,0,0,.7)}.tableWrapper .dataset-paper{background-color:#e4ffee;color:rgba(0,0,0,.7)}.tableWrapper td.focused-cell{outline:2px solid #9ecaed;border-radius:7px;box-shadow:0 0 10px #9ecaed}.tableWrapper span.text-bold{font-weight:700}.tableWrapper span.text-italic{font-style:italic}.tableWrapper span.text-red{color:red}.tableWrapper span.text-green{color:green}.tableWrapper span.text-blue{color:#00f}.predict-dataset,.predict-dataset-metric,.predict-model-competing,.predict-model-paper,.predict-model-params,.predict-table-meta{outline:2px solid #9ecaed;border-radius:7px;box-shadow:0 0 10px #9ecaed}.tableWrapper .predict-model-params{background-color:#209cee;color:rgba(0,0,0,.7)}.tableWrapper .predict-table-meta{background-color:#fff3c5;color:rgba(0,0,0,.7)}.tableWrapper .predict-model-competing{background-color:#ffdd57;color:rgba(0,0,0,.7)}.tableWrapper .predict-model-paper{background-color:#ff3860;color:#fff}.tableWrapper .predict-dataset-metric{background-color:#209cee;color:#fff}.tableWrapper .predict-dataset{background-color:#02bd43;color:#fff}.tableWrapper td{border:inherit}.tableWrapper table tr td.border-l{border-left:1px solid #000}.tableWrapper table tr td.border-r{border-right:1px solid #000}.tableWrapper table tr td.border-t{border-top:1px solid #000}.tableWrapper table tr td.border-b{border-bottom:1px solid #000}.tableWrapper table tr td.border-ll{border-left:2px solid #000}.tableWrapper table tr td.border-rr{border-right:2px solid #000}.tableWrapper table tr td.border-tt{border-top:2px solid #000}.tableWrapper table tr td.border-bb{border-bottom:2px solid #000}.tableWrapper table tr td.align-left{text-align:left}.tableWrapper table tr td.align-right{text-align:right}.tableWrapper table tr td.align-center{text-align:center}.tableWrapper table tr td.align-justify{text-align:justify}div.form-group>input.form-control.input-sm{border-radius:2px;font-size:.75rem;background-color:#fff;color:#363636;box-shadow:inset 0 1px 2px rgba(10,10,10,.1);max-width:100%;width:100%;height:2.25em;padding:calc(.375em - 1px) calc(.625em - 1px);position:relative;border:1px solid #b5b5b5}div.form-group>input.form-control.input-sm:focus{border-color:#3273dc;box-shadow:0 0 0 .125em rgba(50,115,220,.25)}.ht_clone_top{z-index:20}.evaluation-tables{overflow:scroll;max-height:20vh;border-top:1px solid #a9a9a9}.navbar.is-fixed-bottom,.navbar.is-fixed-top{z-index:200}body{padding-bottom:20vh}\n",
       ".tableWrapper .final-proposal{ background: lightgreen }\n",
       "</style>\n",
       "\n",
       "<div class=\"tableWrapper\">\n",
       "<table>\n",
       "<tr>\n",
       "<td class=\" border-t align-center header \" title=\"\">Rank</td>\n",
       "<td class=\" border-t align-left header \" title=\"\">Team</td>\n",
       "<td class=\" border-r border-t align-center header \" title=\"\">Single Model</td>\n",
       "<td class=\" border-t align-center header \" title=\"\">Final Score</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" border-t align-center header \" title=\"\">1</td>\n",
       "<td class=\" border-t align-left header \" title=\"\">CASIA_IVA_JD</td>\n",
       "<td class=\" border-r border-t align-center \" title=\"\">✗</td>\n",
       "<td class=\" border-t align-center \" title=\"\">0.5547</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" align-center header \" title=\"\">2</td>\n",
       "<td class=\" align-left header \" title=\"\">WinterIsComing</td>\n",
       "<td class=\" border-r align-center \" title=\"\">✗</td>\n",
       "<td class=\" align-center \" title=\"\">0.5544</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" align-center header \" title=\"\">-</td>\n",
       "<td class=\" align-left header \" title=\"\">PSPNet [<a title=\"bib-bib38\">38</a>]</td>\n",
       "<td class=\" border-r align-center \" title=\"\">ResNet-269</td>\n",
       "<td class=\" align-center \" title=\"\">0.5538</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" align-center header \" title=\"\">-</td>\n",
       "<td class=\" align-left header \" title=\"\">EncNet [<a title=\"bib-bib36\">36</a>]</td>\n",
       "<td class=\" border-r align-center \" title=\"\">ResNet-101</td>\n",
       "<td class=\" align-center \" title=\"\">0.5567</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" border-b border-t align-center header \" title=\"\">-</td>\n",
       "<td class=\" border-b border-t align-left header \" title=\"\">Ours</td>\n",
       "<td class=\" border-b border-r border-t align-center \" title=\"\">ResNet-101</td>\n",
       "<td class=\" border-b border-t align-center \" title=\"\"><span class=\"text-bold\">0.5584</span></td>\n",
       "</tr>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<axcell.data.table.Table at 0x7feccce790f0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper.tables[4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As *FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation* (Wu et al., 2019) is present in our **SegmentedTables** dataset, we can use `PaperCollection` to import annotations (table segmentation and results):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from axcell.helpers.datasets import read_tables_annotations\n",
    "\n",
    "V1_URL = 'https://github.com/paperswithcode/axcell/releases/download/v1.0/'\n",
    "SEGMENTED_TABLES_URL = V1_URL + 'segmented-tables.json.xz'\n",
    "\n",
    "segmented_tables = read_tables_annotations(SEGMENTED_TABLES_URL)\n",
    "\n",
    "pc = PaperCollection.from_files(PAPERS_PATH, annotations=segmented_tables.to_dict('record'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "body{margin:0;padding:0;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen,Ubuntu,Cantarell,Fira Sans,Droid Sans,Helvetica Neue,sans-serif;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}code{font-family:source-code-pro,Menlo,Monaco,Consolas,Courier New,monospace}.tableWrapper{-overflow:auto}.tableWrapper .has-annotations{color:#ff3860}.tableWrapper .model-params{background-color:#209cee;color:rgba(0,0,0,.7)}.tableWrapper .table-meta{background-color:#fff3c5;color:rgba(0,0,0,.7)}.tableWrapper .model-best{background-color:#ff3860;color:rgba(0,0,0,.7)}.tableWrapper .model-ensemble{background-color: #aa38ff;color: #fff;}.tableWrapper .model-competing{background-color:#ffdd57;color:rgba(0,0,0,.7)}.tableWrapper .model-paper{background-color:#ff3860;color:#fff}.tableWrapper .dataset-sub{background-color:#23d160;color:#fff}.tableWrapper .dataset-metric{background-color:#209cee;color:#fff}.tableWrapper .dataset{background-color:#02bd43;color:#fff}.tableWrapper .trash{background-color:#363636;color:#f5f5f5}.tableWrapper .wtf{background-color:#f0f;color:#f5f5f5}.tableWrapper .dataset-task{background-color:#77ecdd;color:rgba(0,0,0,.7)}.tableWrapper .dataset-paper{background-color:#e4ffee;color:rgba(0,0,0,.7)}.tableWrapper td.focused-cell{outline:2px solid #9ecaed;border-radius:7px;box-shadow:0 0 10px #9ecaed}.tableWrapper span.text-bold{font-weight:700}.tableWrapper span.text-italic{font-style:italic}.tableWrapper span.text-red{color:red}.tableWrapper span.text-green{color:green}.tableWrapper span.text-blue{color:#00f}.predict-dataset,.predict-dataset-metric,.predict-model-competing,.predict-model-paper,.predict-model-params,.predict-table-meta{outline:2px solid #9ecaed;border-radius:7px;box-shadow:0 0 10px #9ecaed}.tableWrapper .predict-model-params{background-color:#209cee;color:rgba(0,0,0,.7)}.tableWrapper .predict-table-meta{background-color:#fff3c5;color:rgba(0,0,0,.7)}.tableWrapper .predict-model-competing{background-color:#ffdd57;color:rgba(0,0,0,.7)}.tableWrapper .predict-model-paper{background-color:#ff3860;color:#fff}.tableWrapper .predict-dataset-metric{background-color:#209cee;color:#fff}.tableWrapper .predict-dataset{background-color:#02bd43;color:#fff}.tableWrapper td{border:inherit}.tableWrapper table tr td.border-l{border-left:1px solid #000}.tableWrapper table tr td.border-r{border-right:1px solid #000}.tableWrapper table tr td.border-t{border-top:1px solid #000}.tableWrapper table tr td.border-b{border-bottom:1px solid #000}.tableWrapper table tr td.border-ll{border-left:2px solid #000}.tableWrapper table tr td.border-rr{border-right:2px solid #000}.tableWrapper table tr td.border-tt{border-top:2px solid #000}.tableWrapper table tr td.border-bb{border-bottom:2px solid #000}.tableWrapper table tr td.align-left{text-align:left}.tableWrapper table tr td.align-right{text-align:right}.tableWrapper table tr td.align-center{text-align:center}.tableWrapper table tr td.align-justify{text-align:justify}div.form-group>input.form-control.input-sm{border-radius:2px;font-size:.75rem;background-color:#fff;color:#363636;box-shadow:inset 0 1px 2px rgba(10,10,10,.1);max-width:100%;width:100%;height:2.25em;padding:calc(.375em - 1px) calc(.625em - 1px);position:relative;border:1px solid #b5b5b5}div.form-group>input.form-control.input-sm:focus{border-color:#3273dc;box-shadow:0 0 0 .125em rgba(50,115,220,.25)}.ht_clone_top{z-index:20}.evaluation-tables{overflow:scroll;max-height:20vh;border-top:1px solid #a9a9a9}.navbar.is-fixed-bottom,.navbar.is-fixed-top{z-index:200}body{padding-bottom:20vh}\n",
       ".tableWrapper .final-proposal{ background: lightgreen }\n",
       "</style>\n",
       "\n",
       "<div class=\"tableWrapper\">\n",
       "<table>\n",
       "<tr>\n",
       "<td class=\"table-meta border-t align-center header \" title=\"\">Rank</td>\n",
       "<td class=\"table-meta border-t align-left header \" title=\"\">Team</td>\n",
       "<td class=\"model-params border-r border-t align-center header \" title=\"\">Single Model</td>\n",
       "<td class=\"dataset-metric border-t align-center header \" title=\"\">Final Score</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" border-t align-center header \" title=\"\">1</td>\n",
       "<td class=\"model-competing border-t align-left header \" title=\"\">CASIA_IVA_JD</td>\n",
       "<td class=\" border-r border-t align-center \" title=\"\">✗</td>\n",
       "<td class=\" border-t align-center \" title=\"\">0.5547</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" align-center header \" title=\"\">2</td>\n",
       "<td class=\"model-competing align-left header \" title=\"\">WinterIsComing</td>\n",
       "<td class=\" border-r align-center \" title=\"\">✗</td>\n",
       "<td class=\" align-center \" title=\"\">0.5544</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" align-center header \" title=\"\">-</td>\n",
       "<td class=\"model-competing align-left header \" title=\"\">PSPNet [<a title=\"bib-bib38\">38</a>]</td>\n",
       "<td class=\" border-r align-center \" title=\"\">ResNet-269</td>\n",
       "<td class=\" align-center \" title=\"\">0.5538</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" align-center header \" title=\"\">-</td>\n",
       "<td class=\"model-competing align-left header \" title=\"\">EncNet [<a title=\"bib-bib36\">36</a>]</td>\n",
       "<td class=\" border-r align-center \" title=\"\">ResNet-101</td>\n",
       "<td class=\" align-center \" title=\"\">0.5567</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\" border-b border-t align-center header \" title=\"\">-</td>\n",
       "<td class=\"model-best border-b border-t align-left header \" title=\"\">Ours</td>\n",
       "<td class=\" border-b border-r border-t align-center \" title=\"\">ResNet-101</td>\n",
       "<td class=\" border-b border-t align-center \" title=\"\"><span class=\"text-bold\">0.5584</span></td>\n",
       "</tr>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<axcell.data.table.Table at 0x7feccd878518>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper = pc.get_by_id('1903.11816')\n",
    "paper.tables[4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "body{margin:0;padding:0;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen,Ubuntu,Cantarell,Fira Sans,Droid Sans,Helvetica Neue,sans-serif;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}code{font-family:source-code-pro,Menlo,Monaco,Consolas,Courier New,monospace}.tableWrapper{-overflow:auto}.tableWrapper .has-annotations{color:#ff3860}.tableWrapper .model-params{background-color:#209cee;color:rgba(0,0,0,.7)}.tableWrapper .table-meta{background-color:#fff3c5;color:rgba(0,0,0,.7)}.tableWrapper .model-best{background-color:#ff3860;color:rgba(0,0,0,.7)}.tableWrapper .model-ensemble{background-color: #aa38ff;color: #fff;}.tableWrapper .model-competing{background-color:#ffdd57;color:rgba(0,0,0,.7)}.tableWrapper .model-paper{background-color:#ff3860;color:#fff}.tableWrapper .dataset-sub{background-color:#23d160;color:#fff}.tableWrapper .dataset-metric{background-color:#209cee;color:#fff}.tableWrapper .dataset{background-color:#02bd43;color:#fff}.tableWrapper .trash{background-color:#363636;color:#f5f5f5}.tableWrapper .wtf{background-color:#f0f;color:#f5f5f5}.tableWrapper .dataset-task{background-color:#77ecdd;color:rgba(0,0,0,.7)}.tableWrapper .dataset-paper{background-color:#e4ffee;color:rgba(0,0,0,.7)}.tableWrapper td.focused-cell{outline:2px solid #9ecaed;border-radius:7px;box-shadow:0 0 10px #9ecaed}.tableWrapper span.text-bold{font-weight:700}.tableWrapper span.text-italic{font-style:italic}.tableWrapper span.text-red{color:red}.tableWrapper span.text-green{color:green}.tableWrapper span.text-blue{color:#00f}.predict-dataset,.predict-dataset-metric,.predict-model-competing,.predict-model-paper,.predict-model-params,.predict-table-meta{outline:2px solid #9ecaed;border-radius:7px;box-shadow:0 0 10px #9ecaed}.tableWrapper .predict-model-params{background-color:#209cee;color:rgba(0,0,0,.7)}.tableWrapper .predict-table-meta{background-color:#fff3c5;color:rgba(0,0,0,.7)}.tableWrapper .predict-model-competing{background-color:#ffdd57;color:rgba(0,0,0,.7)}.tableWrapper .predict-model-paper{background-color:#ff3860;color:#fff}.tableWrapper .predict-dataset-metric{background-color:#209cee;color:#fff}.tableWrapper .predict-dataset{background-color:#02bd43;color:#fff}.tableWrapper td{border:inherit}.tableWrapper table tr td.border-l{border-left:1px solid #000}.tableWrapper table tr td.border-r{border-right:1px solid #000}.tableWrapper table tr td.border-t{border-top:1px solid #000}.tableWrapper table tr td.border-b{border-bottom:1px solid #000}.tableWrapper table tr td.border-ll{border-left:2px solid #000}.tableWrapper table tr td.border-rr{border-right:2px solid #000}.tableWrapper table tr td.border-tt{border-top:2px solid #000}.tableWrapper table tr td.border-bb{border-bottom:2px solid #000}.tableWrapper table tr td.align-left{text-align:left}.tableWrapper table tr td.align-right{text-align:right}.tableWrapper table tr td.align-center{text-align:center}.tableWrapper table tr td.align-justify{text-align:justify}div.form-group>input.form-control.input-sm{border-radius:2px;font-size:.75rem;background-color:#fff;color:#363636;box-shadow:inset 0 1px 2px rgba(10,10,10,.1);max-width:100%;width:100%;height:2.25em;padding:calc(.375em - 1px) calc(.625em - 1px);position:relative;border:1px solid #b5b5b5}div.form-group>input.form-control.input-sm:focus{border-color:#3273dc;box-shadow:0 0 0 .125em rgba(50,115,220,.25)}.ht_clone_top{z-index:20}.evaluation-tables{overflow:scroll;max-height:20vh;border-top:1px solid #a9a9a9}.navbar.is-fixed-bottom,.navbar.is-fixed-top{z-index:200}body{padding-bottom:20vh}\n",
       ".tableWrapper .final-proposal{ background: lightgreen }\n",
       "</style>\n",
       "\n",
       "<div class=\"tableWrapper\">\n",
       "<table>\n",
       "<tr>\n",
       "<td class=\"  \" title=\"\">Tag</td>\n",
       "<td class=\"  \" title=\"\">description</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"model-best  \" title=\"\">model-best</td>\n",
       "<td class=\"  \" title=\"\">the best performing model introduced in the paper</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"model-paper  \" title=\"\">model-paper</td>\n",
       "<td class=\"  \" title=\"\">model introduced in the paper</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"model-ensemble  \" title=\"\">model-ensemble</td>\n",
       "<td class=\"  \" title=\"\">ensemble of models introduced in the paper</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"model-competing  \" title=\"\">model-competing</td>\n",
       "<td class=\"  \" title=\"\">model from another paper used for comparison</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"dataset-task  \" title=\"\">dataset-task</td>\n",
       "<td class=\"  \" title=\"\">Task</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"dataset  \" title=\"\">dataset</td>\n",
       "<td class=\"  \" title=\"\">Dataset</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"dataset-sub  \" title=\"\">dataset-sub</td>\n",
       "<td class=\"  \" title=\"\">Subdataset</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"dataset-metric  \" title=\"\">dataset-metric</td>\n",
       "<td class=\"  \" title=\"\">Metric</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"model-params  \" title=\"\">model-params</td>\n",
       "<td class=\"  \" title=\"\">Params, f.e., number of layers or inference time</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"table-meta  \" title=\"\">table-meta</td>\n",
       "<td class=\"  \" title=\"\">Cell describing other header cells</td>\n",
       "</tr>\n",
       "<tr>\n",
       "<td class=\"trash  \" title=\"\">trash</td>\n",
       "<td class=\"  \" title=\"\">Parsing erros</td>\n",
       "</tr>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pc.cells_gold_tags_legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>task</th>\n",
       "      <th>dataset</th>\n",
       "      <th>metric</th>\n",
       "      <th>format</th>\n",
       "      <th>model</th>\n",
       "      <th>raw_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cell_ext_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>table_05.csv/5.3</th>\n",
       "      <td>Semantic Segmentation</td>\n",
       "      <td>ADE20K test</td>\n",
       "      <td>Test Score</td>\n",
       "      <td>NaN</td>\n",
       "      <td>EncNet + JPU</td>\n",
       "      <td>0.5584</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   task      dataset      metric  format  \\\n",
       "cell_ext_id                                                                \n",
       "table_05.csv/5.3  Semantic Segmentation  ADE20K test  Test Score     NaN   \n",
       "\n",
       "                         model raw_value  \n",
       "cell_ext_id                               \n",
       "table_05.csv/5.3  EncNet + JPU    0.5584  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paper.tables[4].sota_records"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parallel Extraction\n",
    "\n",
    "For a single paper extraction can take from several seconds to a few minutes (the longest phase of converting LaTeX source into HTML is timed-out after 5 minutes), so to process multiple files we run extraction in parallel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 100 ms, sys: 40.5 ms, total: 141 ms\n",
      "Wall time: 30.1 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "from joblib import delayed, Parallel\n",
    "\n",
    "# access extract from the global context to avoid serialization\n",
    "def extract_single(file): return extract(file)\n",
    "\n",
    "files = sorted([path for path in SOURCES_PATH.glob('**/*') if path.is_file()])\n",
    "\n",
    "statuses = Parallel(backend='multiprocessing', n_jobs=-1)(delayed(extract_single)(file) for file in files)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}