notebooks/translate-goe.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Translate Grade of Execution\n",
    "\n",
    "At the end of each competition, the International Skating Union (ISU) releases an event protocol PDF, which is the full compendium of scores given to skaters during a competition. (See, for instance, [this event protocol from the 2017-18 Grand Prix final](http://www.isuresults.com/results/season1718/gpf1718/gpf2017_protocol.pdf).) These PDFs only include each judge's *Grade of Execution (GOE)* for each performed element, but not the equivalent points. This notebook takes the data available in `data/raw/` and translates the *Grade of Execution* for each technical element judgement, based on the publicly available translation tables that the ISU publishes at the beginning of each competition season. Finally, it checks to make sure that when the translated GOEs are combined, they equal the skater's overall translated GOE for each element. (Unlike the per-judgment GOEs, the overall translated GOE is included in the PDFs.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1,726 performances\n",
      "23,932 aspects\n",
      "214,531 scores\n"
     ]
    }
   ],
   "source": [
    "performances = pd.read_csv(\"../data/raw/performances.csv\")\n",
    "print(\"{:,} performances\".format(len(performances)))\n",
    "\n",
    "aspects = pd.read_csv(\"../data/raw/judged-aspects.csv\")\n",
    "print(\"{:,} aspects\".format(len(aspects)))\n",
    "\n",
    "scores = pd.read_csv(\"../data/raw/judge-scores.csv\")\n",
    "print(\"{:,} scores\".format(len(scores)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "214531"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scored = scores.pipe(\n",
    "    pd.merge,\n",
    "    aspects,\n",
    "    on = \"aspect_id\",\n",
    "    how = \"left\"\n",
    ").pipe(\n",
    "    pd.merge,\n",
    "    performances,\n",
    "    on = \"performance_id\",\n",
    "    how = \"left\"\n",
    ").assign(\n",
    "    is_junior = lambda x: x[\"program\"].str.contains(\"JUNIOR\")\n",
    ")\n",
    "\n",
    "len(scored)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare the data\n",
    "\n",
    "Only **technical elements** (as opposed to artistic components) receive a Grade of Execution from the judges. Our analysis only deals with the senior level competitions, so we remove the junior level programs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "129814"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "elements = scored[\n",
    "    (scored[\"section\"] == \"elements\") &\n",
    "    (~scored[\"is_junior\"])\n",
    "].copy()\n",
    "len(elements)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different **competition seasons** use different Scale of Values translation tables. BuzzFeed News downloaded the PDFs containing these translation tables and transformed them into CSVs. Here are the PDFs: [Figure Skating 2016/17](http://www.isu.org/docman-documents-links/isu-files/documents-communications/isu-communications/459-2000-sptc-sov-and-goe-2016-2017-revised-july-14/file), [Figure Skating 2017/18](http://www.isu.org/docman-documents-links/isu-files/documents-communications/isu-communications/14352-isu-communication-2089/file), [Ice Dancing 2016/17](http://isu.org/docman-documents-links/isu-files/documents-communications/isu-communications/476-isu-communication-2015/file), [Ice Dancing 2017/18](http://www.isu.org/docman-documents-links/isu-files/documents-communications/isu-communications/589-isu-communication-2094/file). The function below assigns each competition to a season."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def find_comp_season(competition):\n",
    "        if competition in [\n",
    "            \"Grand Prix Final 2017 Senior and Junior\",\n",
    "            \"ISU GP 2017 Bridgestone Skate America\",\n",
    "            \"ISU GP 2017 Skate Canada International\",\n",
    "            \"ISU GP NHK Trophy 2017\",\n",
    "            \"ISU GP Audi Cup of China 2017\",\n",
    "            \"ISU GP Rostelecom Cup 2017\",\n",
    "            \"ISU GP Internationaux de France de Patinage 2017\"\n",
    "        ]:\n",
    "            return 2017\n",
    "        elif competition in [\n",
    "            \"ISU Grand Prix of Figure Skating Final 2016\",\n",
    "            \"ISU GP Trophee de France 2016\",\n",
    "            \"ISU GP NHK Trophy 2016\",\n",
    "            \"ISU GP 2016 Skate Canada International\",\n",
    "            \"ISU GP Rostelecom Cup 2016\",\n",
    "            \"ISU GP Audi Cup of China 2016\",\n",
    "            \"ISU World Figure Skating Championships 2017\",\n",
    "            \"ISU Four Continents Championships 2017\",\n",
    "            \"ISU GP 2016 Progressive Skate America\",\n",
    "            \"ISU European Figure Skating Championships 2017\"\n",
    "        ]:\n",
    "            return 2016\n",
    "        else:\n",
    "            print(\"Competition {} not found!\".format(competition))\n",
    "            return None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "elements[\"season\"] = elements[\"competition\"].apply(find_comp_season)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load conversion tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def convert_to_float(x):\n",
    "    return float(x.replace(\",\", \".\")) if not pd.isnull(x) else None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fs_goe_adj_2016_17 = pd.read_csv(\n",
    "    \"../data/processed/figure-skating-goe-adj-2016-17.csv\", \n",
    "    names = [\n",
    "        \"name\", \"code\", \"+3\", \"+2\", \"+1\", \n",
    "        \"base\", \"v\", \"v1\", \"-1\", \"-2\", \"-3\"\n",
    "    ],\n",
    "    decimal = \",\"\n",
    ")\n",
    "\n",
    "fs_goe_adj_2016_17[\"base\"] = fs_goe_adj_2016_17[\"base\"]\\\n",
    "    .apply(convert_to_float)\n",
    "    \n",
    "fs_goe_adj_2016_17[\"v\"] = fs_goe_adj_2016_17[\"v\"]\\\n",
    "    .apply(convert_to_float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fs_goe_adj_2017_18 = pd.read_csv(\n",
    "    \"../data/processed/figure-skating-goe-adj-2017-18.csv\", \n",
    "    names = [\n",
    "        \"name\", \"code\", \"+3\", \"+2\", \"+1\", \n",
    "        \"base\", \"v\", \"v1\", \"-1\", \"-2\", \"-3\"\n",
    "    ],\n",
    "    decimal = \",\"\n",
    ")\n",
    "\n",
    "fs_goe_adj_2017_18[\"base\"] = fs_goe_adj_2017_18[\"base\"]\\\n",
    "    .apply(convert_to_float)\n",
    "    \n",
    "fs_goe_adj_2017_18[\"v\"] = fs_goe_adj_2017_18[\"v\"]\\\n",
    "    .apply(convert_to_float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "id_goe_adj_2017_18 = pd.read_csv(\n",
    "    \"../data/processed/ice-dancing-goe-adj-2017-18.csv\",\n",
    "    names = [\n",
    "        \"name\", \"code\", \"+3\", \"+2\", \"+1\", \n",
    "        \"base\", \"-1\", \"-2\", \"-3\"\n",
    "    ],\n",
    "    dtype = {\n",
    "        \"+3\": float, \"+2\": float, \"+1\": float, \"base\": float,\n",
    "        \"-3\": float, \"-2\": float, \"-1\": float \n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "id_goe_adj_2016_17 = pd.read_csv(\n",
    "    \"../data/processed/ice-dancing-goe-adj-2016-17.csv\",\n",
    "    names=[\n",
    "        \"name\", \"code\", \"+3\", \"+2\", \"+1\", \n",
    "        \"base\", \"-1\", \"-2\", \"-3\"\n",
    "    ],\n",
    "    dtype = {\n",
    "        \"+3\": float, \"+2\": float, \"+1\": float, \"base\": float,\n",
    "        \"-3\": float, \"-2\": float, \"-1\": float \n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Translate the Grade of Execution\n",
    "\n",
    "The functions below translate the individual grade of execution that each judge gave a skater.\n",
    "\n",
    "- `clean_fs_aspect`: Cleans the aspect string that is given in the protocol PDF\n",
    "- `downgrade_fs_aspect`: Converts a downgraded aspect to the aspect used for the translation\n",
    "- `translate_sk_goe`: Translates the GOE for an individual figure skating aspect\n",
    "- `translate_id_goe`: Translates the GOE for an individual ice dancing aspect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def clean_fs_aspect(aspect_string):\n",
    "    trimmed_aspect = aspect_string.rstrip(\"<\").rstrip(\"e\")\n",
    "    convert_aspect = {\n",
    "        \"CCoSp4\": \"(F)CCoSp4\",\n",
    "        \"FCCoSp4\": \"(F)CCoSp4\",\n",
    "        \"FCoSp4\": \"(F)CCoSp4\",\n",
    "        \"CCoSp3\": \"(F)CCoSp3\",\n",
    "        \"FCCoSp3\": \"(F)CCoSp3\",\n",
    "        \"FCCoSp1\": \"(F)CCoSp1\",\n",
    "        \"CSSp4\": \"(F)CSSp4\",\n",
    "        \"CCSp4\": \"(F)CCSp4\",\n",
    "        \"CCSp1\": \"(F)CCSp1\",\n",
    "        \"ChSq1\": \"ChSq\",\n",
    "        \"5RLi4\": \"5A/RLi4\",\n",
    "        \"5RLiB\": \"5A/RLiB\",\n",
    "        \"5ALi4\": \"5A/RLi4\",\n",
    "        \"5TLi3\": \"5T/SLi3\",\n",
    "        \"3FTh\": \"3F/LzTh\",\n",
    "        \"3LzTh\": \"3F/LzTh\",\n",
    "        \"CSSp3\": \"(F)CSSp3\",\n",
    "        \"CCSp3\": \"(F)CCSp3\",\n",
    "        \"BiDs4\": \"Fi/BiDs4\",\n",
    "        \"BiDs3\": \"Fi/BiDs3\",\n",
    "        \"BoDs3\": \"Fo/BoDs3\",\n",
    "        \"FiDs4\": \"Fi/BiDs4\",\n",
    "        \"FiDs2\": \"F/BiDs2\",\n",
    "        \"BoDs4\": \"Fo/BoDs4\",\n",
    "        \"FiDs3\": \"Fi/BiDs3\",\n",
    "        \"FiDs1\": \"Fi/BiDs1\",\n",
    "        \"BiDsB\": \"Fi/BiDsB\",\n",
    "        \"CCoSp2\": \"(F)CCoSp2\",\n",
    "        \"CCoSp1\": \"(F)CCoSp1\",\n",
    "        \"5ALi1\": \"5A/RLi1\",\n",
    "        \"5ALi3\": \"5A/RLi3\",\n",
    "        \"5ALiB\": \"5A/RLiB\",\n",
    "        \"5RLi3\": \"5A/RLi3\",\n",
    "        \"5RLi2\": \"5A/RLi2\",\n",
    "        \"BoDs2\": \"Fo/BoDs2\",\n",
    "        \"BoDs1\": \"Fo/BoDs1\",\n",
    "        \"BiDs2\": \"Fi/BiDs2\",\n",
    "        \"BiDs2\": \"Fi/BiDs2\",\n",
    "        \"FCCoSp2\": \"(F)CCoSp2\",\n",
    "        \"CSSp2\": \"(F)CSSp2\",\n",
    "        \"CSSp1\": \"(F)CSSp1\",\n",
    "        \"FCSSp4\": \"(F)CSSp4\",\n",
    "        \"FCSSp3\": \"(F)CSSp3\",\n",
    "        \"CCSp2\": \"(F)CCSp2\",\n",
    "        \"5ALi2\": \"5A/RLi2\",\n",
    "        \"5SLi4\": \"5T/SLi4\",\n",
    "        \"BiDs1\": \"Fi/BiDs1\",\n",
    "        \"BiDs2\": \"F/BiDs2\",\n",
    "        \"BoDs1\": \"Fo/BoDs1\",\n",
    "        \"CCoSp3V\": \"(F)CCoSp3\",\n",
    "        \"CCoSp2V\": \"(F)CCoSp2\",\n",
    "        \"CCoSp4V\": \"(F)CCoSp4\",\n",
    "        \"CCoSp1V\": \"(F)CCoSp1\",\n",
    "        \"FCCoSp1V\": \"(F)CCoSp1\",\n",
    "        \"FCCoSp2V\": \"(F)CCoSp2\",\n",
    "        \"FCCoSp3V\": \"(F)CCoSp3\",\n",
    "        \"FCCoSp4V\": \"(F)CCoSp4\",\n",
    "        \"PCoSp3V\": \"PCoSp3\",\n",
    "        \"PCoSp2V\": \"PCoSp2\",\n",
    "        \"PCoSp4V\": \"PCoSp4\",\n",
    "        \"PCoSp1V\": \"PCoSp1\",\n",
    "        \"FCUSp4\": \"(F)CUSp4\",\n",
    "        \"BoDsB\": \"Fo/BoDsB\",\n",
    "        \"PCoSpBV\": \"PCoSpB\",\n",
    "        \"5SLi1\": \"5T/SLi1\",\n",
    "        \"5SLi2\": \"5T/SLi2\",\n",
    "        \"5SLi3\": \"5T/SLi3\",\n",
    "        \"T\": \"1T\"\n",
    "    }\n",
    "    if trimmed_aspect in convert_aspect.keys():\n",
    "        trimmed_aspect = convert_aspect[trimmed_aspect]\n",
    "    return trimmed_aspect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def downgrade_fs_aspect(aspect_string):\n",
    "    trimmed_aspect = re.sub(r\"<<$\", \"\", aspect_string).rstrip(\"e\")\n",
    "    downgrade_dict = {\n",
    "        \"4F\": \"3F\",\n",
    "        \"3F\": \"2F\",\n",
    "        \"2F\": \"1F\",\n",
    "        \"4Lz\": \"3Lz\",\n",
    "        \"3Lz\": \"2Lz\",\n",
    "        \"4Lo\": \"3Lo\",\n",
    "        \"3Lo\": \"2Lo\",\n",
    "        \"2Lo\": \"1Lo\",\n",
    "        \"4S\": \"3S\",\n",
    "        \"3S\": \"2S\",\n",
    "        \"2S\": \"1S\",\n",
    "        \"4T\": \"3T\",\n",
    "        \"3T\": \"2T\",\n",
    "        \"2T\": \"1T\",\n",
    "        \"3A\": \"2A\",\n",
    "        \"2A\": \"1A\",\n",
    "        \"3TwB\": \"2TwB\",\n",
    "        \"1T\": None,\n",
    "        \"1Lo\": None\n",
    "    }\n",
    "    if trimmed_aspect in downgrade_dict.keys():\n",
    "        trimmed_aspect = downgrade_dict[trimmed_aspect]\n",
    "    return trimmed_aspect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def translate_sk_goe(row):\n",
    "    if row[\"season\"] == 2016:\n",
    "        sov_df = fs_goe_adj_2016_17\n",
    "    else:\n",
    "        sov_df = fs_goe_adj_2017_18\n",
    "    # If score is 0, then the translated GOE is 0\n",
    "    score_col = str(int(row[\"score\"]))\n",
    "    if row[\"score\"] == 0:\n",
    "        return 0\n",
    "    # positives needs a + in front of them for lookup\n",
    "    elif (\"-\" not in score_col):\n",
    "        score_col = \"+\" + score_col\n",
    "    value = None\n",
    "    # No info_flag is simple case, ! is a warning and doesn't affect the translation\n",
    "    if pd.isnull(row[\"info_flag\"]) or (row[\"info_flag\"] == \"!\"):\n",
    "        aspect = clean_fs_aspect(row[\"aspect_desc\"])\n",
    "        if \"+\" in aspect:\n",
    "            aspects = aspect.split(\"+\")\n",
    "            highest_bv = 0\n",
    "            value_highest_element = 0\n",
    "            for a in aspects:\n",
    "                if a in [\"COMBO\", \"REP\", \"SEQ\"]:\n",
    "                    pass\n",
    "                else:\n",
    "                    a = clean_fs_aspect(a)\n",
    "                    aspect_bv = sov_df.set_index(\"code\").loc[a.strip(\"!\")][\"base\"]\n",
    "                    if aspect_bv > highest_bv:\n",
    "                        highest_bv = aspect_bv\n",
    "                        value_highest_element = sov_df.set_index(\"code\").loc[a.strip(\"!\")][score_col]\n",
    "            value = value_highest_element\n",
    "        # There was one case of a downgraded jump with an edge warning\n",
    "        elif \"<<\" in row[\"aspect_desc\"]:\n",
    "            a = downgrade_fs_aspect(row[\"aspect_desc\"])\n",
    "            value = sov_df.set_index(\"code\").loc[a][score_col]\n",
    "        else:\n",
    "            value = sov_df.set_index(\"code\").loc[clean_fs_aspect(aspect.strip(\"!\"))][score_col]\n",
    "        \n",
    "\n",
    "    # These are single jumps with flags\n",
    "    elif (\"+\" not in row[\"aspect_desc\"]):\n",
    "        aspect = clean_fs_aspect(row[\"aspect_desc\"])\n",
    "        # This is an underrotated jump, it affects base value\n",
    "        if (row[\"info_flag\"] == \"<\"):\n",
    "            value = sov_df.set_index(\"code\").loc[aspect][score_col]\n",
    "        # The edge call affects the base value\n",
    "        elif (row[\"info_flag\"] == \"e\"):\n",
    "            # Edge calls can have a downgrade too\n",
    "            if \"<<\" in row[\"aspect_desc\"]:\n",
    "                aspect = downgrade_fs_aspect(row[\"aspect_desc\"])\n",
    "                value = sov_df.set_index(\"code\").loc[aspect][score_col]\n",
    "            else:\n",
    "                value = sov_df.set_index(\"code\").loc[aspect][score_col]\n",
    "        elif (row[\"info_flag\"] == \"<<\"):\n",
    "            aspect = downgrade_fs_aspect(row[\"aspect_desc\"])\n",
    "            value = sov_df.set_index(\"code\").loc[aspect][score_col]\n",
    "    \n",
    "    else:\n",
    "        aspects = row[\"aspect_desc\"].split(\"+\")\n",
    "        highest_bv = 0\n",
    "        value_highest_element = 0\n",
    "        for a in aspects:\n",
    "            if a in [\"COMBO\", \"REP\", \"SEQ\"]:\n",
    "                pass\n",
    "            else:\n",
    "                if \"*\" in a:\n",
    "                    aspect_bv = 0\n",
    "                elif \"<<\" in a:\n",
    "                    a = downgrade_fs_aspect(a)\n",
    "                    # Sometimes the lowest-difficulty jumps get downgraded; \n",
    "                    # in that case, `downgrade_fs_aspect` returns None\n",
    "                    if a == None:\n",
    "                        aspect_bv = 0\n",
    "                    else:\n",
    "                        aspect_bv = sov_df.set_index(\"code\").loc[a][\"base\"]\n",
    "                elif \"<\" in a:\n",
    "                    a = clean_fs_aspect(a)\n",
    "                    aspect_bv = sov_df.set_index(\"code\").loc[a][\"base\"] * 0.7\n",
    "                else:\n",
    "                    a = clean_fs_aspect(a)\n",
    "                    aspect_bv = sov_df.set_index(\"code\").loc[a][\"base\"]\n",
    "                if \"e\" in (a or \"\"):\n",
    "                    aspect_bv = aspect_bv * 0.7\n",
    "                if aspect_bv > highest_bv:\n",
    "                    highest_bv = aspect_bv\n",
    "                    value_highest_element = sov_df.set_index(\"code\").loc[a][score_col]\n",
    "        value = value_highest_element          \n",
    "    return value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def translate_id_goe(row):\n",
    "    if row[\"season\"] == 2016:\n",
    "        sov_df = id_goe_adj_2016_17\n",
    "    else:\n",
    "        sov_df = id_goe_adj_2017_18\n",
    "    trimmed_aspect = row[\"aspect_desc\"].rstrip(\"<\").strip()\n",
    "    # If score is 0, then the GOE is 0\n",
    "    score_col = str(int(row[\"score\"]))\n",
    "    if row[\"score\"] == 0:\n",
    "        return 0\n",
    "    # Positives needs a + in front of them for lookup\n",
    "    elif (\"-\" not in score_col):\n",
    "        score_col = \"+\" + score_col\n",
    "    value = None\n",
    "    if \"kp\" in trimmed_aspect:\n",
    "        # Key Points don't affect score, just the initial grade\n",
    "        aspect = trimmed_aspect.split(\"+\")[0]\n",
    "        value = sov_df.set_index(\"code\").loc[aspect][score_col]\n",
    "    elif \"+\" in trimmed_aspect:\n",
    "        aspects = trimmed_aspect.split(\"+\")\n",
    "        total_value = 0\n",
    "        for a in aspects:\n",
    "            total_value += sov_df.set_index(\"code\").loc[a][score_col]\n",
    "        value = total_value\n",
    "    else:\n",
    "        value = sov_df.set_index(\"code\").loc[trimmed_aspect][score_col]\n",
    "    return value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def translate_goe(row):\n",
    "    if \"ICE DANCE\" in row[\"program\"]:\n",
    "        return translate_id_goe(row)\n",
    "    else:\n",
    "        return translate_sk_goe(row)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "elements[\"judge_goe\"] = elements.apply(translate_goe, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example translations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>season</th>\n",
       "      <th>aspect_desc</th>\n",
       "      <th>info_flag</th>\n",
       "      <th>score</th>\n",
       "      <th>judge_goe</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>2017</td>\n",
       "      <td>3Tw2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>-0.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>2017</td>\n",
       "      <td>3Tw2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>2017</td>\n",
       "      <td>3Tw2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>2017</td>\n",
       "      <td>3Tw2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-2.0</td>\n",
       "      <td>-1.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>2016</td>\n",
       "      <td>CoSp4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>2016</td>\n",
       "      <td>CoSp4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>2016</td>\n",
       "      <td>FCCoSp4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    season aspect_desc info_flag  score  judge_goe\n",
       "36    2017        3Tw2       NaN   -1.0       -0.7\n",
       "39    2017        3Tw2       NaN    0.0        0.0\n",
       "41    2017        3Tw2       NaN    1.0        0.7\n",
       "43    2017        3Tw2       NaN   -2.0       -1.4\n",
       "45    2016       CoSp4       NaN    2.0        1.2\n",
       "46    2016       CoSp4       NaN    1.0        0.6\n",
       "72    2016     FCCoSp4       NaN    1.0        0.5"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "elements[[ \n",
    "    \"season\",\n",
    "    \"aspect_desc\", \n",
    "    \"info_flag\", \n",
    "    \"score\", \n",
    "    \"judge_goe\" \n",
    "]].head(20).drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check that the translated GOEs are correct\n",
    "\n",
    "The protocol PDFs do include the final *overall* translated Grade of Execution for each technical element. In order to check that we have correctly translated the Grade of Execution for each individual judge, this code recalculates the overall translated Grade of Execution from our computed values and then checks that it is equal to the overall GOE in the protocol PDF."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def round_goe(goe):\n",
    "    return float(\"{:.2f}\".format(goe))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "aspect_grp = elements.groupby([\"aspect_id\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "check_df = pd.DataFrame({\n",
    "    \"translated_goes\": aspect_grp[\"judge_goe\"].apply(lambda x: x.values),\n",
    "    \"base_value\": aspect_grp[\"base_value\"].first(),\n",
    "    \"scores_of_panel\": aspect_grp[\"scores_of_panel\"].first(),\n",
    "    \"given_goe\": aspect_grp[\"goe\"].first()\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The overall translated GOE for a technical element is calculated by dropping the highest and lowest scores and then taking the mean of the remaining scores (i.e., the \"trimmed mean\"). The cell below goes through that process for our translated GOEs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "check_df[\"sorted_goes\"] = check_df[\"translated_goes\"].apply(sorted)\n",
    "check_df[\"trimmed_goes\"] = check_df[\"sorted_goes\"].apply(lambda x: x[1:-1])\n",
    "check_df[\"calculated_total_goe\"] = check_df[\"trimmed_goes\"].apply(lambda x: round_goe(sum(x)/len(x)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The overall translated GOE given in the protocol PDF should be equal to the difference between the overall score of the panel minus the base value of the element. The rest of this notebook calculates the overall translated GOE using those two aspects of the protocol PDF and then checks it against the overall translated GOE in the protocol PDF and the overall translated GOE calculated from our individual judge GOE translations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "check_df[\"goe_from_protocol\"] = (check_df[\"scores_of_panel\"] - check_df[\"base_value\"]).apply(round_goe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>calculated_total_goe</th>\n",
       "      <th>goe_from_protocol</th>\n",
       "      <th>given_goe</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>aspect_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>000df5399a</th>\n",
       "      <td>-0.20</td>\n",
       "      <td>-0.20</td>\n",
       "      <td>-0.20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000f259b7c</th>\n",
       "      <td>0.77</td>\n",
       "      <td>0.77</td>\n",
       "      <td>0.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0015b0e54a</th>\n",
       "      <td>0.36</td>\n",
       "      <td>0.36</td>\n",
       "      <td>0.36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>00184539f1</th>\n",
       "      <td>0.14</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>001d80257c</th>\n",
       "      <td>0.90</td>\n",
       "      <td>0.90</td>\n",
       "      <td>0.90</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            calculated_total_goe  goe_from_protocol  given_goe\n",
       "aspect_id                                                     \n",
       "000df5399a                 -0.20              -0.20      -0.20\n",
       "000f259b7c                  0.77               0.77       0.77\n",
       "0015b0e54a                  0.36               0.36       0.36\n",
       "00184539f1                  0.14               0.14       0.14\n",
       "001d80257c                  0.90               0.90       0.90"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "check_df[[\"calculated_total_goe\", \"goe_from_protocol\", \"given_goe\"]].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "assert (check_df[\"calculated_total_goe\"] != check_df[\"given_goe\"]).sum() == 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "assert (check_df[\"calculated_total_goe\"] != check_df[\"goe_from_protocol\"]).sum() == 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "elements[[\"aspect_id\", \"judge\", \"judge_goe\"]].to_csv(\"../data/processed/judge-goe.csv\", index=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "---\n",
    "\n",
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}