Lecture_06.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 6: Monty Hall, Simpson's Paradox\n",
    "\n",
    "\n",
    "## Stat 110, Prof. Joe Blitzstein, Harvard University\n",
    "\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Monty Hall Problem\n",
    "\n",
    "In case you did not grow up watching way too much daytime television in America during the 70's and early 80's, here is [Monty Hall on YouTube talking about the background of this math problem involving his popular game show, Let's Make A Deal](https://www.youtube.com/watch?v=c1BSkquWkDo).\n",
    "\n",
    "* There are three doors.\n",
    "* A car is behind one of the doors.\n",
    "* The other two doors have goats behind them.\n",
    "* You choose a door, but before you see what's behind your choice, Monty opens one of the other doors to reveal a goat.\n",
    "* Monty offers you the chance to switch doors.\n",
    "\n",
    "_Should you switch?_\n",
    "\n",
    "### Defining the problem\n",
    "\n",
    "Let $S$ be the event of winning when you switch.\n",
    "\n",
    "Let $D_j$ be the event of the car being behind door $j$.\n",
    "\n",
    "### Solving with a probability tree\n",
    "\n",
    "![title](images/L0601.png)\n",
    "\n",
    "With a probability tree, it is easy to represent the case where you condition on Monty opening door 2. Given that you initially choose door 1, you can quickly see that if you stick with door 1, you have a $\\frac{1}{3}~$ chance of winning.\n",
    "\n",
    "You have a $\\frac{2}{3}~$ chance of winning if you switch.\n",
    "\n",
    "### Solving with the Law of Total Probability\n",
    "\n",
    "This is even easier to solve using the Law of Total Probability.\n",
    "\n",
    "\\begin{align}\n",
    "    P(S) &= P(S|D_1)P(D_1) + P(S|D_2)P(D_2) + P(S|D_3)P(D_3) \\\\\n",
    "    &= 0 \\frac{1}{3} + 1 \\frac{1}{3} + 1 \\frac{1}{3}  \\\\\n",
    "    &= \\frac{2}{3}\n",
    "\\end{align}\n",
    "\n",
    "### A more general solution\n",
    "\n",
    "Let $n = 7$ be the number of doors in the game.\n",
    "\n",
    "Let $m=3$ be the number of doors with goats that Monty opens after you select your initial door choice.\n",
    "\n",
    "Let $S$ be the event where you win _by sticking with your original door choice of door 1_.\n",
    "\n",
    "Let $C_j$ be the event that the car is actually behind door $j$.\n",
    "\n",
    "Conditioning only on which door has the car, we have\n",
    "\\begin{align}\n",
    "    & &P(S) &= P(S|C_1)P(C_1) + \\dots + P(S|C_n)P(C_n) & &\\text{Law of Total Probability} \\\\\n",
    "    & &    &= P(C_1) \\\\\n",
    "    & &    &= \\frac{1}{7} \\\\\n",
    "\\end{align}\n",
    "\n",
    "Let $M_{i,j,k}$ be the event that Monty opens doors $i,j,k$. Conditioning on Monty opening up doors $i,j,k$, we have\n",
    "\n",
    "\\begin{align}\n",
    "    & &P(S) &= \\sum_{i,j,k} P(S|M_{i,j,k})P(M_{i,j,k}) & &\\text{summed over all i, j, k with } 2 \\le i \\lt j \\lt k \\le 7 \\\\\n",
    "    \\\\\n",
    "    & &\\Rightarrow P(S|M_{i,j,k}) &= P(S) & &\\text{by symmetry} \\\\\n",
    "    & & &=\\frac{1}{7}\n",
    "\\end{align}\n",
    "\n",
    "Note that we can now generalize this  to the case where:\n",
    "* there are $n \\ge 3$ doors\n",
    "* after you choose a door, Monty opens $m$ of the remaining doors $n-1$ doors to reveal a goat (with $1 \\le m \\le n-m-2$)\n",
    "\n",
    "The probability of winning with the strategy of _sticking to your initial choice_ is $\\frac{1}{n}$, whether __unconditional or conditioning on the doors Monty opens__.\n",
    "\n",
    "After Monty opens $m$ doors, each of the remaining $n-m-1$ doors has __conditional__ probability of $\\left(\\frac{n-1}{n-m-1}\\right) \\left(\\frac{1}{n}\\right)$.\n",
    "\n",
    "Since $\\frac{1}{n} \\lt \\left(\\frac{n-1}{n-m-1}\\right) \\left(\\frac{1}{n}\\right)$, you will always have a greater chance of winning if you switch.\n",
    "\n",
    "\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simpson's Paradox\n",
    "\n",
    "_Is it possible for a certain set of events to be more (or less) probable than another without conditioning, and then be less (or more) probable with conditioning?_\n",
    "\n",
    "![title](images/L0602.png)\n",
    "\n",
    "Assume that we have the above rates of success/failure for Drs. [Hibbert](http://simpsons.wikia.com/wiki/Julius_Hibbert) and [Nick](http://simpsons.wikia.com/wiki/Nick_Riviera) for two types of surgery: heart surgery and band-aid removal.\n",
    "\n",
    "### Defining the problem\n",
    "\n",
    "Let $A$ be the event of a successful operation.\n",
    "\n",
    "Let $B$ be the event of treatment by Dr. Nick.\n",
    "\n",
    "Let $C$ be the event of heart surgery.\n",
    "\n",
    "\\begin{align}\n",
    "    P(A|B,C) &< P(A|B^c,C) & &\\text{Dr. Nick is not as skilled as Dr. Hibbert in heart surgery} \\\\\n",
    "    P(A|B,C^c) &< P(A|B^c,C^c) & &\\text{neither is he all that good at band-aid removal} \\\\\n",
    "\\end{align}\n",
    "\n",
    "And yet $P(A|B) > P(A|B^c)$?\n",
    "\n",
    "### Explaining with the Law of Total Probability\n",
    "\n",
    "To explain this paradox, let's try to use the Law of Total Probability.\n",
    "\n",
    "\\begin{align}\n",
    "    P(A|B) &= P(A|B,C)P(C|B) + P(A|B,C^c)P(C^c|B) \\\\\n",
    "    \\\\\n",
    "    \\text{but } P(A|B,C) &< P(A|B^c,C) \\\\\n",
    "    \\text{and } P(A|B,C^c) &< P(A|B^c,C^c)\n",
    "\\end{align}\n",
    "\n",
    "Look at $P(C|B$ and $P(C|B^c)$. These weights are what makes this paradox possible, as they are what make the inequality relation sign flip.\n",
    "\n",
    "Event $C$ is a case of __confounding__\n",
    "\n",
    "\n",
    "### Another example\n",
    "\n",
    "_Is it possible to have events $A_1, A_2, B, C$ such that_ \n",
    "\n",
    "\\begin{align}\n",
    "    P(A_1|B) &\\gt P(A_1|C) \\text{ and } P(A_2|B) \\gt P(A_2|C) & &\\text{ ... yet...} \\\\ \n",
    "    P(A_1 \\cup A_2|B) &\\lt P(A_1 \\cup A_2|C)\n",
    "\\end{align}\n",
    "\n",
    "Yes, and this is just another case of Simpson's Paradox.\n",
    "\n",
    "Note that \n",
    "\n",
    "\\begin{align}\n",
    "    P(A_1 \\cup A_2|B) &= P(A_1|B) + P(A_2|B) - P(A_1 \\cap A_2|B)\n",
    "\\end{align}\n",
    "\n",
    "So this is _not_ possible if $A_1$ and $A_2$ are disjoint and $P(A_1 \\cup A_2|B) = P(A_1|B) + P(A_2|B)$.\n",
    "\n",
    "It is crucial, therefore, to consider the _intersection_ $P(A_1 \\cap A_2|B)$, so let's look at the following example where $P(A_1 \\cap A_2|B) \\gg P(A_1 \\cap A_2|C)$ in order to offset the other inequalities.\n",
    "\n",
    "Consider two basketball players each shooting a pair of free throws.\n",
    "\n",
    "Let $A_j$ be the event basketball free throw scores on the $j^{th}$ try.\n",
    "\n",
    "Player $B$ always either makes both $P(A_1 \\cap A_2|B) = 0.8$, or misses both.\n",
    "\n",
    "\\begin{align}\n",
    "    P(A_1|B) = P(A_2|B) = P(A_1 \\cap A_2|B) = P(A_1 \\cup A_2|B) = 0.8\n",
    "\\end{align}    \n",
    "\n",
    "Player $C$ makes free throw shots with probability $P(A_j|C) = 0.7$, independently, so we have\n",
    "\n",
    "\\begin{align}\n",
    "    P(A_1|C) &= P(A_2|C) = 0.7  \\\\\n",
    "    P(A_1 \\cap A_2|C) &= P(A_1|C) P(A_2|C) = 0.49 \\\\\n",
    "    P(A_1 \\cup A_2|C) &= P(A_1|C) + P(A_2|C) - P(A_1 \\cap A_2|C) \\\\\n",
    "                      &= 2 \\times 0.7 - 0.49 \\\\\n",
    "                      &= 0.91\n",
    "\\end{align}    \n",
    "\n",
    "And so we have our case where\n",
    "\n",
    "\\begin{align}\n",
    "    P(A_1|B) = 0.8 &\\gt P(A_1|C) = 0.7 \\\\\n",
    "    P(A_2|B) = 0.8 &\\gt P(A_2|C) = 0.7 \\\\\n",
    "    \\\\\n",
    "    \\text{ ... and  yet... } \\\\\n",
    "    \\\\\n",
    "    P(A_1 \\cup A_2|B) &\\lt P(A_1 \\cup A_2|C) ~~~~ \\blacksquare\n",
    "\\end{align}\n",
    "\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "View [Lecture 6: Monty Hall, Simpson's Paradox | Statistics 110](http://bit.ly/2nPjJ7S) on YouTube."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}