confidence_intervals.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Link to videos and excercises](https://www.khanacademy.org/math/statistics-probability/confidence-intervals-one-sample)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# a cell to import modules and define helper functions\n",
    "import math\n",
    "import scipy.stats as ss\n",
    "\n",
    "def z_table_probability(z):\n",
    "    # we need to find probability from -math.inf to -z to find probability of a tail\n",
    "    # once we know the probability of one of the tails\n",
    "    # we can calculate probability of z standard deviations from the mean\n",
    "    p = 1 - (ss.norm.cdf(-z) * 2)\n",
    "    print(\"For critical value z* %.3f, probability is: %.2f\" % (z, p))\n",
    "    return p\n",
    "\n",
    "def z_table_zscore(p):\n",
    "    # we need to transform p ppf calculates from -inf to the boundary\n",
    "    # but we need two-tailed estimation\n",
    "    z = ss.norm.ppf(p/2+0.5)\n",
    "    print(\"For confidence interval of %.2f, we need to be in < %.3f standard deviations from the mean\" % (p, z))\n",
    "    return z\n",
    "\n",
    "def t_table_tscore(p, n):\n",
    "    # we need to transform p ppf calculates from -inf to the boundary\n",
    "    # but we need two-tailed estimation\n",
    "    t = ss.t.ppf(p/2+0.5, n-1)\n",
    "    print(\"t-score for confidence level of %d%% with df = %d is: %.3f\" % (p*100, n-1, t))\n",
    "    return t\n",
    "\n",
    "def transform_margin_to_interval(mu, interval):\n",
    "    print(\"The probability interval is: %.3f - %.3f\" % (mu-interval, mu+interval))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Estimating a population proportion\n",
    "\n",
    "Conditions for inference on a proportion:\n",
    "\n",
    "1. Random \n",
    "2. Normal (proportion will always be normal if population is normally distributed and randomness condition is satisfied)\n",
    "    * (expect > 10 sucesses || failures) or np > 10 and n(1-p) > 10\n",
    "    * have more > 30 samples\n",
    "3. Independent \n",
    "    * if sampling without replacement, sample should be < 10% of population\n",
    "    \n",
    "    \n",
    "How to choose if you should use a z distribution or a t distribution\n",
    "\n",
    "* if you know population's standard deviation && sample size > 30: use z distribution\n",
    "* if you don't know population's standard deviation or sample size is < 30: use t distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating confidence levels and critical values of z*\n",
    "### Problem 1: Calculate crical value z*\n",
    "![](img/confidence_intervals_p1.png)\n",
    "![](img/confidence_intervals_c1.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For confidence interval of 0.92, we need to be in < 1.751 standard deviations from the mean\n"
     ]
    }
   ],
   "source": [
    "z = z_table_zscore(0.92)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 2: Calculate confidence interval\n",
    "![](img/confidence_intervals_p2.png)\n",
    "![](img/confidence_intervals_c2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For critical value z* 1.476, probability is: 0.86\n"
     ]
    }
   ],
   "source": [
    "p = z_table_probability(1.476)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 3: Calculate crical value z*\n",
    "![](img/confidence_intervals_p3.png)\n",
    "\n",
    "$(statistic)  \\pm (critical\\ value)(standard\\ deviation\\ of\\ statistic)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\hat{p} \\pm z^*\\sqrt{\\frac{p(1-p)}{n}}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For confidence interval of 0.90, we need to be in < 1.645 standard deviations from the mean\n",
      "Probability of success is 0.48 \n",
      "Margin of error for 0.90 confidence interval is 0.058\n",
      "The probability interval is: 0.422 - 0.538\n"
     ]
    }
   ],
   "source": [
    "CONFIDENCE_LEVEL = 0.9\n",
    "SAMPLE_SIZE = 200\n",
    "SUCCESSES_NUMBER = 96\n",
    "\n",
    "p = SUCCESSES_NUMBER / SAMPLE_SIZE\n",
    "interval = z_table_zscore(CONFIDENCE_LEVEL) * math.sqrt((p*(1-p))/SAMPLE_SIZE)\n",
    "\n",
    "print(\"Probability of success is %.2f \" % p)\n",
    "print(\"Margin of error for %.2f confidence interval is %.3f\" % (CONFIDENCE_LEVEL, interval))\n",
    "\n",
    "transform_margin_to_interval(p, interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 4: Calculate minimal sample size for a margin of error\n",
    "![](img/confidence_intervals_p4.png)\n",
    "\n",
    "We need to solve \n",
    "\n",
    "$z^*\\sqrt{\\frac{p(1-p)}{n}} \\leq margin\\ of \\ error$ \n",
    "\n",
    "which results in\n",
    "\n",
    "$n \\geq \\frac{z^2}{margin\\ of\\ error^2} p(1-p)\n",
    "$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For confidence interval of 0.95, we need to be in < 1.960 standard deviations from the mean\n",
      "If we want margin of error less than 2%, sample size should be > 864.33\n"
     ]
    }
   ],
   "source": [
    "CONFIDENCE_LEVEL = 0.95\n",
    "MARGIN_OF_ERROR = 0.02\n",
    "# set to None if success probability is unknown\n",
    "SUCCESS_PROBABILITY = 0.9\n",
    "\n",
    "# if we don't know p and need to maximize p(1-p) the best value for p is 0.5\n",
    "if not SUCCESS_PROBABILITY:\n",
    "    SUCCESS_PROBABILITY = 0.5\n",
    "\n",
    "# solving algebraically for n\n",
    "min_sample_size = (z_table_zscore(CONFIDENCE_LEVEL)**2 / MARGIN_OF_ERROR**2) \\\n",
    "    * SUCCESS_PROBABILITY * (1-SUCCESS_PROBABILITY)\n",
    "print(\"If we want margin of error less than %d%%, sample size should be > %.2f\" % \\\n",
    "      (MARGIN_OF_ERROR*100, min_sample_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Estimating a population mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To estimate margin of error we can use the formula\n",
    "\n",
    "$\\sigma_\\bar{p} = \\frac{\\sigma}{\\sqrt{n}}$\n",
    "\n",
    "However we usually don't know the population standard deviation $\\sigma$, so we substitute the sample standard deviation $s_x$ as an estimate for $\\sigma$. When we do this, we call it the **standard error** of $\\bar{x}$ to distinguish it from the standard deviation.\n",
    "\n",
    "So formula for standard error of $\\bar{x}$ is:\n",
    "\n",
    "$\\sigma_\\bar{x} \\approx \\frac{s_x}{\\sqrt{n}}$\n",
    "\n",
    "Calculating t-interval for a mean:\n",
    "\n",
    "$\\bar{x} \\pm t^*\\frac{s_x}{\\sqrt{n}}$\n",
    "\n",
    "to find t we need to know: degree of freedom (df, which here could be n-1) and confidence level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 1: Finding the critical value t* for a desired confidence level\n",
    "![](img/confidence_intervals_p5.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t-score for confidence level of 90% with df = 17 is: 1.740\n"
     ]
    }
   ],
   "source": [
    "CONFIDENCE_LEVEL = 0.9\n",
    "SAMPLE_SIZE = 18\n",
    "\n",
    "t = t_table_zscore(CONFIDENCE_LEVEL, SAMPLE_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 2: Finding the critical value t* for a desired confidence level\n",
    "![](img/confidence_intervals_p6.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t-score for confidence level of 90% with df = 11 is: 1.796\n",
      "Margin of error is: 0.104\n",
      "The probability interval is: 2.186 - 2.394\n"
     ]
    }
   ],
   "source": [
    "SAMPLE_SIZE = 12\n",
    "SAMPLE_MU = 2.29\n",
    "SAMPLE_SIGMA = 0.2\n",
    "CONFIDENCE_LEVEL = 0.9\n",
    "\n",
    "interval = t_table_zscore(CONFIDENCE_LEVEL, SAMPLE_SIZE) * (SAMPLE_SIGMA / math.sqrt(SAMPLE_SIZE))\n",
    "\n",
    "print(\"Margin of error is: %.3f\" % interval)\n",
    "transform_margin_to_interval(SAMPLE_MU, interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 3: Finding the critical value t* for a desired confidence level\n",
    "![](img/confidence_intervals_p7.png)\n",
    "\n",
    "We need to solve \n",
    "\n",
    "$t^*\\frac{s_x}{\\sqrt{n}} \\leq margin\\ of \\ error$\n",
    "\n",
    "However getting a t score involves knowing sample size, which is exactly what we are trying to get.\n",
    "We'll have to use estimated standard deviation of the population and z instead of t:\n",
    "\n",
    "$z^*\\frac{\\sigma}{\\sqrt{n}} \\leq margin\\ of \\ error$\n",
    "\n",
    "which results in\n",
    "\n",
    "$n \\geq (\\frac{z^* * \\sigma}{margin\\ of\\ error})^2\n",
    "$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For confidence interval of 0.90, we need to be in < 1.645 standard deviations from the mean\n",
      "If we want margin of error less than 20, sample size should be > 16.91\n"
     ]
    }
   ],
   "source": [
    "CONFIDENCE_LEVEL = 0.9\n",
    "MARGIN_OF_ERROR = 20\n",
    "ESTIMATED_SIGMA = 50\n",
    "\n",
    "z = z_table_zscore(CONFIDENCE_LEVEL)\n",
    "min_sample_size = (z * ESTIMATED_SIGMA / MARGIN_OF_ERROR)**2\n",
    "\n",
    "print(\"If we want margin of error less than %d, sample size should be > %.2f\" % \\\n",
    "      (MARGIN_OF_ERROR, min_sample_size))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}