SKN  Statystyki - Python 3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Warsztaty Python w Data Science"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# https://github.com/MichalKorzycki/WarsztatPythonDataScience"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Data Wrangling 1\n",
    "- Tidy Data – co to jest\n",
    "- Data wrangling, munging, tidying\n",
    "- Biblioteka Pandas – wprowadzenie. Czytanie danych"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Zadanie 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stworz funkcję która zwróci drugi najmniejszy element listy\n",
    "\n",
    "druginajmniejszy([1, 2, 3, 4, 6])\n",
    "\n",
    "2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def druginajmniejszy(lista):\n",
    "    if len(lista) < 2:\n",
    "        return lista[0]\n",
    "    if lista[0] > lista[1]:\n",
    "        pierwszy, drugi = lista[1], lista[0]\n",
    "    else:\n",
    "        pierwszy, drugi = lista[0], lista[1]\n",
    "    for x in lista[2:]:\n",
    "        #print x, pierwszy, drugi\n",
    "        if x < drugi:\n",
    "            if x < pierwszy:\n",
    "                pierwszy, drugi = x, pierwszy\n",
    "            else:\n",
    "                drugi = x\n",
    "    return drugi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "druginajmniejszy([2,1,3,4,5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Zadanie 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stworz funkcję która usunie z listy najmniejszy element listy\n",
    "\n",
    "l = [4, 1, 2, 3]\n",
    "\n",
    "usun_min(l)\n",
    "\n",
    "l\n",
    "\n",
    "[4, 2, 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def usun_min(lista):\n",
    "    for i in range(len(lista)):\n",
    "        if lista[i]==min(lista):\n",
    "            del lista[i]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = [4, 1, 2, 3]\n",
    "usun_min(l)\n",
    "l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def usun_min(lista):\n",
    "    do_usuniecia = set()\n",
    "    for i in range(len(lista)):\n",
    "        if lista[i]==min(lista):\n",
    "            do_usuniecia.add(i)\n",
    "    for i in do_usuniecia:\n",
    "        del lista[i]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = [4, 1, 2, 3]\n",
    "usun_min(l)\n",
    "l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Zadanie 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stworz funkcję która usunie \"wyplaszczy\" zagnieżdzone listy\n",
    "\n",
    "l = [4, 1, [ 2, [7] ], 3]\n",
    "\n",
    "wyplaszcz(l)\n",
    "\n",
    "l\n",
    "\n",
    "[4, 1, 2, 7, 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def wyplaszcz(lista):\n",
    "    ret = []\n",
    "    for x in lista:\n",
    "        if type(x)==type([]):\n",
    "            for y in wyplaszcz(x):\n",
    "                ret.append(y)\n",
    "        else:\n",
    "            ret.append(x)\n",
    "    return ret"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = [4, 1, [ 2, [7] ], 3]\n",
    "l = wyplaszcz(l)\n",
    "l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def wyplaszcz(lista):\n",
    "    kopia = list(lista)\n",
    "    for i in range(len(lista)-1,-1,-1):\n",
    "        del lista[i]\n",
    "    for x in kopia:\n",
    "        if type(x)==type([]):\n",
    "            for y in wyplaszcz(x):\n",
    "                lista.append(y)\n",
    "        else:\n",
    "            lista.append(x)\n",
    "    return lista"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = [4, 1, [ 2, [7] ], 3]\n",
    "wyplaszcz(l)\n",
    "l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Wrangling, Munging, Tidying "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Wrangling\n",
    "- __Discovering__\n",
    "- __Structuring__\n",
    "- __Cleaning__\n",
    "- Enriching\n",
    "- Validating\n",
    "- Publishing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tidy Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wickham, Hadley - _\"Tidy Data\"_\n",
    "https://www.jstatsoft.org/index.php/jss/article/view/v059i10/v59i10.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- __Each variable you measure should be in one column.__\n",
    "- __Each different observation of that variable should be in a different row.__\n",
    "- There should be one table for each \"kind\" of variable.\n",
    "- If you have multiple tables, they should include a column in the table that allows them to be linked."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = {'year': [2010, 2011, 2012, 2011, 2012, 2010, 2011, 2012],\n",
    "        'team': ['Bears', 'Bears', 'Bears', 'Packers', 'Packers', 'Lions',\n",
    "                 'Lions', 'Lions'],\n",
    "        'wins': [11, 8, 10, 15, 11, 6, 10, 4],\n",
    "        'losses': [5, 8, 6, 1, 5, 10, 6, 12]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas\n",
    "\n",
    "football = pandas.DataFrame(data)\n",
    "print football"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football['year']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football.year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football[['year', 'wins', 'losses']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "index = pd.date_range('1/1/2000', periods=8)\n",
    "df = pd.DataFrame(np.random.randn(8, 3), index=index, columns=['A', 'B', 'C'])\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "newcols = []\n",
    "for i in range(len(df.columns)):\n",
    "    newcols.append(df.columns[i].lower())\n",
    "df.columns = newcols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Row Selection\n",
    "1. Slicing\n",
    "2. Individual index (iloc / loc)\n",
    "3. Boolean indexing\n",
    "4. Combination"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Slicing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[1:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football[3:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Individual index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football['wins']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iloc\n",
    "- An integer, e.g. 5.\n",
    "- A list or array of integers, e.g. [4, 3, 0].\n",
    "- A slice object with ints, e.g. 1:7.\n",
    "- A boolean array.\n",
    "- A function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football.iloc[[0,3]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loc\n",
    "- A single label\n",
    "- A list or array of labels, e.g. ['a', 'b', 'c'].\n",
    "- A slice object with labels, e.g. 'a':'f' __(WARNING)__\n",
    "- A boolean array\n",
    "- A callable function "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.loc['2000-01-03']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.loc['2000-01-03': '2000-01-04'] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Boolean indexing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football[football.wins > 10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combination"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "football[(football.wins > 10) & (football.team == \"Packers\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Nowe kolumny"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.suma = df.a + df.b + df.c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['suma'] = df.a + df.b + df.c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.suma"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Zadanie 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Podsumować dane dla zespołu `Packers`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Zadanie 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dodać do DataFrame `football` kolumnę `games_played`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Zadanie 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dodać do DataFrame `football` kolumnę `percentage_games_won`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Zadanie 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stworzyć funkcję `get_subsets(l)` która wygeneruje wszystkie podzbiory elementów z `l` korzystając z `frozenset`\n",
    "\n",
    "\n",
    "Przykład:\n",
    "    \n",
    "`get_subsets(['a', 'b', 'c', 'd'])`\n",
    "\n",
    "\n",
    "`{frozenset(),\n",
    " frozenset({'a', 'b', 'c'}),\n",
    " frozenset({'d'}),\n",
    " frozenset({'b'}),\n",
    " frozenset({'a'}),\n",
    " frozenset({'b', 'd'}),\n",
    " frozenset({'a', 'd'}),\n",
    " frozenset({'c'}),\n",
    " frozenset({'a', 'c', 'd'}),\n",
    " frozenset({'b', 'c'}),\n",
    " frozenset({'b', 'c', 'd'}),\n",
    " frozenset({'a', 'b'}),\n",
    " frozenset({'a', 'b', 'd'}),\n",
    " frozenset({'a', 'c'}),\n",
    " frozenset({'c', 'd'}),\n",
    " frozenset({'a', 'b', 'c', 'd'})}`"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}