12_Tips.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Tips\n",
    "## Deep copy\n",
    "### Problem definition\n",
    "Both lists and dicts have methods for creating copies of these objects:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating a copy of a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list1 = [1, 2, 3]\n",
    "list2 = list1[:]\n",
    "\n",
    "list2 is list1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Copy of a dict:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dict1 = dict(a=1, b=2, c=3)\n",
    "dict2 = dict1.copy()\n",
    "\n",
    "dict2 is dict1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "However this could be a problem in case your list (or dict) is not just a list, but a complex structure. Both `copy()`s actually copy only the referenced object contents. And because in a complex structure, elements store references to other complex (and usually mutable) objects, the copied object store them as well. This can lead to an undesired behaviour that we usually call **bugs**. Example demonstrating the problem:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "crew = [\n",
    "    dict(firstname='Tomas', lastname='Anderson'),\n",
    "    dict(firstname='Morpheus'),\n",
    "    dict(firstname='Trinity'),\n",
    "    dict(firstname='Cypher'),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "because `database` is a list, we use `[:]` to get a copy of it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "replica = crew[:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we may be thinking that `replica` is a full copy of `crew`, we use it to address and modify values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Morpheus'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "replica[1]['firstname']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We can safely delete or replace items in `replica`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "replica[2] = dict(firstname='Mouse')\n",
    "del replica[3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look, original `crew` is kept untouched:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'firstname': 'Tomas', 'lastname': 'Anderson'},\n",
       " {'firstname': 'Morpheus'},\n",
       " {'firstname': 'Trinity'},\n",
       " {'firstname': 'Cypher'}]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "crew"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "However if we attempt to modify a deeper value, like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "replica[0]['firstname'] = 'Neo'\n",
    "replica[0]['lastname'] = 'The One'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`crew` appears to be modified, which is probably undesired."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'firstname': 'Neo', 'lastname': 'The One'},\n",
       " {'firstname': 'Morpheus'},\n",
       " {'firstname': 'Trinity'},\n",
       " {'firstname': 'Cypher'}]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "crew"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The better way: deepcopy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from copy import deepcopy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Defining `crew` once again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "crew = [\n",
    "    dict(firstname='Tomas', lastname='Anderson'),\n",
    "    dict(firstname='Morpheus'),\n",
    "    dict(firstname='Trinity'),\n",
    "    dict(firstname='Cypher'),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "replica = deepcopy(crew)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'firstname': 'Neo', 'lastname': 'The One'},\n",
       " {'firstname': 'Morpheus'},\n",
       " {'firstname': 'Trinity'},\n",
       " {'firstname': 'Cypher'}]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "replica[0]['firstname'] = 'Neo'\n",
    "replica[0]['lastname'] = 'The One'\n",
    "\n",
    "replica"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`crew` remains unmodified:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'firstname': 'Tomas', 'lastname': 'Anderson'},\n",
       " {'firstname': 'Morpheus'},\n",
       " {'firstname': 'Trinity'},\n",
       " {'firstname': 'Cypher'}]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "crew"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## collections.defaultdict\n",
    "\n",
    "It's similar to `dict`, but the only difference is that a `defaultdict` will have a default value if that key has not been set yet. In other words, subscript `d['key']` creates an innitializing value and returns it.\n",
    "\n",
    "> defaultdict(default_factory[, ...]) --> dict with default factory. The default factory is called without arguments to produce a new value when a key is not present\n",
    "\n",
    "So `default_factory` is a callable (typically a class constructor) or it can be an user defined function that will be called on element access and returned value will be returned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "So this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([], {'a': 1, 'b': 2, 'c': []})"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dd = defaultdict(list, a=1, b=2)\n",
    "\n",
    "#\n",
    "dd['c'], dict(dd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "is equivalent to this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([], {'a': 1, 'b': 2, 'c': []})"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = dict(a=1, b=2)\n",
    "d['c'] = d.get('c', [])\n",
    "\n",
    "#\n",
    "d['c'], d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "When `defaultdict` can be handy (using `list` constructor):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'desserts': ['icecream', 'cake'],\n",
       " 'main courses': ['steak', 'grilled chicken'],\n",
       " 'soups': ['borsch']}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "menu = defaultdict(list)\n",
    "\n",
    "# default_factory=list creates an empty list for 'soups' and 'main courses' first\n",
    "menu['soups'].append('borsch')\n",
    "menu['main courses'] += ['steak', 'grilled chicken']\n",
    "\n",
    "# default_factory is not actually called\n",
    "menu['desserts'] = ['icecream', 'cake']\n",
    "\n",
    "dict(menu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "or using `dict` constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'desserts': {'cake': 7, 'icecream': 2},\n",
       " 'main courses': {'grilled chicken': 8, 'steak': 10},\n",
       " 'soups': {'borsch': 5}}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "menu = defaultdict(dict)\n",
    "\n",
    "# default_factory=list creates an empty list for 'soups' and 'main courses' first\n",
    "menu['soups'].update({'borsch': 5})\n",
    "menu['main courses']['steak'] = 10\n",
    "menu['main courses']['grilled chicken'] = 8\n",
    "\n",
    "# default_factory is not actually called\n",
    "menu['desserts'] = {'icecream': 2, 'cake': 7}\n",
    "\n",
    "dict(menu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## collections.OrderedDict\n",
    "\n",
    "Keeps the order of insertions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import OrderedDict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using usual `dict`, order of inserted elements is not guaranteed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: 0, 1: 1, 2: 2}"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = dict()\n",
    "\n",
    "d[1] = 1\n",
    "d[2] = 2\n",
    "d[0] = 0\n",
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 1, 2]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "`OrderedDict` is a drop-in replacement for `dict` that you can use if the order of inserted elements is important."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 0]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "od = OrderedDict()\n",
    "\n",
    "od[1] = 1\n",
    "od[2] = 2\n",
    "od[0] = 0\n",
    "\n",
    "od.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## collections.Counter\n",
    "\n",
    "A dict-like structure for keeping things counted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Creating an empty counter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter()"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = Counter()\n",
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'cats': 10})"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c['cats'] = 10\n",
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'cats': 10, 'dogs': 10})"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c['dogs'] += 10\n",
    "c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Or it can be initialized using a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'Estonia': 7, 'Latvia': 100, 'Lithuania': 10})"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "website_visitors = Counter({'Latvia': 100, 'Lithuania': 10, 'Estonia': 7})\n",
    "website_visitors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Can also be initialized from a sequence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'a': 5, 'b': 2, 'c': 1, 'd': 1, 'r': 2})"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "letters = Counter('abracadabra')\n",
    "letters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({1: 2, 20: 1, 100: 3})"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frequencies = Counter([100, 1, 20, 100, 1, 100])\n",
    "frequencies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "`Counter` supports addition and subtraction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'a': 1,\n",
       "         'c': 1,\n",
       "         'e': 2,\n",
       "         'k': 1,\n",
       "         'l': 3,\n",
       "         'm': 1,\n",
       "         'r': 1,\n",
       "         'w': 1,\n",
       "         'x': 1})"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "firstname_freq = Counter('clerk')\n",
    "lastname_freq = Counter('maxwell')\n",
    "\n",
    "firstname_freq + lastname_freq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'c': 1, 'k': 1, 'r': 1})"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# elements not found in the firstname are excluded\n",
    "# elements with negative count are excluded as well\n",
    "firstname_freq - lastname_freq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "It acts like a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('a', 5), ('r', 2), ('b', 2), ('c', 1), ('d', 1)]"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "letters = Counter('abracadabra')\n",
    "letters.items()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's easy to get total number of all counts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# it's actually just the len of 'abracadabra'\n",
    "sum(Counter('abracadabra').values())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "`Counter.elements()` returns the counted object with elements repeated but in an unpredicted order:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', 'a', 'a', 'a', 'a', 'r', 'r', 'b', 'b', 'c', 'd']"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(Counter('abracadabra').elements())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Apparently the most useful method is `Counter.most_common()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('a', 5)]"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "letters = Counter('abracadabra')\n",
    "\n",
    "# top 1\n",
    "letters.most_common(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.16"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}