1-introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bandicoot notebook\n",
    "\n",
    "_bandicoot_ is an open-source python toolbox to analyze mobile phone metadata. For more information, see: http://bandicoot.mit.edu/\n",
    "\n",
    "You can download this notebook on Github at: [https://github.com/cynddl/bandicoot-training](https://github.com/cynddl/bandicoot-training)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input files\n",
    "\n",
    "<img src=\"mini-mockups-01.png\" width=\"80%\" style=\"border: 1px solid #aaa\" />\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Records for the user 'ego'\n",
    "!head -n 5 data/ego.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# GPS locations of cell towers\n",
    "!head -n 5 data/antennas.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading a user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import bandicoot as bc\n",
    "\n",
    "U = bc.read_csv('ego', 'data/', 'data/antennas.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading your own data\n",
    "\n",
    "<br />\n",
    "<div class=\"alert alert-info\" role=\"alert\">\n",
    "<p><strong>Try bandicoot on your phone!</strong></p>\n",
    "<p>To use bandicoot with your own data, download our Android app at <a href=\"http://bandicoot.mit.edu/android\">bandicoot.mit.edu/android</a></p>\n",
    "</div>\n",
    "\n",
    "![](http://bandicoot.mit.edu/images/android-app.png)\n",
    "\n",
    "\n",
    "### Exercice\n",
    "\n",
    "If you have an Android phone, export your own metadata:\n",
    "\n",
    "1. Go to http://bandicoot.mit.edu/android and install the application on your phone;\n",
    "2. Export your data, send them by email to yourself, and upload the CSV file to the notebook.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Use the same syntax, such as:\n",
    "# U = bc.read_csv(\"my_metadata_file\", \"\")\n",
    "\n",
    "# It looks for the file my_metadata_file.csv at the root of the notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualization\n",
    "\n",
    "Export and serve an interactive visualization using:\n",
    "\n",
    "```python\n",
    "bc.visualization.run(U)\n",
    "```\n",
    "\n",
    "or export to the disk using:\n",
    "\n",
    "```python\n",
    "bc.visualization.export(U, 'my-viz-path')\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Export to the directory where the notebook is stored, in a new directory called \"viz\"\n",
    "bc.visualization.export(U, \"viz\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "hide_input": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from IPython.display import IFrame\n",
    "\n",
    "# Display the exported visualization\n",
    "IFrame(\"../files/viz/index.html\", \"100%\", 700)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Individual and spatial indicators\n",
    "\n",
    "Using bandicoot, compute aggregated indicators from ``bc.individual`` and ``bc.spatial``"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.percent_initiated_conversations(U)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.spatial.number_of_antennas(U)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.spatial.radius_of_gyration(U)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Let's play with indicators\n",
    "\n",
    "The signature of the ``active_days`` indicators is:\n",
    "\n",
    "```python\n",
    "bc.individual.active_days(user, groupby='week', interaction='callandtext', summary='default', split_week=False, split_day=False, filter_empty=True, datatype=None)\n",
    "```\n",
    "\n",
    "What does that mean?\n",
    "\n",
    "\n",
    "<hr />\n",
    "\n",
    "\n",
    "## The *‘groupby’* keyword\n",
    "\n",
    "<br />\n",
    "<div class=\"alert alert-info\" role=\"alert\">\n",
    "<p><strong>Weekly aggregation</strong></p>\n",
    "<p>By default, _bandicoot_ computes the indicators on a weekly basis and returns the average (mean) over all the weeks available and its standard deviation (std) in a nested dictionary.</p>\n",
    "\n",
    "<img src=\"mini-mockups-02.png\" width=\"80%\" style=\"border: 1px solid #aaa\" />\n",
    "\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.active_days(U)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``groupby`` keyword controls the aggregation:\n",
    "\n",
    "- ``groupby='week'`` to divide by week (by default),\n",
    "- ``groupby='month'`` to divide by month,\n",
    "- ``groupby=None`` to aggregate all values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.active_days(U, groupby='week')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.active_days(U, groupby='month')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.active_days(U, groupby=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The *‘summary’* keyword\n",
    "\n",
    "Some indicators such as *active\\_days* returns one number. Others, such as *duration\\_of\\_calls* returns a distribution.\n",
    "\n",
    "The summary keyword can take three values:\n",
    "\n",
    " - `summary='default'` to return mean and standard deviation,\n",
    " - `summary='extended'` for the second type of indicators, to return mean, sem, median, skewness and std of the distribution,\n",
    " - `summary=None` to return the full distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.call_duration(U)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.call_duration(U, summary='extended')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "bc.individual.call_duration(U, summary=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Splitting days and weeks\n",
    "\n",
    "- **``split_week``** divide records by 'all week', 'weekday', and 'weekend'.\n",
    "- **``split_day``** divide records by 'all day', 'day', and 'night'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.individual.active_days(U, split_week=True, split_day=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exporting indicators\n",
    "\n",
    "The function ``bc.utils.all`` computes automatically all indicators for a single user.\n",
    "\n",
    "You can use the same keywords to group by week/month/all time range, or return extended statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "features = bc.utils.all(U, groupby=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exporting in CSV and JSON\n",
    "\n",
    "bandicoot supports exports in CSV and JSON format. Both ``to_csv`` and ``to_json`` functions require either a single feature dictionnary, or a list of dictionnaries (for multiple users)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "bc.to_csv(features, 'demo_export_user.csv')\n",
    "bc.to_json(features, 'demo_export_user.json')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "!head demo_export_user.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!head -n 15 demo_export_user.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  },
  "toc": {
   "toc_cell": false,
   "toc_number_sections": false,
   "toc_threshold": 6,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}