1-reading-writing-files.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Analysis Using Python: A Beginner’s Guide Featuring NYC Open Data\n",
    "## Part 1: Reading and Writing Files in Python\n",
    "\n",
    "Mark Bauer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Table of Contents\n",
    "=================\n",
    "\n",
    "   Getting Started\n",
    " \n",
    "   1. Reading In Data from Local Folder\n",
    "       * 1.1 Reading in a CSV file  \n",
    "       * 1.2 Reading in an Excel file  \n",
    "       * 1.3 Reading in a JSON file  \n",
    "       * 1.4 Reading in a Shapefile  \n",
    "       * 1.5 Reading in a GeoJSON file   \n",
    "   2. Reading In Data from the Web         \n",
    "       * 2.1 Unzipping and reading in data as CSV to local folders  \n",
    "       * 2.2 Unzipping and reading in data as CSV from local folder  \n",
    "       * 2.3 Unzipping and reading in data as CSV in memory  \n",
    "   3. Reading In Data from NYC Open Data  \n",
    "       * 3.1 Reading in data as CSV in static form  \n",
    "       * 3.2 Reading in data as JSON in static form  \n",
    "       * 3.3 Reading in Shapefile data  \n",
    "       * 3.4 Reading in data from Socrata Open Data API (SODA)    \n",
    "   4. Writing Out Data  \n",
    "       * 4.1 Writing to a CSV file  \n",
    "       * 4.2 Writing to a Excel (xlsx) file  \n",
    "       * 4.3 Writing to a JSON file  \n",
    "       * 4.4 Writing to a Shapefile  \n",
    "   5. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Goal:** In this notebook, we will review various ways to read (load) and write (save) data from NYC Open Data. Specifically, we will focus on reading our data into a pandas dataframe.\n",
    "\n",
    "**Main Library:** [pandas](https://pandas.pydata.org/) is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import geopandas as gpd\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "import urllib\n",
    "import json\n",
    "import requests\n",
    "from io import BytesIO\n",
    "from sodapy import Socrata\n",
    "import zipfile\n",
    "from zipfile import ZipFile\n",
    "from os.path import basename\n",
    "from openpyxl import Workbook\n",
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Printing verions of Python modules and packages with **watermark** - the IPython magic extension.\n",
    "\n",
    "Documention for installing watermark: https://github.com/rasbt/watermark"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python implementation: CPython\n",
      "Python version       : 3.8.13\n",
      "IPython version      : 8.4.0\n",
      "\n",
      "numpy     : 1.23.1\n",
      "pandas    : 1.4.3\n",
      "geopandas : 0.11.1\n",
      "matplotlib: 3.5.2\n",
      "json      : 2.0.9\n",
      "requests  : 2.28.1\n",
      "sodapy    : 2.1.1\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "\n",
    "%watermark -v -p numpy,pandas,geopandas,matplotlib,json,requests,sodapy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started: Accessing the Building Footprints Dataset\n",
    "\n",
    "I've provided sample data in the [data folder](https://github.com/mebauer/data-analysis-using-python/tree/master/1-reading-writing-files/data), but follow these steps to access it to follow along. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Search NYC Open Data in Google"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints](images/1.nyc-open-data-google.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Search \"Building Footprints\" in NYC Open Data search bar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints](images/2.building-footprints-opendata-search.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Select \"Building Footprints\" Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints](images/3.building-footprints-dataset-link.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. The Building Footprints Dataset Page"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints](images/4.data-homepage.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dataset Link: \n",
    "https://data.cityofnewyork.us/Housing-Development/Building-Footprints/nqwf-w8eh\n",
    "\n",
    "Documentation/Metadata: \n",
    "https://github.com/CityOfNewYork/nyc-geo-metadata/blob/master/Metadata/Metadata_BuildingFootprints.md"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Building Footprints Dataset Identification**\n",
    "\n",
    "> **Here are a few things to note about the data:**\n",
    ">\n",
    "> - **Purpose:** This feature class is used by the NYC DOITT GIS group to maintain and distribute an accurate 'basemap' for NYC. The basemap provides the foundation upon virtually all other geospatial data with New York.\n",
    "> - **Description:** Building footprints represent the full perimeter outline of each building as viewed from directly above. Additional attribute information maintained for each feature includes: Building Identification Number (BIN); Borough, Block, and Lot information(BBL); ground elevation at building base; roof height above ground elevation; construction year, and feature type.\n",
    "> - **Source(s):** Annually captured aerial imagery, NYC Research of DOB records, or other image resources.\n",
    "> - **Publication Dates:** **Data**: 05/03/16<br>\n",
    "> - **Last Update:** Weekly<br>\n",
    "> - **Metadata:** 12/22/2016<br>\n",
    "> - **Update Frequency:** Features are updated daily by DoITT staff and a public release is available weekly on NYC Open Data. Every four years a citywide review is made of the building footprints and features are updated photogrammetrically.\n",
    "> - **Available Formats:** File Geodatabase Feature Class as part of the Planimetrics geodatabase and individual shapefile on the [NYC Open Data Portal](https://data.cityofnewyork.us/Housing-Development/Building-Footprints/nqwf-w8eh)\n",
    "> - **Use Limitations:** Open Data policies and restrictions apply. See [Terms of Use](http://www.nyc.gov/html/data/terms.html)\n",
    "> - **Access Rights:** Public\n",
    "> - **Links:** https://data.cityofnewyork.us/Housing-Development/Building-Footprints/nqwf-w8eh\n",
    "> - **Tags:** Buildings, Building footprint, BIN, Structure\n",
    "\n",
    "**Source:** \n",
    "https://github.com/CityOfNewYork/nyc-geo-metadata/blob/master/Metadata/Metadata_BuildingFootprints.md"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Reading In Data from a Local Folder\n",
    "\n",
    "Before we dive into retrieving data on NYC Open Data, let's practice reading data into a pandas dataframe from a local folder. If you'd like to see how I exported this sample data, visit the [data-wrangling notebook](https://github.com/mebauer/data-analysis-using-python/blob/master/1-reading-writing-files/data-wrangling.ipynb). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                nta_shape.shx            sample-data.json\r\n",
      "building-footprints.csv  sample-buildings.zip     sample-data.prj\r\n",
      "nta_shape.cpg            sample-data.cpg          sample-data.shp\r\n",
      "nta_shape.dbf            sample-data.csv          sample-data.shx\r\n",
      "nta_shape.geojson        sample-data.dbf          sample-data.xlsx\r\n",
      "nta_shape.prj            sample-data.geojson      \u001b[34munzipped-data\u001b[m\u001b[m/\r\n",
      "nta_shape.shp            sample-data.gpkg\r\n"
     ]
    }
   ],
   "source": [
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 Reading in a CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.7940773567428 40.680625171...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4260357</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/10/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1184712</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.7940773567428 40.680625171...  NaN  4260357   \n",
       "3  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "4  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/10/2017 12:00:00 AM  Constructed   1184712   29.000000   \n",
       "3      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN      BASE_BBL    MPLUTO_BBL  \\\n",
       "0     2100.0        15.0           0          0  3.044521e+09  3.044521e+09   \n",
       "1     2100.0        37.0           0          0  4.018780e+09  4.018780e+09   \n",
       "2     2100.0        20.0           0          0  4.120060e+09  4.120060e+09   \n",
       "3     2100.0       112.0           0          0  4.030600e+09  4.030600e+09   \n",
       "4     2100.0        74.0           0          0  4.066560e+09  4.066560e+09   \n",
       "\n",
       "   GEOMSOURCE  \n",
       "0  Other (Man  \n",
       "1  Other (Man  \n",
       "2  Other (Man  \n",
       "3  Photogramm  \n",
       "4  Photogramm  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_csv('data/sample-data.csv')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 15,498 rows and 15 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Sanity check**\n",
    "\n",
    "We use pandas `.head()` method to preview the first five rows of the dataframe.\n",
    "\n",
    "We use pandas `.shape` method to print the dimensions of the dataframe (i.e. number of rows, number of columns).\n",
    "\n",
    "We will use these two methods throughout the examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 Reading in a JSON file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>None</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.7940773567428 40.680625171...</td>\n",
       "      <td>None</td>\n",
       "      <td>4260357</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/10/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1184712</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>None</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>None</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom  NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  None  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  None  4540159   \n",
       "2  MULTIPOLYGON (((-73.7940773567428 40.680625171...  None  4260357   \n",
       "3  MULTIPOLYGON (((-73.87805078807256 40.71475698...  None  4540051   \n",
       "4  MULTIPOLYGON (((-73.81520745135124 40.73053646...  None  4545453   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0        2011  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1        2010  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2        2010  08/10/2017 12:00:00 AM  Constructed   1184712   29.000000   \n",
       "3        2010  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "4        2012  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN      BASE_BBL    MPLUTO_BBL  \\\n",
       "0     2100.0        15.0           0          0  3.044521e+09  3.044521e+09   \n",
       "1     2100.0        37.0           0          0  4.018780e+09  4.018780e+09   \n",
       "2     2100.0        20.0           0          0  4.120060e+09  4.120060e+09   \n",
       "3     2100.0       112.0           0          0  4.030600e+09  4.030600e+09   \n",
       "4     2100.0        74.0           0          0  4.066560e+09  4.066560e+09   \n",
       "\n",
       "   GEOMSOURCE  \n",
       "0  Other (Man  \n",
       "1  Other (Man  \n",
       "2  Other (Man  \n",
       "3  Photogramm  \n",
       "4  Photogramm  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_json('data/sample-data.json')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 15,498 rows and 15 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Reading in an Excel file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.7940773567428 40.680625171...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4260357</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/10/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1184712</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.7940773567428 40.680625171...  NaN  4260357   \n",
       "3  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "4  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0        2011  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1        2010  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2        2010  08/10/2017 12:00:00 AM  Constructed   1184712   29.000000   \n",
       "3        2010  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "4        2012  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN      BASE_BBL    MPLUTO_BBL  \\\n",
       "0     2100.0        15.0           0          0  3.044521e+09  3.044521e+09   \n",
       "1     2100.0        37.0           0          0  4.018780e+09  4.018780e+09   \n",
       "2     2100.0        20.0           0          0  4.120060e+09  4.120060e+09   \n",
       "3     2100.0       112.0           0          0  4.030600e+09  4.030600e+09   \n",
       "4     2100.0        74.0           0          0  4.066560e+09  4.066560e+09   \n",
       "\n",
       "   GEOMSOURCE  \n",
       "0  Other (Man  \n",
       "1  Other (Man  \n",
       "2  Other (Man  \n",
       "3  Photogramm  \n",
       "4  Photogramm  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_excel('data/sample-data.xlsx')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 15,498 rows and 15 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.4 Reading in a Shapefile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>bin</th>\n",
       "      <th>cnstrct_yr</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>geomsource</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>date_lstmo</th>\n",
       "      <th>time_lstmo</th>\n",
       "      <th>lststatype</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>name</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3044520924</td>\n",
       "      <td>3394834.0</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>1250314.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>15.0</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.87172 40.65519, -73.87179 40.655...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4018780115</td>\n",
       "      <td>4540159.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>201366.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>37.0</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.86650 40.74236, -73.86645 40.742...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4120060029</td>\n",
       "      <td>4260357.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1184712.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>20.0</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2017-08-10</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4120060029</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.79408 40.68063, -73.79407 40.680...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4030600139</td>\n",
       "      <td>4540051.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1171655.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>112.0</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.87805 40.71476, -73.87787 40.714...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4066560052</td>\n",
       "      <td>4545453.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>1118502.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>74.0</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.81521 40.73054, -73.81546 40.730...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     base_bbl        bin  cnstrct_yr   doitt_id  feat_code  geomsource  \\\n",
       "0  3044520924  3394834.0      2011.0  1250314.0     2100.0  Other (Man   \n",
       "1  4018780115  4540159.0      2010.0   201366.0     2100.0  Other (Man   \n",
       "2  4120060029  4260357.0      2010.0  1184712.0     2100.0  Other (Man   \n",
       "3  4030600139  4540051.0      2010.0  1171655.0     2100.0  Photogramm   \n",
       "4  4066560052  4545453.0      2012.0  1118502.0     2100.0  Photogramm   \n",
       "\n",
       "   groundelev  heightroof  date_lstmo    time_lstmo   lststatype  mpluto_bbl  \\\n",
       "0        15.0   26.000000  2017-08-22  00:00:00.000  Constructed  3044520924   \n",
       "1        37.0   28.000000  2017-08-22  00:00:00.000  Constructed  4018780115   \n",
       "2        20.0   29.000000  2017-08-10  00:00:00.000  Constructed  4120060029   \n",
       "3       112.0   28.330225  2017-08-22  00:00:00.000  Constructed  4030600139   \n",
       "4        74.0   16.640000  2017-08-22  00:00:00.000  Constructed  4066560052   \n",
       "\n",
       "   name  shape_area  shape_len  \\\n",
       "0  None         0.0        0.0   \n",
       "1  None         0.0        0.0   \n",
       "2  None         0.0        0.0   \n",
       "3  None         0.0        0.0   \n",
       "4  None         0.0        0.0   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-73.87172 40.65519, -73.87179 40.655...  \n",
       "1  POLYGON ((-73.86650 40.74236, -73.86645 40.742...  \n",
       "2  POLYGON ((-73.79408 40.68063, -73.79407 40.680...  \n",
       "3  POLYGON ((-73.87805 40.71476, -73.87787 40.714...  \n",
       "4  POLYGON ((-73.81521 40.73054, -73.81546 40.730...  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a geodataframe\n",
    "path = 'data/sample-data.shp'\n",
    "gdf = gpd.read_file(path)\n",
    "\n",
    "# previewing first five rows in data\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 15,498 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = gdf.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 Reading in a GeoJSON file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>bin</th>\n",
       "      <th>cnstrct_yr</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>geomsource</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>date_lstmo</th>\n",
       "      <th>time_lstmo</th>\n",
       "      <th>lststatype</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>name</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3044520924</td>\n",
       "      <td>3394834.0</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>1250314.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>15.0</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.87172 40.65519, -73.87179 40.655...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4018780115</td>\n",
       "      <td>4540159.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>201366.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>37.0</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.86650 40.74236, -73.86645 40.742...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4120060029</td>\n",
       "      <td>4260357.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1184712.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>20.0</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2017-08-10</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4120060029</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.79408 40.68063, -73.79407 40.680...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4030600139</td>\n",
       "      <td>4540051.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1171655.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>112.0</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.87805 40.71476, -73.87787 40.714...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4066560052</td>\n",
       "      <td>4545453.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>1118502.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>74.0</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.81521 40.73054, -73.81546 40.730...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     base_bbl        bin  cnstrct_yr   doitt_id  feat_code  geomsource  \\\n",
       "0  3044520924  3394834.0      2011.0  1250314.0     2100.0  Other (Man   \n",
       "1  4018780115  4540159.0      2010.0   201366.0     2100.0  Other (Man   \n",
       "2  4120060029  4260357.0      2010.0  1184712.0     2100.0  Other (Man   \n",
       "3  4030600139  4540051.0      2010.0  1171655.0     2100.0  Photogramm   \n",
       "4  4066560052  4545453.0      2012.0  1118502.0     2100.0  Photogramm   \n",
       "\n",
       "   groundelev  heightroof  date_lstmo time_lstmo   lststatype  mpluto_bbl  \\\n",
       "0        15.0   26.000000  2017-08-22   00:00:00  Constructed  3044520924   \n",
       "1        37.0   28.000000  2017-08-22   00:00:00  Constructed  4018780115   \n",
       "2        20.0   29.000000  2017-08-10   00:00:00  Constructed  4120060029   \n",
       "3       112.0   28.330225  2017-08-22   00:00:00  Constructed  4030600139   \n",
       "4        74.0   16.640000  2017-08-22   00:00:00  Constructed  4066560052   \n",
       "\n",
       "   name  shape_area  shape_len  \\\n",
       "0  None         0.0        0.0   \n",
       "1  None         0.0        0.0   \n",
       "2  None         0.0        0.0   \n",
       "3  None         0.0        0.0   \n",
       "4  None         0.0        0.0   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-73.87172 40.65519, -73.87179 40.655...  \n",
       "1  POLYGON ((-73.86650 40.74236, -73.86645 40.742...  \n",
       "2  POLYGON ((-73.79408 40.68063, -73.79407 40.680...  \n",
       "3  POLYGON ((-73.87805 40.71476, -73.87787 40.714...  \n",
       "4  POLYGON ((-73.81521 40.73054, -73.81546 40.730...  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = 'data/sample-data.geojson'\n",
    "gdf = gpd.read_file(path)\n",
    "\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 15,498 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = gdf.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Reading in Data from the Web"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another popular dataset is NYC's PLUTO dataset. We will use this one because it comes in a zip file.\n",
    "\n",
    "- **Description**: Extensive land use and geographic data at the tax lot level in comma–separated values (CSV) file format. The PLUTO files contain more than seventy fields derived from data maintained by city agencies.\n",
    "- **Dataset Link**: https://www1.nyc.gov/site/planning/data-maps/open-data/dwn-pluto-mappluto.page\n",
    "- **Data Dictionary**: https://www1.nyc.gov/assets/planning/download/pdf/data-maps/open-data/pluto_datadictionary.pdf?v=20v1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints_csv](images/pluto-csv.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Unzipping and reading in data as CSV to local folder\n",
    "\n",
    "We will retrieve, unzip and read in data in our downloads folder.  \n",
    "**Note: right-click the Download icon and copy the link address as the url.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('/Users/geribauer/Downloads/PLUTO.gz',\n",
       " <http.client.HTTPMessage at 0x16ca48250>)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# download url of dataset\n",
    "url = 'https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/nyc_pluto_20v1_csv.zip'\n",
    "\n",
    "# a path to our downloads folder \n",
    "downloads_path = str(Path.home() / \"Downloads\")\n",
    "\n",
    "# specify file name\n",
    "filename = 'PLUTO.gz'\n",
    "\n",
    "# a path to our file from our downloads path\n",
    "filepath = os.path.join(downloads_path, filename)\n",
    "\n",
    "# retrieving data \n",
    "urllib.request.urlretrieve(url, filepath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['pluto_20v1.csv', 'PLUTODD20v1.pdf', 'PlutoReadme20v1.pdf']\n"
     ]
    }
   ],
   "source": [
    "# open zipfile and saving items in our zipfolder\n",
    "items = zipfile.ZipFile(filepath)\n",
    "\n",
    "# available files in the container\n",
    "print(items.namelist())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# opening zipfile using 'with' keyword in read mode\n",
    "with zipfile.ZipFile(filepath, 'r') as file:\n",
    "    \n",
    "    # extract all files inside the zip file\n",
    "    file.extractall(downloads_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>borough</th>\n",
       "      <th>block</th>\n",
       "      <th>lot</th>\n",
       "      <th>cd</th>\n",
       "      <th>ct2010</th>\n",
       "      <th>cb2010</th>\n",
       "      <th>schooldist</th>\n",
       "      <th>council</th>\n",
       "      <th>zipcode</th>\n",
       "      <th>firecomp</th>\n",
       "      <th>...</th>\n",
       "      <th>dcasdate</th>\n",
       "      <th>zoningdate</th>\n",
       "      <th>landmkdate</th>\n",
       "      <th>basempdate</th>\n",
       "      <th>masdate</th>\n",
       "      <th>polidate</th>\n",
       "      <th>edesigdate</th>\n",
       "      <th>geom</th>\n",
       "      <th>dcpedited</th>\n",
       "      <th>notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BK</td>\n",
       "      <td>834</td>\n",
       "      <td>46</td>\n",
       "      <td>307.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>2001.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>11220.0</td>\n",
       "      <td>L114</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>QN</td>\n",
       "      <td>4042</td>\n",
       "      <td>106</td>\n",
       "      <td>407.0</td>\n",
       "      <td>929.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>11356.0</td>\n",
       "      <td>E297</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000007...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BK</td>\n",
       "      <td>4679</td>\n",
       "      <td>17</td>\n",
       "      <td>317.0</td>\n",
       "      <td>866.0</td>\n",
       "      <td>3002.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>11203.0</td>\n",
       "      <td>L174</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000006...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>6</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>7</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 99 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  borough  block  lot     cd  ct2010  cb2010  schooldist  council  zipcode  \\\n",
       "0      BK    834   46  307.0   106.0  2001.0        20.0     38.0  11220.0   \n",
       "1      QN   4042  106  407.0   929.0  3000.0        25.0     19.0  11356.0   \n",
       "2      BK   4679   17  317.0   866.0  3002.0        18.0     41.0  11203.0   \n",
       "3      BK   7831    6  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "4      BK   7831    7  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "\n",
       "  firecomp  ...  dcasdate  zoningdate  landmkdate basempdate masdate polidate  \\\n",
       "0     L114  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "1     E297  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "2     L174  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "3     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "4     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "\n",
       "  edesigdate                                               geom dcpedited  \\\n",
       "0        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "1        NaN  0106000020E61000000100000001030000000100000007...       NaN   \n",
       "2        NaN  0106000020E61000000100000001030000000100000006...       NaN   \n",
       "3        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "4        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "\n",
       "  notes  \n",
       "0   NaN  \n",
       "1   NaN  \n",
       "2   NaN  \n",
       "3   NaN  \n",
       "4   NaN  \n",
       "\n",
       "[5 rows x 99 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# a path to our file from our downloads path\n",
    "file = 'pluto_20v1.csv'\n",
    "filepath = os.path.join(downloads_path, file)\n",
    "pluto_data = pd.read_csv(filepath, low_memory=False)\n",
    "\n",
    "pluto_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 859,172 rows and 99 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = pluto_data.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Unzipping and reading in data as CSV from local folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# note: ive already placed data into this zip file\n",
    "filename = 'data/sample-buildings.zip'\n",
    "\n",
    "# opening zip using 'with' keyword in read mode\n",
    "with zipfile.ZipFile(filename, 'r') as file:\n",
    "    \n",
    "    # extracing all items in our zipfile\n",
    "    file.extractall('data/unzipped-data')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sample-data.csv\r\n"
     ]
    }
   ],
   "source": [
    "# list files in this file path\n",
    "%ls data/unzipped-data/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>3.044521e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>4.018780e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.7940773567428 40.680625171...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4260357</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/10/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1184712</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>4.120060e+09</td>\n",
       "      <td>Other (Man</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>4.030600e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>4.066560e+09</td>\n",
       "      <td>Photogramm</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.7940773567428 40.680625171...  NaN  4260357   \n",
       "3  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "4  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/10/2017 12:00:00 AM  Constructed   1184712   29.000000   \n",
       "3      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN      BASE_BBL    MPLUTO_BBL  \\\n",
       "0     2100.0        15.0           0          0  3.044521e+09  3.044521e+09   \n",
       "1     2100.0        37.0           0          0  4.018780e+09  4.018780e+09   \n",
       "2     2100.0        20.0           0          0  4.120060e+09  4.120060e+09   \n",
       "3     2100.0       112.0           0          0  4.030600e+09  4.030600e+09   \n",
       "4     2100.0        74.0           0          0  4.066560e+09  4.066560e+09   \n",
       "\n",
       "   GEOMSOURCE  \n",
       "0  Other (Man  \n",
       "1  Other (Man  \n",
       "2  Other (Man  \n",
       "3  Photogramm  \n",
       "4  Photogramm  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "path = 'data/unzipped-data/sample-data.csv'\n",
    "sample_buidlings = pd.read_csv(path)\n",
    "\n",
    "# previewing first five rows of data\n",
    "sample_buidlings.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 15,498 rows and 15 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = sample_buidlings.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Unzipping and reading in data as a CSV in-memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File in zip: pluto_20v1.csv\n",
      "File in zip: PLUTODD20v1.pdf\n",
      "File in zip: PlutoReadme20v1.pdf\n"
     ]
    }
   ],
   "source": [
    "# our download link\n",
    "url = 'https://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/nyc_pluto_20v1_csv.zip'\n",
    "\n",
    "# reading in our zipfile data in-memory\n",
    "content = requests.get(url)\n",
    "zf = ZipFile(BytesIO(content.content))\n",
    "\n",
    "# printing files in our zipfile\n",
    "for item in zf.namelist():\n",
    "    print(\"File in zip: {}\".format(item))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>borough</th>\n",
       "      <th>block</th>\n",
       "      <th>lot</th>\n",
       "      <th>cd</th>\n",
       "      <th>ct2010</th>\n",
       "      <th>cb2010</th>\n",
       "      <th>schooldist</th>\n",
       "      <th>council</th>\n",
       "      <th>zipcode</th>\n",
       "      <th>firecomp</th>\n",
       "      <th>...</th>\n",
       "      <th>dcasdate</th>\n",
       "      <th>zoningdate</th>\n",
       "      <th>landmkdate</th>\n",
       "      <th>basempdate</th>\n",
       "      <th>masdate</th>\n",
       "      <th>polidate</th>\n",
       "      <th>edesigdate</th>\n",
       "      <th>geom</th>\n",
       "      <th>dcpedited</th>\n",
       "      <th>notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BK</td>\n",
       "      <td>834</td>\n",
       "      <td>46</td>\n",
       "      <td>307.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>2001.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>11220.0</td>\n",
       "      <td>L114</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>QN</td>\n",
       "      <td>4042</td>\n",
       "      <td>106</td>\n",
       "      <td>407.0</td>\n",
       "      <td>929.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>11356.0</td>\n",
       "      <td>E297</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000007...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BK</td>\n",
       "      <td>4679</td>\n",
       "      <td>17</td>\n",
       "      <td>317.0</td>\n",
       "      <td>866.0</td>\n",
       "      <td>3002.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>11203.0</td>\n",
       "      <td>L174</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000006...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>6</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BK</td>\n",
       "      <td>7831</td>\n",
       "      <td>7</td>\n",
       "      <td>318.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>1002.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>11234.0</td>\n",
       "      <td>L159</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0106000020E61000000100000001030000000100000005...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 99 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  borough  block  lot     cd  ct2010  cb2010  schooldist  council  zipcode  \\\n",
       "0      BK    834   46  307.0   106.0  2001.0        20.0     38.0  11220.0   \n",
       "1      QN   4042  106  407.0   929.0  3000.0        25.0     19.0  11356.0   \n",
       "2      BK   4679   17  317.0   866.0  3002.0        18.0     41.0  11203.0   \n",
       "3      BK   7831    6  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "4      BK   7831    7  318.0   676.0  1002.0        22.0     46.0  11234.0   \n",
       "\n",
       "  firecomp  ...  dcasdate  zoningdate  landmkdate basempdate masdate polidate  \\\n",
       "0     L114  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "1     E297  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "2     L174  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "3     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "4     L159  ...       NaN         NaN         NaN        NaN     NaN      NaN   \n",
       "\n",
       "  edesigdate                                               geom dcpedited  \\\n",
       "0        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "1        NaN  0106000020E61000000100000001030000000100000007...       NaN   \n",
       "2        NaN  0106000020E61000000100000001030000000100000006...       NaN   \n",
       "3        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "4        NaN  0106000020E61000000100000001030000000100000005...       NaN   \n",
       "\n",
       "  notes  \n",
       "0   NaN  \n",
       "1   NaN  \n",
       "2   NaN  \n",
       "3   NaN  \n",
       "4   NaN  \n",
       "\n",
       "[5 rows x 99 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read our csv data into a dataframe from our zipfile\n",
    "file = 'pluto_20v1.csv'\n",
    "pluto_data = pd.read_csv(zf.open(file), low_memory=False)\n",
    "\n",
    "# previewing the first five rows of data\n",
    "pluto_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 859,172 rows and 99 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = pluto_data.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Reading in data from NYC Open Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Reading in data as CSV in static form"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints_csv](images/building-footprints-csv.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Note: \n",
    "The buildings footprints `dataset identifier` changes weekly, and so does the data api path. Click on the API Docs page and verify the correct dataset identifier. If you're not working with the correct id, you will receive a `HTTP Error`. Screenshots below:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Click on API Docs**\n",
    "![building_footprints_csv](images/api_docs.png)\n",
    "\n",
    "**Grab the updated dataset identifier**\n",
    "![building_footprints_csv](images/api_docs_dataset_id.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `dataset identifier` is inserted into the api path below:  \n",
    "url = `https://data.cityofnewyork.us/api/views/{DATASET_IDENTIFIER}/rows.csv?accessType=DOWNLOAD`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.96664570466969 40.62599676...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3170958</td>\n",
       "      <td>1925.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>96807</td>\n",
       "      <td>29.749853</td>\n",
       "      <td>2100</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-74.16790202462265 40.63936048...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5028452</td>\n",
       "      <td>1965.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>326368</td>\n",
       "      <td>22.630000</td>\n",
       "      <td>2100</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F5F8CDA5-69E2-46F8-8F69-BA95C025B520}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-74.19510813278613 40.55610681...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5078368</td>\n",
       "      <td>1970.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746627</td>\n",
       "      <td>35.760000</td>\n",
       "      <td>2100</td>\n",
       "      <td>51.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{9F644794-F72C-4582-9E5E-B337E2B97068}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.96113466505085 40.57743931...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3245111</td>\n",
       "      <td>1928.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>786626</td>\n",
       "      <td>37.500000</td>\n",
       "      <td>2100</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F916B22D-E25B-44AE-9FA9-2A51191B9CDF}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.75421559146166 40.75591276...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4161096</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746409</td>\n",
       "      <td>18.015113</td>\n",
       "      <td>2100</td>\n",
       "      <td>93.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.96664570466969 40.62599676...  NaN  3170958   \n",
       "1  MULTIPOLYGON (((-74.16790202462265 40.63936048...  NaN  5028452   \n",
       "2  MULTIPOLYGON (((-74.19510813278613 40.55610681...  NaN  5078368   \n",
       "3  MULTIPOLYGON (((-73.96113466505085 40.57743931...  NaN  3245111   \n",
       "4  MULTIPOLYGON (((-73.75421559146166 40.75591276...  NaN  4161096   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      1925.0  08/22/2017 12:00:00 AM  Constructed     96807   29.749853   \n",
       "1      1965.0  08/22/2017 12:00:00 AM  Constructed    326368   22.630000   \n",
       "2      1970.0  08/22/2017 12:00:00 AM  Constructed    746627   35.760000   \n",
       "3      1928.0  08/22/2017 12:00:00 AM  Constructed    786626   37.500000   \n",
       "4      1950.0  08/22/2017 12:00:00 AM  Constructed    746409   18.015113   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        40.0           0          0  3065220021  3065220021   \n",
       "1       2100        39.0           0          0  5012640036  5012640036   \n",
       "2       2100        51.0           0          0  5060190091  5060190091   \n",
       "3       2100         6.0           0          0  3086910048  3086910048   \n",
       "4       2100        93.0           0          0  4075020005  4075020005   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Photogramm  {31298F86-3088-4F53-B3DB-71A9EFA6FA1F}  \n",
       "1  Photogramm  {F5F8CDA5-69E2-46F8-8F69-BA95C025B520}  \n",
       "2  Photogramm  {9F644794-F72C-4582-9E5E-B337E2B97068}  \n",
       "3  Photogramm  {F916B22D-E25B-44AE-9FA9-2A51191B9CDF}  \n",
       "4  Photogramm  {525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reading in data as a url from NYC Open Data\n",
    "url = 'https://data.cityofnewyork.us/api/views/qb5r-6dgf/rows.csv?accessType=DOWNLOAD'\n",
    "\n",
    "# saving data as a pandas dataframe named 'df_csv'\n",
    "df_csv = pd.read_csv(url)\n",
    "\n",
    "# previewing the first five rows \n",
    "df_csv.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 1,083,347 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df_csv.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Reading in data as JSON in static form\n",
    "\n",
    "I don't use this method often in my data wrangling process, but it is available and the steps are below. It's a bit of a manual process and best to avoid if you can."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints_csv](images/building-footprints-json.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url = 'https://data.cityofnewyork.us/api/views/qb5r-6dgf/rows.json?accessType=DOWNLOAD'\n",
    "\n",
    "# loads a json object as a python object\n",
    "with urllib.request.urlopen(url) as url:\n",
    "    data = json.loads(url.read().decode())\n",
    "    \n",
    "# identifying type of python object\n",
    "type(data)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['meta', 'data'])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reviewing the dictionairy's keys\n",
    "data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# looking at the type of the data key\n",
    "type(data['data'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1083347"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# previewing the length of items in the data list\n",
    "len(data['data'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['row-bgk2-jf54-8tu8',\n",
       " '00000000-0000-0000-0336-65AE1353DFEE',\n",
       " 0,\n",
       " 1713361986,\n",
       " None,\n",
       " 1713361986,\n",
       " None,\n",
       " '{ }',\n",
       " 'MULTIPOLYGON (((-73.96664570466969 40.62599676998366, -73.96684846176461 40.625977490862574, -73.96685938726297 40.62604419372411, -73.96661621040211 40.62606731716107, -73.96660638332114 40.626007324369795, -73.96664680403327 40.626003480977275, -73.96664570466969 40.62599676998366)))',\n",
       " None,\n",
       " '3170958',\n",
       " '1925',\n",
       " '2017-08-22T00:00:00',\n",
       " 'Constructed',\n",
       " '96807',\n",
       " '29.74985318',\n",
       " '2100',\n",
       " '40',\n",
       " '0.0',\n",
       " '0.0',\n",
       " '3065220021',\n",
       " '3065220021',\n",
       " 'Photogramm',\n",
       " '{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}']"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# previewing the first row of our data\n",
    "data['data'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieving column names from the meta data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['view'])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['meta'].keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id\n",
      "name\n",
      "assetType\n",
      "averageRating\n",
      "createdAt\n",
      "displayType\n",
      "downloadCount\n",
      "hideFromCatalog\n",
      "hideFromDataJson\n",
      "newBackend\n",
      "numberOfComments\n",
      "oid\n",
      "provenance\n",
      "publicationAppendEnabled\n",
      "publicationDate\n",
      "publicationGroup\n",
      "publicationStage\n",
      "rowsUpdatedAt\n",
      "rowsUpdatedBy\n",
      "tableId\n",
      "totalTimesRated\n",
      "viewCount\n",
      "viewLastModified\n",
      "viewType\n",
      "approvals\n",
      "clientContext\n",
      "columns\n",
      "grants\n",
      "metadata\n",
      "owner\n",
      "query\n",
      "rights\n",
      "tableAuthor\n",
      "flags\n"
     ]
    }
   ],
   "source": [
    "keys = data['meta']['view'].keys()\n",
    "\n",
    "for key in keys:\n",
    "    print(key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "length of columns: 24\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'id': -1,\n",
       "  'name': 'sid',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':sid',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'id',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':id',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'position',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':position',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'created_at',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':created_at',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']},\n",
       " {'id': -1,\n",
       "  'name': 'created_meta',\n",
       "  'dataTypeName': 'meta_data',\n",
       "  'fieldName': ':created_meta',\n",
       "  'position': 0,\n",
       "  'renderTypeName': 'meta_data',\n",
       "  'format': {},\n",
       "  'flags': ['hidden']}]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# locating our columns (i.e. field names) and saving as a new variable called 'cols'\n",
    "cols = data['meta']['view']['columns']\n",
    "print('length of columns: {}'.format(len(cols)))\n",
    "\n",
    "# previewing first five\n",
    "cols[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ":sid\n",
      ":id\n",
      ":position\n",
      ":created_at\n",
      ":created_meta\n",
      ":updated_at\n",
      ":updated_meta\n",
      ":meta\n",
      "the_geom\n",
      "name\n",
      "bin\n",
      "cnstrct_yr\n",
      "lstmoddate\n",
      "lststatype\n",
      "doitt_id\n",
      "heightroof\n",
      "feat_code\n",
      "groundelev\n",
      "shape_area\n",
      "shape_len\n",
      "base_bbl\n",
      "mpluto_bbl\n",
      "geomsource\n",
      "globalid\n"
     ]
    }
   ],
   "source": [
    "col_names = []\n",
    "\n",
    "for col in cols:\n",
    "    print(col['fieldName'])\n",
    "    col_names.append(col['fieldName'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>:sid</th>\n",
       "      <th>:id</th>\n",
       "      <th>:position</th>\n",
       "      <th>:created_at</th>\n",
       "      <th>:created_meta</th>\n",
       "      <th>:updated_at</th>\n",
       "      <th>:updated_meta</th>\n",
       "      <th>:meta</th>\n",
       "      <th>the_geom</th>\n",
       "      <th>name</th>\n",
       "      <th>...</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>geomsource</th>\n",
       "      <th>globalid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>row-bgk2-jf54-8tu8</td>\n",
       "      <td>00000000-0000-0000-0336-65AE1353DFEE</td>\n",
       "      <td>0</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-73.96664570466969 40.62599676...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>96807</td>\n",
       "      <td>29.74985318</td>\n",
       "      <td>2100</td>\n",
       "      <td>40</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>row-qkid-3vfc.5kxn</td>\n",
       "      <td>00000000-0000-0000-310F-DC93101B7FB2</td>\n",
       "      <td>0</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-74.16790202462265 40.63936048...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>326368</td>\n",
       "      <td>22.63</td>\n",
       "      <td>2100</td>\n",
       "      <td>39</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F5F8CDA5-69E2-46F8-8F69-BA95C025B520}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>row-cw7t-bpix_8nr6</td>\n",
       "      <td>00000000-0000-0000-27DC-95188F2F91C9</td>\n",
       "      <td>0</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-74.19510813278613 40.55610681...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>746627</td>\n",
       "      <td>35.76</td>\n",
       "      <td>2100</td>\n",
       "      <td>51</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{9F644794-F72C-4582-9E5E-B337E2B97068}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>row-xa8y-grfu.cue8</td>\n",
       "      <td>00000000-0000-0000-713B-3335FE6E4BFC</td>\n",
       "      <td>0</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-73.96113466505085 40.57743931...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>786626</td>\n",
       "      <td>37.5</td>\n",
       "      <td>2100</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F916B22D-E25B-44AE-9FA9-2A51191B9CDF}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>row-d98p.ghx9-25dk</td>\n",
       "      <td>00000000-0000-0000-1E0D-153CE9C006DD</td>\n",
       "      <td>0</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>1713361986</td>\n",
       "      <td>None</td>\n",
       "      <td>{ }</td>\n",
       "      <td>MULTIPOLYGON (((-73.75421559146166 40.75591276...</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>746409</td>\n",
       "      <td>18.01511294</td>\n",
       "      <td>2100</td>\n",
       "      <td>93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 :sid                                   :id  :position  \\\n",
       "0  row-bgk2-jf54-8tu8  00000000-0000-0000-0336-65AE1353DFEE          0   \n",
       "1  row-qkid-3vfc.5kxn  00000000-0000-0000-310F-DC93101B7FB2          0   \n",
       "2  row-cw7t-bpix_8nr6  00000000-0000-0000-27DC-95188F2F91C9          0   \n",
       "3  row-xa8y-grfu.cue8  00000000-0000-0000-713B-3335FE6E4BFC          0   \n",
       "4  row-d98p.ghx9-25dk  00000000-0000-0000-1E0D-153CE9C006DD          0   \n",
       "\n",
       "   :created_at :created_meta  :updated_at :updated_meta :meta  \\\n",
       "0   1713361986          None   1713361986          None   { }   \n",
       "1   1713361986          None   1713361986          None   { }   \n",
       "2   1713361986          None   1713361986          None   { }   \n",
       "3   1713361986          None   1713361986          None   { }   \n",
       "4   1713361986          None   1713361986          None   { }   \n",
       "\n",
       "                                            the_geom  name  ... doitt_id  \\\n",
       "0  MULTIPOLYGON (((-73.96664570466969 40.62599676...  None  ...    96807   \n",
       "1  MULTIPOLYGON (((-74.16790202462265 40.63936048...  None  ...   326368   \n",
       "2  MULTIPOLYGON (((-74.19510813278613 40.55610681...  None  ...   746627   \n",
       "3  MULTIPOLYGON (((-73.96113466505085 40.57743931...  None  ...   786626   \n",
       "4  MULTIPOLYGON (((-73.75421559146166 40.75591276...  None  ...   746409   \n",
       "\n",
       "    heightroof feat_code groundelev shape_area shape_len    base_bbl  \\\n",
       "0  29.74985318      2100         40        0.0       0.0  3065220021   \n",
       "1        22.63      2100         39        0.0       0.0  5012640036   \n",
       "2        35.76      2100         51        0.0       0.0  5060190091   \n",
       "3         37.5      2100          6        0.0       0.0  3086910048   \n",
       "4  18.01511294      2100         93        0.0       0.0  4075020005   \n",
       "\n",
       "   mpluto_bbl  geomsource                                globalid  \n",
       "0  3065220021  Photogramm  {31298F86-3088-4F53-B3DB-71A9EFA6FA1F}  \n",
       "1  5012640036  Photogramm  {F5F8CDA5-69E2-46F8-8F69-BA95C025B520}  \n",
       "2  5060190091  Photogramm  {9F644794-F72C-4582-9E5E-B337E2B97068}  \n",
       "3  3086910048  Photogramm  {F916B22D-E25B-44AE-9FA9-2A51191B9CDF}  \n",
       "4  4075020005  Photogramm  {525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# constructing a dataframe\n",
    "df_json = pd.DataFrame(data['data'], columns=col_names)\n",
    "\n",
    "df_json.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1083347 entries, 0 to 1083346\n",
      "Data columns (total 16 columns):\n",
      " #   Column      Non-Null Count    Dtype \n",
      "---  ------      --------------    ----- \n",
      " 0   the_geom    1083347 non-null  object\n",
      " 1   name        2254 non-null     object\n",
      " 2   bin         1083347 non-null  object\n",
      " 3   cnstrct_yr  1072987 non-null  object\n",
      " 4   lstmoddate  1083347 non-null  object\n",
      " 5   lststatype  1083025 non-null  object\n",
      " 6   doitt_id    1083347 non-null  object\n",
      " 7   heightroof  1080700 non-null  object\n",
      " 8   feat_code   1083347 non-null  object\n",
      " 9   groundelev  1082783 non-null  object\n",
      " 10  shape_area  1083347 non-null  object\n",
      " 11  shape_len   1083347 non-null  object\n",
      " 12  base_bbl    1083347 non-null  object\n",
      " 13  mpluto_bbl  1083347 non-null  object\n",
      " 14  geomsource  1083029 non-null  object\n",
      " 15  globalid    1083347 non-null  object\n",
      "dtypes: object(16)\n",
      "memory usage: 132.2+ MB\n"
     ]
    }
   ],
   "source": [
    "# removing columns that start with \":\"\n",
    "df_json = df_json.loc[:, ~df_json.columns.str.startswith(':')]\n",
    "\n",
    "df_json.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 1,083,347 rows and 16 columns.\n"
     ]
    }
   ],
   "source": [
    "# printing dimensions of data\n",
    "rows, columns = df_json.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 Reading in Shapefile data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints_csv](images/boros-shp.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>boro_code</th>\n",
       "      <th>boro_name</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_leng</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.0</td>\n",
       "      <td>Staten Island</td>\n",
       "      <td>1.623621e+09</td>\n",
       "      <td>325917.353950</td>\n",
       "      <td>MULTIPOLYGON (((-74.05051 40.56642, -74.05047 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>Manhattan</td>\n",
       "      <td>6.365205e+08</td>\n",
       "      <td>357713.308162</td>\n",
       "      <td>MULTIPOLYGON (((-74.01093 40.68449, -74.01193 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.0</td>\n",
       "      <td>Bronx</td>\n",
       "      <td>1.187175e+09</td>\n",
       "      <td>463180.579449</td>\n",
       "      <td>MULTIPOLYGON (((-73.89681 40.79581, -73.89694 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.0</td>\n",
       "      <td>Brooklyn</td>\n",
       "      <td>1.934138e+09</td>\n",
       "      <td>728146.574928</td>\n",
       "      <td>MULTIPOLYGON (((-73.86327 40.58388, -73.86381 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.0</td>\n",
       "      <td>Queens</td>\n",
       "      <td>3.041419e+09</td>\n",
       "      <td>888199.731385</td>\n",
       "      <td>MULTIPOLYGON (((-73.82645 40.59053, -73.82642 ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   boro_code      boro_name    shape_area     shape_leng  \\\n",
       "0        5.0  Staten Island  1.623621e+09  325917.353950   \n",
       "1        1.0      Manhattan  6.365205e+08  357713.308162   \n",
       "2        2.0          Bronx  1.187175e+09  463180.579449   \n",
       "3        3.0       Brooklyn  1.934138e+09  728146.574928   \n",
       "4        4.0         Queens  3.041419e+09  888199.731385   \n",
       "\n",
       "                                            geometry  \n",
       "0  MULTIPOLYGON (((-74.05051 40.56642, -74.05047 ...  \n",
       "1  MULTIPOLYGON (((-74.01093 40.68449, -74.01193 ...  \n",
       "2  MULTIPOLYGON (((-73.89681 40.79581, -73.89694 ...  \n",
       "3  MULTIPOLYGON (((-73.86327 40.58388, -73.86381 ...  \n",
       "4  MULTIPOLYGON (((-73.82645 40.59053, -73.82642 ...  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url = 'https://data.cityofnewyork.us/api/geospatial/tqmj-j8zm?method=export&format=Shapefile'\n",
    "\n",
    "# reading in data as a geodataframe\n",
    "gdf = gpd.read_file(url)\n",
    "\n",
    "# printing the first five rows\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 5 rows and 5 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = gdf.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQsAAAD4CAYAAAD7JMNRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABH7klEQVR4nO2dd3ib1fm/7yPJ8t57O3EcZ09nAQlJIIwkJJSyoc23zG4KXVBW6frR0pZRWiizjELDDjuQQCAJWc52huPYcbz3Xprn94cUxUO25SnZPvd1+bL0vue872NZ+uiMZwgpJQqFQtEbGncboFAoRgZKLBQKhUsosVAoFC6hxEKhULiEEguFQuESOncb4IyIiAiZkpLibjMUilHN3r17q6SUka6290ixSElJITMz091mKBSjGiHE6b60V9MQhULhEkosFAqFSyixUCgULqHEQqFQuIQSC4VC4RJKLBQKhUsosVAoFC6hxEKhULiEEguFx7P3dC2Pfn6CioY2d5sypvFID06Foj3vHyjmpR2neWpLLqtnxLJqRixL06PQaoS7TRtTKLFQeDwFNS0AGC1W3tlfzDv7i4kI8ObqjASumZdIcrh/n695sqKR1MgAhFCC4ypqGqLwaEwWK4eL67scr2oy8K8tuZz/yBau+fcO3t5bRJvJ4tI1T1Y08cnhMiUUfcRlsRBCaIUQ+4UQH9qfhwkhPhdC5Nh/h3bT7w4hRJYQ4ogQ4meDZLdijJBd1khVk7HHNrtO1fDzNw+y6P9t5qEPjnC0pKHbtlar5I3MQj48VMrVT+/gg4MlGM3WwTZ7VNKXkcUdwLF2z+8GNksp04DN9ucdEEJMA24F5gMzgdVCiLT+m6sYa7x/sMTltrUtJl7cns/KJ7Zy2T+28fKOfOpbTB3aFNa28NI3+WSXN7I7v4afvL6fv3x6nL2nawfb9FGHS2IhhEgAVgHPtTu8FnjJ/vgl4HInXScDO6WULVJKM/AV8K1+W6sYc3zYB7Foz+Hieh7YcIT5f9rEXW8cYEduNVJK/vTxMQydRhLPbTvFc1vzBsPcUY2rC5yPAb8CAtsdi5ZSlgJIKUuFEFFO+mUBfxRChAOtwEpAJapQuERNs5HSAW6XGsxW3tlXzDv7iokL9qGq2fmUZvPxCorrWokP8R3Q/UYzvY4shBCrgQop5d6+XlxKeQz4M/A58ClwEDB3c5/bhBCZQojMysrKvt5KMQrZdrKKwSxrU1Lf1u36hNFspbHN5PScwoYr05BzgTVCiHzgf8ByIcSrQLkQIhbA/rvCWWcp5fNSyjlSyiVADZDTTbtnpJQZUsqMyEiXM30pRjHbcob3SyO/qmVY7zfS6FUspJT3SCkTpJQpwLXAF1LKG4H3gXX2ZuuADc76n5meCCGSgCuA1wfBbsUox2qVfHHc6ffPkPHYphN92oIdawzEz+JhYIUQIgdYYX+OECJOCPFxu3ZvCyGOAh8AP5JSqmVnRa8cLW3odct0sDle1sjP3zxIxh82cf97WRwrbcBksZJb2TSsdngqwhNrnWZkZEiVsHds84/NOfzt8xPuNoNJMYHkVTUzLS6IK+YkcOm0GMIDvN1t1qAghNgrpcxwtb1y91Z4JF9kD+8UpDuOlzUCsK+gjn0FdTywIYsF48JZPTOWi6fGEDFKhMMVlFgoPI7Cmhb2F9S52wynWCXsyKtmR141972XxfyUMFbPjGPlKBpxdIeKDVF4HH3x2nQnUtpcze9/L4t5f9zE3z7LdrdJQ4oSC4VHIaXk3f3F7jajzwghWDIxkr2na6hobKOmyYDJMrpiTpRYKDyK6mYjU+OCOhzLSA4lKczPTRa5RoivF4U1Lax7YQ8vbs/no8OlaEdZVKsSC4XH8MnhUn72vwMsSYvg2nmJaDWCjJRQMk/XUt9qJMjHc5fYrp2fSHFtK00GM09tyeXNzCKqmwzuNmtQ8dxXXzGmKG9o4471BzCarcxJDmVGQghL0sIprjMQ4uvF5mMV+Ht75tv1vAkRfO/ccTy2ybbVqxHwwGVT0GnPjiwa2kwE+Xi5y8RBwTNffcWY49mv8xxxGy9sO0WovxeFNa0AXDE7nivnJvDm3iJ3mtgt962ezJNfnOQtu30+Xlo2Hinj0umxhPrbdkiCfLyobzUR7DtyBUNNQxRux2qVbGi3A9JkMDuEAqC2xUh9q2cFeV08NRp/vZafXZjGgYI6/vNNPm0mm9i1GC38b3choX76Dn1GslCAGlkoPACjxUplY/fz+yUTI3nyi5PDaFHvbM2p4qcXpPGt2fFc9fQOx/HVM2J5YPUU9DoNIZ3E4gxSyhGZ0k+JhcLtfJJV2u25MH89UYE+VHeTh8JdtBgt/POLk5yqbHYkFAa4+9JJRAX5dNvPbN9Obb+eMVJQ0xCFWzGarTy+yWnWAjQCnrx+Nn/ZeHyYreqe8RFnM4k3GsyszyzscN5Pb/v+tVolFmvXuCudVoNOOzI/diPTasWo4cXtp8ivdp5H4qZzU3h+2ylOd3N+uFk1I5aNdy7hlvPGATArMaRLm1NVtgjVVpOFZsPZPE+jIexdTUMUbqOuxci/v+6a+zIu2IekcD9mJobw3Lb84TesGz46VMrEqEB+eUk6sSG+6LWCA4V1jvP3XDqJuclhlNS18tquAn5xcbrjnLdOw9Nf5XLFnHiiArufpngyamShcBtPfZVLjZO1CIPZyn2rpvCLNw+5waqe2ZpTibdOy83njWPeuLAO5xaMC+NfW05yoryRn180scM5IQQbj5Tx4IYjeGJaCFdQYqFwC5WNBl7cnu/03HXzErn3vawuWbg9gbp2W7j+9vUJP72W129dyKSYIDbsL6G0vg0hBCV1LV3yen6SVcanWWXDavNgocRC4Rae/irXafLcUD8vgvy8ONhueO8pPLRmKk/fOMfx/Izvx/hIfxaMC0OnFUyNC2LjkTKklMSF+BFo99q0WCVaIfjZhWlcOj3WLfYPFCUWimGloc3ET17fz/PbTjk9f9vi8Ty1JXeYrXKNpDA/JkSdrYYxLT6Y1TNiySlvwmzf+bh6XiJbsiu55aXMDmKoERDi50V0D9uqno4SC8WwUlbfxgfd5KuIDvJGqxXUtniWt+YZ/ruroMuxqzMSeWjNVPQ625Zond32L7MrWJ9Z6FifEEKwYko0oX4j14tTiYViWAnoIRjse+em8Fg3PheeQJCPjjaTxeFYBTbv0mvmJTqebzpWDtgyam06Wt7B10Ig8PHSDp/Bg4wSC8WwEhPk41Qw4kN8Ka5ro8Xomf4IGmHLivXKjtNoNR29L9u7bt+yeBxnTq/qtDZxVUbCiI4PUWKhGFZyK5toMnQtSnfx1Gg+Oey5uwRWCUsmRvCtOfEIIWhoM7HhQDGtncQtNTKABePCAWg2mjG2G4UIIZidFDqsdg8mSiwUw8rLO047PT4pNpAqD08W8/WJKsL9bcFhXhoNAd46yjvVYvXSalh3TgoAL27Px0c3cqcdnVFioRg2rFbJh4e6Lm6ekxrONyer3WDRWXy8NEyODeqxzZS4IEftVV+9lgsmR5PSLlbkDJdMi+E7C5PRCBiBwaXdosRCMWzkVTU53elYNT2WDw51H3k6HLSZrPjpex4FTI0LQqNx7dP/k+UTePL6OSMyFL07VGyIYtj4Jtf56EGv0ziN0BxudL0IQZ2LW7qnq5tJDvfvMVR9JKJGFophwWyx8p9v8p2eq21xf66KJRMjOVra0GObg0V1Ll0rObzr1GQ0oMRCMSxsOFBCXmWz03Nebs7vEOBt859obOu6S9Oe/QV17Mxz79qKO1FioRhytp+s4u89FDnWuHle32ayuFwQ6Jmv8yiua+294ShEiYViSDlYWMcNz+3q8QPm7hRzZqskrJt8mZ1ZMSUaAbw3AqumDRS1wKkYUj5xIRzbE/YLGtpMxAT5UNbJb6IzL2w7xe8/PIqPl5bLZsZ18eYczaiRhWJI6W2HAXBEbLqTfQV1zE3u3bsyp6KJFqOFmmYjp6udr8GMVpRYKIaMIyX1PLO1a9q8znR2mXYHSydG9rnPJ1llWD1A6IYLJRaKIaHNZOGu9QedJrjpjLurjes0gvtXT+Gzo67HpnhpBY9szCbjj5t4ftspt/8Nw4ESC8WQ8OzXeWSXN7rUVj/IW6feOg03nzeOK2bHOz0vhM0R7AwX2t22Y4Jdd6LSazVcOTeBmmYjh4vqaDG4f3Q01KgFTsWgU9di5FkXph9nGOyRvMFs5fltp1g7K47r5ifx+u6OSWu8tBqSwvxICffjy+xK1syKo6rJQGld18VNvVbD4rQIFqdFMCEqkLyqJg4X1bPhQAkzE4LRCsFv10zFtxdX8dGAEgvFoPPs1jwaenFwak+Az9B80DYcKGH9bQvJq2wit7LZEdVqNFs5WdFEekwgv74knQsnR2O2WvH10tJoD5+/YnY8K6ZEMyc5tEMqvPPSIgA4d0IEP3/zIK/funBMCAUosVAMMrXNRl7+xnkYendohMBLKzBZBn+xcMuJStbfvgiAguoWHvrgCJuPVwCQHh3IbUtSAdCj4fqFSfz7qzziQ3x55KqZPW6LXj47niBfHXvya/jDR0e5am4C1y9IHtVbqWrNQjGo/P3zE45vZ1f5764C7rggrU99UiP9WTIxkvGR/j2GgbdPu58U7tfh3JNfnqShXar+ldNi8dIKDGYrrnzml0+K5kfLJvDnb8/gg0OlrHpiKy90k4h4NODyyEIIoQUygWIp5WohRBiwHkgB8oGrpZS1TvrdCdwCSOAw8D0pZc+eL4oRSU55I//d1bdRBcCRkgYumBTNvJRQ9uR3eQt1YHFaBP93TgrnpUXgrdNitljZnlvN91/ZS6uTEoGdFzl/u2Yqc1NCqW4yEhXoTavRQpA9Xf/MxBB23HMBO3KrsUpw1bF0cmwQb9y+iL9/lt0lGc5oQrhaHUkIcReQAQTZxeIvQI2U8mEhxN1AqJTy1536xAPbgClSylYhxBvAx1LK//R0r4yMDJmZmdmPP0fhTm59OZPPj5b3q2+4v563vr+If3+dR15lE7udiMZ5EyJ46ab5NLaZqG0xMa5d4pkvj1fwRmYhMxND2HikjP0FdcSH+LL55+eP6CS5Q4kQYq+UMsPV9i6NLIQQCcAq4I/AXfbDa4Gl9scvAVuAX3fua7+HrxDCBPgBzvPAK0Y0e0/X9FsoAKqbjdS0mNh4pAyj2WpP4Hs2niQiQM+j18xCqxFszaniUFEd966a4ji/bFIUyyZFAbb0/DXNBoJ99YMiFN/kVnG0pIGbzxs3qpLZ9BVXpyGPAb8CAtsdi5ZSlgJIKUuFEFGdO0kpi4UQfwUKgFbgMynlZ85uIIS4DbgNICkpyeU/QOF+pJT8dWP3UaWuklt5NpNWqD+cPzGSbSer0ArB366eRWSgN2Dzi1hpz5xd32rik8OlpEUHMDfZVns0zF9PmL9rgWFnMFmsWKyyg7jUtRg5VtrID17dR32riW9yq/nJ8gmkxwTipx97ewO9/sVCiNVAhZRyrxBiaV8uLoQIxTYCGQfUAW8KIW6UUr7aua2U8hngGbBNQ/pyH4V72ZpTxY4B5nkQAqbHnc2BWVTbSlFtK49fM4up8UEdKoEJAe/uL+bKuQk8tukEL27Px9dLy9PfmcuStAiXv/3bTBbeP1DChVOiWb+nkJK6Vn5/+TTHue88v5vDxfWO9l8cr2BLdgVeWg3zUsKYEBVAs8HMhKgArpmXSEinyNXGNhN///wEbSYrv1s71e15OwaKK/J4LrBGCLES8AGChBCvAuVCiFj7qCIWqHDS90LglJSyEkAI8Q5wDtBFLBQjE6tV8udPjw/oGnqthvtWTybHSXKcA0V1rO20SOnjpeXKuQkA7DttW9toNVlY98JuQvy8+NbseH59yaQepyB/+PAo8aG+PPTBUfw/0NJstBDgreO+1ZPx1mnx1mmIDvLpIBZgcyAzmK1sO1nFtpNVjuOPbMzm5Zvnc05qhOOY2SL54GApVU0GogK9uXNFx8rqI41epU5KeY+UMkFKmQJcC3whpbwReB9YZ2+2DtjgpHsBsFAI4Sdscn8BcGxQLFd4BO8fLOFISc/p6HoiwFvHa7cuYM3MOKoaDQS2K0AU7OvF/9nT6ne3EG/pdLyuxcSL2/O59eVMCmtaur3v4eJ6HvrgKADN9kA2g9lCRYMBq1Xy5t6iPmXFMlslccG+judWq6S+1YTRbLv2a7sLqPOA9IEDYSATr4eBN4QQN2MThasAhBBxwHNSypVSyl1CiLeAfYAZ2I99qqEY+RjN1h4zYLnCj5dPIC06kNX/2EpJO3frIB8d/7ttIbHBvjz6+Qle2XmapemRLBgXxrJJUUQF2rwqf7d2Gute2N0lJd7WnCpueSmTjXcu6XLPoyUN7C+o63Jcr9WQXdaIwWzhV28d6vPfsiW7gpr9JnbmVWMwWcgqaXAkIq5sNPDZ0XKuzkjs5SqeS5/EQkq5BduuB1LKamwjhc5tSoCV7Z4/CDw4ECMVnsn6zEIKevj27g1fLy3XzUvi4U+OU1jTMZPWA5dNZXJsEPe9d5hXd9piO97ZV8w7+4qJDvLm5ZsWkB4TyJykUK5fkMRL3+TTZuoY+dlqr0uq1QjHOkZdi5HPj5YT6KOjurnjN32z0cItL2f2WI+1J35rH6l0RiMgIdSPpj64wHsiY29JVzEo1LeYeHxT/0cVGgG/XTOFkvrWLoFey9Ij+faceL48XuG0cnl5g4FbX87kq18uRQjBPZdO5u5LJpFb2cSuUzVUNBhICPXFapUcL2tkWnywo+8v3zrU6xavs/KKA8EqYXJsIDedN25QrzvcjOzlWYXbeG13AVVN/ZuDCwGv37qQa+YloddpmJkQ7AhT1whYd04KO/Kq+eVbB3G2VJEa6c/1C5KoaDQ48kgIIZgQFcj185O4bn4Sl0yL4dFNOWSXdQyT//75qb1WHhsKNh4pZ1MPItVqtHS7LuMpuOzBOZwoD07PpsVo5pyHv3C56E5nIgL0ZN63gtPVzcQE++Ct01LVZOAfm3O4ZfF4KpsMfPf53U6/4VMjbV6bhTWteHtpePzaWSyfFA3YfCVyK5u45t87iQ/xZVZSCH9YO61LFbHyhjZ8vLS8tqtgwDs5fSHY14sPf3IeiWEdY1QsVslFj37F5bPi+fHyCcPm+NVXD041slD0CYtV8s6+4n4LBUBVk5GvT1QSHWQTCoCIAG8eWjuN6mYj33luV7dTgdzKZnIrmzFarDS2mTlgX6gsq28j/b5PuOSxrdS3mjha2sDbe4t4ftspzJ2yWEUH+RDgrXMaSzKU1Lea+N2HR7tUX9t0rJzcymYe35zDW3uLqG/1zF0TJRaKPlHR2MZjA1irOMPR0gZ8vLR8dKiUKQ98yt8/P8E/vzzJ5f/c7tjKdIX86hbyKpsID9B3CQ83mK388eNjXP6v7eRWNnU4p9UIzpsQwXCzLaeqixBuya4EbNuv92/I4uJHt/LVicpht603lFgoXMZilby4Pb/faxXtmWFfdPx/nxyjxWjhic05PLIxu8/X+ehwKWUNbbSaLN2m58sqbuDSx7byxOYcDOazQuRKftDBptVk4dWdHSNzF44PczxuM1kpa2hj3Qu7+eeXJz1qHUOJhcIlTBYrGw4U85/t+QO+1nXzk0gM86OqyUBypxwTfcVildy5/gAzfvtZjyMSo8XmE7Ly8a3syqtGSkmJmyqLPbUll5MVZxdeU7qpjfrIxmx++N99tBg9Y8tViYWiV6SU5FY2sflYBcYBZrGekxTC7UvG8e2nvqHNZOHl780n1M9rQNcsbzC43La62YhV2nZPpsQN/64I2LZmv/v8bkfui8c353Tb9pOsMq59ZqfbhK09SiwUvVLbYqK8wcDHWaUDuo5eq+HvV8/kvveO8KNlEzhd3cLaf213RJoOB1PjgpiZaJsCRQf5uC0NXkl9Gz/87z6klCSF9Ty6OlRUz5ont7P3dM0wWeccJRaKXimpa2Xv6VqnPg994Y4L0zhQWI9epyHUX88Nz+0iq7j/cSV9QacRPHXDHJ6+ca4jvDwy0JsfLZtAVKA3K6fHkBDq28tVBpe9p2u56T97aHNhV6aqycC1z+zs4sA2nCixUPRIeUMbp6tbeHtv0YCuMzMxhDWzYnlicw53rpjIve8eHiQLXcNslezOr8FPr+uwaHjXiol8+rMlHC9tdEt19C+zK/noUCmxLtQsMVkk97xzmDvXH3BJYAYbJRaKHjGarfz50+MD+iAFeOv421UzueftLG5ePI7XdhV0CfwaDl7deZpNx856UUopKapt4cqnviGvqnnAI6f+cibBcUSAawl73t1fzPXP7qS6yfW1msFAxYYouuVYaQMbDpQMKFhMI+Af183mUFEdvnotE6MCuffdrEG00nVMFsntr+xlcVoEl06LpdVk4emvcqlsHN4PnTNK69sI8Na5XBJhX0Edl/9rO898J2PY3NeVWCi6JcBbxzNf5w7oGj+/KJ0JUQFc+8xO3v/Judz43O5Bsq7/bM2pYmtOVe8Nh5kmgxl/vRaTxbUpRmFNK99+6hv+cd1sLpgcPcTWqWmIogfe3lc0oNKC0+ODueW8cdz3XhYPXDaFDftLOFY6PAuaI5W+eK8CtBgt3PpyJk9/lTvkFd2VWCic0may8OzXrtcr7YyfXsu/bpjDf3cVMH9cGNPig3l0gIlyFM6xSnj4k+Pc/ureQQ+vb48SC4VT3j9Y0udvufbcu2oy/t46yhva+MH5qdy1/kCfK5Up+sbnR8t56Zv8Ibu+EgtFF9pMFr445iz/smuckxrO9fOTMJgt/PLidDYeKWPXKfc6FI0VKoawIpoSC0UXapqNaPr5ztBqBL9bOxWLVRIb7Etmfi13vXGQKHvND8XQUjGEOztqN0TRBZ1W9LlIzxl+vGxChxofm4+X02qyDHvuiLHKUPqvqJGFogtWK/1yUAr01vH981Mdz4+W1PO/PYXdtj+T9UoxeNQ0D13iHCUWii5YpOzXcHZSbCC++rOFfTJP13b7TRfoowa1Q0FF49CtWaj/mKILVqtkWz+cltpXNQc4UFDHrYvH0Ww0IyX46XVsya6wxWcghy2IbCwR7OuFlHJI8ngqsVB0ISHUF2sf5yFajSC00zrHb9dO5fInt5NXdbYs4YWTo1k4Poy39w0sME3hnDtXTByyhL9qGqLoQnWzEUMfU85ZrJLvLkrpcOzRz090EAqAQ0V1/PHjYySE+g446Y2iI+Mj/bl0WuyQXV+JhaIDRrO1xxqhPdF+j19K6TQDeEWjASlh07EKlkyM7Lediq78ZPmEIU3mo8RC0YGi2hZe2XG694ZOeH7bKcdjKWHhuLBu20oJzQazI3GvYmCMj/Bnzcz43hsOACUWig54aTVdivK4ykeHSx0VwjQaQVhAz45Ym45V0Nhm4pp5idxy3jjSogL6dd+xTqifF7+8OH3IUwSqBU5FB6wu5ITsjmBfL7zapeO3SklkoHeP+SJOVbdwqto27bl18XhK69uGNBhqNBAd5M1lM+JYOD6c9JhA4kN8+y3wfUGJhaIDyeH+/f6G6jwy2HS0nGBfL5eTy+wrqOW5dRk8tukEBwvrlddnJ1ZOj+HGBcksGB/ulkTDSiwUHSisaWF3P4O+2mdsKqpu5tMjZX1yP957upbyhjZ25qmgs848cd1s1syMc6sNas1C4UBKSWObuUupP1eZYhcLi1Xy4PtH+hWn8PWJSoapLvCI4mRF//4ng4kSC4WDQ0X13PSfPZTV989leL5992NLdgWbs/tWq1MIuGBSFDXNRpanR/Xr/qMZd9cMATUNUbQjv7qZsn7mQ0gJ93O4e7+Z2XfvzIzkUPKrm5FSkhjm53Li2rHC4aL6IXPjdhU1slA4iOxlq7MnVs+IQwiB2WLtcwVwL60g0MeL3MpmCmtaqWsxKaHoREObeUBZ1gcDJRYKBxsOlPS5jxA2h6DrFyQBcLCo77sYXloNzfbtUpNVUt1sdFtZQU/maIl7A+/UNETh4GBRXbfnxkf6c96ECKbFBTM+0p+9p2tJCPVjZmIwCaE2v4yKxjaOl/X9Dd1itFBSf7aIUUObiRBfL6qHMDfDSKRlADlRBwMlFgoADGZLt7sXV8yJ58/fntHB4Soj5awrt5QSo8VKVnG903gQVyisaScWrWbC/PRMignkeFljv643GjFbB1bBfqC4LBZCCC2QCRRLKVcLIcKA9UAKkA9cLaWs7dQn3d7mDOOBB6SUjw3MbMVgoxUCva7jrNRPr+X2JaksnhiBl1ZDVWMb2eVNVDYa+DqnktzKZgwmC+enR/KT5WmcKG/iYGEdEQF6qpoGNiqoaTFS12pkfkoou/Nre+8wBjD2MRJ4sOnLyOIO4BhwxvPmbmCzlPJhIcTd9ue/bt9BSpkNzAKH2BQD7w7QZsUQoNNquPvSSdz+yl4A4kN8+ecNc5iVGALYKqn/v4+P8cGh0i5986ubWZYexdzkUL44XjFow2WrhN35tcSH+lJcO/xFiz2NvqYNGGxcWuAUQiQAq4Dn2h1eC7xkf/wScHkvl7kAyJVS9i+kUTEg2kwWXttV0OO308VTY/jrVTO5fcl41t++kOn2iFCj2cqx0gY2HXdeHqDNZOWBDVk8sTkHKeWgz60TQnwH9XojlWbDyFizeAz4FRDY7li0lLIUQEpZKoTozZPmWuD17k4KIW4DbgNISkpy0SyFqzz6+QkqmwyckxpOSkT3iXKvnJvQ5di+glreO1BMaw8icKK8iRPlQ+NleLysgenxQRwe42n43B0r0+vIQgixGqiQUu7t702EEHpgDfBmd22klM9IKTOklBmRkSopymBz9bxEfnXxpB6FwhkWq2RcuF+/cnIOFvWtZg4XN7Cgh/wYYwGzxfPXLM4F1gghVgI+QJAQ4lWgXAgRax9VxAI9lbC6FNgnpSwfuMmK/pAa2XuuCItVohF08BI8XFTHqzsLqO3nLsdg0pfQ9dRIf8L89Viskn0FdUNn1DBidLNY9DqykFLeI6VMkFKmYJtKfCGlvBF4H1hnb7YO2NDDZa6jhymIwv28siOf1N98zEWPfk12u+3KQ8X1ZJd7xvD/SEkDc5NCem23YFwY+VXN7MmvdbvX42Di7t2QgXhwPgysEELkACvszxFCxAkhPj7TSAjhZz//zkAMVfSPNpOFK5/6pseoxaMlDfzx42MA5FQ08dUJ2yDxaGkDtc1GwHO8KQ8U1hHs232i34zkUHadquGMt3hVk7FDLZORjLvFok9OWVLKLcAW++NqbDscnduUACvbPW8BwgdipKL/+Hhpefa7GV18KM5Q02zk1pczaTNZ0WoE81PCWDvLlsvx6+xK3tlXzGkP+na2SEgK8+NwcX2Xc/NTwtid3zU6MzHUd8gWX4cTd1ehVx6cY4DO9Tza88jG4xTXtTI7KYTfr51GekwgWiF4cdspPjpc6lFCcYYzeT69dRoMZith/npSI/3Z40QogB5HIiOJejevGymxGOPcung8C8eHs2p6LDqthhajmV0Fdbyy6zR5lc29X8ANHC9rZF5KKFVNRoJ9vDhR3sCeHrw8PWcSNTDcvcCpxGKMMz4ygPH2nZLKRgM/f/MgK6fFeKxQnKEncehMXlUzc5ND2Xt6ZLuNy/5Uqx5ElFgoAGgxmrniqe0U1rTydR/zUXg6VU1GogK90QibC/lIZUS4eytGP20mq9tX24eSo6WNxIeObLfxSTGBvTcaQpRYKAAI89czNzkUH6/R+5YYSCYwT+DCKdFuvf/ofWco+szSiVFcNCWm223WkY5OO7L/rmlx7i31OLJfPcWgYbVKwvz1/OzCNEcMRkSAN15a4UidN9LJr2pmXkqou83oNz1tgQ8HSiwUgK02aWl9K0lhfvztqpksS49k5fQYAH66PG1U7D9WNBo45uY8lv3FW6ch0Nu9+xFqN0RxFiHQaTXsOlXDP2+YQ5vJyq2Lx/Obdw97/FaqqzQZLcQE+/S7Noq78NNrh6WeaU+okYXCwbL0SN7aW0R4gB4/vY7KRgMnyhtZtyjF3aYNGgIId/Nwvj/UtZpobFMenAoPISHUj7WzfKhqMvCbdw8T6K1je24VNQPMp+lJzB8Xxq5uarl2F1sCtm3LykaD2zKO6zSiQ8Jkt9jg1rsrPIryhjbe3V9MTbOR6fHBPLAha1QV+wnw1tLTXxMe0HXEEeit46bzxrEnv4bxkf746XVs7GPB58HAZJFszalihRu3T5VYKACbK3FhTQtbcyrJzK91u7fgYBMb7MOi1HDe2VfcbZsz3tTjIvypajTQaDBz7oQIPj5cSo4HFCZ+e2+RW8VCrVkoAJsr8fGyRrLLGkedUIAt8rS3tHxWu1poBA5vz/SYQI8QCui5CNRwoMRCQWObiQc3HOGzo+UDrvfhbvz1WqKDunpqHi9r5Jmv83r0s2g1WdDrNPjqtRTX2UoPNBnMeHuIk1ppfVufUgsONp7xKijcRkldKzvzatidXzMqAsiajRbC/J27dedWNnNeWgQZyaHodRpm2muinCGnvImJ0QEU1rSSHm2Lw8gqrme2C6n8hhqdRrB2VpxbhUutWYwQztTj8B9ExxwpJdVNtkxZI5VgXy+sUnZYcCysaSE9OpDs8q6lD3fn1bD+9kVUNhq48fldHc6VNbQxIyGYrOIGVs+IpbLJQKifnonRAezMc75LMpQEeOu4OiORafFBnJMaQUywz7Db0B4lFiOAIyX1/P7Do+zJryU53I+EUD/uWzWZidEDi0I8WmrzZvT10rq9JkV/aTNZuGvFRP7fJ8cdx5oMZkL8nGfHmpEYglYj+OpEhdO8pBdOiebS6TGsmBLDtfNt9WssVsnOUzXs7mbLdaDMSAimoKalQ53YVdNj+cPl09zu4t0eJRa9UNHQhl6nIcRPz9GSBqbEBfXeqY8YzBbe3ltMSV0rJouVmGAfIgK8KW9oY2tOFV+1mx7kVTaTV9nMVYV1PHHdbBZPiOiXZ19Vk4G4YF++/+reESsUYFuYffqrXJLD/ThdfTYF4P6COn6+YiKPb87B3C6JRXmDzXNz0fgIIgO9qWw0oBG2UgkzE0NICfdnvpOF0Ge+M5f3D5bwry9zKWsYXO/Pw8X1XDsvkSMlDRwqsuUW/cHSVI8SClBi0S0tRjN//OgYmfaMTL+8OJ071x/gv7cuYEZCCFJKhBD8+LV93LAgmUWpfc9JbLVKCmtb+MGr+xzf8q5S32pi3Qu7eeeH5zAnyfXgKCkl92/IYkduNSV1bSNaKM5Q22IiOsiH125ZwPf+sweD2YrRYmVGYgjLJkXx+dFy9DoNS9IiHJGbSeF+fHrHYqqajMSF+BDo03OezhA/Pd9dlMINC5J5cfsp/rUll5pBctCSEl7fXcj1C5Ioqm2lptmIj5fnZSQX7k7V5YyMjAyZmem+ebTBbOGWlzLZ6qQK17gIf3y8tMxKDOaaeUnUthhZND68yz9XSsnp6hZ2n6rh82PlhPh68d1FKUyICqDRYOKVHafJKW/iy+yKfm9VzkwI5tFrZjnS4rlCQXULSx75sl/383RmJgSzdlY8v/vwKGDzuvz5Renctf4A//7uXM5JjRi0e1U1GfjH5hxe3VWAZZDSb2k1givnJLA+s5DM+y4kYojzbwgh9kopM1xtr0YWnTBbrNzx+gGnQgFwqsoWUHWstIFNxyr45/VzANu8trHNREFNC1tzqiiqbeH13YUd+r65twg/vW19YDA02t9b1yehkFKyv3Bk56HsiYNF9cxOCmXFlGg+P1rO8bJGaluMbP31MkL8nA/p86uau5R0PFRUx6SYIPQ6DXvya5iX0nVaEhHgzUNrp3HjwmTufS9r0NYzNBoI8fOivKFtyMWiryix6MQfPjrGp0fKXGpb2Wjgumd34q3TMH9cGNVNRnIrm3qsIj6YFcYjArwd0yFXEELwyWHX/raRyn++yefBy6YwOSaQkvo2psYFdSsUTQYzq57Yyn9vXcisdtuoW7IrefiT4zy/bh5HiuuZFBPY7TQlLTqQN25fxJ78Gu5553CPxZx6Iz06kKlxwdx9yWTe3lfEVDcnu+mMEot2nKxo5LXdBX3qY7HatjS3ZA+/j0J9q8lloQCoazGy+fjoLzfrpdVw10Xpvbb7KruSeePCmBDVcXR2weQo/v75Ceb/cRMGs5W06EDOndDzFGZeShif37mEO9cf4P2DJX1ODBwX7MOPl09g5fRY6ltMRAZ61qgClFg4yC5r5OdvHhhRSWu9tH3bBfn4cNmICgwL9Nb1qQqXl1Zw/+op3Lgwuds2+wtqeXZrHlNig9BpNfxm5WQC2vmuNLSZmBIb5NgpAThUVN+rWFisEquUPHrNLC6dHsu972ZR1WTo1eZxEf7csngcV8xOcJRZDPbz4rKZca78ycOKEgs792/IIqt4ZGVRamgz02Qwd3iz98SGA90HUXkiK6fHMik2kL9/fsKlKM+7L53Md3vJvfFNbjVl9W2snhHHJVNjumw7C6C62egQCoDsMtv74mRFIynh/l1yeVqtEgGOEPKLp8awKDWcpY9s6XbHJD7Elx8tm8DVGQkjJjeoEgtsjj2Z3eQx8GTK6ttoM1kwW6zdzsvPIKV0Wh/Uk1mfWUhSmB+/WTmZg4V1rM8sdLowHBGg5zsLU9B1429yuKieafFBCCH40bIJ/GjZBMD2ITdbrOi0GsfaT6CPF18fKu3Qf/PxCpoNZvYV1OGr1xEf0rGkgDM/l0BvHSF+XtQ0G9FqRIcdkzsvnMjt54/3yO3RnhgZkjbENBvMI7L4jNliJSLAu1ehAFuQ1GAurg4XBTUt3PPOYYrrWvnrlTPJSO7oU+Kv17L+9kXccWEa685J6XCuzWThN+8e5rIntzn9htdohONbfX9hHW12n5MlEyM6xI00tpm58fldXDkngROdXMibDGY+PFTSZfv0X1tyyats5k/fmk7Wby/mHLsfTkKoLz9ePmHECQUosQBApxmZL0NpQxul9a0utR0sByJ3sTWnirvfOcRlM+N4/NpZxAT5IAT85II0Up1sH5+saGTtk9t5bVcBF02JxmixFVE6Iwgmi9Xx897+YuYkhTo+wFlF9bQaO057Supa2ZNfQ055o6Mw8+Zj5ax6YmuXxe1Ps8r462fZTI0L4tp5ifjqtTx141zSogK4cm4CWjfn0uwvahoCNBrcm9uwv0gJf914gr9dPbPXtq6ua3gyJovkDx8d5d6Vk/nLlTOIDvIh3UmVrsz8Gm76zx4a7Osc+dXNRAX6oNUIpJRkFdej0woe35TDUzfO5fyJkY6+ZfVtfJNbTXWnUH1vnS3L1tUZiXjZpy0/fX0/zUYLF02JRiPgN+8exmKRvHugGCnhNysnO6Yogd46blyYTFK439C9QEPMyPxKHWTiQ3xH7Ifp7X1FPPr5Ccy9VNj299Z5TF6GgWCySH77wVFueSnTaTnC+lYT33vxrFAAnK5uwWy1vT7NRgvv7i/GaLby0NqpQMd6HIW1LWw6Vt4l12ZBTQsfHSrt4G+REOpHaqQ/yyZFsb+wjtd2FbA+sxCj2crFU6M77KBIICrQm4KakZslfeS/ewYB28LWyBQLgMc35/Cfb/J7bOOl1XTxJxhJdP7/xIb44K/vOu/PrWzqsN2aEu7HPZdOwltnaxvgreP+1VOYkRBCVGDXkO9mg5k2J9vnGgH3rZ7smEIIIbhxYRIxwT6ckxrBm5lFHdrXtnQcrWo1ggsmR/P4ppOOqdBIY+R+QgYRg9kyovwrnPGHj44RHeTT4/58WlQAR0ZYkR0hbNOtHyxN5ePDpY7t7WaDBauEzq4mU2KD+NmFaVitkiUTI5mbHOqy41qbycLz205hscouYftWCaV1bR1cw9fMisfbS8uhojre299xW3pPfg0bDhSzZmac4/4VjW3MTQ6lptlIXMjIK9KsRhbAu/uK3ZbifTD5zTuHaTF2749wVUbiMFozcCZGB3CefSj/l0+zWT4pmii7Z2NVk4Ef/XcfH3Xa5vTx0vKzCydy10XpZKSE9SoUVqvkpW/yaTGaqWsx8cS1s3jrB4vYc9+F/P7yaR12X3734VHqW8+OGIJ9vfj2nATeyCzsEr0rJTz4/hEaWm3/j9zKJp784iRWq+SNzI4xQyMFJRYw6PkJ3EWjwcy/v8rr9vy5EyL44dLUYbRoYMSH+HbIOfnE5hx+uDTVUcZve24V//4615EvszPdRVQ3GcycrGjijT2FWKXkyS9Pcv2zu7jk8a+paDQSFehDgLeO7yxM5teXTmJqXBDBvl746bX4dZr6aARMjg1iUruFVo2Aa+cl8tFPFxNsT8JTUNPCzrxqNh+v4OUdp/nzp8c5ZE/AK6Xkm5NV3PXGAR7ZeBxPRYkFZxOijAae3ZrXxRegPf93bsqIqciVkRLG/oK6Dsfe2V/MhfZ0+I1tZqbFBxMb5Dzd3O8+PIrVasXayQfihW2n+MGre0kM86PZaKGy0UBWcT3nT4wkPSaQb05W0WI0s/d0DbMTQ1i3yPaafZVd2cXxy2SRTI0L5vFrZ3HzeeOIDfbh1ZsX8PC3ZxAf4usQAX+9jvd/ch4Z9mnIU1tyuek/mZz78BfM/cMmrn9uF+/sK+afX+ay97RnRgarNQvA12v0vAwtRgt//+wE/7h+ttMKVlGBPjx6zSz+78XdHu2INjMhmMh2IdqBPjpajBYqGgwOf4gwfz3fX5LabaawSTGBfHi4lHNTIwhvd63LZsZxxZx4/PU6/vLpcfRaDRNjAnhg9RSOlzXw3Rd2863Z8WSVNPD0jXMI8fMir6qZlHC/DtMao9nK7z48Sk55IxoheP22hXz//FRHEFh2WSM3v5RJq8nCeRMimJcSSnbZWSHvLnbkxe2nmJvsekKj4cLlT4kQQgtkAsVSytVCiDBgPZAC5ANXSym7SKIQIgR4DpiGbQfpJinljgFbPohcOCWKF7afcrcZg8anR8r408fHePCyqU7PL5kYyXPrMvjrxhOcrGjC2Mu2qzs4WFTPOROamZ0Uwv6COp64djbeOg3Fda1MjA7kdE0LKyZH46vX8s3JKnbn15AeHcil02Md17hoSgwWKQnrNJIaZ1+kNJosRAbouX5+EmEBesIDvCmoaeEHS1NZlh7F1zmVnK5u4ZGN2QDcs3IyjW0mx/apyWLlk8MlBPp4cfls28Jy+2jRn76+37GWER/iS3WzkaYe1pTO8OGhUhalnuaGBd0HxLmDvnyl3gEcA84kobwb2CylfFgIcbf9+a+d9Hsc+FRKeaUQQg94nFfKOakR3HzeOJ7fNnoEI6e857wKyydFs3xSNGaLldM1LezKq+G13afJq2x2i1v4Y9fM4i+fHqekXXXzp7bk8puVk/jh0glMig10VD5/ecdpQvy8OFJSz9GSBkeiohsXJnUQi1B/PS1GM8V1rSSE+lFY00J8iK9jJKL30nLb+amYLVaCfG2CMjsplNn2NIVzkkMxmq3UthjRCNtoZ+GfNnPvqilcvyAJf28dmfetAOgQs/LZkTLOnRCB3u7XkhjmS0ZKGCV1rS4nPbr33SzSowPJcJJ4x124JBZCiARgFfBH4C774bXAUvvjl4AtdBILIUQQsAT4PwAppRHwyG2H288fzwvbTw1KBitPoLyhzaXEODqthtTIAFIjA7h+QRJtJpvT0l8+Pd7FV2AoefqrXB64bAoPf3KcX1ycTmldG09szuFPH9sW/Pz02l5F7GBhx0A5k8VKcW0radGBZBXXs+bJbQBEB/lw76rJrJ4Rh5++54/A/oJajGYrGSlhRAX6MCEqgE+ySrl+gS3z95nXVwhbrM5/dxVQ1WRgZ14Nh4vrWTAujCUTI/k6p5KmPtZH9bSFd1cXOB8DfgW0H69GSylLAey/o5z0Gw9UAi8KIfYLIZ4TQvg7aYcQ4jYhRKYQIrOycvgTyUQF+jAj3rMyEw2Ea+cn9WtNwsdLy3Xzk/j4jsVMix/8TObdcbyskR+/tp87V0wkIcSX9w4U88Blk0mwe2m6Mto5XFzfYWvTZLE66qyEB+gJ9dPb/CXq2/jTR8ccMR6dsVglrUYLXxwvZ2tOFY0GMxOjA9BoBH++cgaXTrONXoxmK3/7LJvvv7KXq/+9g8V/+ZJPs8qICvTmhe2nmBAVwCs3L2BnXjUfHCzBS6shuQ/u3u6umt6ZXkcWQojVQIWUcq8QYmk/rj8H+ImUcpcQ4nFs05X7OzeUUj4DPAO2hL19vM+AsVplnxKteDK3nz+em88bN6BrxAb78sbti/i/F/cMWb2Mzpitkl+8eZB/XDebFqOFB98/yj2XTuL+DUdcvsbe0zUsn2TbLfHT6zCYbIIQG+zL366eyZNfnCTET8+PlqV2+DD++6tcsssbuW/VFG59OZPDRfWYrVas0rYVetti25bzpJggJsXYRPStvUX844uTHe5/7bwknrNPZ09WNHHJ41+TV2lz8e6pKHNnpsQGEdRLxvHhxpVpyLnAGiHESsAHCBJCvAqUCyFipZSlQohYoMJJ3yKgSEp5pvTTW9jEwuP47Gi54586klmcFsGvLp40KNfy0+t44trZLP3rl7SZhm4RNNBbx8rpsazPLLTFfrx/lF9eks5L3+Tz6ZEyJscGcczFUgl5lc0sb/fnt4/7WJoexdL0rgPgg4V1jiJFLUYL4yL8WTAujMQwP8wWK3qdpkMAmJQ2x6qdedXMSAh21PoA+CSrlIqGs7scfXlPaTWCNTPjmBoXRKvR0sWnw930KhZSynuAewDsI4tfSClvFEI8AqwDHrb/3uCkb5kQolAIkS6lzAYuAI4OnvmDQ35VMw++n+VuMwbMrMQQ/v2duYMaAh0T7MM/rpvD/e9lDdkcutFgdkw3wDZX//Vbh3jwsil8eKiUxFBfl8ViT34Ntywe3+FYZaOBMH99t6/LlLggMpJDyTxdy6dZZfh6adl3/wqMFis+XhpHXAnYhOLVXQV8fKgUnVZ0mR4dL+vex6U9gd46vHQaR+qAiABv/vO9eUzz4KnwQBwMHgbeEELcDBQAVwEIIeKA56SUK+3tfgL8174Tkgd8bwD3HHTqW03cvyGL8obe8yV6MoE+Ov55w5xeF+z6w+K0CC6eGs1LO04P+rXPkFVS7/B3yCpuwGyVPPHFSV6/dQHPb8t3+Tpfn6jCaLY6diIOFdVxw7O7SI0K4N0fnuN0wddLq+F/ty3krb1F/OGjY0yPD0av03CkpL7DbsSJ8kZ+/sZBR8YxjaDXdaGkMD8ykkOJDfEhJsiHuBBfksP9GB9hWwMpb2jjVFUzk2NtXqKeTJ/eWVLKLdh2PZBSVmMbKXRuUwKsbPf8AOByIZPh5nR1c7c1QkYSty8Z3yXd22AhBEyNC+a7i5J5eYgEQ0p4dl0G617Y7TgWF+wDCBrsi5bjI/ypbDJwbmpEt+UaWk0WKhrbSAj1w2KVfP+VvTQazBworGNLdiXLJjlbh7ftCl07P4kr5yZgtkq0GkFGShht9hov/9tTwEMfdBwUOxOKUD8vpsYFc15aBJdOiyE53Ol6voPoIB+iu/FA9TRGj+tiPxmJ6c06891FyY68kkOBwWzly+wKPskaupoje0/X4qXT8MOlqVQ1GQj08WJchD8fHirh/PRIrpufxJzkEHQaDV5awdp/bu+wVtCeV3ac5p6Vk6lrMXbw23CldqhOq+HMrOPvn2Xz0o7TWKyyQ4xKe0L9vDh3QgQLx4czf1wYaVEBfSrPMJIY82LROSPSSGPBuDDuXz1lSN+gJyua2Gj/Jr9oSjSHi+spre/b+oVeqyE+1JfIAG9MVitS2qaAZquV8noDEltdE3+9liaDma05VVQ3GRECdv3mgi55RtfMjOtWLAprbQWSwwO8CfPXU9di5LuLUpiZ0Pt6QJvJwqZj5byzr5gvsyu6+N3MSQohNTKAKXFBnJMaQWpk12zfo5UxLxb7CjwzaMcV/PVaHrhsypDsx+dXNfPu/mLiQ31JDPXjH9fNIdTfi9hgX5b/bQtRgd5UNLq+zjMlLoj4EF8Sw/xYlBrO5JhALFJitkgMZgsJoX4dRnknKxrZllNFY5uZoyUNnNOpbkdP1ex/uPTsKOu+VZMJ9dezrNMuyNacSj44WEJMkA8JoX40tJkc+TI6u7+nRvqzakYc35od73AVH4uMebHYfGxkVujSagT/unFutyXu/rXlZIcPTV94M7OQ37x7uENBohA/Ly6YFM3S9Ej+ePl0PjtaRkUfqrAdKKzjQGEdYPPW7Mzuey/oIBYTogKZENU1v+YZ9E4EMjLQm58un9BhR+GKOQkd2jS0mfjthiO8s79nnwe9TsOKKdHcuCCZheN7z4sxFhjTYmG1SopqXcuO7Wn8fu00zrWnl3fGQIrqvrOvuEvlsroWE2/vK+LtfUXd9BoYhwrruXCK6wt9MxJCmJEQzOK0CCbFBBEV6M2spJAO25yd2ZlXzc/fONht/guA5HA/frI8jVXTYx0VwhQ2xrRY5Fc392ko7QloNYLfrpnqiE3ojqsHkBVrRmIwO/Kq+92/O7x1GibFBHLxtBhajRYsVltEaIvR4jT5bk/odRre//F5LrU1mG1h+89szes29icxzJebzh3H9QuSehScscyYFotsFx1oPInfrZ065KHLP7tgIrMTQ9lfUMsbmYUuB5QF+uhIjQxgVmIICaG+xIX4Eh3kQ0KoLzqNINRP323uiaFiX0Etd799iBPdROFOignkh8smsGp67Iit5zFcjGmxGGnJa392Ydqw5Djw1Wu5ZFoMl0yL4ebF41j1hC1a02i24q/XkhTux4SoAGKDfYkP8SXUX0+CfSFUpxHDLgjOaDGa+evGE7z4jfNI4lmJIfxgaSorJkd7hL0jgTEtFptG0OLmhZOj+enytGG/b1SgDzvuXt7hW/fMYp/JYnVrZKTZYqWgpoWUcH/yqprw1es4UdZIdbORxzadcLoetTgtgjsuSPOoPBEjhTErFrmVTS778bubdYuSuadddStXqWk2EuSjG7AfQHf93R1CfdcbB3n/YAneOg0Gs9VRNqAzscE+LE2P4vJZcSwY3/2isKJnxqxYfHCwxN0muERGcigPXja1X0NlrUbQbLAQ7DcynYaklHydU8XWE5X8aNkEhwemlJL/7Snko8O2MgAGe82XzkIxKzGEWxaP4+KpMW4XttHAmBWLkxU9p53zBL41O54fLZvQ7zm1pwcmOaPZYKakrpWCmhbW7ynks6O2qeJ7B4q5cWEyxbWtVDUZ+LIbHw8h4LIZcfx4+YRR7XrtDsasWLS6Ic9kXwjz1/O7tVM71NYc7XyaVcad6w90KdgDUNVk5LFNOd32TQj15ZqMRK6YmzBkAXVjnTEpFm0mC5keWpsBbGHNL/xfxpgRCqtV8tRXuY4s2q4S6KPj4qkxXJ2RyNzkULX1OcSMarGobzXx0aFSQv28mJ0UipdWYJGSX791qEOuRk9Cr9Xwo2WpPbo6jxZqm4388q1D7DpVTaMLyWyXTIxk1fQYhBCkRQUwOTZoVEQNjxRGrVg0Gcxc8LctVLWLKk2N9Ke4rnVIU8QNhKXpkcxJCuWaeT17Z44WHv7kuEvb18vSI7lrRTrTXYgaVQwdo1Is/rXlJO8fKOkgFAC5Hpxjc9WMWJ68bvaoKUXgCr1F/C4aH86vLkl31PFQuJdRJxaFNS28saeQ/OoWd5viMj5eGh6056QYS4v3ft7O334Lx4fx42VpnDshXO1meBCjRiwKa1poM1n4/qt7R4xQRAd5c+m0WBaODyNqhKRWG0w6Fyw+JzWcX1yczhw1kvBIRoVYFNe1cvk/t1Pd7HlZr9bMjOOOC9MI8vHiz58e5629thDvWYkhPPzt6Y4aFGORulbb/2vR+HB+sDSVxWkRaiThwYwKsXhua55HCsWi8eH84VvTHMVi/vSt6ezMqyYjOZRHr5k15j8YaVGBPHndHGYmhrjbFIULjAof2HWLUtB52B67ViP45w1zOlSV0us0TI0L4qE108a8UEgpefrGuUooRhCjQixSIvyZFOtZfgkWq8RoPrtFa7VKXth2iiUTIwn2GxvOVj0hhHDU9lCMDEbNfysj2bNCji+YFIW/t81hSErJxiNlrN9TyDUDyGClULiTUbFmAXDHBWl8mlXm9jL10UHe/PSCNK6YncDh4nq+PlFJm8nC67sLeOK62WMmbbxi9DFqxCLUX09GSigfHip1mw3xIb48ty6DybG2HQ4h4MkvbVW2r5gdzwWTo91mm0IxUEaNWABujTacFh/Ea7cu7LCgmZEcSkKoLQ/l7y6f5jbbFIrBYFSJxVcnXK9jMZhcMSee366Z2kEowLaI9/YPziHQRzckBYsViuFk1LyDmw1mssuHL02eEHDbkvFcOi2WWT1s/42UorcKRW+MGrGobTGiFQLzMERixYf48s8b5vQoEgrFaGPUiEVCqB9h/vohLRq0cHwYv1s7jeggnxGZsk6hGAijRiwAHr1mFute2I3ZOjijC61GcMXseMZF+nOkuIFHr5mlHIkUY5YRLxbSPu146qtcNmaVDZpQ6DSCF783j8VpkY77jHUXbcXYZsSLxbHSRl7bfZpXdxYM6nUfvGyKQygAJRSKMc+IFoui2ha+8/yuQYk49dIKR+Xw6xck8Z1FKQO+pkIxmhjRYvHaroIBCcW18xK5YHI0c5JCeGdfMX/8+BjT4oN48LIpg2ilQjE6GLFiYbZY2ZFX3e/+3zs3hQfsqewAbl0ynrToAFIjA/DWqYzRCkVnRqxYvL6nkP0Fdf3qmxzux10rJnZZh1iaHjUIlikUoxOXxUIIoQUygWIp5WohRBiwHkgB8oGrpZRd0jULIfKBRsACmKWUGQM3G+cVcF1g0fhwXvzePFVvQqHoI31xGrgDONbu+d3AZillGrDZ/rw7lkkpZw2aUADjIwP61D422Iflk6L4y5UzlFAoFP3ApZGFECIBWAX8EbjLfngtsNT++CVgC/DrwTWve85JDSctKoAcJwWOw/31rJ4Ry7Xzk0gJ96ekvpXxEf5q+1OhGACujiweA34FtC/lFS2lLAWw/+5uwi+Bz4QQe4UQt3V3AyHEbUKITCFEZmVl79GjQggevWYWgT4d9W7h+DC2/HIpD62dxuTYIHz1WlIjVTVthWKg9DqyEEKsBiqklHuFEEv7cY9zpZQlQogo4HMhxHEp5dedG0kpnwGeAcjIyHBpQWJafDD3XDqZF7af4hcXpRMZqGdidOCYKSisUAwnrkxDzgXWCCFWAj5AkBDiVaBcCBErpSwVQsQCFc46SylL7L8rhBDvAvOBLmLRX65fkMT1C8ZGbVCFwp30Og2RUt4jpUyQUqYA1wJfSClvBN4H1tmbrQM2dO4rhPAXQgSeeQxcBGQNku0KhWIYGUgI5cPACiFEDrDC/hwhRJwQ4mN7m2hgmxDiILAb+EhK+elADFYoFO6hT05ZUsot2HY9kFJWAxc4aVMCrLQ/zgNmDtRIhULhflRyBoVC4RJKLBQKhUsosVAoFC6hxEKhULiEEguFQuESSiwUCoVLCDkMdTb6ihCiEjg9zLeNAKqG+Z49oezpGU+yx5NsAdftSZZSRvbezIZHioU7EEJkDmYI/UBR9vSMJ9njSbbA0NmjpiEKhcIllFgoFAqXUGJxlmfcbUAnlD0940n2eJItMET2qDULhULhEmpkoVAoXEKJhUKhcIkxJRZCiPVCiAP2n3whxIFO55OEEE1CiF900/8RIcRxIcQhIcS7QogQN9tzlRDiiBDCKoQY8FbZINgTJoT4XAiRY/8dOti2CCHmtzt+UAjxrW76zxRC7BBCHBZCfCCECOqvLYNkzywhxE57u0whxHw329Pj/9opUsox+QP8DXig07G3gTeBX3TT5yJAZ3/8Z+DPbrZnMpCOLcdIhge8Pn8B7rY/vnuwXp/2tgB+7f4HZ9I56pz02QOcb398E/D7oXht+mDPZ8Cl9scrgS3utKe3/7WznxFbkWwgCFuq76uB5e2OXQ7kAc3d9ZNSftbu6U7gSjfbc8zedjDMGLA9DEF5iM62SClb2p32wZY93hnpnM31+jmwEbh/ILYM0B4JnBndBAMlA7VlgPY47d8TY2oa0o7FQLmUMgcc+UF/DTzUh2vcBHziQfYMJv21x9XyEP22xW7PAiHEEeAw8H0ppdlJvyxgjf3xVUDiINgyEHt+BjwihCgE/grc42Z7uu3fHaNuZCGE2ATEODl1r5TyTFLh64DX2517CHhUStnkyre0EOJewAz81xPs6QueZE8/bUFKuQuYKoSYDLwkhPhEStnW6Ro3AU8IIR7Allza6GZ7fgDcKaV8WwhxNfA8cKEb7TlDl/7dMljzppHyg00gy4GEdse2YqvXmg/UATXAj7vpvw7YAfh5gj329lsYpDWLgdgDZAOx9sexQPZg2+KkzZe9/e3ARGD3ULw2rtoD1HPWr0kADe60x9X+7X9G3cjCBS4Ejkspi84ckFIuPvNYCPFboElK+WTnjkKIS7ANx8+XHeeGbrFniBiIPWfKQzxMN+UhBmqLEGIcUCilNAshkrGtTeR37iiEiJK2WjUa4D7g6QHaMiB7sK1RnI9N2JcDvQ77h9gep/17YiyuWVyLq8MuQAjxXLttySeBQGyV1Q4IIQbjDdhve4QQ3xJCFAGLgI+EEBvdaQ/dlIcYZFvOAw7at/reBX4opaxyYst1QogTwHFsH9QXB2jLQO25FfibsJXF+BPQbSnPYbKnu/7doty9FQqFS4zFkYVCoegHSiwUCoVLKLFQKBQuocRCoVC4hBILhULhEkosFAqFSyixUCgULvH/AXfdDUuQQj64AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plotting boros\n",
    "gdf.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 Reading in data from Socrata Open Data API (SODA)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note: If you haven't signed up for an app token, there might be a 1,000 rows limit. Of course, verify on the SODA documentation page.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![building_footprints_csv](images/building-footprints-soda-api.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Requests made without an app_token will be subject to strict throttling limits.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'domain': 'data.cityofnewyork.us',\n",
       " 'session': <requests.sessions.Session at 0x1712fd0a0>,\n",
       " 'uri_prefix': 'https://',\n",
       " 'timeout': 10}"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# nyc open data domain\n",
    "socrata_domain = 'data.cityofnewyork.us' \n",
    "\n",
    "# building footprints dataset identifier\n",
    "socrata_dataset_identifier = 'qb5r-6dgf' \n",
    "\n",
    "# The main class that interacts with the SODA API.     \n",
    "client = Socrata(socrata_domain, None)\n",
    "\n",
    "client.__dict__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Source: https://github.com/xmunoz/sodapy/blob/master/examples/basic_queries.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Domain: data.cityofnewyork.us    \n",
      "Session: <requests.sessions.Session object at 0x1712fd0a0>    \n",
      "URI Prefix: https://\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    \"Domain: {domain:}\\\n",
    "    \\nSession: {session:}\\\n",
    "    \\nURI Prefix: {uri_prefix:}\".format(**client.__dict__)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are setting the **limit** at **1,000 rows** (i.e. the full data set)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>bin</th>\n",
       "      <th>cnstrct_yr</th>\n",
       "      <th>lstmoddate</th>\n",
       "      <th>lststatype</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>geomsource</th>\n",
       "      <th>globalid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>3170958</td>\n",
       "      <td>1925</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>96807</td>\n",
       "      <td>29.74985318</td>\n",
       "      <td>2100</td>\n",
       "      <td>40</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>3065220021</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{31298F86-3088-4F53-B3DB-71A9EFA6FA1F}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>5028452</td>\n",
       "      <td>1965</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>326368</td>\n",
       "      <td>22.63</td>\n",
       "      <td>2100</td>\n",
       "      <td>39</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>5012640036</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F5F8CDA5-69E2-46F8-8F69-BA95C025B520}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>5078368</td>\n",
       "      <td>1970</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746627</td>\n",
       "      <td>35.76</td>\n",
       "      <td>2100</td>\n",
       "      <td>51</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>5060190091</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{9F644794-F72C-4582-9E5E-B337E2B97068}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>3245111</td>\n",
       "      <td>1928</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>786626</td>\n",
       "      <td>37.5</td>\n",
       "      <td>2100</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>3086910048</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{F916B22D-E25B-44AE-9FA9-2A51191B9CDF}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>{'type': 'MultiPolygon', 'coordinates': [[[[-7...</td>\n",
       "      <td>4161096</td>\n",
       "      <td>1950</td>\n",
       "      <td>2017-08-22T00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>746409</td>\n",
       "      <td>18.01511294</td>\n",
       "      <td>2100</td>\n",
       "      <td>93</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>4075020005</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom      bin cnstrct_yr  \\\n",
       "0  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  3170958       1925   \n",
       "1  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  5028452       1965   \n",
       "2  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  5078368       1970   \n",
       "3  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  3245111       1928   \n",
       "4  {'type': 'MultiPolygon', 'coordinates': [[[[-7...  4161096       1950   \n",
       "\n",
       "                lstmoddate   lststatype doitt_id   heightroof feat_code  \\\n",
       "0  2017-08-22T00:00:00.000  Constructed    96807  29.74985318      2100   \n",
       "1  2017-08-22T00:00:00.000  Constructed   326368        22.63      2100   \n",
       "2  2017-08-22T00:00:00.000  Constructed   746627        35.76      2100   \n",
       "3  2017-08-22T00:00:00.000  Constructed   786626         37.5      2100   \n",
       "4  2017-08-22T00:00:00.000  Constructed   746409  18.01511294      2100   \n",
       "\n",
       "  groundelev shape_area shape_len    base_bbl  mpluto_bbl  geomsource  \\\n",
       "0         40        0.0       0.0  3065220021  3065220021  Photogramm   \n",
       "1         39        0.0       0.0  5012640036  5012640036  Photogramm   \n",
       "2         51        0.0       0.0  5060190091  5060190091  Photogramm   \n",
       "3          6        0.0       0.0  3086910048  3086910048  Photogramm   \n",
       "4         93        0.0       0.0  4075020005  4075020005  Photogramm   \n",
       "\n",
       "                                 globalid  \n",
       "0  {31298F86-3088-4F53-B3DB-71A9EFA6FA1F}  \n",
       "1  {F5F8CDA5-69E2-46F8-8F69-BA95C025B520}  \n",
       "2  {9F644794-F72C-4582-9E5E-B337E2B97068}  \n",
       "3  {F916B22D-E25B-44AE-9FA9-2A51191B9CDF}  \n",
       "4  {525F2C24-616B-4F29-98A3-8FEA5D4B1A7D}  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieving data as a dictionary \n",
    "results = client.get(socrata_dataset_identifier, limit=1000)\n",
    "\n",
    "# creating a dataframe from our dictionary\n",
    "df_api = pd.DataFrame.from_dict(results)\n",
    "\n",
    "# ending our API request\n",
    "client.close()\n",
    "\n",
    "# printing first five rows of data\n",
    "df_api.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This dataset has 1,000 rows and 15 columns.\n"
     ]
    }
   ],
   "source": [
    "rows, columns = df_api.shape\n",
    "print('This dataset has {:,} rows and {:,} columns.'.format(rows, columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Useful resources:**\n",
    "    \n",
    "- API Docs: https://dev.socrata.com/foundry/data.cityofnewyork.us/i62d-kjv8\n",
    "- Sign up for app token: https://data.cityofnewyork.us/profile/edit/developer_settings\n",
    "- Python client for the Socrata Open Data API: https://github.com/xmunoz/sodapy\n",
    "- Examples: https://github.com/xmunoz/sodapy/tree/master/examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Writing Out Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>the_geom</th>\n",
       "      <th>NAME</th>\n",
       "      <th>BIN</th>\n",
       "      <th>CNSTRCT_YR</th>\n",
       "      <th>LSTMODDATE</th>\n",
       "      <th>LSTSTATYPE</th>\n",
       "      <th>DOITT_ID</th>\n",
       "      <th>HEIGHTROOF</th>\n",
       "      <th>FEAT_CODE</th>\n",
       "      <th>GROUNDELEV</th>\n",
       "      <th>SHAPE_AREA</th>\n",
       "      <th>SHAPE_LEN</th>\n",
       "      <th>BASE_BBL</th>\n",
       "      <th>MPLUTO_BBL</th>\n",
       "      <th>GEOMSOURCE</th>\n",
       "      <th>GLOBALID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>MULTIPOLYGON (((-73.87172426474349 40.65519420...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3394834</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1250314</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{C045C815-79DB-4644-AD9D-C34AC03D1AB4}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MULTIPOLYGON (((-73.86650099829305 40.74236058...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540159</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>201366</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{FDF673E7-FF92-4A7A-AF6D-C49D77343C47}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>MULTIPOLYGON (((-73.87805078807256 40.71475698...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4540051</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1171655</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2100</td>\n",
       "      <td>112.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{788E5D72-46C1-443F-8BC9-6B97F329BFED}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MULTIPOLYGON (((-73.81520745135124 40.73053646...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4545453</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1118502</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2100</td>\n",
       "      <td>74.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>{789A5A51-5B12-46DC-AE85-B06820A3225E}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MULTIPOLYGON (((-73.84769179857282 40.87911947...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2118998</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>08/22/2017 12:00:00 AM</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>1254551</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>2100</td>\n",
       "      <td>154.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>2047220003</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>{BB58FD7B-CC22-4896-901D-F8BAFF4AC129}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            the_geom NAME      BIN  \\\n",
       "0  MULTIPOLYGON (((-73.87172426474349 40.65519420...  NaN  3394834   \n",
       "1  MULTIPOLYGON (((-73.86650099829305 40.74236058...  NaN  4540159   \n",
       "2  MULTIPOLYGON (((-73.87805078807256 40.71475698...  NaN  4540051   \n",
       "3  MULTIPOLYGON (((-73.81520745135124 40.73053646...  NaN  4545453   \n",
       "4  MULTIPOLYGON (((-73.84769179857282 40.87911947...  NaN  2118998   \n",
       "\n",
       "   CNSTRCT_YR              LSTMODDATE   LSTSTATYPE  DOITT_ID  HEIGHTROOF  \\\n",
       "0      2011.0  08/22/2017 12:00:00 AM  Constructed   1250314   26.000000   \n",
       "1      2010.0  08/22/2017 12:00:00 AM  Constructed    201366   28.000000   \n",
       "2      2010.0  08/22/2017 12:00:00 AM  Constructed   1171655   28.330225   \n",
       "3      2012.0  08/22/2017 12:00:00 AM  Constructed   1118502   16.640000   \n",
       "4      2012.0  08/22/2017 12:00:00 AM  Constructed   1254551   33.000000   \n",
       "\n",
       "   FEAT_CODE  GROUNDELEV  SHAPE_AREA  SHAPE_LEN    BASE_BBL  MPLUTO_BBL  \\\n",
       "0       2100        15.0           0          0  3044520924  3044520924   \n",
       "1       2100        37.0           0          0  4018780115  4018780115   \n",
       "2       2100       112.0           0          0  4030600139  4030600139   \n",
       "3       2100        74.0           0          0  4066560052  4066560052   \n",
       "4       2100       154.0           0          0  2047220003  2047220003   \n",
       "\n",
       "   GEOMSOURCE                                GLOBALID  \n",
       "0  Other (Man  {C045C815-79DB-4644-AD9D-C34AC03D1AB4}  \n",
       "1  Other (Man  {FDF673E7-FF92-4A7A-AF6D-C49D77343C47}  \n",
       "2  Photogramm  {788E5D72-46C1-443F-8BC9-6B97F329BFED}  \n",
       "3  Photogramm  {789A5A51-5B12-46DC-AE85-B06820A3225E}  \n",
       "4  Other (Man  {BB58FD7B-CC22-4896-901D-F8BAFF4AC129}  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a dataframe\n",
    "df = pd.read_csv('data/sample-data.csv')\n",
    "\n",
    "# previewing first five rows in data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1 Writing to a CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                nta_shape.shx            sample-data.gpkg\r\n",
      "building-footprints.csv  output.csv               sample-data.json\r\n",
      "nta_shape.cpg            sample-buildings.zip     sample-data.prj\r\n",
      "nta_shape.dbf            sample-data.cpg          sample-data.shp\r\n",
      "nta_shape.geojson        sample-data.csv          sample-data.shx\r\n",
      "nta_shape.prj            sample-data.dbf          sample-data.xlsx\r\n",
      "nta_shape.shp            sample-data.geojson      \u001b[34munzipped-data\u001b[m\u001b[m/\r\n"
     ]
    }
   ],
   "source": [
    "# writing files as a csv\n",
    "df.to_csv('data/output.csv', index=False)\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Writing to an Excel (xlsx) file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                output.csv               sample-data.json\r\n",
      "building-footprints.csv  output.xlsx              sample-data.prj\r\n",
      "nta_shape.cpg            sample-buildings.zip     sample-data.shp\r\n",
      "nta_shape.dbf            sample-data.cpg          sample-data.shx\r\n",
      "nta_shape.geojson        sample-data.csv          sample-data.xlsx\r\n",
      "nta_shape.prj            sample-data.dbf          \u001b[34munzipped-data\u001b[m\u001b[m/\r\n",
      "nta_shape.shp            sample-data.geojson\r\n",
      "nta_shape.shx            sample-data.gpkg\r\n"
     ]
    }
   ],
   "source": [
    "# writing files as an excel file\n",
    "df.to_excel('data/output.xlsx', index=False)\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 Writing to a JSON file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                output.csv               sample-data.gpkg\r\n",
      "building-footprints.csv  output.json              sample-data.json\r\n",
      "nta_shape.cpg            output.xlsx              sample-data.prj\r\n",
      "nta_shape.dbf            sample-buildings.zip     sample-data.shp\r\n",
      "nta_shape.geojson        sample-data.cpg          sample-data.shx\r\n",
      "nta_shape.prj            sample-data.csv          sample-data.xlsx\r\n",
      "nta_shape.shp            sample-data.dbf          \u001b[34munzipped-data\u001b[m\u001b[m/\r\n",
      "nta_shape.shx            sample-data.geojson\r\n"
     ]
    }
   ],
   "source": [
    "# writing files as json\n",
    "df.to_json('data/output.json')\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.4 Writing to a Shapefile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>base_bbl</th>\n",
       "      <th>bin</th>\n",
       "      <th>cnstrct_yr</th>\n",
       "      <th>doitt_id</th>\n",
       "      <th>feat_code</th>\n",
       "      <th>geomsource</th>\n",
       "      <th>groundelev</th>\n",
       "      <th>heightroof</th>\n",
       "      <th>date_lstmo</th>\n",
       "      <th>time_lstmo</th>\n",
       "      <th>lststatype</th>\n",
       "      <th>mpluto_bbl</th>\n",
       "      <th>name</th>\n",
       "      <th>shape_area</th>\n",
       "      <th>shape_len</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3044520924</td>\n",
       "      <td>3394834.0</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>1250314.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>15.0</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>3044520924</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.87172 40.65519, -73.87179 40.655...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4018780115</td>\n",
       "      <td>4540159.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>201366.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>37.0</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4018780115</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.86650 40.74236, -73.86645 40.742...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4120060029</td>\n",
       "      <td>4260357.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1184712.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Other (Man</td>\n",
       "      <td>20.0</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>2017-08-10</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4120060029</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.79408 40.68063, -73.79407 40.680...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4030600139</td>\n",
       "      <td>4540051.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1171655.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>112.0</td>\n",
       "      <td>28.330225</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4030600139</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.87805 40.71476, -73.87787 40.714...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4066560052</td>\n",
       "      <td>4545453.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>1118502.0</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>Photogramm</td>\n",
       "      <td>74.0</td>\n",
       "      <td>16.640000</td>\n",
       "      <td>2017-08-22</td>\n",
       "      <td>00:00:00.000</td>\n",
       "      <td>Constructed</td>\n",
       "      <td>4066560052</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>POLYGON ((-73.81521 40.73054, -73.81546 40.730...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     base_bbl        bin  cnstrct_yr   doitt_id  feat_code  geomsource  \\\n",
       "0  3044520924  3394834.0      2011.0  1250314.0     2100.0  Other (Man   \n",
       "1  4018780115  4540159.0      2010.0   201366.0     2100.0  Other (Man   \n",
       "2  4120060029  4260357.0      2010.0  1184712.0     2100.0  Other (Man   \n",
       "3  4030600139  4540051.0      2010.0  1171655.0     2100.0  Photogramm   \n",
       "4  4066560052  4545453.0      2012.0  1118502.0     2100.0  Photogramm   \n",
       "\n",
       "   groundelev  heightroof  date_lstmo    time_lstmo   lststatype  mpluto_bbl  \\\n",
       "0        15.0   26.000000  2017-08-22  00:00:00.000  Constructed  3044520924   \n",
       "1        37.0   28.000000  2017-08-22  00:00:00.000  Constructed  4018780115   \n",
       "2        20.0   29.000000  2017-08-10  00:00:00.000  Constructed  4120060029   \n",
       "3       112.0   28.330225  2017-08-22  00:00:00.000  Constructed  4030600139   \n",
       "4        74.0   16.640000  2017-08-22  00:00:00.000  Constructed  4066560052   \n",
       "\n",
       "   name  shape_area  shape_len  \\\n",
       "0  None         0.0        0.0   \n",
       "1  None         0.0        0.0   \n",
       "2  None         0.0        0.0   \n",
       "3  None         0.0        0.0   \n",
       "4  None         0.0        0.0   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((-73.87172 40.65519, -73.87179 40.655...  \n",
       "1  POLYGON ((-73.86650 40.74236, -73.86645 40.742...  \n",
       "2  POLYGON ((-73.79408 40.68063, -73.79407 40.680...  \n",
       "3  POLYGON ((-73.87805 40.71476, -73.87787 40.714...  \n",
       "4  POLYGON ((-73.81521 40.73054, -73.81546 40.730...  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read data as a geodataframe\n",
    "gdf = gpd.read_file('data/sample-data.shp')\n",
    "\n",
    "# previewing first five rows in data\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "README.md                output.dbf               sample-data.geojson\r\n",
      "building-footprints.csv  output.json              sample-data.gpkg\r\n",
      "nta_shape.cpg            output.prj               sample-data.json\r\n",
      "nta_shape.dbf            output.shp               sample-data.prj\r\n",
      "nta_shape.geojson        output.shx               sample-data.shp\r\n",
      "nta_shape.prj            output.xlsx              sample-data.shx\r\n",
      "nta_shape.shp            sample-buildings.zip     sample-data.xlsx\r\n",
      "nta_shape.shx            sample-data.cpg          \u001b[34munzipped-data\u001b[m\u001b[m/\r\n",
      "output.cpg               sample-data.csv\r\n",
      "output.csv               sample-data.dbf\r\n"
     ]
    }
   ],
   "source": [
    "gdf.to_file('data/output.shp')\n",
    "\n",
    "# listing items in data folder\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we reviewed various ways to read (load) and write (save) data from NYC Open Data. Specifically, we focused on reading our data into a pandas dataframe. We also went over common file formats that you might encounter - csv, json, shapefiles, and zip files. In Part II, we will focus on basic data inspection and wrangling techniques in the data analysis workflow."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}