ibis-basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "099be667",
   "metadata": {},
   "source": [
    "# Ibis Basics\n",
    "Author: Mark Bauer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "338585f7",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "This tutorial was inspired by my recent but intense interest in composable SQL and Python dataframe code, and of course, Ibis. Learning how well DuckDB integrates with Ibis was even more of a reason to delve deeper into this portable Python dataframe library. And lastly, anything that Wes McKinney is passionate about is probably worth learning.\n",
    "\n",
    "**Resources**  \n",
    "Tutorials from Ibis contributors served as great starting points. Much of what I know about Ibis either came from the offical docs or these tutorials:\n",
    "- Official Docs Tutorial: getting started: https://ibis-project.org/tutorials/getting_started\n",
    "- Ibis Tutorial PyCon 2024: https://github.com/ibis-project/ibis-tutorial\n",
    "- A repository of runnable examples using ibis: https://github.com/ibis-project/ibis-examples\n",
    "\n",
    "**Ibis Documentation**  \n",
    "Always start at the Official Docs: https://ibis-project.org/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b400573d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import ibis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "088fced8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python implementation: CPython\n",
      "Python version       : 3.8.13\n",
      "IPython version      : 8.4.0\n",
      "\n",
      "ibis: 3.2.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# to reproduce\n",
    "%reload_ext watermark\n",
    "%watermark -v -p ibis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cd07e097",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34mraw\u001b[m\u001b[m/                  storm-events.db       storm-events.parquet\r\n"
     ]
    }
   ],
   "source": [
    "# list items in data directory\n",
    "%ls data/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b72559ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "# we will use Ibis interactive mode\n",
    "ibis.options.interactive = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b29a9fc",
   "metadata": {},
   "source": [
    "# Connect to a database. I'm using DuckDB."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d44358f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.backends.duckdb.Backend"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reconnect to the persisted database called storm-events\n",
    "con = ibis.duckdb.connect(\"data/storm-events.db\")\n",
    "\n",
    "type(con)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "225566f0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['storm_events']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# list tables in .db file\n",
    "con.list_tables()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b4d3d29f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/geribauer/anaconda3/envs/geopandas_env/lib/python3.8/site-packages/duckdb_engine/__init__.py:229: DuckDBEngineWarning: duckdb-engine doesn't yet support reflection on indices\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.relations.Table"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a table expression and assign to storm_events alias\n",
    "storm_events = con.table(\"storm_events\")\n",
    "\n",
    "type(storm_events)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c7969ca",
   "metadata": {},
   "source": [
    "Note: DuckDB will limit table preview to 10,000 rows."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fe28c0f",
   "metadata": {},
   "source": [
    "# Familiarize with Ibis table "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a7b7a3b8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.Schema {\n",
       "  EVENT_ID                   int32\n",
       "  STATE                      string\n",
       "  STATE_FIPS                 int32\n",
       "  YEAR                       int32\n",
       "  MONTH_NAME                 string\n",
       "  EVENT_TYPE                 string\n",
       "  CZ_TYPE                    string\n",
       "  CZ_FIPS                    int32\n",
       "  CZ_NAME                    string\n",
       "  WFO                        string\n",
       "  BEGIN_DATE_TIME            string\n",
       "  END_DATE_TIME              string\n",
       "  INJURIES_DIRECT            int32\n",
       "  INJURIES_INDIRECT          int32\n",
       "  DEATHS_DIRECT              int32\n",
       "  DEATHS_INDIRECT            int32\n",
       "  DAMAGE_PROPERTY            string\n",
       "  DAMAGE_CROPS               string\n",
       "  SOURCE                     string\n",
       "  DAMAGE_PROPERTY_CONVERTED  float64\n",
       "}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get table schema\n",
    "storm_events.schema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "fc6d7c98",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[3m                         Summary of storm_events                          \u001b[0m\n",
      "\u001b[3m                               1295193 rows                               \u001b[0m\n",
      "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┓\n",
      "┃\u001b[1m \u001b[0m\u001b[1mName                     \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mType                  \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m# Nulls\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m% Nulls\u001b[0m\u001b[1m \u001b[0m┃\n",
      "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━┩\n",
      "│ EVENT_ID                  │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ STATE                     │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │       0 │    0.00 │\n",
      "│ STATE_FIPS                │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ YEAR                      │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ MONTH_NAME                │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │       0 │    0.00 │\n",
      "│ EVENT_TYPE                │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │       0 │    0.00 │\n",
      "│ CZ_TYPE                   │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │       0 │    0.00 │\n",
      "│ CZ_FIPS                   │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ CZ_NAME                   │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │    1557 │    0.12 │\n",
      "│ WFO                       │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │  125574 │    9.70 │\n",
      "│ BEGIN_DATE_TIME           │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │       0 │    0.00 │\n",
      "│ END_DATE_TIME             │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │       0 │    0.00 │\n",
      "│ INJURIES_DIRECT           │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ INJURIES_INDIRECT         │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ DEATHS_DIRECT             │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ DEATHS_INDIRECT           │ \u001b[1;35mInt32\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m   │       0 │    0.00 │\n",
      "│ DAMAGE_PROPERTY           │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │       0 │    0.00 │\n",
      "│ DAMAGE_CROPS              │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │  144672 │   11.17 │\n",
      "│ SOURCE                    │ \u001b[1;35mString\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m  │  260208 │   20.09 │\n",
      "│ DAMAGE_PROPERTY_CONVERTED │ \u001b[1;35mFloat64\u001b[0m\u001b[1m(\u001b[0m\u001b[33mnullable\u001b[0m=\u001b[3;92mTrue\u001b[0m\u001b[1m)\u001b[0m │       0 │    0.00 │\n",
      "└───────────────────────────┴────────────────────────┴─────────┴─────────┘\n"
     ]
    }
   ],
   "source": [
    "# summarize table\n",
    "storm_events.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "97ff93f9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['EVENT_ID',\n",
       " 'STATE',\n",
       " 'STATE_FIPS',\n",
       " 'YEAR',\n",
       " 'MONTH_NAME',\n",
       " 'EVENT_TYPE',\n",
       " 'CZ_TYPE',\n",
       " 'CZ_FIPS',\n",
       " 'CZ_NAME',\n",
       " 'WFO',\n",
       " 'BEGIN_DATE_TIME',\n",
       " 'END_DATE_TIME',\n",
       " 'INJURIES_DIRECT',\n",
       " 'INJURIES_INDIRECT',\n",
       " 'DEATHS_DIRECT',\n",
       " 'DEATHS_INDIRECT',\n",
       " 'DAMAGE_PROPERTY',\n",
       " 'DAMAGE_CROPS',\n",
       " 'SOURCE',\n",
       " 'DAMAGE_PROPERTY_CONVERTED']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# columns attribute\n",
    "storm_events.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "485b828a",
   "metadata": {},
   "source": [
    "# Examine Ibis types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d7556b9a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.strings.StringColumn"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.EVENT_TYPE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ba122a3e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.numeric.IntegerColumn"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.YEAR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f8e8aa83",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.numeric.FloatingColumn"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.DAMAGE_PROPERTY_CONVERTED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "ba661ee8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.logical.BooleanColumn"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.DAMAGE_PROPERTY_CONVERTED > 100_000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "94b7f1b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.relations.Table"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.select('YEAR'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9d10504f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.numeric.IntegerColumn"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.select('YEAR').YEAR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "a2ed19f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.relations.Table"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.select('YEAR', 'EVENT_TYPE'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86760d61",
   "metadata": {},
   "source": [
    "# Preview data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "8bad001a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>BEGIN_DATE_TIME</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME EVENT_TYPE CZ_TYPE  \\\n",
       "0  10096222      OKLAHOMA          40  1950      April    Tornado       C   \n",
       "1  10120412         TEXAS          48  1950      April    Tornado       C   \n",
       "2  10104927  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "3  10104928  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "4  10104929  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "\n",
       "   CZ_FIPS   CZ_NAME   WFO     BEGIN_DATE_TIME       END_DATE_TIME  \\\n",
       "0      149   WASHITA  None  28-APR-50 14:45:00  28-APR-50 14:45:00   \n",
       "1       93  COMANCHE  None  29-APR-50 15:30:00  29-APR-50 15:30:00   \n",
       "2       77    LEHIGH  None  05-JUL-50 18:00:00  05-JUL-50 18:00:00   \n",
       "3       43   DAUPHIN  None  05-JUL-50 18:30:00  05-JUL-50 18:30:00   \n",
       "4       39  CRAWFORD  None  24-JUL-50 14:40:00  24-JUL-50 14:40:00   \n",
       "\n",
       "   INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                0                  0              0                0   \n",
       "1                0                  0              0                0   \n",
       "2                2                  0              0                0   \n",
       "3                0                  0              0                0   \n",
       "4                0                  0              0                0   \n",
       "\n",
       "  DAMAGE_PROPERTY DAMAGE_CROPS SOURCE  DAMAGE_PROPERTY_CONVERTED  \n",
       "0            250K            0   None                   250000.0  \n",
       "1             25K            0   None                    25000.0  \n",
       "2             25K            0   None                    25000.0  \n",
       "3            2.5K            0   None                     2500.0  \n",
       "4            2.5K            0   None                     2500.0  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# preview first five rows\n",
    "storm_events.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "a70885ea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.relations.Table"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ed20033d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>BEGIN_DATE_TIME</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME EVENT_TYPE CZ_TYPE  \\\n",
       "0  10096222      OKLAHOMA          40  1950      April    Tornado       C   \n",
       "1  10120412         TEXAS          48  1950      April    Tornado       C   \n",
       "2  10104927  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "3  10104928  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "4  10104929  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "\n",
       "   CZ_FIPS   CZ_NAME   WFO     BEGIN_DATE_TIME       END_DATE_TIME  \\\n",
       "0      149   WASHITA  None  28-APR-50 14:45:00  28-APR-50 14:45:00   \n",
       "1       93  COMANCHE  None  29-APR-50 15:30:00  29-APR-50 15:30:00   \n",
       "2       77    LEHIGH  None  05-JUL-50 18:00:00  05-JUL-50 18:00:00   \n",
       "3       43   DAUPHIN  None  05-JUL-50 18:30:00  05-JUL-50 18:30:00   \n",
       "4       39  CRAWFORD  None  24-JUL-50 14:40:00  24-JUL-50 14:40:00   \n",
       "\n",
       "   INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                0                  0              0                0   \n",
       "1                0                  0              0                0   \n",
       "2                2                  0              0                0   \n",
       "3                0                  0              0                0   \n",
       "4                0                  0              0                0   \n",
       "\n",
       "  DAMAGE_PROPERTY DAMAGE_CROPS SOURCE  DAMAGE_PROPERTY_CONVERTED  \n",
       "0            250K            0   None                   250000.0  \n",
       "1             25K            0   None                    25000.0  \n",
       "2             25K            0   None                    25000.0  \n",
       "3            2.5K            0   None                     2500.0  \n",
       "4            2.5K            0   None                     2500.0  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transform to pandas df\n",
    "storm_events.head().execute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "acb45174",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.head().execute())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "70d4a8dd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1295193"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# numer of rows\n",
    "storm_events.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "588e9ef9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# numer of columns\n",
    "len(storm_events.schema())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "831fddf9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1295193, 20)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# row, columns - similar to pandas df.shape attribute\n",
    "(storm_events.count().execute(), len(storm_events.schema()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "9867dcd7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>BEGIN_DATE_TIME</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME EVENT_TYPE CZ_TYPE  \\\n",
       "0  10096222      OKLAHOMA          40  1950      April    Tornado       C   \n",
       "1  10120412         TEXAS          48  1950      April    Tornado       C   \n",
       "2  10104927  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "3  10104928  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "4  10104929  PENNSYLVANIA          42  1950       July    Tornado       C   \n",
       "\n",
       "   CZ_FIPS   CZ_NAME   WFO     BEGIN_DATE_TIME       END_DATE_TIME  \\\n",
       "0      149   WASHITA  None  28-APR-50 14:45:00  28-APR-50 14:45:00   \n",
       "1       93  COMANCHE  None  29-APR-50 15:30:00  29-APR-50 15:30:00   \n",
       "2       77    LEHIGH  None  05-JUL-50 18:00:00  05-JUL-50 18:00:00   \n",
       "3       43   DAUPHIN  None  05-JUL-50 18:30:00  05-JUL-50 18:30:00   \n",
       "4       39  CRAWFORD  None  24-JUL-50 14:40:00  24-JUL-50 14:40:00   \n",
       "\n",
       "   INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                0                  0              0                0   \n",
       "1                0                  0              0                0   \n",
       "2                2                  0              0                0   \n",
       "3                0                  0              0                0   \n",
       "4                0                  0              0                0   \n",
       "\n",
       "  DAMAGE_PROPERTY DAMAGE_CROPS SOURCE  DAMAGE_PROPERTY_CONVERTED  \n",
       "0            250K            0   None                   250000.0  \n",
       "1             25K            0   None                    25000.0  \n",
       "2             25K            0   None                    25000.0  \n",
       "3            2.5K            0   None                     2500.0  \n",
       "4            2.5K            0   None                     2500.0  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# limit number of rows\n",
    "storm_events.limit(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38036504",
   "metadata": {},
   "source": [
    "# Methods\n",
    "## Select"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "fb578b8c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>YEAR</th>\n",
       "      <th>STATE</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1950</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1950</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1950</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1950</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1950</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>1956</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>1956</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>1956</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>1956</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>1956</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>Hail</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      YEAR         STATE         EVENT_TYPE  DAMAGE_PROPERTY_CONVERTED\n",
       "0     1950      OKLAHOMA            Tornado                   250000.0\n",
       "1     1950         TEXAS            Tornado                    25000.0\n",
       "2     1950  PENNSYLVANIA            Tornado                    25000.0\n",
       "3     1950  PENNSYLVANIA            Tornado                     2500.0\n",
       "4     1950  PENNSYLVANIA            Tornado                     2500.0\n",
       "...    ...           ...                ...                        ...\n",
       "9995  1956      OKLAHOMA  Thunderstorm Wind                        0.0\n",
       "9996  1956      OKLAHOMA  Thunderstorm Wind                        0.0\n",
       "9997  1956      OKLAHOMA  Thunderstorm Wind                        0.0\n",
       "9998  1956      OKLAHOMA            Tornado                   250000.0\n",
       "9999  1956      OKLAHOMA               Hail                        0.0\n",
       "\n",
       "[10000 rows x 4 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# select columns\n",
    "storm_events.select(\n",
    "    \"YEAR\",\n",
    "    \"STATE\",\n",
    "    \"EVENT_TYPE\",\n",
    "    \"DAMAGE_PROPERTY_CONVERTED\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9baf3813",
   "metadata": {},
   "source": [
    "## Drop\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "68e9787d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>1950</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>1950</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>1950</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>1950</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>1950</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10093724</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>1956</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10093725</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>1956</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10093726</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>1956</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>10093727</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>1956</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10093769</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>1956</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID         STATE  YEAR  DAMAGE_PROPERTY_CONVERTED\n",
       "0     10096222      OKLAHOMA  1950                   250000.0\n",
       "1     10120412         TEXAS  1950                    25000.0\n",
       "2     10104927  PENNSYLVANIA  1950                    25000.0\n",
       "3     10104928  PENNSYLVANIA  1950                     2500.0\n",
       "4     10104929  PENNSYLVANIA  1950                     2500.0\n",
       "...        ...           ...   ...                        ...\n",
       "9995  10093724      OKLAHOMA  1956                        0.0\n",
       "9996  10093725      OKLAHOMA  1956                        0.0\n",
       "9997  10093726      OKLAHOMA  1956                        0.0\n",
       "9998  10093727      OKLAHOMA  1956                   250000.0\n",
       "9999  10093769      OKLAHOMA  1956                        0.0\n",
       "\n",
       "[10000 rows x 4 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# columns to drop\n",
    "cols = [\n",
    "    'STATE_FIPS',\n",
    "    'MONTH_NAME',\n",
    "    'EVENT_TYPE',\n",
    "    'CZ_TYPE',\n",
    "    'CZ_FIPS',\n",
    "    'CZ_NAME',\n",
    "    'WFO',\n",
    "    'BEGIN_DATE_TIME',\n",
    "    'END_DATE_TIME',\n",
    "    'INJURIES_DIRECT',\n",
    "    'INJURIES_INDIRECT',\n",
    "    'DEATHS_DIRECT',\n",
    "    'DEATHS_INDIRECT',\n",
    "    'DAMAGE_PROPERTY',\n",
    "    'DAMAGE_CROPS',\n",
    "    'SOURCE',\n",
    "]\n",
    "\n",
    "storm_events.drop(cols)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d090f90b",
   "metadata": {},
   "source": [
    "## Filter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "8dad8ab1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>BEGIN_DATE_TIME</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10075659</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1952</td>\n",
       "      <td>May</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>31</td>\n",
       "      <td>ESSEX</td>\n",
       "      <td>None</td>\n",
       "      <td>06-MAY-52 13:00:00</td>\n",
       "      <td>06-MAY-52 13:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10075660</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1954</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>123</td>\n",
       "      <td>YATES</td>\n",
       "      <td>None</td>\n",
       "      <td>14-JUL-54 16:30:00</td>\n",
       "      <td>14-JUL-54 16:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5M</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10075661</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1954</td>\n",
       "      <td>September</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>89</td>\n",
       "      <td>ST. LAWRENCE</td>\n",
       "      <td>None</td>\n",
       "      <td>19-SEP-54 17:30:00</td>\n",
       "      <td>19-SEP-54 17:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10075701</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1958</td>\n",
       "      <td>June</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>29</td>\n",
       "      <td>ERIE</td>\n",
       "      <td>None</td>\n",
       "      <td>25-JUN-58 18:30:00</td>\n",
       "      <td>25-JUN-58 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10075702</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1958</td>\n",
       "      <td>June</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>29</td>\n",
       "      <td>ERIE</td>\n",
       "      <td>None</td>\n",
       "      <td>25-JUN-58 18:30:00</td>\n",
       "      <td>25-JUN-58 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>10075703</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1958</td>\n",
       "      <td>June</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>55</td>\n",
       "      <td>MONROE</td>\n",
       "      <td>None</td>\n",
       "      <td>25-JUN-58 19:00:00</td>\n",
       "      <td>25-JUN-58 19:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>10075704</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1958</td>\n",
       "      <td>June</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>51</td>\n",
       "      <td>LIVINGSTON</td>\n",
       "      <td>None</td>\n",
       "      <td>25-JUN-58 19:00:00</td>\n",
       "      <td>25-JUN-58 19:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>10075697</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1958</td>\n",
       "      <td>May</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>17</td>\n",
       "      <td>CHENANGO</td>\n",
       "      <td>None</td>\n",
       "      <td>18-MAY-58 14:00:00</td>\n",
       "      <td>18-MAY-58 14:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>10075698</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1958</td>\n",
       "      <td>May</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>11</td>\n",
       "      <td>CAYUGA</td>\n",
       "      <td>None</td>\n",
       "      <td>18-MAY-58 14:15:00</td>\n",
       "      <td>18-MAY-58 14:15:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10075699</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>1958</td>\n",
       "      <td>May</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>117</td>\n",
       "      <td>WAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>18-MAY-58 14:30:00</td>\n",
       "      <td>18-MAY-58 14:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   EVENT_ID     STATE  STATE_FIPS  YEAR MONTH_NAME         EVENT_TYPE CZ_TYPE  \\\n",
       "0  10075659  NEW YORK          36  1952        May            Tornado       C   \n",
       "1  10075660  NEW YORK          36  1954       July            Tornado       C   \n",
       "2  10075661  NEW YORK          36  1954  September            Tornado       C   \n",
       "3  10075701  NEW YORK          36  1958       June  Thunderstorm Wind       C   \n",
       "4  10075702  NEW YORK          36  1958       June  Thunderstorm Wind       C   \n",
       "5  10075703  NEW YORK          36  1958       June  Thunderstorm Wind       C   \n",
       "6  10075704  NEW YORK          36  1958       June  Thunderstorm Wind       C   \n",
       "7  10075697  NEW YORK          36  1958        May  Thunderstorm Wind       C   \n",
       "8  10075698  NEW YORK          36  1958        May  Thunderstorm Wind       C   \n",
       "9  10075699  NEW YORK          36  1958        May               Hail       C   \n",
       "\n",
       "   CZ_FIPS       CZ_NAME   WFO     BEGIN_DATE_TIME       END_DATE_TIME  \\\n",
       "0       31         ESSEX  None  06-MAY-52 13:00:00  06-MAY-52 13:00:00   \n",
       "1      123         YATES  None  14-JUL-54 16:30:00  14-JUL-54 16:30:00   \n",
       "2       89  ST. LAWRENCE  None  19-SEP-54 17:30:00  19-SEP-54 17:30:00   \n",
       "3       29          ERIE  None  25-JUN-58 18:30:00  25-JUN-58 18:30:00   \n",
       "4       29          ERIE  None  25-JUN-58 18:30:00  25-JUN-58 18:30:00   \n",
       "5       55        MONROE  None  25-JUN-58 19:00:00  25-JUN-58 19:00:00   \n",
       "6       51    LIVINGSTON  None  25-JUN-58 19:00:00  25-JUN-58 19:00:00   \n",
       "7       17      CHENANGO  None  18-MAY-58 14:00:00  18-MAY-58 14:00:00   \n",
       "8       11        CAYUGA  None  18-MAY-58 14:15:00  18-MAY-58 14:15:00   \n",
       "9      117         WAYNE  None  18-MAY-58 14:30:00  18-MAY-58 14:30:00   \n",
       "\n",
       "   INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                0                  0              0                0   \n",
       "1                0                  0              0                0   \n",
       "2                0                  0              0                0   \n",
       "3                0                  0              0                0   \n",
       "4                0                  0              0                0   \n",
       "5                0                  0              0                0   \n",
       "6                0                  0              0                0   \n",
       "7                0                  0              0                0   \n",
       "8                0                  0              0                0   \n",
       "9                0                  0              0                0   \n",
       "\n",
       "  DAMAGE_PROPERTY DAMAGE_CROPS SOURCE  DAMAGE_PROPERTY_CONVERTED  \n",
       "0             25K            0   None                    25000.0  \n",
       "1            2.5M            0   None                  2500000.0  \n",
       "2             25K            0   None                    25000.0  \n",
       "3               0            0   None                        0.0  \n",
       "4               0            0   None                        0.0  \n",
       "5               0            0   None                        0.0  \n",
       "6               0            0   None                        0.0  \n",
       "7               0            0   None                        0.0  \n",
       "8               0            0   None                        0.0  \n",
       "9               0            0   None                        0.0  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# filter events only contain New York\n",
    "(storm_events\n",
    " .filter(storm_events.STATE == \"NEW YORK\")\n",
    " .limit(10)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "7be3f84c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>BEGIN_DATE_TIME</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5516309</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2006</td>\n",
       "      <td>June</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>107</td>\n",
       "      <td>TIOGA</td>\n",
       "      <td>BGM</td>\n",
       "      <td>27-JUN-06 14:10:00</td>\n",
       "      <td>28-JUN-06 14:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>100M</td>\n",
       "      <td>None</td>\n",
       "      <td>EMERGENCY MANAGER</td>\n",
       "      <td>100000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5519036</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2006</td>\n",
       "      <td>June</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>105</td>\n",
       "      <td>SULLIVAN</td>\n",
       "      <td>BGM</td>\n",
       "      <td>27-JUN-06 04:00:00</td>\n",
       "      <td>28-JUN-06 14:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>100M</td>\n",
       "      <td>None</td>\n",
       "      <td>EMERGENCY MANAGER</td>\n",
       "      <td>100000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5519040</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2006</td>\n",
       "      <td>June</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>7</td>\n",
       "      <td>BROOME</td>\n",
       "      <td>BGM</td>\n",
       "      <td>27-JUN-06 16:00:00</td>\n",
       "      <td>28-JUN-06 14:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>200M</td>\n",
       "      <td>None</td>\n",
       "      <td>EMERGENCY MANAGER</td>\n",
       "      <td>200000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5519034</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2006</td>\n",
       "      <td>June</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>25</td>\n",
       "      <td>DELAWARE</td>\n",
       "      <td>BGM</td>\n",
       "      <td>27-JUN-06 08:10:00</td>\n",
       "      <td>28-JUN-06 13:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>250M</td>\n",
       "      <td>None</td>\n",
       "      <td>LAW ENFORCEMENT</td>\n",
       "      <td>250000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1266</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2006</td>\n",
       "      <td>October</td>\n",
       "      <td>Lake-Effect Snow</td>\n",
       "      <td>Z</td>\n",
       "      <td>10</td>\n",
       "      <td>NORTHERN ERIE</td>\n",
       "      <td>BUF</td>\n",
       "      <td>12-OCT-06 14:30:00</td>\n",
       "      <td>13-OCT-06 07:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>130.00M</td>\n",
       "      <td>0.00K</td>\n",
       "      <td>Law Enforcement</td>\n",
       "      <td>130000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>206970</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2009</td>\n",
       "      <td>December</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>Z</td>\n",
       "      <td>7</td>\n",
       "      <td>JEFFERSON</td>\n",
       "      <td>BUF</td>\n",
       "      <td>09-DEC-09 13:00:00</td>\n",
       "      <td>10-DEC-09 09:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>100.00M</td>\n",
       "      <td>0.00K</td>\n",
       "      <td>Utility Company</td>\n",
       "      <td>100000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>348760</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2011</td>\n",
       "      <td>September</td>\n",
       "      <td>Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>107</td>\n",
       "      <td>TIOGA</td>\n",
       "      <td>BGM</td>\n",
       "      <td>07-SEP-11 16:54:00</td>\n",
       "      <td>11-SEP-11 03:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>309.00M</td>\n",
       "      <td>0.00K</td>\n",
       "      <td>River/Stream Gage</td>\n",
       "      <td>309000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>348756</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2011</td>\n",
       "      <td>September</td>\n",
       "      <td>Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>7</td>\n",
       "      <td>BROOME</td>\n",
       "      <td>BGM</td>\n",
       "      <td>07-SEP-11 14:23:00</td>\n",
       "      <td>11-SEP-11 03:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>170.00M</td>\n",
       "      <td>0.00K</td>\n",
       "      <td>River/Stream Gage</td>\n",
       "      <td>170000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>349312</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2011</td>\n",
       "      <td>September</td>\n",
       "      <td>Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>7</td>\n",
       "      <td>BROOME</td>\n",
       "      <td>BGM</td>\n",
       "      <td>07-SEP-11 12:22:00</td>\n",
       "      <td>11-SEP-11 23:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>160.00M</td>\n",
       "      <td>0.00K</td>\n",
       "      <td>River/Stream Gage</td>\n",
       "      <td>160000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>995048</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>36</td>\n",
       "      <td>2021</td>\n",
       "      <td>September</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>WESTCHESTER</td>\n",
       "      <td>OKX</td>\n",
       "      <td>01-SEP-21 21:30:00</td>\n",
       "      <td>01-SEP-21 22:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>92.60M</td>\n",
       "      <td>0.00K</td>\n",
       "      <td>Emergency Manager</td>\n",
       "      <td>92600000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   EVENT_ID     STATE  STATE_FIPS  YEAR MONTH_NAME        EVENT_TYPE CZ_TYPE  \\\n",
       "0   5516309  NEW YORK          36  2006       June       Flash Flood       C   \n",
       "1   5519036  NEW YORK          36  2006       June       Flash Flood       C   \n",
       "2   5519040  NEW YORK          36  2006       June       Flash Flood       C   \n",
       "3   5519034  NEW YORK          36  2006       June       Flash Flood       C   \n",
       "4      1266  NEW YORK          36  2006    October  Lake-Effect Snow       Z   \n",
       "5    206970  NEW YORK          36  2009   December         High Wind       Z   \n",
       "6    348760  NEW YORK          36  2011  September             Flood       C   \n",
       "7    348756  NEW YORK          36  2011  September             Flood       C   \n",
       "8    349312  NEW YORK          36  2011  September             Flood       C   \n",
       "9    995048  NEW YORK          36  2021  September       Flash Flood       C   \n",
       "\n",
       "   CZ_FIPS        CZ_NAME  WFO     BEGIN_DATE_TIME       END_DATE_TIME  \\\n",
       "0      107          TIOGA  BGM  27-JUN-06 14:10:00  28-JUN-06 14:00:00   \n",
       "1      105       SULLIVAN  BGM  27-JUN-06 04:00:00  28-JUN-06 14:00:00   \n",
       "2        7         BROOME  BGM  27-JUN-06 16:00:00  28-JUN-06 14:00:00   \n",
       "3       25       DELAWARE  BGM  27-JUN-06 08:10:00  28-JUN-06 13:00:00   \n",
       "4       10  NORTHERN ERIE  BUF  12-OCT-06 14:30:00  13-OCT-06 07:00:00   \n",
       "5        7      JEFFERSON  BUF  09-DEC-09 13:00:00  10-DEC-09 09:00:00   \n",
       "6      107          TIOGA  BGM  07-SEP-11 16:54:00  11-SEP-11 03:00:00   \n",
       "7        7         BROOME  BGM  07-SEP-11 14:23:00  11-SEP-11 03:45:00   \n",
       "8        7         BROOME  BGM  07-SEP-11 12:22:00  11-SEP-11 23:40:00   \n",
       "9      119    WESTCHESTER  OKX  01-SEP-21 21:30:00  01-SEP-21 22:00:00   \n",
       "\n",
       "   INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                0                  0              0                0   \n",
       "1                0                  0              1                0   \n",
       "2                0                  0              0                0   \n",
       "3                0                  0              2                0   \n",
       "4                0                  0              1                0   \n",
       "5                0                  0              0                0   \n",
       "6                0                  0              0                0   \n",
       "7                0                  0              0                0   \n",
       "8                0                  0              0                0   \n",
       "9                0                  0              0                0   \n",
       "\n",
       "  DAMAGE_PROPERTY DAMAGE_CROPS             SOURCE  DAMAGE_PROPERTY_CONVERTED  \n",
       "0            100M         None  EMERGENCY MANAGER                100000000.0  \n",
       "1            100M         None  EMERGENCY MANAGER                100000000.0  \n",
       "2            200M         None  EMERGENCY MANAGER                200000000.0  \n",
       "3            250M         None    LAW ENFORCEMENT                250000000.0  \n",
       "4         130.00M        0.00K    Law Enforcement                130000000.0  \n",
       "5         100.00M        0.00K    Utility Company                100000000.0  \n",
       "6         309.00M        0.00K  River/Stream Gage                309000000.0  \n",
       "7         170.00M        0.00K  River/Stream Gage                170000000.0  \n",
       "8         160.00M        0.00K  River/Stream Gage                160000000.0  \n",
       "9          92.60M        0.00K  Emergency Manager                 92600000.0  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "storm_events.filter(\n",
    "    (storm_events.STATE == \"NEW YORK\")\n",
    "    & (storm_events.DAMAGE_PROPERTY_CONVERTED > 90_000_000)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "0d9dd406",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>STATE</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>2006</td>\n",
       "      <td>100M</td>\n",
       "      <td>100000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>2006</td>\n",
       "      <td>100M</td>\n",
       "      <td>100000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>2006</td>\n",
       "      <td>200M</td>\n",
       "      <td>200000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>2006</td>\n",
       "      <td>250M</td>\n",
       "      <td>250000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Lake-Effect Snow</td>\n",
       "      <td>2006</td>\n",
       "      <td>130.00M</td>\n",
       "      <td>130000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>2009</td>\n",
       "      <td>100.00M</td>\n",
       "      <td>100000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flood</td>\n",
       "      <td>2011</td>\n",
       "      <td>309.00M</td>\n",
       "      <td>309000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flood</td>\n",
       "      <td>2011</td>\n",
       "      <td>170.00M</td>\n",
       "      <td>170000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flood</td>\n",
       "      <td>2011</td>\n",
       "      <td>160.00M</td>\n",
       "      <td>160000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>2021</td>\n",
       "      <td>92.60M</td>\n",
       "      <td>92600000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      STATE        EVENT_TYPE  YEAR DAMAGE_PROPERTY  DAMAGE_PROPERTY_CONVERTED\n",
       "0  NEW YORK       Flash Flood  2006            100M                100000000.0\n",
       "1  NEW YORK       Flash Flood  2006            100M                100000000.0\n",
       "2  NEW YORK       Flash Flood  2006            200M                200000000.0\n",
       "3  NEW YORK       Flash Flood  2006            250M                250000000.0\n",
       "4  NEW YORK  Lake-Effect Snow  2006         130.00M                130000000.0\n",
       "5  NEW YORK         High Wind  2009         100.00M                100000000.0\n",
       "6  NEW YORK             Flood  2011         309.00M                309000000.0\n",
       "7  NEW YORK             Flood  2011         170.00M                170000000.0\n",
       "8  NEW YORK             Flood  2011         160.00M                160000000.0\n",
       "9  NEW YORK       Flash Flood  2021          92.60M                 92600000.0"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# chaining with select and filter methods\n",
    "new_york = (\n",
    "    storm_events\n",
    "    .select([\"STATE\", \"EVENT_TYPE\", \"YEAR\", \"DAMAGE_PROPERTY\", \"DAMAGE_PROPERTY_CONVERTED\"])\n",
    "    .filter(\n",
    "        (storm_events[\"STATE\"] == \"NEW YORK\")\n",
    "        & (storm_events.DAMAGE_PROPERTY_CONVERTED > 90_000_000))\n",
    ")\n",
    "    \n",
    "new_york"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2cc748a",
   "metadata": {},
   "source": [
    "## Mutate\n",
    "Adding columns is performed using the `mutate` method. Adding other ways to mutate columns as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "b2d5a2db",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_K</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>250.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>2.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>2.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10093724</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>103</td>\n",
       "      <td>NOBLE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10093725</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>65</td>\n",
       "      <td>JACKSON</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10093726</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>83</td>\n",
       "      <td>LOGAN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:02:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>10093727</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>250.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10093769</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>May</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>13-MAY-56 19:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME         EVENT_TYPE  \\\n",
       "0     10096222      OKLAHOMA          40  1950      April            Tornado   \n",
       "1     10120412         TEXAS          48  1950      April            Tornado   \n",
       "2     10104927  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "3     10104928  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "4     10104929  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "...        ...           ...         ...   ...        ...                ...   \n",
       "9995  10093724      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9996  10093725      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9997  10093726      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9998  10093727      OKLAHOMA          40  1956      April            Tornado   \n",
       "9999  10093769      OKLAHOMA          40  1956        May               Hail   \n",
       "\n",
       "     CZ_TYPE  CZ_FIPS   CZ_NAME   WFO  ...       END_DATE_TIME  \\\n",
       "0          C      149   WASHITA  None  ...  28-APR-50 14:45:00   \n",
       "1          C       93  COMANCHE  None  ...  29-APR-50 15:30:00   \n",
       "2          C       77    LEHIGH  None  ...  05-JUL-50 18:00:00   \n",
       "3          C       43   DAUPHIN  None  ...  05-JUL-50 18:30:00   \n",
       "4          C       39  CRAWFORD  None  ...  24-JUL-50 14:40:00   \n",
       "...      ...      ...       ...   ...  ...                 ...   \n",
       "9995       C      103     NOBLE  None  ...  08-APR-56 23:00:00   \n",
       "9996       C       65   JACKSON  None  ...  08-APR-56 23:00:00   \n",
       "9997       C       83     LOGAN  None  ...  08-APR-56 23:02:00   \n",
       "9998       C      119     PAYNE  None  ...  08-APR-56 23:30:00   \n",
       "9999       C      119     PAYNE  None  ...  13-MAY-56 19:00:00   \n",
       "\n",
       "     INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                  0                  0              0                0   \n",
       "1                  0                  0              0                0   \n",
       "2                  2                  0              0                0   \n",
       "3                  0                  0              0                0   \n",
       "4                  0                  0              0                0   \n",
       "...              ...                ...            ...              ...   \n",
       "9995               0                  0              0                0   \n",
       "9996               0                  0              0                0   \n",
       "9997               0                  0              0                0   \n",
       "9998               0                  0              0                0   \n",
       "9999               0                  0              0                0   \n",
       "\n",
       "      DAMAGE_PROPERTY DAMAGE_CROPS SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0                250K            0   None                  250000.0   \n",
       "1                 25K            0   None                   25000.0   \n",
       "2                 25K            0   None                   25000.0   \n",
       "3                2.5K            0   None                    2500.0   \n",
       "4                2.5K            0   None                    2500.0   \n",
       "...               ...          ...    ...                       ...   \n",
       "9995                0            0   None                       0.0   \n",
       "9996                0            0   None                       0.0   \n",
       "9997                0            0   None                       0.0   \n",
       "9998             250K            0   None                  250000.0   \n",
       "9999                0            0   None                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_K  \n",
       "0                           250.0  \n",
       "1                            25.0  \n",
       "2                            25.0  \n",
       "3                             2.5  \n",
       "4                             2.5  \n",
       "...                           ...  \n",
       "9995                          0.0  \n",
       "9996                          0.0  \n",
       "9997                          0.0  \n",
       "9998                        250.0  \n",
       "9999                          0.0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add column\n",
    "conversion = storm_events.DAMAGE_PROPERTY_CONVERTED / 1000\n",
    "\n",
    "storm_events.mutate(DAMAGE_PROPERTY_CONVERTED_K = conversion)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "be04686e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10093724</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>103</td>\n",
       "      <td>NOBLE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10093725</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>65</td>\n",
       "      <td>JACKSON</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10093726</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>83</td>\n",
       "      <td>LOGAN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:02:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>10093727</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10093769</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>May</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>13-MAY-56 19:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME         EVENT_TYPE  \\\n",
       "0     10096222      OKLAHOMA          40  1950      April            Tornado   \n",
       "1     10120412         TEXAS          48  1950      April            Tornado   \n",
       "2     10104927  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "3     10104928  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "4     10104929  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "...        ...           ...         ...   ...        ...                ...   \n",
       "9995  10093724      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9996  10093725      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9997  10093726      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9998  10093727      OKLAHOMA          40  1956      April            Tornado   \n",
       "9999  10093769      OKLAHOMA          40  1956        May               Hail   \n",
       "\n",
       "     CZ_TYPE  CZ_FIPS   CZ_NAME   WFO  ...       END_DATE_TIME  \\\n",
       "0          C      149   WASHITA  None  ...  28-APR-50 14:45:00   \n",
       "1          C       93  COMANCHE  None  ...  29-APR-50 15:30:00   \n",
       "2          C       77    LEHIGH  None  ...  05-JUL-50 18:00:00   \n",
       "3          C       43   DAUPHIN  None  ...  05-JUL-50 18:30:00   \n",
       "4          C       39  CRAWFORD  None  ...  24-JUL-50 14:40:00   \n",
       "...      ...      ...       ...   ...  ...                 ...   \n",
       "9995       C      103     NOBLE  None  ...  08-APR-56 23:00:00   \n",
       "9996       C       65   JACKSON  None  ...  08-APR-56 23:00:00   \n",
       "9997       C       83     LOGAN  None  ...  08-APR-56 23:02:00   \n",
       "9998       C      119     PAYNE  None  ...  08-APR-56 23:30:00   \n",
       "9999       C      119     PAYNE  None  ...  13-MAY-56 19:00:00   \n",
       "\n",
       "     INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                  0                  0              0                0   \n",
       "1                  0                  0              0                0   \n",
       "2                  2                  0              0                0   \n",
       "3                  0                  0              0                0   \n",
       "4                  0                  0              0                0   \n",
       "...              ...                ...            ...              ...   \n",
       "9995               0                  0              0                0   \n",
       "9996               0                  0              0                0   \n",
       "9997               0                  0              0                0   \n",
       "9998               0                  0              0                0   \n",
       "9999               0                  0              0                0   \n",
       "\n",
       "      DAMAGE_PROPERTY DAMAGE_CROPS SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0                250K            0   None                  250000.0   \n",
       "1                 25K            0   None                   25000.0   \n",
       "2                 25K            0   None                   25000.0   \n",
       "3                2.5K            0   None                    2500.0   \n",
       "4                2.5K            0   None                    2500.0   \n",
       "...               ...          ...    ...                       ...   \n",
       "9995                0            0   None                       0.0   \n",
       "9996                0            0   None                       0.0   \n",
       "9997                0            0   None                       0.0   \n",
       "9998             250K            0   None                  250000.0   \n",
       "9999                0            0   None                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_DOLLARS  \n",
       "0                               $250000  \n",
       "1                                $25000  \n",
       "2                                $25000  \n",
       "3                                 $2500  \n",
       "4                                 $2500  \n",
       "...                                 ...  \n",
       "9995                                 $0  \n",
       "9996                                 $0  \n",
       "9997                                 $0  \n",
       "9998                            $250000  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add a \"$\" to the beginning of the damage field\n",
    "cast_col = storm_events.DAMAGE_PROPERTY_CONVERTED.cast(\"int\").cast(\"str\")\n",
    "dollars = (\"$\" + cast_col).name(\"DAMAGE_PROPERTY_CONVERTED_DOLLARS\")\n",
    "\n",
    "# pass name expression \"dollars\" to mutate fuction to create new column\n",
    "storm_events = storm_events.mutate(dollars)\n",
    "\n",
    "storm_events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "f74c080a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>aaa</th>\n",
       "      <th>bbb</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>25000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>2500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>250000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>Hail</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                    aaa       bbb\n",
       "0               Tornado  250000.0\n",
       "1               Tornado   25000.0\n",
       "2               Tornado   25000.0\n",
       "3               Tornado    2500.0\n",
       "4               Tornado    2500.0\n",
       "...                 ...       ...\n",
       "9995  Thunderstorm Wind       0.0\n",
       "9996  Thunderstorm Wind       0.0\n",
       "9997  Thunderstorm Wind       0.0\n",
       "9998            Tornado  250000.0\n",
       "9999               Hail       0.0\n",
       "\n",
       "[10000 rows x 2 columns]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# rename columns, example below\n",
    "temp_table = storm_events.select(\"EVENT_TYPE\", \"DAMAGE_PROPERTY_CONVERTED\")\n",
    "\n",
    "renamed_table = temp_table.relabel(dict(\n",
    "    EVENT_TYPE=\"aaa\",\n",
    "    DAMAGE_PROPERTY_CONVERTED=\"bbb\"\n",
    "))\n",
    "\n",
    "renamed_table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "93661169",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.Schema {\n",
       "  aaa  string\n",
       "  bbb  float64\n",
       "}"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sanity check\n",
    "renamed_table.schema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "17d33696",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event_id</th>\n",
       "      <th>state</th>\n",
       "      <th>state_fips</th>\n",
       "      <th>year</th>\n",
       "      <th>month_name</th>\n",
       "      <th>event_type</th>\n",
       "      <th>cz_type</th>\n",
       "      <th>cz_fips</th>\n",
       "      <th>cz_name</th>\n",
       "      <th>wfo</th>\n",
       "      <th>...</th>\n",
       "      <th>end_date_time</th>\n",
       "      <th>injuries_direct</th>\n",
       "      <th>injuries_indirect</th>\n",
       "      <th>deaths_direct</th>\n",
       "      <th>deaths_indirect</th>\n",
       "      <th>damage_property</th>\n",
       "      <th>damage_crops</th>\n",
       "      <th>source</th>\n",
       "      <th>damage_property_converted</th>\n",
       "      <th>damage_property_converted_dollars</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10093724</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>103</td>\n",
       "      <td>NOBLE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10093725</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>65</td>\n",
       "      <td>JACKSON</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10093726</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>83</td>\n",
       "      <td>LOGAN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:02:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>10093727</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10093769</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>May</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>13-MAY-56 19:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      event_id         state  state_fips  year month_name         event_type  \\\n",
       "0     10096222      OKLAHOMA          40  1950      April            Tornado   \n",
       "1     10120412         TEXAS          48  1950      April            Tornado   \n",
       "2     10104927  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "3     10104928  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "4     10104929  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "...        ...           ...         ...   ...        ...                ...   \n",
       "9995  10093724      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9996  10093725      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9997  10093726      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9998  10093727      OKLAHOMA          40  1956      April            Tornado   \n",
       "9999  10093769      OKLAHOMA          40  1956        May               Hail   \n",
       "\n",
       "     cz_type  cz_fips   cz_name   wfo  ...       end_date_time  \\\n",
       "0          C      149   WASHITA  None  ...  28-APR-50 14:45:00   \n",
       "1          C       93  COMANCHE  None  ...  29-APR-50 15:30:00   \n",
       "2          C       77    LEHIGH  None  ...  05-JUL-50 18:00:00   \n",
       "3          C       43   DAUPHIN  None  ...  05-JUL-50 18:30:00   \n",
       "4          C       39  CRAWFORD  None  ...  24-JUL-50 14:40:00   \n",
       "...      ...      ...       ...   ...  ...                 ...   \n",
       "9995       C      103     NOBLE  None  ...  08-APR-56 23:00:00   \n",
       "9996       C       65   JACKSON  None  ...  08-APR-56 23:00:00   \n",
       "9997       C       83     LOGAN  None  ...  08-APR-56 23:02:00   \n",
       "9998       C      119     PAYNE  None  ...  08-APR-56 23:30:00   \n",
       "9999       C      119     PAYNE  None  ...  13-MAY-56 19:00:00   \n",
       "\n",
       "     injuries_direct  injuries_indirect  deaths_direct  deaths_indirect  \\\n",
       "0                  0                  0              0                0   \n",
       "1                  0                  0              0                0   \n",
       "2                  2                  0              0                0   \n",
       "3                  0                  0              0                0   \n",
       "4                  0                  0              0                0   \n",
       "...              ...                ...            ...              ...   \n",
       "9995               0                  0              0                0   \n",
       "9996               0                  0              0                0   \n",
       "9997               0                  0              0                0   \n",
       "9998               0                  0              0                0   \n",
       "9999               0                  0              0                0   \n",
       "\n",
       "      damage_property damage_crops source damage_property_converted  \\\n",
       "0                250K            0   None                  250000.0   \n",
       "1                 25K            0   None                   25000.0   \n",
       "2                 25K            0   None                   25000.0   \n",
       "3                2.5K            0   None                    2500.0   \n",
       "4                2.5K            0   None                    2500.0   \n",
       "...               ...          ...    ...                       ...   \n",
       "9995                0            0   None                       0.0   \n",
       "9996                0            0   None                       0.0   \n",
       "9997                0            0   None                       0.0   \n",
       "9998             250K            0   None                  250000.0   \n",
       "9999                0            0   None                       0.0   \n",
       "\n",
       "      damage_property_converted_dollars  \n",
       "0                               $250000  \n",
       "1                                $25000  \n",
       "2                                $25000  \n",
       "3                                 $2500  \n",
       "4                                 $2500  \n",
       "...                                 ...  \n",
       "9995                                 $0  \n",
       "9996                                 $0  \n",
       "9997                                 $0  \n",
       "9998                            $250000  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert columns to lowercase, source: stackoverflow from Ian Cook\n",
    "table_cols_lower = storm_events.relabel(dict(zip(\n",
    "    storm_events.columns,\n",
    "    [x.lower() for x in storm_events.columns]\n",
    ")))\n",
    "\n",
    "table_cols_lower"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b350bfe1",
   "metadata": {},
   "source": [
    "## Order By"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "d8bb79ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>YEAR</th>\n",
       "      <th>STATE</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2007</td>\n",
       "      <td>ALABAMA</td>\n",
       "      <td>Hail</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2007</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>Heavy Rain</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2007</td>\n",
       "      <td>KENTUCKY</td>\n",
       "      <td>Drought</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2007</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>Coastal Flood</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2007</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>Coastal Flood</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2007</td>\n",
       "      <td>MONTANA</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2007</td>\n",
       "      <td>MONTANA</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2007</td>\n",
       "      <td>MONTANA</td>\n",
       "      <td>Hail</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2007</td>\n",
       "      <td>MONTANA</td>\n",
       "      <td>Hail</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2007</td>\n",
       "      <td>COLORADO</td>\n",
       "      <td>Hail</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   YEAR      STATE         EVENT_TYPE  DAMAGE_PROPERTY_CONVERTED\n",
       "0  2011  LOUISIANA  Thunderstorm Wind                        0.0\n",
       "1  2011   NEBRASKA            Tornado                        0.0\n",
       "2  2011   NEBRASKA               Hail                        0.0\n",
       "3  2011   NEBRASKA               Hail                        0.0\n",
       "4  2011   NEBRASKA               Hail                        0.0\n",
       "5  2011   NEBRASKA               Hail                        0.0\n",
       "6  2011   NEBRASKA               Hail                        0.0\n",
       "7  2011    WYOMING  Thunderstorm Wind                        0.0\n",
       "8  2011   NEBRASKA  Thunderstorm Wind                        0.0\n",
       "9  2011   NEBRASKA  Thunderstorm Wind                        0.0"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# order by DAMAGE_PROPERTY_CONVERTED in ascending order\n",
    "(storm_events\n",
    " .sort_by(storm_events.DAMAGE_PROPERTY_CONVERTED)\n",
    " .select(\n",
    "    \"YEAR\",\n",
    "    \"STATE\",\n",
    "    \"EVENT_TYPE\",\n",
    "    \"DAMAGE_PROPERTY_CONVERTED\")\n",
    " .limit(10)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "691886cc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>YEAR</th>\n",
       "      <th>STATE</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020</td>\n",
       "      <td>WASHINGTON</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>950000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013</td>\n",
       "      <td>ILLINOIS</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>910000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2012</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>Hail</td>\n",
       "      <td>900000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane</td>\n",
       "      <td>900000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2010</td>\n",
       "      <td>ARIZONA</td>\n",
       "      <td>Hail</td>\n",
       "      <td>900000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2004</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>881000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2005</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>850000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2005</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>850000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2020</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane</td>\n",
       "      <td>800000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2017</td>\n",
       "      <td>PUERTO RICO</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>750000000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   YEAR        STATE           EVENT_TYPE  DAMAGE_PROPERTY_CONVERTED\n",
       "0  2020   WASHINGTON            High Wind                950000000.0\n",
       "1  2013     ILLINOIS              Tornado                910000000.0\n",
       "2  2012        TEXAS                 Hail                900000000.0\n",
       "3  2010      ARIZONA                 Hail                900000000.0\n",
       "4  2020    LOUISIANA            Hurricane                900000000.0\n",
       "5  2004      FLORIDA            High Wind                881000000.0\n",
       "6  2005    LOUISIANA  Hurricane (Typhoon)                850000000.0\n",
       "7  2005    LOUISIANA  Hurricane (Typhoon)                850000000.0\n",
       "8  2020    LOUISIANA            Hurricane                800000000.0\n",
       "9  2017  PUERTO RICO          Flash Flood                750000000.0"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# order by DAMAGE_PROPERTY_CONVERTED in descending order\n",
    "(storm_events\n",
    " .sort_by(ibis.desc(\"DAMAGE_PROPERTY_CONVERTED\"))\n",
    " .select(\n",
    "    \"YEAR\",\n",
    "    \"STATE\",\n",
    "    \"EVENT_TYPE\",\n",
    "    \"DAMAGE_PROPERTY_CONVERTED\")\n",
    " .limit(10)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "dd551dda",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>YEAR</th>\n",
       "      <th>STATE</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020</td>\n",
       "      <td>WASHINGTON</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>950000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013</td>\n",
       "      <td>ILLINOIS</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>910000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane</td>\n",
       "      <td>900000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2012</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>Hail</td>\n",
       "      <td>900000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2010</td>\n",
       "      <td>ARIZONA</td>\n",
       "      <td>Hail</td>\n",
       "      <td>900000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2004</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>881000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2005</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>850000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2005</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>850000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2020</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Hurricane</td>\n",
       "      <td>800000000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2021</td>\n",
       "      <td>LOUISIANA</td>\n",
       "      <td>Storm Surge/Tide</td>\n",
       "      <td>750000000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   YEAR       STATE           EVENT_TYPE  DAMAGE_PROPERTY_CONVERTED\n",
       "0  2020  WASHINGTON            High Wind                950000000.0\n",
       "1  2013    ILLINOIS              Tornado                910000000.0\n",
       "2  2020   LOUISIANA            Hurricane                900000000.0\n",
       "3  2012       TEXAS                 Hail                900000000.0\n",
       "4  2010     ARIZONA                 Hail                900000000.0\n",
       "5  2004     FLORIDA            High Wind                881000000.0\n",
       "6  2005   LOUISIANA  Hurricane (Typhoon)                850000000.0\n",
       "7  2005   LOUISIANA  Hurricane (Typhoon)                850000000.0\n",
       "8  2020   LOUISIANA            Hurricane                800000000.0\n",
       "9  2021   LOUISIANA     Storm Surge/Tide                750000000.0"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# order by DAMAGE_PROPERTY_CONVERTED and YEAR both in descending order\n",
    "(storm_events\n",
    " .sort_by([\n",
    "     ibis.desc(\"DAMAGE_PROPERTY_CONVERTED\"),\n",
    "     ibis.desc(\"YEAR\")])\n",
    " .select(\n",
    "    \"YEAR\",\n",
    "    \"STATE\",\n",
    "    \"EVENT_TYPE\",\n",
    "    \"DAMAGE_PROPERTY_CONVERTED\")\n",
    " .limit(10)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f15e2f5",
   "metadata": {},
   "source": [
    "## Aggregate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "69d535bf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1295193"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# count rows\n",
    "storm_events.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "93d157d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "storm_events.DAMAGE_PROPERTY_CONVERTED.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "1efcd084",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "950000000.0"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "storm_events.DAMAGE_PROPERTY_CONVERTED.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "c6c7cb44",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "222760.43361035068"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "storm_events.DAMAGE_PROPERTY_CONVERTED.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "25ab3bb1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "288517754289.09094"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "storm_events.DAMAGE_PROPERTY_CONVERTED.sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "195ecdcc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>222760.43361</td>\n",
       "      <td>950000000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           mean          max\n",
       "0  222760.43361  950000000.0"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# mean and max of DAMAGE_PROPERTY_CONVERTED column\n",
    "storm_events.aggregate([\n",
    "    storm_events.DAMAGE_PROPERTY_CONVERTED.mean(),\n",
    "    storm_events.DAMAGE_PROPERTY_CONVERTED.max()\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4ec0ca3",
   "metadata": {},
   "source": [
    "## Group By"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "4b12f66b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ibis.expr.types.groupby.GroupedTable"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(storm_events.group_by(\"EVENT_TYPE\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "0fff4d06",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>2017</td>\n",
       "      <td>1.911869e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>2005</td>\n",
       "      <td>1.111273e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2021</td>\n",
       "      <td>7.588400e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2018</td>\n",
       "      <td>7.143780e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2020</td>\n",
       "      <td>6.702710e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>2011</td>\n",
       "      <td>3.884651e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>2008</td>\n",
       "      <td>3.663098e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.645402e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Tropical Storm</td>\n",
       "      <td>2017</td>\n",
       "      <td>3.645017e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Flood</td>\n",
       "      <td>2011</td>\n",
       "      <td>3.621686e+09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            EVENT_TYPE  YEAR  DAMAGE_PROPERTY\n",
       "0          Flash Flood  2017     1.911869e+10\n",
       "1  Hurricane (Typhoon)  2005     1.111273e+10\n",
       "2            Hurricane  2021     7.588400e+09\n",
       "3            Hurricane  2018     7.143780e+09\n",
       "4            Hurricane  2020     6.702710e+09\n",
       "5              Tornado  2011     3.884651e+09\n",
       "6  Hurricane (Typhoon)  2008     3.663098e+09\n",
       "7          Flash Flood  2016     3.645402e+09\n",
       "8       Tropical Storm  2017     3.645017e+09\n",
       "9                Flood  2011     3.621686e+09"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sum of DAMAGE_PROPERTY by EVENT_TYPE and YEAR\n",
    "(storm_events\n",
    " .group_by([\"EVENT_TYPE\", \"YEAR\"])\n",
    " .aggregate(storm_events.DAMAGE_PROPERTY_CONVERTED.sum().name(\"DAMAGE_PROPERTY\"))\n",
    " .sort_by(ibis.desc(\"DAMAGE_PROPERTY\"))\n",
    " .limit(10)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "4876e7d4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>DAMAGE_PROPERTY_AVG</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2016</td>\n",
       "      <td>2.017333e+08</td>\n",
       "      <td>6.052000e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2021</td>\n",
       "      <td>1.996947e+08</td>\n",
       "      <td>7.588400e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2020</td>\n",
       "      <td>1.136053e+08</td>\n",
       "      <td>6.702710e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2018</td>\n",
       "      <td>1.066236e+08</td>\n",
       "      <td>7.143780e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>2017</td>\n",
       "      <td>8.210649e+07</td>\n",
       "      <td>3.037940e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Tsunami</td>\n",
       "      <td>2009</td>\n",
       "      <td>8.100000e+07</td>\n",
       "      <td>8.100000e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Storm Surge/Tide</td>\n",
       "      <td>2021</td>\n",
       "      <td>7.290867e+07</td>\n",
       "      <td>3.572525e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>2008</td>\n",
       "      <td>5.087635e+07</td>\n",
       "      <td>3.663098e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Storm Surge/Tide</td>\n",
       "      <td>2018</td>\n",
       "      <td>4.773509e+07</td>\n",
       "      <td>1.050172e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>1997</td>\n",
       "      <td>4.722357e+07</td>\n",
       "      <td>6.611300e+08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            EVENT_TYPE  YEAR  DAMAGE_PROPERTY_AVG  DAMAGE_PROPERTY\n",
       "0            Hurricane  2016         2.017333e+08     6.052000e+08\n",
       "1            Hurricane  2021         1.996947e+08     7.588400e+09\n",
       "2            Hurricane  2020         1.136053e+08     6.702710e+09\n",
       "3            Hurricane  2018         1.066236e+08     7.143780e+09\n",
       "4            Hurricane  2017         8.210649e+07     3.037940e+09\n",
       "5              Tsunami  2009         8.100000e+07     8.100000e+07\n",
       "6     Storm Surge/Tide  2021         7.290867e+07     3.572525e+09\n",
       "7  Hurricane (Typhoon)  2008         5.087635e+07     3.663098e+09\n",
       "8     Storm Surge/Tide  2018         4.773509e+07     1.050172e+09\n",
       "9  Hurricane (Typhoon)  1997         4.722357e+07     6.611300e+08"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sum and mean of DAMAGE_PROPERTY by EVENT_TYPE and YEAR\n",
    "(storm_events\n",
    " .group_by([\"EVENT_TYPE\", \"YEAR\"])\n",
    " .aggregate([\n",
    "     storm_events.DAMAGE_PROPERTY_CONVERTED.mean().name(\"DAMAGE_PROPERTY_AVG\"),\n",
    "     storm_events.DAMAGE_PROPERTY_CONVERTED.sum().name(\"DAMAGE_PROPERTY\")])\n",
    " .sort_by(ibis.desc(\"DAMAGE_PROPERTY_AVG\"))\n",
    " .limit(10)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "13611a0d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>DAMAGE_PROPERTY_AVG</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Hurricane</td>\n",
       "      <td>1.052364e+08</td>\n",
       "      <td>2.515150e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hurricane (Typhoon)</td>\n",
       "      <td>2.342431e+07</td>\n",
       "      <td>2.464237e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Storm Surge/Tide</td>\n",
       "      <td>1.045442e+07</td>\n",
       "      <td>9.837608e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Tsunami</td>\n",
       "      <td>4.146368e+06</td>\n",
       "      <td>1.575620e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Tropical Storm</td>\n",
       "      <td>2.325792e+06</td>\n",
       "      <td>9.733441e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Wildfire</td>\n",
       "      <td>1.789479e+06</td>\n",
       "      <td>1.014098e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Ice Storm</td>\n",
       "      <td>9.281769e+05</td>\n",
       "      <td>5.897636e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>9.240670e+05</td>\n",
       "      <td>6.136359e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Coastal Flood</td>\n",
       "      <td>7.023198e+05</td>\n",
       "      <td>1.927868e+09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Debris Flow</td>\n",
       "      <td>6.796346e+05</td>\n",
       "      <td>1.296743e+09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            EVENT_TYPE  DAMAGE_PROPERTY_AVG  DAMAGE_PROPERTY\n",
       "0            Hurricane         1.052364e+08     2.515150e+10\n",
       "1  Hurricane (Typhoon)         2.342431e+07     2.464237e+10\n",
       "2     Storm Surge/Tide         1.045442e+07     9.837608e+09\n",
       "3              Tsunami         4.146368e+06     1.575620e+08\n",
       "4       Tropical Storm         2.325792e+06     9.733441e+09\n",
       "5             Wildfire         1.789479e+06     1.014098e+10\n",
       "6            Ice Storm         9.281769e+05     5.897636e+09\n",
       "7              Tornado         9.240670e+05     6.136359e+10\n",
       "8        Coastal Flood         7.023198e+05     1.927868e+09\n",
       "9          Debris Flow         6.796346e+05     1.296743e+09"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sum and mean of DAMAGE_PROPERTY by EVENT_TYPE\n",
    "(storm_events\n",
    " .group_by([\"EVENT_TYPE\"])\n",
    " .aggregate([\n",
    "     storm_events.DAMAGE_PROPERTY_CONVERTED.mean().name(\"DAMAGE_PROPERTY_AVG\"),\n",
    "     storm_events.DAMAGE_PROPERTY_CONVERTED.sum().name(\"DAMAGE_PROPERTY\")])\n",
    " .sort_by(ibis.desc(\"DAMAGE_PROPERTY_AVG\"))\n",
    " .limit(10)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7185671",
   "metadata": {},
   "source": [
    "## Cast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "ae84cd18",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_INT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "      <td>25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "      <td>25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "      <td>2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "      <td>2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10093724</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>103</td>\n",
       "      <td>NOBLE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10093725</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>65</td>\n",
       "      <td>JACKSON</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10093726</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>83</td>\n",
       "      <td>LOGAN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>10093727</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10093769</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>May</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME         EVENT_TYPE  \\\n",
       "0     10096222      OKLAHOMA          40  1950      April            Tornado   \n",
       "1     10120412         TEXAS          48  1950      April            Tornado   \n",
       "2     10104927  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "3     10104928  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "4     10104929  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "...        ...           ...         ...   ...        ...                ...   \n",
       "9995  10093724      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9996  10093725      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9997  10093726      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9998  10093727      OKLAHOMA          40  1956      April            Tornado   \n",
       "9999  10093769      OKLAHOMA          40  1956        May               Hail   \n",
       "\n",
       "     CZ_TYPE  CZ_FIPS   CZ_NAME   WFO  ... INJURIES_DIRECT INJURIES_INDIRECT  \\\n",
       "0          C      149   WASHITA  None  ...               0                 0   \n",
       "1          C       93  COMANCHE  None  ...               0                 0   \n",
       "2          C       77    LEHIGH  None  ...               2                 0   \n",
       "3          C       43   DAUPHIN  None  ...               0                 0   \n",
       "4          C       39  CRAWFORD  None  ...               0                 0   \n",
       "...      ...      ...       ...   ...  ...             ...               ...   \n",
       "9995       C      103     NOBLE  None  ...               0                 0   \n",
       "9996       C       65   JACKSON  None  ...               0                 0   \n",
       "9997       C       83     LOGAN  None  ...               0                 0   \n",
       "9998       C      119     PAYNE  None  ...               0                 0   \n",
       "9999       C      119     PAYNE  None  ...               0                 0   \n",
       "\n",
       "      DEATHS_DIRECT  DEATHS_INDIRECT  DAMAGE_PROPERTY  DAMAGE_CROPS SOURCE  \\\n",
       "0                 0                0             250K             0   None   \n",
       "1                 0                0              25K             0   None   \n",
       "2                 0                0              25K             0   None   \n",
       "3                 0                0             2.5K             0   None   \n",
       "4                 0                0             2.5K             0   None   \n",
       "...             ...              ...              ...           ...    ...   \n",
       "9995              0                0                0             0   None   \n",
       "9996              0                0                0             0   None   \n",
       "9997              0                0                0             0   None   \n",
       "9998              0                0             250K             0   None   \n",
       "9999              0                0                0             0   None   \n",
       "\n",
       "     DAMAGE_PROPERTY_CONVERTED DAMAGE_PROPERTY_CONVERTED_DOLLARS  \\\n",
       "0                     250000.0                           $250000   \n",
       "1                      25000.0                            $25000   \n",
       "2                      25000.0                            $25000   \n",
       "3                       2500.0                             $2500   \n",
       "4                       2500.0                             $2500   \n",
       "...                        ...                               ...   \n",
       "9995                       0.0                                $0   \n",
       "9996                       0.0                                $0   \n",
       "9997                       0.0                                $0   \n",
       "9998                  250000.0                           $250000   \n",
       "9999                       0.0                                $0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_INT  \n",
       "0                            250000  \n",
       "1                             25000  \n",
       "2                             25000  \n",
       "3                              2500  \n",
       "4                              2500  \n",
       "...                             ...  \n",
       "9995                              0  \n",
       "9996                              0  \n",
       "9997                              0  \n",
       "9998                         250000  \n",
       "9999                              0  \n",
       "\n",
       "[10000 rows x 22 columns]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# cast DAMAGE_PROPERTY_CONVERTED to int\n",
    "expr = storm_events.DAMAGE_PROPERTY_CONVERTED.cast(\"int32\")\n",
    "\n",
    "storm_events.mutate(DAMAGE_PROPERTY_CONVERTED_INT=expr)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06bf0a0b",
   "metadata": {},
   "source": [
    "## Other"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "484db5bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Tornado</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Frost/Freeze</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Extreme Cold/Wind Chill</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Dense Smoke</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Cold/Wind Chill</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>Coastal Flood</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>Avalanche</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>Astronomical Low Tide</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>Freezing Fog</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>Marine Tropical Storm</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>69 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                  EVENT_TYPE\n",
       "0               Frost/Freeze\n",
       "1                    Tornado\n",
       "2    Extreme Cold/Wind Chill\n",
       "3                Dense Smoke\n",
       "4   Marine Thunderstorm Wind\n",
       "..                       ...\n",
       "64               Debris Flow\n",
       "65                  Blizzard\n",
       "66                 Avalanche\n",
       "67              Freezing Fog\n",
       "68     Marine Tropical Storm\n",
       "\n",
       "[69 rows x 1 columns]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# distinct values\n",
    "storm_events.select(\"EVENT_TYPE\").distinct()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "b0107c44",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>408310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Hail</td>\n",
       "      <td>242063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>79247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Tornado</td>\n",
       "      <td>66406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>High Wind</td>\n",
       "      <td>59177</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Winter Weather</td>\n",
       "      <td>52422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Flood</td>\n",
       "      <td>51073</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Winter Storm</td>\n",
       "      <td>49599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Drought</td>\n",
       "      <td>43288</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Heavy Snow</td>\n",
       "      <td>38561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Marine Thunderstorm Wind</td>\n",
       "      <td>24613</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Strong Wind</td>\n",
       "      <td>21038</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Heavy Rain</td>\n",
       "      <td>20189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Heat</td>\n",
       "      <td>14816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Lightning</td>\n",
       "      <td>13901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Excessive Heat</td>\n",
       "      <td>13829</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Frost/Freeze</td>\n",
       "      <td>10910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Extreme Cold/Wind Chill</td>\n",
       "      <td>10194</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Dense Fog</td>\n",
       "      <td>9537</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Blizzard</td>\n",
       "      <td>9231</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  EVENT_TYPE   count\n",
       "0          Thunderstorm Wind  408310\n",
       "1                       Hail  242063\n",
       "2                Flash Flood   79247\n",
       "3                    Tornado   66406\n",
       "4                  High Wind   59177\n",
       "5             Winter Weather   52422\n",
       "6                      Flood   51073\n",
       "7               Winter Storm   49599\n",
       "8                    Drought   43288\n",
       "9                 Heavy Snow   38561\n",
       "10  Marine Thunderstorm Wind   24613\n",
       "11               Strong Wind   21038\n",
       "12                Heavy Rain   20189\n",
       "13                      Heat   14816\n",
       "14                 Lightning   13901\n",
       "15            Excessive Heat   13829\n",
       "16              Frost/Freeze   10910\n",
       "17   Extreme Cold/Wind Chill   10194\n",
       "18                 Dense Fog    9537\n",
       "19                  Blizzard    9231"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# value counts of EVENT_TYPE\n",
    "(storm_events\n",
    " .EVENT_TYPE\n",
    " .value_counts()\n",
    " .sort_by(ibis.desc(\"count\"))\n",
    " .limit(20)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "4f28867b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>major_damage</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>False</td>\n",
       "      <td>1294724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>True</td>\n",
       "      <td>469</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   major_damage    count\n",
       "0         False  1294724\n",
       "1          True      469"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# value counts of boolean expression\n",
    "value_counts = (storm_events.DAMAGE_PROPERTY_CONVERTED > 100_000_000).name(\"major_damage\")\n",
    "\n",
    "storm_events.mutate(value_counts).major_damage.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "9c9c7134",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'SELECT\\n  t0.\"YEAR\",\\n  t0.\"EVENT_TYPE\",\\n  t0.sum\\nFROM (\\n  SELECT\\n    t1.\"YEAR\" AS \"YEAR\",\\n    t1.\"EVENT_TYPE\" AS \"EVENT_TYPE\",\\n    SUM(t1.\"DAMAGE_PROPERTY_CONVERTED\") AS sum\\n  FROM (\\n    SELECT\\n      t2.\"EVENT_ID\" AS \"EVENT_ID\",\\n      t2.\"STATE\" AS \"STATE\",\\n      t2.\"STATE_FIPS\" AS \"STATE_FIPS\",\\n      t2.\"YEAR\" AS \"YEAR\",\\n      t2.\"MONTH_NAME\" AS \"MONTH_NAME\",\\n      t2.\"EVENT_TYPE\" AS \"EVENT_TYPE\",\\n      t2.\"CZ_TYPE\" AS \"CZ_TYPE\",\\n      t2.\"CZ_FIPS\" AS \"CZ_FIPS\",\\n      t2.\"CZ_NAME\" AS \"CZ_NAME\",\\n      t2.\"WFO\" AS \"WFO\",\\n      t2.\"BEGIN_DATE_TIME\" AS \"BEGIN_DATE_TIME\",\\n      t2.\"END_DATE_TIME\" AS \"END_DATE_TIME\",\\n      t2.\"INJURIES_DIRECT\" AS \"INJURIES_DIRECT\",\\n      t2.\"INJURIES_INDIRECT\" AS \"INJURIES_INDIRECT\",\\n      t2.\"DEATHS_DIRECT\" AS \"DEATHS_DIRECT\",\\n      t2.\"DEATHS_INDIRECT\" AS \"DEATHS_INDIRECT\",\\n      t2.\"DAMAGE_PROPERTY\" AS \"DAMAGE_PROPERTY\",\\n      t2.\"DAMAGE_CROPS\" AS \"DAMAGE_CROPS\",\\n      t2.\"SOURCE\" AS \"SOURCE\",\\n      t2.\"DAMAGE_PROPERTY_CONVERTED\" AS \"DAMAGE_PROPERTY_CONVERTED\",\\n      CONCAT(CAST(\\'$\\' AS TEXT), CAST(CAST(t2.\"DAMAGE_PROPERTY_CONVERTED\" AS BIGINT) AS TEXT)) AS \"DAMAGE_PROPERTY_CONVERTED_DOLLARS\"\\n    FROM storm_events AS t2\\n  ) AS t1\\n  GROUP BY\\n    t1.\"YEAR\",\\n    t1.\"EVENT_TYPE\"\\n) AS t0\\nORDER BY\\n  t0.sum DESC\\nLIMIT 10'"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert to sql statement\n",
    "expr = (\n",
    "    storm_events\n",
    "    .group_by([\"YEAR\", \"EVENT_TYPE\"])\n",
    "    .aggregate(storm_events.DAMAGE_PROPERTY_CONVERTED.sum())\n",
    "    .sort_by(ibis.desc(\"sum\"))\n",
    "    .limit(10)\n",
    ")\n",
    "\n",
    "ibis.to_sql(expr, dialect=\"postgres\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "1309a6d1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10071654</td>\n",
       "      <td>NEBRASKA</td>\n",
       "      <td>31</td>\n",
       "      <td>1958</td>\n",
       "      <td>June</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>165</td>\n",
       "      <td>SIOUX</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-58 14:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10071660</td>\n",
       "      <td>NEBRASKA</td>\n",
       "      <td>31</td>\n",
       "      <td>1958</td>\n",
       "      <td>June</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>145</td>\n",
       "      <td>RED WILLOW</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>30-JUN-58 19:10:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10121841</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1958</td>\n",
       "      <td>July</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>153</td>\n",
       "      <td>FLOYD</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>31-JUL-58 19:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10124075</td>\n",
       "      <td>TENNESSEE</td>\n",
       "      <td>47</td>\n",
       "      <td>1958</td>\n",
       "      <td>April</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>145</td>\n",
       "      <td>ROANE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>24-APR-58 17:50:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10028944</td>\n",
       "      <td>KANSAS</td>\n",
       "      <td>20</td>\n",
       "      <td>1958</td>\n",
       "      <td>May</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>167</td>\n",
       "      <td>RUSSELL</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>24-MAY-58 15:57:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10136016</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1969</td>\n",
       "      <td>June</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>205</td>\n",
       "      <td>HARTLEY</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>17-JUN-69 18:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10136017</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1969</td>\n",
       "      <td>June</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>375</td>\n",
       "      <td>POTTER</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>17-JUN-69 20:55:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10136018</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1969</td>\n",
       "      <td>June</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>191</td>\n",
       "      <td>HALL</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>17-JUN-69 22:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>10136019</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1969</td>\n",
       "      <td>June</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>75</td>\n",
       "      <td>CHILDRESS</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>17-JUN-69 23:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10136020</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1969</td>\n",
       "      <td>June</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>307</td>\n",
       "      <td>MCCULLOCH</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-69 16:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID      STATE  STATE_FIPS  YEAR MONTH_NAME EVENT_TYPE CZ_TYPE  \\\n",
       "0     10071654   NEBRASKA          31  1958       June       Hail       C   \n",
       "1     10071660   NEBRASKA          31  1958       June       Hail       C   \n",
       "2     10121841      TEXAS          48  1958       July       Hail       C   \n",
       "3     10124075  TENNESSEE          47  1958      April       Hail       C   \n",
       "4     10028944     KANSAS          20  1958        May       Hail       C   \n",
       "...        ...        ...         ...   ...        ...        ...     ...   \n",
       "9995  10136016      TEXAS          48  1969       June       Hail       C   \n",
       "9996  10136017      TEXAS          48  1969       June       Hail       C   \n",
       "9997  10136018      TEXAS          48  1969       June       Hail       C   \n",
       "9998  10136019      TEXAS          48  1969       June       Hail       C   \n",
       "9999  10136020      TEXAS          48  1969       June       Hail       C   \n",
       "\n",
       "      CZ_FIPS     CZ_NAME   WFO  ...       END_DATE_TIME INJURIES_DIRECT  \\\n",
       "0         165       SIOUX  None  ...  18-JUN-58 14:00:00               0   \n",
       "1         145  RED WILLOW  None  ...  30-JUN-58 19:10:00               0   \n",
       "2         153       FLOYD  None  ...  31-JUL-58 19:45:00               0   \n",
       "3         145       ROANE  None  ...  24-APR-58 17:50:00               0   \n",
       "4         167     RUSSELL  None  ...  24-MAY-58 15:57:00               0   \n",
       "...       ...         ...   ...  ...                 ...             ...   \n",
       "9995      205     HARTLEY  None  ...  17-JUN-69 18:00:00               0   \n",
       "9996      375      POTTER  None  ...  17-JUN-69 20:55:00               0   \n",
       "9997      191        HALL  None  ...  17-JUN-69 22:45:00               0   \n",
       "9998       75   CHILDRESS  None  ...  17-JUN-69 23:45:00               0   \n",
       "9999      307   MCCULLOCH  None  ...  18-JUN-69 16:45:00               0   \n",
       "\n",
       "      INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  DAMAGE_PROPERTY  \\\n",
       "0                     0              0                0                0   \n",
       "1                     0              0                0                0   \n",
       "2                     0              0                0                0   \n",
       "3                     0              0                0                0   \n",
       "4                     0              0                0                0   \n",
       "...                 ...            ...              ...              ...   \n",
       "9995                  0              0                0                0   \n",
       "9996                  0              0                0                0   \n",
       "9997                  0              0                0                0   \n",
       "9998                  0              0                0                0   \n",
       "9999                  0              0                0                0   \n",
       "\n",
       "     DAMAGE_CROPS SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0               0   None                       0.0   \n",
       "1               0   None                       0.0   \n",
       "2               0   None                       0.0   \n",
       "3               0   None                       0.0   \n",
       "4               0   None                       0.0   \n",
       "...           ...    ...                       ...   \n",
       "9995            0   None                       0.0   \n",
       "9996            0   None                       0.0   \n",
       "9997            0   None                       0.0   \n",
       "9998            0   None                       0.0   \n",
       "9999            0   None                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_DOLLARS  \n",
       "0                                    $0  \n",
       "1                                    $0  \n",
       "2                                    $0  \n",
       "3                                    $0  \n",
       "4                                    $0  \n",
       "...                                 ...  \n",
       "9995                                 $0  \n",
       "9996                                 $0  \n",
       "9997                                 $0  \n",
       "9998                                 $0  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# is in\n",
    "storm_events[storm_events.EVENT_TYPE.isin([\"Hail\"])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "f1d34ab1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10029164</td>\n",
       "      <td>KANSAS</td>\n",
       "      <td>20</td>\n",
       "      <td>1962</td>\n",
       "      <td>July</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>145</td>\n",
       "      <td>PAWNEE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>14-JUL-62 21:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10029165</td>\n",
       "      <td>KANSAS</td>\n",
       "      <td>20</td>\n",
       "      <td>1962</td>\n",
       "      <td>July</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>9</td>\n",
       "      <td>BARTON</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>14-JUL-62 21:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10029166</td>\n",
       "      <td>KANSAS</td>\n",
       "      <td>20</td>\n",
       "      <td>1962</td>\n",
       "      <td>July</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>169</td>\n",
       "      <td>SALINE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>14-JUL-62 23:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>9988125</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>12</td>\n",
       "      <td>1962</td>\n",
       "      <td>May</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>105</td>\n",
       "      <td>POLK</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>11-MAY-62 17:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10029167</td>\n",
       "      <td>KANSAS</td>\n",
       "      <td>20</td>\n",
       "      <td>1962</td>\n",
       "      <td>July</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>65</td>\n",
       "      <td>GRAHAM</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>16-JUL-62 20:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME         EVENT_TYPE  \\\n",
       "0     10096222      OKLAHOMA          40  1950      April            Tornado   \n",
       "1     10120412         TEXAS          48  1950      April            Tornado   \n",
       "2     10104927  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "3     10104928  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "4     10104929  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "...        ...           ...         ...   ...        ...                ...   \n",
       "9995  10029164        KANSAS          20  1962       July  Thunderstorm Wind   \n",
       "9996  10029165        KANSAS          20  1962       July  Thunderstorm Wind   \n",
       "9997  10029166        KANSAS          20  1962       July  Thunderstorm Wind   \n",
       "9998   9988125       FLORIDA          12  1962        May  Thunderstorm Wind   \n",
       "9999  10029167        KANSAS          20  1962       July  Thunderstorm Wind   \n",
       "\n",
       "     CZ_TYPE  CZ_FIPS   CZ_NAME   WFO  ...       END_DATE_TIME  \\\n",
       "0          C      149   WASHITA  None  ...  28-APR-50 14:45:00   \n",
       "1          C       93  COMANCHE  None  ...  29-APR-50 15:30:00   \n",
       "2          C       77    LEHIGH  None  ...  05-JUL-50 18:00:00   \n",
       "3          C       43   DAUPHIN  None  ...  05-JUL-50 18:30:00   \n",
       "4          C       39  CRAWFORD  None  ...  24-JUL-50 14:40:00   \n",
       "...      ...      ...       ...   ...  ...                 ...   \n",
       "9995       C      145    PAWNEE  None  ...  14-JUL-62 21:00:00   \n",
       "9996       C        9    BARTON  None  ...  14-JUL-62 21:30:00   \n",
       "9997       C      169    SALINE  None  ...  14-JUL-62 23:30:00   \n",
       "9998       C      105      POLK  None  ...  11-MAY-62 17:00:00   \n",
       "9999       C       65    GRAHAM  None  ...  16-JUL-62 20:00:00   \n",
       "\n",
       "     INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                  0                  0              0                0   \n",
       "1                  0                  0              0                0   \n",
       "2                  2                  0              0                0   \n",
       "3                  0                  0              0                0   \n",
       "4                  0                  0              0                0   \n",
       "...              ...                ...            ...              ...   \n",
       "9995               0                  0              0                0   \n",
       "9996               0                  0              0                0   \n",
       "9997               0                  0              0                0   \n",
       "9998               0                  0              0                0   \n",
       "9999               0                  0              0                0   \n",
       "\n",
       "      DAMAGE_PROPERTY DAMAGE_CROPS SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0                250K            0   None                  250000.0   \n",
       "1                 25K            0   None                   25000.0   \n",
       "2                 25K            0   None                   25000.0   \n",
       "3                2.5K            0   None                    2500.0   \n",
       "4                2.5K            0   None                    2500.0   \n",
       "...               ...          ...    ...                       ...   \n",
       "9995                0            0   None                       0.0   \n",
       "9996                0            0   None                       0.0   \n",
       "9997                0            0   None                       0.0   \n",
       "9998                0            0   None                       0.0   \n",
       "9999                0            0   None                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_DOLLARS  \n",
       "0                               $250000  \n",
       "1                                $25000  \n",
       "2                                $25000  \n",
       "3                                 $2500  \n",
       "4                                 $2500  \n",
       "...                                 ...  \n",
       "9995                                 $0  \n",
       "9996                                 $0  \n",
       "9997                                 $0  \n",
       "9998                                 $0  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# not in\n",
    "storm_events[storm_events.EVENT_TYPE.notin([\"Hail\"])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "e9dd0591",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10096222</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>149</td>\n",
       "      <td>WASHITA</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>28-APR-50 14:45:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10120412</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1950</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>COMANCHE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>29-APR-50 15:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10104927</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>77</td>\n",
       "      <td>LEHIGH</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:00:00</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10104928</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>DAUPHIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-JUL-50 18:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10104929</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1950</td>\n",
       "      <td>July</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>39</td>\n",
       "      <td>CRAWFORD</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>24-JUL-50 14:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>$2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>10093724</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>103</td>\n",
       "      <td>NOBLE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>10093725</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>65</td>\n",
       "      <td>JACKSON</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>10093726</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Thunderstorm Wind</td>\n",
       "      <td>C</td>\n",
       "      <td>83</td>\n",
       "      <td>LOGAN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:02:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>10093727</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>08-APR-56 23:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>10093769</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1956</td>\n",
       "      <td>May</td>\n",
       "      <td>Hail</td>\n",
       "      <td>C</td>\n",
       "      <td>119</td>\n",
       "      <td>PAYNE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>13-MAY-56 19:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME         EVENT_TYPE  \\\n",
       "0     10096222      OKLAHOMA          40  1950      April            Tornado   \n",
       "1     10120412         TEXAS          48  1950      April            Tornado   \n",
       "2     10104927  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "3     10104928  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "4     10104929  PENNSYLVANIA          42  1950       July            Tornado   \n",
       "...        ...           ...         ...   ...        ...                ...   \n",
       "9995  10093724      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9996  10093725      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9997  10093726      OKLAHOMA          40  1956      April  Thunderstorm Wind   \n",
       "9998  10093727      OKLAHOMA          40  1956      April            Tornado   \n",
       "9999  10093769      OKLAHOMA          40  1956        May               Hail   \n",
       "\n",
       "     CZ_TYPE  CZ_FIPS   CZ_NAME   WFO  ...       END_DATE_TIME  \\\n",
       "0          C      149   WASHITA  None  ...  28-APR-50 14:45:00   \n",
       "1          C       93  COMANCHE  None  ...  29-APR-50 15:30:00   \n",
       "2          C       77    LEHIGH  None  ...  05-JUL-50 18:00:00   \n",
       "3          C       43   DAUPHIN  None  ...  05-JUL-50 18:30:00   \n",
       "4          C       39  CRAWFORD  None  ...  24-JUL-50 14:40:00   \n",
       "...      ...      ...       ...   ...  ...                 ...   \n",
       "9995       C      103     NOBLE  None  ...  08-APR-56 23:00:00   \n",
       "9996       C       65   JACKSON  None  ...  08-APR-56 23:00:00   \n",
       "9997       C       83     LOGAN  None  ...  08-APR-56 23:02:00   \n",
       "9998       C      119     PAYNE  None  ...  08-APR-56 23:30:00   \n",
       "9999       C      119     PAYNE  None  ...  13-MAY-56 19:00:00   \n",
       "\n",
       "     INJURIES_DIRECT  INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  \\\n",
       "0                  0                  0              0                0   \n",
       "1                  0                  0              0                0   \n",
       "2                  2                  0              0                0   \n",
       "3                  0                  0              0                0   \n",
       "4                  0                  0              0                0   \n",
       "...              ...                ...            ...              ...   \n",
       "9995               0                  0              0                0   \n",
       "9996               0                  0              0                0   \n",
       "9997               0                  0              0                0   \n",
       "9998               0                  0              0                0   \n",
       "9999               0                  0              0                0   \n",
       "\n",
       "      DAMAGE_PROPERTY DAMAGE_CROPS SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0                250K            0   None                  250000.0   \n",
       "1                 25K            0   None                   25000.0   \n",
       "2                 25K            0   None                   25000.0   \n",
       "3                2.5K            0   None                    2500.0   \n",
       "4                2.5K            0   None                    2500.0   \n",
       "...               ...          ...    ...                       ...   \n",
       "9995                0            0   None                       0.0   \n",
       "9996                0            0   None                       0.0   \n",
       "9997                0            0   None                       0.0   \n",
       "9998             250K            0   None                  250000.0   \n",
       "9999                0            0   None                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_DOLLARS  \n",
       "0                               $250000  \n",
       "1                                $25000  \n",
       "2                                $25000  \n",
       "3                                 $2500  \n",
       "4                                 $2500  \n",
       "...                                 ...  \n",
       "9995                                 $0  \n",
       "9996                                 $0  \n",
       "9997                                 $0  \n",
       "9998                            $250000  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# is null\n",
    "storm_events[storm_events.SOURCE.isnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "d3e243d5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>990000001</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>12</td>\n",
       "      <td>1972</td>\n",
       "      <td>June</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>GLADES</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-72 15:17:00</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>190K</td>\n",
       "      <td>5K</td>\n",
       "      <td>Manual Input</td>\n",
       "      <td>190000.0</td>\n",
       "      <td>$190000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>990000002</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>12</td>\n",
       "      <td>1972</td>\n",
       "      <td>June</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>51</td>\n",
       "      <td>HENDRY</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-72 15:13:00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>10K</td>\n",
       "      <td>5K</td>\n",
       "      <td>Manual Input</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>$10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>990000003</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>12</td>\n",
       "      <td>1972</td>\n",
       "      <td>June</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>OKEECHOBEE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-72 22:55:00</td>\n",
       "      <td>44</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>500K</td>\n",
       "      <td>0</td>\n",
       "      <td>Manual Input</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>$500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10032307</td>\n",
       "      <td>Kentucky</td>\n",
       "      <td>21</td>\n",
       "      <td>1974</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>215</td>\n",
       "      <td>SPENCER</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>03-APR-74 17:31:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>WFO</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10032301</td>\n",
       "      <td>Kentucky</td>\n",
       "      <td>21</td>\n",
       "      <td>1974</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>HARDIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>03-APR-74 16:55:00</td>\n",
       "      <td>57</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>WFO</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>5678048</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1998</td>\n",
       "      <td>October</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "      <td>ALFALFA</td>\n",
       "      <td>OUN</td>\n",
       "      <td>...</td>\n",
       "      <td>04-OCT-98 14:43:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>55K</td>\n",
       "      <td>None</td>\n",
       "      <td>UNKNOWN</td>\n",
       "      <td>55000.0</td>\n",
       "      <td>$55000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>5678049</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1998</td>\n",
       "      <td>October</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>11</td>\n",
       "      <td>BLAINE</td>\n",
       "      <td>OUN</td>\n",
       "      <td>...</td>\n",
       "      <td>04-OCT-98 17:12:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50K</td>\n",
       "      <td>None</td>\n",
       "      <td>UNKNOWN</td>\n",
       "      <td>50000.0</td>\n",
       "      <td>$50000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>5673341</td>\n",
       "      <td>NEBRASKA</td>\n",
       "      <td>31</td>\n",
       "      <td>1998</td>\n",
       "      <td>November</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>Z</td>\n",
       "      <td>77</td>\n",
       "      <td>FILLMORE</td>\n",
       "      <td>GID</td>\n",
       "      <td>...</td>\n",
       "      <td>10-NOV-98 16:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>AWOS,ASOS,MESONET,ETC</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>5676260</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1998</td>\n",
       "      <td>November</td>\n",
       "      <td>Heat</td>\n",
       "      <td>Z</td>\n",
       "      <td>67</td>\n",
       "      <td>CHESTER</td>\n",
       "      <td>PHI</td>\n",
       "      <td>...</td>\n",
       "      <td>30-NOV-98 23:59:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>AWOS,ASOS,MESONET,ETC</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>5672925</td>\n",
       "      <td>NEW JERSEY</td>\n",
       "      <td>34</td>\n",
       "      <td>1998</td>\n",
       "      <td>September</td>\n",
       "      <td>Heat</td>\n",
       "      <td>Z</td>\n",
       "      <td>18</td>\n",
       "      <td>CAMDEN</td>\n",
       "      <td>PHI</td>\n",
       "      <td>...</td>\n",
       "      <td>27-SEP-98 18:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>AWOS,ASOS,MESONET,ETC</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME EVENT_TYPE CZ_TYPE  \\\n",
       "0     990000001       FLORIDA          12  1972       June    Tornado       C   \n",
       "1     990000002       FLORIDA          12  1972       June    Tornado       C   \n",
       "2     990000003       FLORIDA          12  1972       June    Tornado       C   \n",
       "3      10032307      Kentucky          21  1974      April    Tornado       C   \n",
       "4      10032301      Kentucky          21  1974      April    Tornado       C   \n",
       "...         ...           ...         ...   ...        ...        ...     ...   \n",
       "9995    5678048      OKLAHOMA          40  1998    October    Tornado       C   \n",
       "9996    5678049      OKLAHOMA          40  1998    October    Tornado       C   \n",
       "9997    5673341      NEBRASKA          31  1998   November  High Wind       Z   \n",
       "9998    5676260  PENNSYLVANIA          42  1998   November       Heat       Z   \n",
       "9999    5672925    NEW JERSEY          34  1998  September       Heat       Z   \n",
       "\n",
       "      CZ_FIPS     CZ_NAME   WFO  ...       END_DATE_TIME INJURIES_DIRECT  \\\n",
       "0          43      GLADES  None  ...  18-JUN-72 15:17:00               6   \n",
       "1          51      HENDRY  None  ...  18-JUN-72 15:13:00               1   \n",
       "2          93  OKEECHOBEE  None  ...  18-JUN-72 22:55:00              44   \n",
       "3         215     SPENCER  None  ...  03-APR-74 17:31:00               0   \n",
       "4          93      HARDIN  None  ...  03-APR-74 16:55:00              57   \n",
       "...       ...         ...   ...  ...                 ...             ...   \n",
       "9995        3     ALFALFA   OUN  ...  04-OCT-98 14:43:00               0   \n",
       "9996       11      BLAINE   OUN  ...  04-OCT-98 17:12:00               0   \n",
       "9997       77    FILLMORE   GID  ...  10-NOV-98 16:00:00               0   \n",
       "9998       67     CHESTER   PHI  ...  30-NOV-98 23:59:00               0   \n",
       "9999       18      CAMDEN   PHI  ...  27-SEP-98 18:00:00               0   \n",
       "\n",
       "      INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  DAMAGE_PROPERTY  \\\n",
       "0                     0              0                0             190K   \n",
       "1                     0              1                0              10K   \n",
       "2                     0              6                0             500K   \n",
       "3                     0              0                0              25K   \n",
       "4                     0              2                0             250K   \n",
       "...                 ...            ...              ...              ...   \n",
       "9995                  0              0                0              55K   \n",
       "9996                  0              0                0              50K   \n",
       "9997                  0              0                0                0   \n",
       "9998                  0              0                0                0   \n",
       "9999                  0              0                0                0   \n",
       "\n",
       "     DAMAGE_CROPS                 SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0              5K           Manual Input                  190000.0   \n",
       "1              5K           Manual Input                   10000.0   \n",
       "2               0           Manual Input                  500000.0   \n",
       "3               0                    WFO                   25000.0   \n",
       "4               0                    WFO                  250000.0   \n",
       "...           ...                    ...                       ...   \n",
       "9995         None                UNKNOWN                   55000.0   \n",
       "9996         None                UNKNOWN                   50000.0   \n",
       "9997            0  AWOS,ASOS,MESONET,ETC                       0.0   \n",
       "9998            0  AWOS,ASOS,MESONET,ETC                       0.0   \n",
       "9999         None  AWOS,ASOS,MESONET,ETC                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_DOLLARS  \n",
       "0                               $190000  \n",
       "1                                $10000  \n",
       "2                               $500000  \n",
       "3                                $25000  \n",
       "4                               $250000  \n",
       "...                                 ...  \n",
       "9995                             $55000  \n",
       "9996                             $50000  \n",
       "9997                                 $0  \n",
       "9998                                 $0  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# not null\n",
    "storm_events[storm_events.SOURCE.notnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "83c1bb9d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>990000001</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>12</td>\n",
       "      <td>1972</td>\n",
       "      <td>June</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>43</td>\n",
       "      <td>GLADES</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-72 15:17:00</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>190K</td>\n",
       "      <td>5K</td>\n",
       "      <td>Manual Input</td>\n",
       "      <td>190000.0</td>\n",
       "      <td>$190000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>990000002</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>12</td>\n",
       "      <td>1972</td>\n",
       "      <td>June</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>51</td>\n",
       "      <td>HENDRY</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-72 15:13:00</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>10K</td>\n",
       "      <td>5K</td>\n",
       "      <td>Manual Input</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>$10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>990000003</td>\n",
       "      <td>FLORIDA</td>\n",
       "      <td>12</td>\n",
       "      <td>1972</td>\n",
       "      <td>June</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>OKEECHOBEE</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>18-JUN-72 22:55:00</td>\n",
       "      <td>44</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>500K</td>\n",
       "      <td>0</td>\n",
       "      <td>Manual Input</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>$500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10032307</td>\n",
       "      <td>Kentucky</td>\n",
       "      <td>21</td>\n",
       "      <td>1974</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>215</td>\n",
       "      <td>SPENCER</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>03-APR-74 17:31:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0</td>\n",
       "      <td>WFO</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10032301</td>\n",
       "      <td>Kentucky</td>\n",
       "      <td>21</td>\n",
       "      <td>1974</td>\n",
       "      <td>April</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>93</td>\n",
       "      <td>HARDIN</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>03-APR-74 16:55:00</td>\n",
       "      <td>57</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>250K</td>\n",
       "      <td>0</td>\n",
       "      <td>WFO</td>\n",
       "      <td>250000.0</td>\n",
       "      <td>$250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>5678048</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1998</td>\n",
       "      <td>October</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "      <td>ALFALFA</td>\n",
       "      <td>OUN</td>\n",
       "      <td>...</td>\n",
       "      <td>04-OCT-98 14:43:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>55K</td>\n",
       "      <td>None</td>\n",
       "      <td>UNKNOWN</td>\n",
       "      <td>55000.0</td>\n",
       "      <td>$55000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>5678049</td>\n",
       "      <td>OKLAHOMA</td>\n",
       "      <td>40</td>\n",
       "      <td>1998</td>\n",
       "      <td>October</td>\n",
       "      <td>Tornado</td>\n",
       "      <td>C</td>\n",
       "      <td>11</td>\n",
       "      <td>BLAINE</td>\n",
       "      <td>OUN</td>\n",
       "      <td>...</td>\n",
       "      <td>04-OCT-98 17:12:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50K</td>\n",
       "      <td>None</td>\n",
       "      <td>UNKNOWN</td>\n",
       "      <td>50000.0</td>\n",
       "      <td>$50000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>5673341</td>\n",
       "      <td>NEBRASKA</td>\n",
       "      <td>31</td>\n",
       "      <td>1998</td>\n",
       "      <td>November</td>\n",
       "      <td>High Wind</td>\n",
       "      <td>Z</td>\n",
       "      <td>77</td>\n",
       "      <td>FILLMORE</td>\n",
       "      <td>GID</td>\n",
       "      <td>...</td>\n",
       "      <td>10-NOV-98 16:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>AWOS,ASOS,MESONET,ETC</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>5676260</td>\n",
       "      <td>PENNSYLVANIA</td>\n",
       "      <td>42</td>\n",
       "      <td>1998</td>\n",
       "      <td>November</td>\n",
       "      <td>Heat</td>\n",
       "      <td>Z</td>\n",
       "      <td>67</td>\n",
       "      <td>CHESTER</td>\n",
       "      <td>PHI</td>\n",
       "      <td>...</td>\n",
       "      <td>30-NOV-98 23:59:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>AWOS,ASOS,MESONET,ETC</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>5672925</td>\n",
       "      <td>NEW JERSEY</td>\n",
       "      <td>34</td>\n",
       "      <td>1998</td>\n",
       "      <td>September</td>\n",
       "      <td>Heat</td>\n",
       "      <td>Z</td>\n",
       "      <td>18</td>\n",
       "      <td>CAMDEN</td>\n",
       "      <td>PHI</td>\n",
       "      <td>...</td>\n",
       "      <td>27-SEP-98 18:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>AWOS,ASOS,MESONET,ETC</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       EVENT_ID         STATE  STATE_FIPS  YEAR MONTH_NAME EVENT_TYPE CZ_TYPE  \\\n",
       "0     990000001       FLORIDA          12  1972       June    Tornado       C   \n",
       "1     990000002       FLORIDA          12  1972       June    Tornado       C   \n",
       "2     990000003       FLORIDA          12  1972       June    Tornado       C   \n",
       "3      10032307      Kentucky          21  1974      April    Tornado       C   \n",
       "4      10032301      Kentucky          21  1974      April    Tornado       C   \n",
       "...         ...           ...         ...   ...        ...        ...     ...   \n",
       "9995    5678048      OKLAHOMA          40  1998    October    Tornado       C   \n",
       "9996    5678049      OKLAHOMA          40  1998    October    Tornado       C   \n",
       "9997    5673341      NEBRASKA          31  1998   November  High Wind       Z   \n",
       "9998    5676260  PENNSYLVANIA          42  1998   November       Heat       Z   \n",
       "9999    5672925    NEW JERSEY          34  1998  September       Heat       Z   \n",
       "\n",
       "      CZ_FIPS     CZ_NAME   WFO  ...       END_DATE_TIME INJURIES_DIRECT  \\\n",
       "0          43      GLADES  None  ...  18-JUN-72 15:17:00               6   \n",
       "1          51      HENDRY  None  ...  18-JUN-72 15:13:00               1   \n",
       "2          93  OKEECHOBEE  None  ...  18-JUN-72 22:55:00              44   \n",
       "3         215     SPENCER  None  ...  03-APR-74 17:31:00               0   \n",
       "4          93      HARDIN  None  ...  03-APR-74 16:55:00              57   \n",
       "...       ...         ...   ...  ...                 ...             ...   \n",
       "9995        3     ALFALFA   OUN  ...  04-OCT-98 14:43:00               0   \n",
       "9996       11      BLAINE   OUN  ...  04-OCT-98 17:12:00               0   \n",
       "9997       77    FILLMORE   GID  ...  10-NOV-98 16:00:00               0   \n",
       "9998       67     CHESTER   PHI  ...  30-NOV-98 23:59:00               0   \n",
       "9999       18      CAMDEN   PHI  ...  27-SEP-98 18:00:00               0   \n",
       "\n",
       "      INJURIES_INDIRECT  DEATHS_DIRECT  DEATHS_INDIRECT  DAMAGE_PROPERTY  \\\n",
       "0                     0              0                0             190K   \n",
       "1                     0              1                0              10K   \n",
       "2                     0              6                0             500K   \n",
       "3                     0              0                0              25K   \n",
       "4                     0              2                0             250K   \n",
       "...                 ...            ...              ...              ...   \n",
       "9995                  0              0                0              55K   \n",
       "9996                  0              0                0              50K   \n",
       "9997                  0              0                0                0   \n",
       "9998                  0              0                0                0   \n",
       "9999                  0              0                0                0   \n",
       "\n",
       "     DAMAGE_CROPS                 SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0              5K           Manual Input                  190000.0   \n",
       "1              5K           Manual Input                   10000.0   \n",
       "2               0           Manual Input                  500000.0   \n",
       "3               0                    WFO                   25000.0   \n",
       "4               0                    WFO                  250000.0   \n",
       "...           ...                    ...                       ...   \n",
       "9995         None                UNKNOWN                   55000.0   \n",
       "9996         None                UNKNOWN                   50000.0   \n",
       "9997            0  AWOS,ASOS,MESONET,ETC                       0.0   \n",
       "9998            0  AWOS,ASOS,MESONET,ETC                       0.0   \n",
       "9999         None  AWOS,ASOS,MESONET,ETC                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_DOLLARS  \n",
       "0                               $190000  \n",
       "1                                $10000  \n",
       "2                               $500000  \n",
       "3                                $25000  \n",
       "4                               $250000  \n",
       "...                                 ...  \n",
       "9995                             $55000  \n",
       "9996                             $50000  \n",
       "9997                                 $0  \n",
       "9998                                 $0  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# dropna\n",
    "storm_events.dropna(subset=[\"SOURCE\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "98fe7993",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>EVENT_ID</th>\n",
       "      <th>STATE</th>\n",
       "      <th>STATE_FIPS</th>\n",
       "      <th>YEAR</th>\n",
       "      <th>MONTH_NAME</th>\n",
       "      <th>EVENT_TYPE</th>\n",
       "      <th>CZ_TYPE</th>\n",
       "      <th>CZ_FIPS</th>\n",
       "      <th>CZ_NAME</th>\n",
       "      <th>WFO</th>\n",
       "      <th>...</th>\n",
       "      <th>END_DATE_TIME</th>\n",
       "      <th>INJURIES_DIRECT</th>\n",
       "      <th>INJURIES_INDIRECT</th>\n",
       "      <th>DEATHS_DIRECT</th>\n",
       "      <th>DEATHS_INDIRECT</th>\n",
       "      <th>DAMAGE_PROPERTY</th>\n",
       "      <th>DAMAGE_CROPS</th>\n",
       "      <th>SOURCE</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED</th>\n",
       "      <th>DAMAGE_PROPERTY_CONVERTED_DOLLARS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10347267</td>\n",
       "      <td>SOUTH CAROLINA</td>\n",
       "      <td>45</td>\n",
       "      <td>1994</td>\n",
       "      <td>April</td>\n",
       "      <td>THUNDERSTORM WINDS/FLOODING</td>\n",
       "      <td>C</td>\n",
       "      <td>33</td>\n",
       "      <td>DILLON</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>15-APR-94 15:40:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>$5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10335845</td>\n",
       "      <td>NEVADA</td>\n",
       "      <td>32</td>\n",
       "      <td>1995</td>\n",
       "      <td>June</td>\n",
       "      <td>HAIL FLOODING</td>\n",
       "      <td>C</td>\n",
       "      <td>0</td>\n",
       "      <td>NVZ003 - 004</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>01-JUN-95 19:10:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10335846</td>\n",
       "      <td>NEVADA</td>\n",
       "      <td>32</td>\n",
       "      <td>1995</td>\n",
       "      <td>August</td>\n",
       "      <td>THUNDERSTORM WINDS/FLASH FLOOD</td>\n",
       "      <td>C</td>\n",
       "      <td>0</td>\n",
       "      <td>NVZ003 - 004</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>05-AUG-95 21:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10355544</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1995</td>\n",
       "      <td>April</td>\n",
       "      <td>THUNDERSTORM WINDS/ FLOOD</td>\n",
       "      <td>C</td>\n",
       "      <td>489</td>\n",
       "      <td>WILLACY</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>04-APR-95 19:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10K</td>\n",
       "      <td>0</td>\n",
       "      <td>None</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>$10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10355545</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>1995</td>\n",
       "      <td>April</td>\n",
       "      <td>THUNDERSTORM WINDS/ FLOOD</td>\n",
       "      <td>C</td>\n",
       "      <td>489</td>\n",
       "      <td>WILLACY</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>04-APR-95 20:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10K</td>\n",
       "      <td>30K</td>\n",
       "      <td>None</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>$10000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9995</th>\n",
       "      <td>5306778</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>2002</td>\n",
       "      <td>July</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>49</td>\n",
       "      <td>BROWN</td>\n",
       "      <td>SJT</td>\n",
       "      <td>...</td>\n",
       "      <td>03-JUL-02 11:30:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7K</td>\n",
       "      <td>None</td>\n",
       "      <td>LAW ENFORCEMENT</td>\n",
       "      <td>7000.0</td>\n",
       "      <td>$7000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9996</th>\n",
       "      <td>5307484</td>\n",
       "      <td>OREGON</td>\n",
       "      <td>41</td>\n",
       "      <td>2002</td>\n",
       "      <td>July</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>63</td>\n",
       "      <td>WALLOWA</td>\n",
       "      <td>PDT</td>\n",
       "      <td>...</td>\n",
       "      <td>19-JUL-02 16:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1K</td>\n",
       "      <td>None</td>\n",
       "      <td>GENERAL PUBLIC</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>$1000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9997</th>\n",
       "      <td>5306783</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>2002</td>\n",
       "      <td>July</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>83</td>\n",
       "      <td>COLEMAN</td>\n",
       "      <td>SJT</td>\n",
       "      <td>...</td>\n",
       "      <td>04-JUL-02 05:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4K</td>\n",
       "      <td>None</td>\n",
       "      <td>LAW ENFORCEMENT</td>\n",
       "      <td>4000.0</td>\n",
       "      <td>$4000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9998</th>\n",
       "      <td>5306523</td>\n",
       "      <td>ALABAMA</td>\n",
       "      <td>1</td>\n",
       "      <td>2002</td>\n",
       "      <td>July</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>73</td>\n",
       "      <td>JEFFERSON</td>\n",
       "      <td>BMX</td>\n",
       "      <td>...</td>\n",
       "      <td>12-JUL-02 22:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25K</td>\n",
       "      <td>0K</td>\n",
       "      <td>EMERGENCY MANAGER</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>$25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9999</th>\n",
       "      <td>5304042</td>\n",
       "      <td>TEXAS</td>\n",
       "      <td>48</td>\n",
       "      <td>2002</td>\n",
       "      <td>July</td>\n",
       "      <td>Flash Flood</td>\n",
       "      <td>C</td>\n",
       "      <td>371</td>\n",
       "      <td>PECOS</td>\n",
       "      <td>MAF</td>\n",
       "      <td>...</td>\n",
       "      <td>29-JUL-02 18:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>LAW ENFORCEMENT</td>\n",
       "      <td>0.0</td>\n",
       "      <td>$0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      EVENT_ID           STATE  STATE_FIPS  YEAR MONTH_NAME  \\\n",
       "0     10347267  SOUTH CAROLINA          45  1994      April   \n",
       "1     10335845          NEVADA          32  1995       June   \n",
       "2     10335846          NEVADA          32  1995     August   \n",
       "3     10355544           TEXAS          48  1995      April   \n",
       "4     10355545           TEXAS          48  1995      April   \n",
       "...        ...             ...         ...   ...        ...   \n",
       "9995   5306778           TEXAS          48  2002       July   \n",
       "9996   5307484          OREGON          41  2002       July   \n",
       "9997   5306783           TEXAS          48  2002       July   \n",
       "9998   5306523         ALABAMA           1  2002       July   \n",
       "9999   5304042           TEXAS          48  2002       July   \n",
       "\n",
       "                          EVENT_TYPE CZ_TYPE  CZ_FIPS       CZ_NAME   WFO  \\\n",
       "0        THUNDERSTORM WINDS/FLOODING       C       33        DILLON  None   \n",
       "1                      HAIL FLOODING       C        0  NVZ003 - 004  None   \n",
       "2     THUNDERSTORM WINDS/FLASH FLOOD       C        0  NVZ003 - 004  None   \n",
       "3          THUNDERSTORM WINDS/ FLOOD       C      489       WILLACY  None   \n",
       "4          THUNDERSTORM WINDS/ FLOOD       C      489       WILLACY  None   \n",
       "...                              ...     ...      ...           ...   ...   \n",
       "9995                     Flash Flood       C       49         BROWN   SJT   \n",
       "9996                     Flash Flood       C       63       WALLOWA   PDT   \n",
       "9997                     Flash Flood       C       83       COLEMAN   SJT   \n",
       "9998                     Flash Flood       C       73     JEFFERSON   BMX   \n",
       "9999                     Flash Flood       C      371         PECOS   MAF   \n",
       "\n",
       "      ...       END_DATE_TIME INJURIES_DIRECT  INJURIES_INDIRECT  \\\n",
       "0     ...  15-APR-94 15:40:00               0                  0   \n",
       "1     ...  01-JUN-95 19:10:00               0                  0   \n",
       "2     ...  05-AUG-95 21:00:00               0                  0   \n",
       "3     ...  04-APR-95 19:30:00               0                  0   \n",
       "4     ...  04-APR-95 20:30:00               0                  0   \n",
       "...   ...                 ...             ...                ...   \n",
       "9995  ...  03-JUL-02 11:30:00               0                  0   \n",
       "9996  ...  19-JUL-02 16:00:00               0                  0   \n",
       "9997  ...  04-JUL-02 05:00:00               0                  0   \n",
       "9998  ...  12-JUL-02 22:00:00               0                  0   \n",
       "9999  ...  29-JUL-02 18:00:00               0                  0   \n",
       "\n",
       "      DEATHS_DIRECT  DEATHS_INDIRECT  DAMAGE_PROPERTY DAMAGE_CROPS  \\\n",
       "0                 0                0               5K            0   \n",
       "1                 0                0                0            0   \n",
       "2                 0                0                0            0   \n",
       "3                 0                0              10K            0   \n",
       "4                 0                0              10K          30K   \n",
       "...             ...              ...              ...          ...   \n",
       "9995              0                0               7K         None   \n",
       "9996              0                0               1K         None   \n",
       "9997              0                0               4K         None   \n",
       "9998              0                0              25K           0K   \n",
       "9999              0                0                0            0   \n",
       "\n",
       "                 SOURCE DAMAGE_PROPERTY_CONVERTED  \\\n",
       "0                  None                    5000.0   \n",
       "1                  None                       0.0   \n",
       "2                  None                       0.0   \n",
       "3                  None                   10000.0   \n",
       "4                  None                   10000.0   \n",
       "...                 ...                       ...   \n",
       "9995    LAW ENFORCEMENT                    7000.0   \n",
       "9996     GENERAL PUBLIC                    1000.0   \n",
       "9997    LAW ENFORCEMENT                    4000.0   \n",
       "9998  EMERGENCY MANAGER                   25000.0   \n",
       "9999    LAW ENFORCEMENT                       0.0   \n",
       "\n",
       "      DAMAGE_PROPERTY_CONVERTED_DOLLARS  \n",
       "0                                 $5000  \n",
       "1                                    $0  \n",
       "2                                    $0  \n",
       "3                                $10000  \n",
       "4                                $10000  \n",
       "...                                 ...  \n",
       "9995                              $7000  \n",
       "9996                              $1000  \n",
       "9997                              $4000  \n",
       "9998                             $25000  \n",
       "9999                                 $0  \n",
       "\n",
       "[10000 rows x 21 columns]"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# contains\n",
    "storm_events[storm_events.EVENT_TYPE.lower().contains(\"flood\")]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31e4e751",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}