07-Back_to_Pandas.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "vSh_WANmJ1m4"
   },
   "source": [
    "# 7. Back to Pandas\n",
    "\n",
    "We have started this course by importing a DataFrame, briefly exploring it and discovering that Numpy was underlying these structures. Then we explored in more details those Numpy arrays. Now we come back to these DataFrames and will see that we can re-use many concepts from Numpy in Pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "executionInfo": {
     "elapsed": 768,
     "status": "ok",
     "timestamp": 1614506291468,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "wdZH4KjvKLtf",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "grv20Sf8KNav"
   },
   "source": [
    "## Creating Dataframes\n",
    "\n",
    "We have seen that we could extract Numpy arrays from a DataFrame, but we can also do the reverse: create a DataFrame from an array. While most of the time you will import data as DataFrames, sometimes it is useful to explicitly create an DataFrame.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vxTdWH6jNxgo"
   },
   "source": [
    "### From a Numpy array\n",
    "\n",
    "We know how to create a 2D Numpy array, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1267,
     "status": "ok",
     "timestamp": 1614506291982,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "cTZOfguLL3Pn",
    "outputId": "590ba508-42d4-4efc-8cd6-8342f4803b4b",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[69,  0,  1, 91, 15],\n",
       "       [68, 89, 69, 58, 50],\n",
       "       [28, 94, 55, 40, 63]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_array = np.random.randint(0, 100, ((3,5)))\n",
    "my_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t0BGSLgGL97P"
   },
   "source": [
    "We can simply transform this array into a DataFrame using the ```pd.DataFrame``` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1256,
     "status": "ok",
     "timestamp": 1614506291983,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "HD3BmoYRMD4g",
    "outputId": "4147f038-f550-4793-d6fe-09a731687f87",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>68</td>\n",
       "      <td>89</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    0   1   2   3   4\n",
       "0  69   0   1  91  15\n",
       "1  68  89  69  58  50\n",
       "2  28  94  55  40  63"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df = pd.DataFrame(my_array)\n",
    "my_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "grZCtQAjUPum"
   },
   "source": [
    "We will see that many of the operations that we have used on Numpy arrays before also work for Pandas. As a start, you can for example get the shape of a DataFrame: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1245,
     "status": "ok",
     "timestamp": 1614506291984,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "RBWRzpD4UeXh",
    "outputId": "f863a9c1-5302-40a5-9dd3-d872f3a231cc",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 5)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FeowNph9MG_J"
   },
   "source": [
    "By default, Pandas uses a numerical index for each row and each column, but if we want we can override this by explicitly providing lists for the ```index``` and ```columns``` options."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1232,
     "status": "ok",
     "timestamp": 1614506291984,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "L9y7bQGwMY07",
    "outputId": "ec5e8e7e-3827-4478-c5ca-019f69ec2d95",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col A</th>\n",
       "      <th>col B</th>\n",
       "      <th>col C</th>\n",
       "      <th>col D</th>\n",
       "      <th>col E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>row a</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>row b</th>\n",
       "      <td>68</td>\n",
       "      <td>89</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>row c</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       col A  col B  col C  col D  col E\n",
       "row a     69      0      1     91     15\n",
       "row b     68     89     69     58     50\n",
       "row c     28     94     55     40     63"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df = pd.DataFrame(my_array, index=['row a','row b','row c'], columns=['col A', 'col B', 'col C','col D','col E'])\n",
    "my_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q8JCPd89MoZM"
   },
   "source": [
    "### From a dictionary\n",
    "\n",
    "There are multiple ways to create DataFrames. One of the most \"natural\" ones is to transform a **dictionary of lists** into an array. Here each element of the dictionary defines a column. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "executionInfo": {
     "elapsed": 1227,
     "status": "ok",
     "timestamp": 1614506291985,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "cictoB8INfDi",
    "tags": []
   },
   "outputs": [],
   "source": [
    "dict_of_list = {\n",
    "    'birth': [1860, 1770, 1858, 1906],\n",
    "    'death':[1911, 1827, 1924, 1975], \n",
    "    'city':['Kaliste', 'Bonn', 'Lucques', 'Saint-Petersburg']\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 173
    },
    "executionInfo": {
     "elapsed": 1215,
     "status": "ok",
     "timestamp": 1614506291985,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "0eKb093JNnBa",
    "outputId": "ca59e6b6-4bd6-4c90-a506-90191cff3f61",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   birth  death              city\n",
       "0   1860   1911           Kaliste\n",
       "1   1770   1827              Bonn\n",
       "2   1858   1924           Lucques\n",
       "3   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(dict_of_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nJbgsCJ6Nprz"
   },
   "source": [
    "## Index and column\n",
    "\n",
    "One of the main differences between Numpy arrays and Pandas DataFrames is that the elements in the table are not located by their \"numerical position\" but by their corresponding index and column. These two elements are in fact just list of values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1203,
     "status": "ok",
     "timestamp": 1614506291986,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "uGRwTSzUOpHe",
    "outputId": "8bc8fc12-0e68-4ed2-a682-876e27e86f48",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['row a', 'row b', 'row c'], dtype='object')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1192,
     "status": "ok",
     "timestamp": 1614506291986,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "EV4IxYpRQqwm",
    "outputId": "4787b80d-2abe-4432-e412-03621b08d7f1",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['col A', 'col B', 'col C', 'col D', 'col E'], dtype='object')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CVMDzJlXOr6h"
   },
   "source": [
    "### Changing indices\n",
    "One can change these properties of the DataFrame directly by reassigning a new list. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1180,
     "status": "ok",
     "timestamp": 1614506291987,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "PyIUHd2zQ4OS",
    "outputId": "6aaa5564-6539-42e0-b3b7-37911b41e8e3",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>new A</th>\n",
       "      <th>new B</th>\n",
       "      <th>new C</th>\n",
       "      <th>new D</th>\n",
       "      <th>new E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>x</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>y</th>\n",
       "      <td>68</td>\n",
       "      <td>89</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>z</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   new A  new B  new C  new D  new E\n",
       "x     69      0      1     91     15\n",
       "y     68     89     69     58     50\n",
       "z     28     94     55     40     63"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df.index = np.array(['x','y','z'])\n",
    "my_df.columns = np.array(['new A','new B','new C','new D','new E'])\n",
    "my_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RB5HhNlDRODn"
   },
   "source": [
    "Alternatively, one can use dedicated functions that give us more flexibility. For example we can change just one column or index name by using a dictionary old -> new and the ```.rename()``` method. We also have to specify which ```axis``` we want to rename, either the ```'columns'``` or the ```'rows'```:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1167,
     "status": "ok",
     "timestamp": 1614506291987,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "NHKd5z2lRbe-",
    "outputId": "9582b528-c269-41df-9ad8-3cac6101e2ec",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>re-name A</th>\n",
       "      <th>new B</th>\n",
       "      <th>new C</th>\n",
       "      <th>new D</th>\n",
       "      <th>new E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>x</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>y</th>\n",
       "      <td>68</td>\n",
       "      <td>89</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>z</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   re-name A  new B  new C  new D  new E\n",
       "x         69      0      1     91     15\n",
       "y         68     89     69     58     50\n",
       "z         28     94     55     40     63"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df = my_df.rename({'new A': 're-name A'},axis='columns')\n",
    "my_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1137,
     "status": "ok",
     "timestamp": 1614506291988,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "IKc0NwUzPBSg",
    "outputId": "7990e9c8-dbef-4dae-bf5f-709e76c3707b",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>re-name A</th>\n",
       "      <th>new B</th>\n",
       "      <th>new C</th>\n",
       "      <th>new D</th>\n",
       "      <th>new E</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>new x</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>y</th>\n",
       "      <td>68</td>\n",
       "      <td>89</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>z</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       re-name A  new B  new C  new D  new E\n",
       "new x         69      0      1     91     15\n",
       "y             68     89     69     58     50\n",
       "z             28     94     55     40     63"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df = my_df.rename({'x': 'new x'},axis='rows')\n",
    "my_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "y7Ji98iJPLIy"
   },
   "source": [
    "### Adding columns\n",
    "\n",
    "It is also very easy to add a new column to an existing DataFrame. For that you just assign a list or a single value to a not yet existing column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1482,
     "status": "ok",
     "timestamp": 1614506292347,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "6uu7G5ZGPVk8",
    "outputId": "a4e46a85-2ef9-414f-808b-404d9538b6df",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>re-name A</th>\n",
       "      <th>new B</th>\n",
       "      <th>new C</th>\n",
       "      <th>new D</th>\n",
       "      <th>new E</th>\n",
       "      <th>new column</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>new x</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>y</th>\n",
       "      <td>68</td>\n",
       "      <td>89</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>z</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       re-name A  new B  new C  new D  new E  new column\n",
       "new x         69      0      1     91     15           1\n",
       "y             68     89     69     58     50           1\n",
       "z             28     94     55     40     63           1"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df['new column'] = 1\n",
    "my_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Rfr0MhPRPgvc"
   },
   "source": [
    "We have seen before that DataFrame can contain different types of data, so we can also fill a column with text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1469,
     "status": "ok",
     "timestamp": 1614506292349,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "y9y9fH2IP_I-",
    "outputId": "0271f7f5-1d32-42aa-858e-b3919b9828a1",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>re-name A</th>\n",
       "      <th>new B</th>\n",
       "      <th>new C</th>\n",
       "      <th>new D</th>\n",
       "      <th>new E</th>\n",
       "      <th>new column</th>\n",
       "      <th>text column</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>new x</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>y</th>\n",
       "      <td>68</td>\n",
       "      <td>89</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>z</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       re-name A  new B  new C  new D  new E  new column text column\n",
       "new x         69      0      1     91     15           1           a\n",
       "y             68     89     69     58     50           1           b\n",
       "z             28     94     55     40     63           1           c"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df['text column'] = ['a', 'b', 'c']\n",
    "my_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AF3dSFFxRaGV"
   },
   "source": [
    "### Droping items\n",
    "Inversely we can remove columns by using the ```drop()``` function. Here also we have to say with ```axis``` if we want to affect ```'rows'``` or ```'columns'```:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 1457,
     "status": "ok",
     "timestamp": 1614506292351,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "ey3103aeS6qB",
    "outputId": "a568b9e0-7ec8-4b82-905d-8594c9619323",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>re-name A</th>\n",
       "      <th>new C</th>\n",
       "      <th>new D</th>\n",
       "      <th>new E</th>\n",
       "      <th>new column</th>\n",
       "      <th>text column</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>new x</th>\n",
       "      <td>69</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>y</th>\n",
       "      <td>68</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>z</th>\n",
       "      <td>28</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       re-name A  new C  new D  new E  new column text column\n",
       "new x         69      1     91     15           1           a\n",
       "y             68     69     58     50           1           b\n",
       "z             28     55     40     63           1           c"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df.drop(labels='new B', axis='columns')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 111
    },
    "executionInfo": {
     "elapsed": 1444,
     "status": "ok",
     "timestamp": 1614506292352,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "fAv6nXylTF6y",
    "outputId": "06bca125-16f8-4c83-f7af-746e435a06bd",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>re-name A</th>\n",
       "      <th>new B</th>\n",
       "      <th>new C</th>\n",
       "      <th>new D</th>\n",
       "      <th>new E</th>\n",
       "      <th>new column</th>\n",
       "      <th>text column</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>new x</th>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>15</td>\n",
       "      <td>1</td>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>z</th>\n",
       "      <td>28</td>\n",
       "      <td>94</td>\n",
       "      <td>55</td>\n",
       "      <td>40</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       re-name A  new B  new C  new D  new E  new column text column\n",
       "new x         69      0      1     91     15           1           a\n",
       "z             28     94     55     40     63           1           c"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_df.drop(labels='y', axis='rows')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1p_8i8DnoCDb"
   },
   "source": [
    "## Importing data\n",
    "\n",
    "Until now we have always used the ```reader_XXX``` functions with default options. However these functions have many options, in particular regarding the choice of indices and column names. For example, we can specify if we want to use **one of the columns as index** when importing data. Each data format will present its own challenges, and we present here some core concepts.\n",
    "\n",
    "Here we are using the dataset that you can find [here](https://github.com/guiwitz/NumpyPandas_course/blob/master/Data/composers.xlsx). It is a simple collection of Excel sheet with information regarding classical composers. These simple and rather short dataset will allow us to explore many of the problems that you might encounter with \"real\" datasets.\n",
    "\n",
    "First we need a new importer function for Excel:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 173
    },
    "executionInfo": {
     "elapsed": 2652,
     "status": "ok",
     "timestamp": 1614506293582,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "fEE5cD20qkhE",
    "outputId": "f0a661cf-988f-4ee5-c5c0-f420b25e2650",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shostakovich</td>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       composer  birth  death              city\n",
       "0        Mahler   1860   1911           Kaliste\n",
       "1     Beethoven   1770   1827              Bonn\n",
       "2       Puccini   1858   1924           Lucques\n",
       "3  Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_url = 'https://github.com/guiwitz/NumpyPandas_course/blob/master/Data/composers.xlsx?raw=true'\n",
    "composers = pd.read_excel(file_url)\n",
    "composers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5PtIZ7x6qrjU"
   },
   "source": [
    "### Setting indices\n",
    "Again, by default Pandas sets a numerical index. However here each composer name is unique, so we could use that as an index. We can specify this to our importer with the ```index_col``` option:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "executionInfo": {
     "elapsed": 2638,
     "status": "ok",
     "timestamp": 1614506293583,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "eB-tf_Caq89E",
    "outputId": "42436a62-0f28-479a-b980-d49a18a9bb40",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "composer                                    \n",
       "Mahler         1860   1911           Kaliste\n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers = pd.read_excel(file_url, index_col='composer')\n",
    "composers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lnuXfd0GrRpL"
   },
   "source": [
    "### Skipping rows\n",
    "\n",
    "Instead of dropping certains rows at the start or end of the table, we can also directly do that at import time by specifying the ```skipfooter```  and ```skiprows``` arguments. Let's for example remove the first line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 324
    },
    "executionInfo": {
     "elapsed": 3447,
     "status": "error",
     "timestamp": 1614506294409,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "0h72CInwrhza",
    "outputId": "972c53b3-0a2b-4426-d90c-a36e087cdf03",
    "tags": []
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Index composer invalid",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[19], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m composers \u001b[38;5;241m=\u001b[39m \u001b[43mpd\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread_excel\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfile_url\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mindex_col\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mcomposer\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mskiprows\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m0\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m      2\u001b[0m composers\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/util/_decorators.py:211\u001b[0m, in \u001b[0;36mdeprecate_kwarg.<locals>._deprecate_kwarg.<locals>.wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    209\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    210\u001b[0m         kwargs[new_arg_name] \u001b[38;5;241m=\u001b[39m new_arg_value\n\u001b[0;32m--> 211\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/util/_decorators.py:331\u001b[0m, in \u001b[0;36mdeprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    325\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(args) \u001b[38;5;241m>\u001b[39m num_allow_args:\n\u001b[1;32m    326\u001b[0m     warnings\u001b[38;5;241m.\u001b[39mwarn(\n\u001b[1;32m    327\u001b[0m         msg\u001b[38;5;241m.\u001b[39mformat(arguments\u001b[38;5;241m=\u001b[39m_format_argument_list(allow_args)),\n\u001b[1;32m    328\u001b[0m         \u001b[38;5;167;01mFutureWarning\u001b[39;00m,\n\u001b[1;32m    329\u001b[0m         stacklevel\u001b[38;5;241m=\u001b[39mfind_stack_level(),\n\u001b[1;32m    330\u001b[0m     )\n\u001b[0;32m--> 331\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/excel/_base.py:490\u001b[0m, in \u001b[0;36mread_excel\u001b[0;34m(io, sheet_name, header, names, index_col, usecols, squeeze, dtype, engine, converters, true_values, false_values, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, parse_dates, date_parser, thousands, decimal, comment, skipfooter, convert_float, mangle_dupe_cols, storage_options)\u001b[0m\n\u001b[1;32m    484\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[1;32m    485\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mEngine should not be specified when passing \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    486\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124man ExcelFile - ExcelFile already has the engine set\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    487\u001b[0m     )\n\u001b[1;32m    489\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 490\u001b[0m     data \u001b[38;5;241m=\u001b[39m \u001b[43mio\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    491\u001b[0m \u001b[43m        \u001b[49m\u001b[43msheet_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msheet_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    492\u001b[0m \u001b[43m        \u001b[49m\u001b[43mheader\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mheader\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    493\u001b[0m \u001b[43m        \u001b[49m\u001b[43mnames\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mnames\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    494\u001b[0m \u001b[43m        \u001b[49m\u001b[43mindex_col\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mindex_col\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    495\u001b[0m \u001b[43m        \u001b[49m\u001b[43musecols\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43musecols\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    496\u001b[0m \u001b[43m        \u001b[49m\u001b[43msqueeze\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msqueeze\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    497\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdtype\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdtype\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    498\u001b[0m \u001b[43m        \u001b[49m\u001b[43mconverters\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconverters\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    499\u001b[0m \u001b[43m        \u001b[49m\u001b[43mtrue_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtrue_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    500\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfalse_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfalse_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    501\u001b[0m \u001b[43m        \u001b[49m\u001b[43mskiprows\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mskiprows\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    502\u001b[0m \u001b[43m        \u001b[49m\u001b[43mnrows\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mnrows\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    503\u001b[0m \u001b[43m        \u001b[49m\u001b[43mna_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mna_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    504\u001b[0m \u001b[43m        \u001b[49m\u001b[43mkeep_default_na\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mkeep_default_na\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    505\u001b[0m \u001b[43m        \u001b[49m\u001b[43mna_filter\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mna_filter\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    506\u001b[0m \u001b[43m        \u001b[49m\u001b[43mverbose\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mverbose\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    507\u001b[0m \u001b[43m        \u001b[49m\u001b[43mparse_dates\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mparse_dates\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    508\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdate_parser\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdate_parser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    509\u001b[0m \u001b[43m        \u001b[49m\u001b[43mthousands\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mthousands\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    510\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdecimal\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdecimal\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    511\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcomment\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcomment\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    512\u001b[0m \u001b[43m        \u001b[49m\u001b[43mskipfooter\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mskipfooter\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    513\u001b[0m \u001b[43m        \u001b[49m\u001b[43mconvert_float\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconvert_float\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    514\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmangle_dupe_cols\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmangle_dupe_cols\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    515\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    516\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m    517\u001b[0m     \u001b[38;5;66;03m# make sure to close opened file handles\u001b[39;00m\n\u001b[1;32m    518\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m should_close:\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/excel/_base.py:1734\u001b[0m, in \u001b[0;36mExcelFile.parse\u001b[0;34m(self, sheet_name, header, names, index_col, usecols, squeeze, converters, true_values, false_values, skiprows, nrows, na_values, parse_dates, date_parser, thousands, comment, skipfooter, convert_float, mangle_dupe_cols, **kwds)\u001b[0m\n\u001b[1;32m   1700\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mparse\u001b[39m(\n\u001b[1;32m   1701\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m   1702\u001b[0m     sheet_name: \u001b[38;5;28mstr\u001b[39m \u001b[38;5;241m|\u001b[39m \u001b[38;5;28mint\u001b[39m \u001b[38;5;241m|\u001b[39m \u001b[38;5;28mlist\u001b[39m[\u001b[38;5;28mint\u001b[39m] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28mlist\u001b[39m[\u001b[38;5;28mstr\u001b[39m] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1721\u001b[0m     \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwds,\n\u001b[1;32m   1722\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m DataFrame \u001b[38;5;241m|\u001b[39m \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mstr\u001b[39m, DataFrame] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28mdict\u001b[39m[\u001b[38;5;28mint\u001b[39m, DataFrame]:\n\u001b[1;32m   1723\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[1;32m   1724\u001b[0m \u001b[38;5;124;03m    Parse specified sheet(s) into a DataFrame.\u001b[39;00m\n\u001b[1;32m   1725\u001b[0m \n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1732\u001b[0m \u001b[38;5;124;03m        DataFrame from the passed in Excel file.\u001b[39;00m\n\u001b[1;32m   1733\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[0;32m-> 1734\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_reader\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mparse\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m   1735\u001b[0m \u001b[43m        \u001b[49m\u001b[43msheet_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msheet_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1736\u001b[0m \u001b[43m        \u001b[49m\u001b[43mheader\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mheader\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1737\u001b[0m \u001b[43m        \u001b[49m\u001b[43mnames\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mnames\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1738\u001b[0m \u001b[43m        \u001b[49m\u001b[43mindex_col\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mindex_col\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1739\u001b[0m \u001b[43m        \u001b[49m\u001b[43musecols\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43musecols\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1740\u001b[0m \u001b[43m        \u001b[49m\u001b[43msqueeze\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43msqueeze\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1741\u001b[0m \u001b[43m        \u001b[49m\u001b[43mconverters\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconverters\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1742\u001b[0m \u001b[43m        \u001b[49m\u001b[43mtrue_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtrue_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1743\u001b[0m \u001b[43m        \u001b[49m\u001b[43mfalse_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfalse_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1744\u001b[0m \u001b[43m        \u001b[49m\u001b[43mskiprows\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mskiprows\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1745\u001b[0m \u001b[43m        \u001b[49m\u001b[43mnrows\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mnrows\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1746\u001b[0m \u001b[43m        \u001b[49m\u001b[43mna_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mna_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1747\u001b[0m \u001b[43m        \u001b[49m\u001b[43mparse_dates\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mparse_dates\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1748\u001b[0m \u001b[43m        \u001b[49m\u001b[43mdate_parser\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdate_parser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1749\u001b[0m \u001b[43m        \u001b[49m\u001b[43mthousands\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mthousands\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1750\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcomment\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcomment\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1751\u001b[0m \u001b[43m        \u001b[49m\u001b[43mskipfooter\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mskipfooter\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1752\u001b[0m \u001b[43m        \u001b[49m\u001b[43mconvert_float\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconvert_float\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1753\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmangle_dupe_cols\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmangle_dupe_cols\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1754\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m   1755\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/excel/_base.py:891\u001b[0m, in \u001b[0;36mBaseExcelReader.parse\u001b[0;34m(self, sheet_name, header, names, index_col, usecols, squeeze, dtype, true_values, false_values, skiprows, nrows, na_values, verbose, parse_dates, date_parser, thousands, decimal, comment, skipfooter, convert_float, mangle_dupe_cols, **kwds)\u001b[0m\n\u001b[1;32m    865\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m    866\u001b[0m     parser \u001b[38;5;241m=\u001b[39m TextParser(\n\u001b[1;32m    867\u001b[0m         data,\n\u001b[1;32m    868\u001b[0m         names\u001b[38;5;241m=\u001b[39mnames,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwds,\n\u001b[1;32m    889\u001b[0m     )\n\u001b[0;32m--> 891\u001b[0m     output[asheetname] \u001b[38;5;241m=\u001b[39m \u001b[43mparser\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread\u001b[49m\u001b[43m(\u001b[49m\u001b[43mnrows\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mnrows\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    893\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m squeeze \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(output[asheetname], DataFrame):\n\u001b[1;32m    894\u001b[0m         \u001b[38;5;28;01mif\u001b[39;00m header_names:\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/parsers/readers.py:1778\u001b[0m, in \u001b[0;36mTextFileReader.read\u001b[0;34m(self, nrows)\u001b[0m\n\u001b[1;32m   1771\u001b[0m nrows \u001b[38;5;241m=\u001b[39m validate_integer(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mnrows\u001b[39m\u001b[38;5;124m\"\u001b[39m, nrows)\n\u001b[1;32m   1772\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m   1773\u001b[0m     \u001b[38;5;66;03m# error: \"ParserBase\" has no attribute \"read\"\u001b[39;00m\n\u001b[1;32m   1774\u001b[0m     (\n\u001b[1;32m   1775\u001b[0m         index,\n\u001b[1;32m   1776\u001b[0m         columns,\n\u001b[1;32m   1777\u001b[0m         col_dict,\n\u001b[0;32m-> 1778\u001b[0m     ) \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_engine\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mread\u001b[49m\u001b[43m(\u001b[49m\u001b[43m  \u001b[49m\u001b[38;5;66;43;03m# type: ignore[attr-defined]\u001b[39;49;00m\n\u001b[1;32m   1779\u001b[0m \u001b[43m        \u001b[49m\u001b[43mnrows\u001b[49m\n\u001b[1;32m   1780\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m   1781\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m:\n\u001b[1;32m   1782\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mclose()\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/parsers/python_parser.py:288\u001b[0m, in \u001b[0;36mPythonParser.read\u001b[0;34m(self, rows)\u001b[0m\n\u001b[1;32m    285\u001b[0m conv_data \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_convert_data(data)\n\u001b[1;32m    286\u001b[0m columns, conv_data \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_do_date_conversions(columns, conv_data)\n\u001b[0;32m--> 288\u001b[0m index, result_columns \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_make_index\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    289\u001b[0m \u001b[43m    \u001b[49m\u001b[43mconv_data\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43malldata\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcolumns\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mindexnamerow\u001b[49m\n\u001b[1;32m    290\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    292\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m index, result_columns, conv_data\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py:379\u001b[0m, in \u001b[0;36mParserBase._make_index\u001b[0;34m(self, data, alldata, columns, indexnamerow)\u001b[0m\n\u001b[1;32m    376\u001b[0m     index \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    378\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_has_complex_date_col:\n\u001b[0;32m--> 379\u001b[0m     simple_index \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_get_simple_index\u001b[49m\u001b[43m(\u001b[49m\u001b[43malldata\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcolumns\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    380\u001b[0m     index \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_agg_index(simple_index)\n\u001b[1;32m    381\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_has_complex_date_col:\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py:411\u001b[0m, in \u001b[0;36mParserBase._get_simple_index\u001b[0;34m(self, data, columns)\u001b[0m\n\u001b[1;32m    409\u001b[0m index \u001b[38;5;241m=\u001b[39m []\n\u001b[1;32m    410\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m idx \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mindex_col:\n\u001b[0;32m--> 411\u001b[0m     i \u001b[38;5;241m=\u001b[39m \u001b[43mix\u001b[49m\u001b[43m(\u001b[49m\u001b[43midx\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    412\u001b[0m     to_remove\u001b[38;5;241m.\u001b[39mappend(i)\n\u001b[1;32m    413\u001b[0m     index\u001b[38;5;241m.\u001b[39mappend(data[i])\n",
      "File \u001b[0;32m~/mambaforge/envs/DAVPy2023/lib/python3.10/site-packages/pandas/io/parsers/base_parser.py:406\u001b[0m, in \u001b[0;36mParserBase._get_simple_index.<locals>.ix\u001b[0;34m(col)\u001b[0m\n\u001b[1;32m    404\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(col, \u001b[38;5;28mstr\u001b[39m):\n\u001b[1;32m    405\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m col\n\u001b[0;32m--> 406\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mIndex \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mcol\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m invalid\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "\u001b[0;31mValueError\u001b[0m: Index composer invalid"
     ]
    }
   ],
   "source": [
    "composers = pd.read_excel(file_url, index_col='composer', skiprows=[0])\n",
    "composers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZRi5mq4CrxbO"
   },
   "source": [
    "We ran into trouble, because we skipped the first row which in reality is just the row with the column names! To skip the actual first row we need:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 173
    },
    "executionInfo": {
     "elapsed": 986,
     "status": "ok",
     "timestamp": 1614506327136,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "JVicv0uVr_A-",
    "outputId": "eec7bb45-79ff-47e6-b5be-613608bf3a0f",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "composer                                    \n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers = pd.read_excel(file_url, index_col='composer', skiprows=[1])\n",
    "composers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B_YZyZausb5r"
   },
   "source": [
    "We can also specify a list of columns that we actually want to use with the ```usecols``` option. For example we could skip the ```city```:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "executionInfo": {
     "elapsed": 972,
     "status": "ok",
     "timestamp": 1614506327988,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "bfmXQWjgsw2X",
    "outputId": "0fa843f0-0168-411f-d1de-6430d2ceec00",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death\n",
       "composer                  \n",
       "Mahler         1860   1911\n",
       "Beethoven      1770   1827\n",
       "Puccini        1858   1924\n",
       "Shostakovich   1906   1975"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers = pd.read_excel(file_url, index_col='composer', usecols=['composer','birth', 'death'])\n",
    "composers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y_afnfGys5Rv"
   },
   "source": [
    "### Format-specific options \n",
    "Finally, for each importer, we have specific options only available in that format. For example with the Excel importer we can specify sheets to import. By default it imports the first one, but we can also indicate the position of the sheet or its name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "executionInfo": {
     "elapsed": 993,
     "status": "ok",
     "timestamp": 1614506332289,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "vtZwADgbtJhG",
    "outputId": "371dae32-bfd0-4e1a-ec84-a79b8bd5d4bf",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>period</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911.0</td>\n",
       "      <td>post-romantic</td>\n",
       "      <td>Austria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>1770</td>\n",
       "      <td>1827.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>1858</td>\n",
       "      <td>1924.0</td>\n",
       "      <td>post-romantic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shostakovich</td>\n",
       "      <td>1906</td>\n",
       "      <td>1975.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>Russia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Verdi</td>\n",
       "      <td>1813</td>\n",
       "      <td>1901.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Dvorak</td>\n",
       "      <td>1841</td>\n",
       "      <td>1904.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Czechia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Schumann</td>\n",
       "      <td>1810</td>\n",
       "      <td>1856.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Stravinsky</td>\n",
       "      <td>1882</td>\n",
       "      <td>1971.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>Russia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Sibelius</td>\n",
       "      <td>1865</td>\n",
       "      <td>1957.0</td>\n",
       "      <td>post-romantic</td>\n",
       "      <td>Finland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Haydn</td>\n",
       "      <td>1732</td>\n",
       "      <td>1809.0</td>\n",
       "      <td>classic</td>\n",
       "      <td>Austria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Mozart</td>\n",
       "      <td>1756</td>\n",
       "      <td>1791.0</td>\n",
       "      <td>classic</td>\n",
       "      <td>Austria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Messiaen</td>\n",
       "      <td>1908</td>\n",
       "      <td>1992.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Prokofiev</td>\n",
       "      <td>1891</td>\n",
       "      <td>1953.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>RUssia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Monteverdi</td>\n",
       "      <td>1567</td>\n",
       "      <td>1643.0</td>\n",
       "      <td>renaissance</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Haendel</td>\n",
       "      <td>1685</td>\n",
       "      <td>1759.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Brahms</td>\n",
       "      <td>1833</td>\n",
       "      <td>1897.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Purcell</td>\n",
       "      <td>1659</td>\n",
       "      <td>1695.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>England</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Charpentier</td>\n",
       "      <td>1643</td>\n",
       "      <td>1704.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Bruckner</td>\n",
       "      <td>1824</td>\n",
       "      <td>1896.0</td>\n",
       "      <td>post-romantic</td>\n",
       "      <td>Austria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Berg</td>\n",
       "      <td>1885</td>\n",
       "      <td>1935.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>Austria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Couperin</td>\n",
       "      <td>1626</td>\n",
       "      <td>1661.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Rameau</td>\n",
       "      <td>1683</td>\n",
       "      <td>1764.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Berlioz</td>\n",
       "      <td>1803</td>\n",
       "      <td>1869.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Gounod</td>\n",
       "      <td>1818</td>\n",
       "      <td>1893.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Massenet</td>\n",
       "      <td>1842</td>\n",
       "      <td>1912.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Boulez</td>\n",
       "      <td>1925</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Palestrina</td>\n",
       "      <td>1525</td>\n",
       "      <td>1594.0</td>\n",
       "      <td>renaissance</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Gesualdo</td>\n",
       "      <td>1566</td>\n",
       "      <td>1613.0</td>\n",
       "      <td>renaissance</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Caldara</td>\n",
       "      <td>1670</td>\n",
       "      <td>1736.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Pergolesi</td>\n",
       "      <td>1710</td>\n",
       "      <td>1736.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>Scarlatti</td>\n",
       "      <td>1685</td>\n",
       "      <td>1757.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>Caccini</td>\n",
       "      <td>1587</td>\n",
       "      <td>1640.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Cimarosa</td>\n",
       "      <td>1749</td>\n",
       "      <td>1801.0</td>\n",
       "      <td>classic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>Donizetti</td>\n",
       "      <td>1797</td>\n",
       "      <td>1848.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>Leoncavallo</td>\n",
       "      <td>1858</td>\n",
       "      <td>1919.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>Bellini</td>\n",
       "      <td>1801</td>\n",
       "      <td>1835.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>Dufay</td>\n",
       "      <td>1397</td>\n",
       "      <td>1474.0</td>\n",
       "      <td>renaissance</td>\n",
       "      <td>Belgium</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>Lassus</td>\n",
       "      <td>1532</td>\n",
       "      <td>1594.0</td>\n",
       "      <td>renaissance</td>\n",
       "      <td>Belgium</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Borodin</td>\n",
       "      <td>1833</td>\n",
       "      <td>1887.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Russia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>Mussorsgsky</td>\n",
       "      <td>1839</td>\n",
       "      <td>1881.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Russia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>Soler</td>\n",
       "      <td>1754</td>\n",
       "      <td>1806.0</td>\n",
       "      <td>classic</td>\n",
       "      <td>Spain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>Albeniz</td>\n",
       "      <td>1860</td>\n",
       "      <td>1909.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Spain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>Granados</td>\n",
       "      <td>1867</td>\n",
       "      <td>1916.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Spain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>Dowland</td>\n",
       "      <td>1563</td>\n",
       "      <td>1626.0</td>\n",
       "      <td>renaissance</td>\n",
       "      <td>England</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>Byrd</td>\n",
       "      <td>1540</td>\n",
       "      <td>1623.0</td>\n",
       "      <td>renaissance</td>\n",
       "      <td>England</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>Walton</td>\n",
       "      <td>1902</td>\n",
       "      <td>1983.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>England</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>Adès</td>\n",
       "      <td>1971</td>\n",
       "      <td>NaN</td>\n",
       "      <td>modern</td>\n",
       "      <td>England</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>Bach</td>\n",
       "      <td>1685</td>\n",
       "      <td>1750.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>Wagner</td>\n",
       "      <td>1813</td>\n",
       "      <td>1883.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>Strauss</td>\n",
       "      <td>1864</td>\n",
       "      <td>1949.0</td>\n",
       "      <td>post-romantic</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>Orff</td>\n",
       "      <td>1895</td>\n",
       "      <td>1982.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>Dusek</td>\n",
       "      <td>1731</td>\n",
       "      <td>1799.0</td>\n",
       "      <td>classic</td>\n",
       "      <td>Czechia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>Smetana</td>\n",
       "      <td>1824</td>\n",
       "      <td>1884.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Czechia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>Janacek</td>\n",
       "      <td>1854</td>\n",
       "      <td>1928.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>Czechia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>Copland</td>\n",
       "      <td>1900</td>\n",
       "      <td>1990.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>Bernstein</td>\n",
       "      <td>1918</td>\n",
       "      <td>1990.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>Glass</td>\n",
       "      <td>1937</td>\n",
       "      <td>NaN</td>\n",
       "      <td>modern</td>\n",
       "      <td>USA</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        composer  birth   death         period  country\n",
       "0         Mahler   1860  1911.0  post-romantic  Austria\n",
       "1      Beethoven   1770  1827.0       romantic  Germany\n",
       "2        Puccini   1858  1924.0  post-romantic    Italy\n",
       "3   Shostakovich   1906  1975.0         modern   Russia\n",
       "4          Verdi   1813  1901.0       romantic    Italy\n",
       "5         Dvorak   1841  1904.0       romantic  Czechia\n",
       "6       Schumann   1810  1856.0       romantic  Germany\n",
       "7     Stravinsky   1882  1971.0         modern   Russia\n",
       "8       Sibelius   1865  1957.0  post-romantic  Finland\n",
       "9          Haydn   1732  1809.0        classic  Austria\n",
       "10        Mozart   1756  1791.0        classic  Austria\n",
       "11      Messiaen   1908  1992.0         modern   France\n",
       "12     Prokofiev   1891  1953.0         modern   RUssia\n",
       "13    Monteverdi   1567  1643.0    renaissance    Italy\n",
       "14       Haendel   1685  1759.0        baroque  Germany\n",
       "15        Brahms   1833  1897.0       romantic  Germany\n",
       "16       Purcell   1659  1695.0        baroque  England\n",
       "17   Charpentier   1643  1704.0        baroque   France\n",
       "18      Bruckner   1824  1896.0  post-romantic  Austria\n",
       "19          Berg   1885  1935.0         modern  Austria\n",
       "20      Couperin   1626  1661.0        baroque   France\n",
       "21        Rameau   1683  1764.0        baroque   France\n",
       "22       Berlioz   1803  1869.0       romantic   France\n",
       "23        Gounod   1818  1893.0       romantic   France\n",
       "24      Massenet   1842  1912.0       romantic   France\n",
       "25        Boulez   1925  2016.0         modern   France\n",
       "26    Palestrina   1525  1594.0    renaissance    Italy\n",
       "27      Gesualdo   1566  1613.0    renaissance    Italy\n",
       "28       Caldara   1670  1736.0        baroque    Italy\n",
       "29     Pergolesi   1710  1736.0        baroque    Italy\n",
       "30     Scarlatti   1685  1757.0        baroque    Italy\n",
       "31       Caccini   1587  1640.0        baroque    Italy\n",
       "32      Cimarosa   1749  1801.0        classic    Italy\n",
       "33     Donizetti   1797  1848.0       romantic    Italy\n",
       "34   Leoncavallo   1858  1919.0       romantic    Italy\n",
       "35       Bellini   1801  1835.0       romantic    Italy\n",
       "36         Dufay   1397  1474.0    renaissance  Belgium\n",
       "37        Lassus   1532  1594.0    renaissance  Belgium\n",
       "38       Borodin   1833  1887.0       romantic   Russia\n",
       "39   Mussorsgsky   1839  1881.0       romantic   Russia\n",
       "40         Soler   1754  1806.0        classic    Spain\n",
       "41       Albeniz   1860  1909.0       romantic    Spain\n",
       "42      Granados   1867  1916.0       romantic    Spain\n",
       "43       Dowland   1563  1626.0    renaissance  England\n",
       "44          Byrd   1540  1623.0    renaissance  England\n",
       "45        Walton   1902  1983.0         modern  England\n",
       "46          Adès   1971     NaN         modern  England\n",
       "47          Bach   1685  1750.0        baroque  Germany\n",
       "48        Wagner   1813  1883.0       romantic  Germany\n",
       "49       Strauss   1864  1949.0  post-romantic  Germany\n",
       "50          Orff   1895  1982.0         modern  Germany\n",
       "51         Dusek   1731  1799.0        classic  Czechia\n",
       "52       Smetana   1824  1884.0       romantic  Czechia\n",
       "53       Janacek   1854  1928.0         modern  Czechia\n",
       "54       Copland   1900  1990.0         modern      USA\n",
       "55     Bernstein   1918  1990.0         modern      USA\n",
       "56         Glass   1937     NaN         modern      USA"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers = pd.read_excel(file_url, sheet_name=4)\n",
    "composers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "96tRltp9EVWs"
   },
   "source": [
    "## Saving tables\n",
    "\n",
    "Most commonly, you don't save many tables. You just keep a notebook or a script for processing and only keep the final output such as a statistics or a plot. However sometimes you want to keep intermediary steps. For that we can use the reverse of the ```read_XXX``` function, i.e the ```to_XXX``` function. Here again, you have plenty of options that you can explore. For example you can avoid saving the index and use ```;``` as separator:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "executionInfo": {
     "elapsed": 928,
     "status": "ok",
     "timestamp": 1614506453025,
     "user": {
      "displayName": "Guillaume Witz",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GgT0K2JVYzEsjzsS5nhkUVjUrSIJ5jHzXnBoYrmVf8=s64",
      "userId": "16033870147214403532"
     },
     "user_tz": -60
    },
    "id": "lzOnjtCnE6RQ",
    "tags": []
   },
   "outputs": [],
   "source": [
    "composers.to_csv('export_test.csv', index=False, sep=';')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sT3AOfgGTHH6"
   },
   "source": [
    "## Exercises\n",
    "1. Import the penguin dataset https://raw.githubusercontent.com/allisonhorst/palmerpenguins/master/inst/extdata/penguins.csv\n",
    "2. Rename the ```bill_length_mm``` and ```bill_depth_mm``` colums into ```length``` and ```depth```.\n",
    "3. Check that the names of the DataFrame have changed. If not, do you understand why?\n",
    "4. Add a column with name ```my_column``` and fill it with default value 'test'.\n",
    "5. We have seen how to remove a given row. Can you find out how to remove the 10 first rows?\n",
    "6. What do you observe happened with the index?\n",
    "7. Import again the data but this time only the 10 first rows. Can you find an option of the ```read_csv``` function to do that?\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyMAnw9J8r77GvgnXoJ4g+Rw",
   "collapsed_sections": [],
   "name": "06-Back_to_Pandas.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}