R/scripts/s02_data_storage_objects.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## data storage objects  \n",
    "There are several objects in R that handle data storage. We will go through them one by one.   \n",
    "But first the concept of placeholders, also called identifiers, has to be explained.  \n",
    "\n",
    "You, as a programmer, are expected to create logical placeholder names. These names should be short, but still clarify what it is. Placeholders are used to store any object type. It is recommended to:  \n",
    "* separate words with underscores: my_variable.  \n",
    "* use small letters  \n",
    "* for functions use verbs: join_tables()  \n",
    "\n",
    "\n",
    "### placeholders  \n",
    "As an example of a placeholder, let us define a variable that enables us to store a time-stamp: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "current_time <- Sys.time()\n",
    "current_time "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: Run the lines several times, see how the placeholder `current_time` stores the output of our `Sys.time()` function?  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also store the output of a calculation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_result <- 5 * (1+5)\n",
    "my_result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And use this later on in our analysis, because it is stored in *R memory*! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_result + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise 3: Explain and then fix the error message you get when running the following line of code: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_other_result + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### vectors\n",
    "A vector object is created with the `c()` command, each entry is separated by a comma:  \n",
    "`my_vector <- c( 4, 6, 2, 4, 1)`   \n",
    "\n",
    "\n",
    "* exercise 4: Now that we know how to create vectors, how to create variables and strings/ characters lets practice this:  \n",
    "Create a vector with the `c()` function containing the sentence; \"hello, my name is \" in the first entry and in the second vector entry a placeholder containing your name.  Make sure to   define the placeholder containing your name first, or you will get an error!  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to create sequences of numbers is with the `seq()` function. Have a look at the help file for `seq` by typing\n",
    "?seq. Note the `from` `to` `by` and `length.out` arguments. To create a sequence of 6 equally spaced numbers from 0 to 10 you can do:\n",
    "`my_seq <- seq(from = 0, to = 10, by = 2)` or you can do `my_seq <- seq(from = 0, to = 10, length.out = 6)` \n",
    "Another useful method to generate sequences of integers is the colon: `1:10`  \n",
    "\n",
    "* exercise: Try all three methods to generate a sequence of numbers from 12 to 2 with steps of 1. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to do calculations with numeric vectors. There is a behavior to be aware of called recycling. \n",
    "* exercise: Calculate the sum of these vectors and explain recycling: `1:6 + 1:2` also try `1:5 + 1:2`, when and why did you get a warning?   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us move on to selection operations on vectors. Square brackets `[]` are used for selection. Selection by dimension (rows/columns/ or more dimensions) can be performed by separating the arguments inside the brackets by comma's. A vector however, has only 1 dimension. To select the second entry of my_seq: `my_seq[2]`   \n",
    "* exercise: Select entry 1, 3 and 5 of `my_seq`:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### matrixes  \n",
    "Single class tables can be stored as matrices. Matrices are not so useful for us biologists, but super useful for fast matrix calculations in the for of linear algebra needed for the various statistical tools available in R.  We will have a quick look at them. To define a matrix:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_matrix <- matrix(1:9, nrow = 3, ncol = 3)\n",
    "my_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_matrix2 <- matrix(1:9, nrow = 3, ncol = 3, byrow = TRUE)\n",
    "my_matrix2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Selection of matrix data is two dimensional, you need row and column identifiers, selection occurs like this: \n",
    "`my_matrix[`*row-numbers*`,` *column-numbers*`]` to select row 2 and 3 of the 1st and 3rd columns: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_matrix[2:3, c(1, 3)] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### data.frames  \n",
    "The data.frame allows mixing data classes, so you can mix numeric, strings and factors. Let's have a look at the already available `iris` data.frame using the `head` function to display only the first 7 rows. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "head(iris, n = 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: Display the last 5 rows of `iris`  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To define your own data frame use the `data.frame()` function:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_dataframe <- data.frame(names = c(\"Steven\", \"Margriet\", \"Gerard\", \"Elke\"),\n",
    "                           hobbies = c(\"R\", \"Python\", \"cheminformatics\", \"pharmacokinetics\"),\n",
    "                           sex = c(\"M\", \"F\", \"M\", \"F\"),\n",
    "                           fakeage = c(38, 30, 34, 28))\n",
    "my_dataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: Define your own data frame, the first column should have 10 random letters, the second column random numbers drawn from the `1:6` sequence. Name the columns as you want.  \n",
    "Hints: check out the `letters` or `LETTERS` object and have a look at the help documentation for the `sample()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check the column classes, for column one and five this would be: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class(iris[, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class(iris[, 5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This mixing of data types is not possible in matrices.  \n",
    "Before we dive deeper into `data.frames`, we will discuss the `factor` class. Let us print the contents of the `Species` column from the `iris` `data.frame`: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris[, 5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the `Levels` at the bottom? Note: (`levels(iris[, 5])`) does displayes the levels specifically. This informs us this column has the class `factor`, and also informs us of the unique factor levels. This is mainly important for statistical analysis, where defining the order is important. However many functions and visualization tools in R also use the factor data type to define for example the order of displaying data labels or the order of plotting nominal values on the x-axis.  \n",
    "You can create the `factor` class with (first we transform the `Species` to character class):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris$Species <- as.character(iris$Species)\n",
    "levels(iris$Species)\n",
    "iris$Species <- factor(iris$Species)\n",
    "levels(iris$Species)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Always think of `data.frames` in a column-wise fashion. A column represents a variable. Next to `iris[, 5]` We can aces `data.frame` columns with `iris$Species` or `iris[, \"Species\"]`  \n",
    "\n",
    "This is an opportune moment to explain Boolean and conditional selection. The Boolean class, in R called the logical class, is another class of data. It has the values `TRUE`, `FALSE` and `NA`. The `NA` is used if the entry does not exist.  \n",
    "With logicals it is possible to perform selection. For example;  \n",
    "`iris[ iris$Petal.Length < 1.4,  ]` displays all rows of the `iris` data for which the Petal Length is smaller than 1.4.  \n",
    "printing the `iris$Petal.Length < 1.4` statement clearly shows what is happening. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris$Petal.Length < 1.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to perform multiple conditional selections by using the `&` sign in between.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: Display the `Petal.Length` for all plants that belong to the `virginica` species. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: Display the `Petal.Length` for all plants that belong to the `virginica` species for which the petal length is smaller than 1.6.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A very useful operator for selection to multiple entries is the `%in%` operator:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris[ iris$Species %in% c(\"virginica\", \"setosa\")  & iris$Sepal.Length %in% c( 7.2, 5.3),]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we will have a quick look how we can perform calculations with data frames. To define a new variable (thus column) in your data frame you can use direct assignment to a new variable: `iris$is_small_Sepal <- iris$Sepal.Length < 5 | iris$Sepal.Width < 3`   \n",
    "Note that you can do calculations on logicals:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sum(c(TRUE, TRUE, FALSE, FALSE, TRUE))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: how many small sepal plants are there for each of the three species? With small as defined above.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note how we used multiple columns in a `data.frame` to calculate a new column and that these operations are performed row-wise, thus for each plant!  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### lists  \n",
    "\n",
    "Lists are another object class in R to store data. Actually any object can be stored in lists, for example functions, data frames, vectors, characters and even other lists can all be placed in a list. If you put 1 object in a list, the list has length 1, putting 2 objects in the list creates a list of length 2, and so on. You can name the entries in the list making access easy and explicit. Let's create a list with named entries:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list <- list(teachers = my_dataframe, stat = t.test( \n",
    "  subset(iris, Species == \"virginica\")[, \"Sepal.Length\"], subset(iris, Species == \"setosa\")[, \"Sepal.Length\"]) )\n",
    "\n",
    "my_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the first entry in the list is our `my_dataframe` and the second entry is the result of the `t.test()` function where we tested for different `Sepal.Length` between the species `virginica` and`setosa`  \n",
    "\n",
    "To access an entry in our list there are several options. One can use the name:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list$teachers\n",
    "my_list$stat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or simply use index numbers:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list[[1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to keep the list attributes use a single bracket:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To select multiple slots simultaneously use single brackets:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list[1:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Accessing objects inside lists is also straightforward, access the list entry and then treat the object selections as you would normally:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list$teachers$hobbies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you could do:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_list[[1]][, \"hobbies\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: Store the iris data frame as and additional slot in the existing list object `my_list`  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* exercise: Display the `Sepal.Lengths` of all `virginca` and `setosa` plants from within the `my_list`  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### arrays  \n",
    "Arrays are higher dimensional  matrices. For example, a three dimensional array can be thought of as multiple matrices stacked behind one another, for 4 3x3 matrices: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_array <- array(1:28, dim = c(3, 3, 4))\n",
    "dim(my_array)\n",
    "my_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To access 2nd row, 2nd column of 2nd matrix: `[row, column, extra_dimension1]`  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_array[ 2, 2, 2 ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or to access the 3rd matrix:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_array[ , , 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But you could also give the matrices themselves indexes in multiple dimensions, for example:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_array <- array(1:28, dim = c(3, 3, 2, 2 ))\n",
    "dim(my_array)\n",
    "my_array"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}