-
Notifications
You must be signed in to change notification settings - Fork 0
/
04_automating-tasks.Rmd
666 lines (502 loc) · 19.5 KB
/
04_automating-tasks.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
Automating Tasks
================
By now, you've learned all of the basic skills necessary to explore a data set
in R. The focus of this chapter is how to organize your code so that it's
concise, clear, and easy to automate. This will help you and your collaborators
avoid tedious, redundant work, reproduce results efficiently, and run code in
specialized environments for scientific computing, such as high-performance
computing clusters.
#### Learning Objectives {-}
* Create code that only runs when a condition is satisfied
* Create custom functions in order to organize and reuse code
* Run code repeatedly in a for-loop
* Describe the different types of loops and how to choose between them
Conditional Expressions
-----------------------
Sometimes you'll need code to do different things, depending on a condition.
_If-statements_ provide a way to write conditional code.
For example, suppose we want to greet one person differently from the others:
```{r}
name = "Nick"
if (name == "Nick") {
# If name is Nick:
message("We went down the TRUE branch")
msg = "Hi Nick, nice to see you again!"
} else {
# Anything else:
msg = "Nice to meet you!"
}
```
Indent code inside of the if-statement by 2 or 4 spaces. Indentation makes your
code easier to read.
The condition in an if-statement has to be a scalar:
```{r, error = TRUE}
name = c("Nick", "Susan")
if (name == "Nick") {
msg = "Hi Nick!"
} else {
msg = "Nice to meet you!"
}
```
You can chain together if-statements:
```{r}
name = "Susan"
if (name == "Nick") {
msg = "Hi Nick, nice to see you again!"
} else if (name == "Peter") {
msg = "Go away Peter, I'm busy!"
} else {
msg = "Nice to meet you!"
}
msg
```
If-statements return the value of the last expression in the evaluated block:
```{r}
name = "Tom"
msg = if (name == "Nick") {
"Hi Nick, nice to see you again!"
} else {
"Nice to meet you!"
}
msg
```
Curly braces `{ }` are optional for single-line expressions:
```{r}
name = "Nick"
msg = if (name == "Nick") "Hi Nick, nice to see you again!" else
"Nice to meet you!"
msg
```
But you have to be careful if you don't use them:
```{r, error = TRUE}
# NO GOOD:
msg = if (name == "Nick")
"Hi Nick, nice to see you again!"
else
"Nice to meet you!"
```
The `else` block is optional:
```{r}
msg = "Hi"
name = "Tom"
if (name == "Nick")
msg = "Hi Nick, nice to see you again!"
msg
```
When there's no `else` block, the value of the `else` block is `NULL`:
```{r}
name = "Tom"
msg = if (name == "Nick")
"Hi Nick, nice to see you again!"
msg
```
Functions
---------
The main way to interact with R is by calling functions, which was first
explained way back in Section \@ref(calling-functions). Since then, you've
learned how to use many of R's built-in functions. This section explains how
you can write your own functions.
To start, let's briefly review what functions are, and some of the jargon
associated with them. It's useful to think of functions as factories: raw
materials (inputs) go in, products (outputs) come out. We can also represent
this visually:
<img src="img/functions.png">
Programmers use several specific terms to describe the parts and usage of
functions:
* _Parameters_ are placeholder variables for inputs.
+ _Arguments_ are the actual values assigned to the parameters in a call.
* The _return value_ is the output.
* The _body_ is the code inside.
* _Calling_ a function means using a function to compute something.
Almost every command in R is a function, even the arithmetic operators and the
parentheses! You can view the body of a function by typing its name without
trailing parentheses (in contrast to how you call functions). The body of a
function is usually surrounded by curly braces `{}`, although they're optional
if the body only contains one line of code. Indenting code inside of curly
braces by 2-4 spaces also helps make it visually distinct from other code.
For example, let's look at the body of the `append` function, which appends a
value to the end of a list or vector:
```{r}
append
```
Don't worry if you can't understand everything the `append` function's code
does yet. It will make more sense later on, after you've written a few
functions of your own.
Many of R's built-in functions are not entirely written in R code. You can spot
these by calls to the special `.Primitive` or `.Internal` functions in their
code.
For instance, the `sum` function is not written in R code:
```{r}
sum
```
The `function` keyword creates a new function. Here's the syntax:
```
function(parameter1, parameter2, ...) {
# Your code goes here
# The result goes here
}
```
A function can have any number of parameters, and will automatically return the
value of the last line of its body.
A function is a value, and like any other value, if you want to reuse it, you
need to assign it to variable. Choosing descriptive variable names is a good
habit. For functions, that means choosing a name that describes what the
function does. It often makes sense to use verbs in function names.
Let's write a function that gets the largest values in a vector. The inputs or
arguments to the function will be the vector in question and also the number of
values to get. Let's call these `vec` and `n`, respectively. The result will be
a vector of the `n` largest elements. Here's one way to write the function:
```{r}
get_largest = function(vec, n) {
sorted = sort(vec, decreasing = TRUE)
head(sorted, n)
}
```
The name of the function, `get_largest`, describes what the function does and
includes a verb. If this function will be used frequently, a shorter name, such
as `largest`, might be preferable (compare to the `head` function).
Any time you write a function, the first thing you should do afterwards is test
that it actually works. Let's try the `get_largest` function on a few test
cases:
```{r}
x = c(1, 10, 20, -3)
get_largest(x, 2)
get_largest(x, 3)
y = c(-1, -2, -3)
get_largest(y, 2)
z = c("d", "a", "t", "a", "l", "a", "b")
get_largest(z, 3)
```
Notice that the parameters `vec` and `n` inside the function do not exist as
variables outside of the function:
```{r, error = TRUE}
vec
```
In general, R keeps parameters and variables you define inside of a function
separate from variables you define outside of a function. You can read more
about the specific rules for how R searches for variables in Section
\@ref(variable-scope-lookup).
As a function for quickly summarizing data, `get_largest` would be more
convenient if the parameter `n` for the number of values to return was optional
(again, compare to the `head` function). You can make the parameter `n`
optional by setting a _default argument_: an argument assigned to the parameter
if no argument is assigned in the call to the function. You can use `=` to
assign default arguments to parameters when you define a function with the
`function` keyword. Here's a new definition of the function with the default `n
= 5`:
```{r}
get_largest = function(vec, n = 5) {
sorted = sort(vec, decreasing = TRUE)
head(sorted, n)
}
```
After making this change, it's a good idea to test the function again:
```{r}
get_largest(x)
get_largest(y)
get_largest(z)
```
### Returning Values
We've already seen that a function will automatically return the value of its
last line.
The `return` keyword causes a function to return a result immediately, without
running any subsequent code in its body. It only makes sense to use `return`
from inside of an if-statement. If your function doesn't have any
if-statements, you don't need to use `return`.
For example, suppose you want the `get_largest` function to immediately return
`NULL` if the argument for `vec` is a list. Here's the code, along with some
test cases:
```{r}
get_largest = function(vec, n = 5) {
if (is.list(vec))
return(NULL)
sorted = sort(vec, decreasing = TRUE)
head(sorted, n)
}
get_largest(x)
get_largest(z)
get_largest(list(1, 2))
```
Alternatively, you could make the function raise an error by calling the `stop`
function. Whether it makes more sense to return `NULL` or print an error
depends on how you plan to use the `get_largest` function.
Notice that the last line of the `get_largest` function still doesn't use the
`return` keyword. It's idiomatic to only use `return` when strictly necessary.
A function returns one R object, but sometimes computations have multiple
results. In that case, return the results in a vector, list, or other data
structure.
For example, let's make a function that computes the mean and median for a
vector. We'll return the results in a named list, although we could also use a
named vector:
```{r}
compute_mean_med = function(x) {
m1 = mean(x)
m2 = median(x)
list(mean = m1, median = m2)
}
compute_mean_med(c(1, 2, 3, 1))
```
The names make the result easier to understand for the caller of the function,
although they certainly aren't required here.
### Planning Your Functions
Before you write a function, it's useful to go through several steps:
1. Write down what you want to do, in detail. It can also help to
draw a picture of what needs to happen.
2. Check whether there's already a built-in function. Search online and in the
R documentation.
3. Write the code to handle a simple case first. For data science
problems, use a small dataset at this step.
Let's apply this in one final example: a function that detects leap years. A
year is a leap year if either of these conditions is true:
* It is divisible by 4 and not 100
* It is divisible by 400
That means the years 2004 and 2000 are leap years, but the year 2200 is not.
Here's the code and a few test cases:
```{r}
# If year is divisible by 4 and not 100 -> leap
# If year is divisible by 400 -> leap
year = 2004
is_leap = function(year) {
if (year %% 4 == 0 & year %% 100 != 0) {
leap = TRUE
} else if (year %% 400 == 0) {
leap = TRUE
} else {
leap = FALSE
}
leap
}
is_leap(400)
is_leap(1997)
```
Functions are the building blocks for solving larger problems. Take a
divide-and-conquer approach, breaking large problems into smaller steps. Use a
short function for each step. This approach makes it easier to:
* Test that each step works correctly.
* Modify, reuse, or repurpose a step.
Loops
-----
One major benefit of using a programming language like R is that repetitive
tasks can be automated. We've already seen two ways to do this:
1. Vectorization, introduced in Section \@ref(vectorization)
2. Apply functions, introduced in Section \@ref(apply-functions)
Both of these are _iteration strategies_. They _iterate_ over some object, and
compute something for each element. Each one of these computations is one
_iteration_. Vectorization is the most efficient iteration strategy, but only
works with vectorized functions and vectors. Apply functions are more
flexible---they work with any function and any data structure with
elements---but less efficient and less concise.
A _loop_ is another iteration strategy, one that's even more flexible than
apply functions. Besides being flexible, loops are a feature of almost all
modern programming languages, so it's useful to understand them. In R, there
are two kinds of loops. We'll learn both.
### For-loops
A _for-loop_ runs a block of code once for each element of a vector or list.
The `for` keyword creates a for-loop. Here's the syntax:
```
for (I in DATA) {
# Your code goes here
}
```
The variable `I` is called the _induction variable_. At the beginning of each
iteration, `I` is assigned the next element of the vector or list `DATA`. The
loop iterates once for each element of `DATA`, unless you use a keyword to exit
the loop early (more about this in Section \@ref(break-next)). As with
if-statements and functions, the curly braces `{ }` are only required if the
body contains multiple lines of code.
Unlike the other iteration strategies, loops do not automatically return a
result. You have complete control over the output, which means that anything
you want to save must be assigned to a variable.
For example, let's make a loop that repeatedly adds a number to a running total
and squares the new total. We'll use a variable `total` to keep track of the
running total as the loop iterates:
```{r}
numbers = c(-1, 1, -3, 2)
total = 0
for (number in numbers) {
total = (total + number)^2
}
total
```
Use for-loops when some or all of the iterations depend on results from other
iterations. If the iterations are not dependent, use one of:
1. Vectorization (because it's faster)
2. Apply functions (because they're idiomatic)
In some cases, you can use vectorization even when the iterations are
dependent. For example, you can use vectorization to compute the sum of the
cubes of several numbers:
```{r}
sum(numbers^3)
```
### While-loops
A _while-loop_ runs a block of code repeatedly as long as some condition is
`TRUE`. The `while` keyword creates a while-loop. Here's the syntax:
```
while (CONDITION) {
# Your code goes here
}
```
The `CONDITION` should be a scalar logical value or an expression that returns
one. At the beginning of each iteration, `CONDITION` is checked, and the loop
exits if it is `FALSE`. As always, the curly braces `{ }` are only required if
the body contains multiple lines of code.
For example, suppose you want to add up numbers from 0 to 50, but stop as soon
as the total is greater than 50:
```{r}
num50 = seq(0, 50)
total = 0
i = 1
while (total < 50) {
total = total + num50[i]
message("i is ", i, " total is ", total)
i = i + 1
}
total
i
```
While-loops are a generalization of for-loops. They tend to be most useful when
you don't know how many iterations will be necessary. For example, suppose you
want to repeat a computation until the result falls within some range of
values.
### Saving Multiple Results
Loops often produce a different result for each iteration. If you want to save
more than one result, there are a few things you must do.
First, set up an index vector. The index vector should usually be congruent to
the number of iterations or the input. The `seq_along` function returns a
congruent index vector when passed a vector or list. For instance, let's make
in index for the `numbers` vector from Section \@ref(for-loops):
```{r}
index = seq_along(numbers)
```
The loop will iterate over the index rather than the input, so the induction
variable will track the current iteration number. On the first iteration, the
induction variable will be 1, on the second it will be 2, and so on. Then you
can use the induction variable and indexing to get the input for each
iteration.
Second, set up an empty output vector or list. This should usually be congruent
to the input, or one element longer (the extra element comes from the initial
value). R has several functions for creating vectors. We've already seen a few,
but here are more:
* `logical`, `integer`, `numeric`, `complex`, and `character` to create an
empty vector with a specific type and length
* `vector` to create an empty vector with a specific type and length
* `rep` to create a vector by repeating elements of some other vector
Empty vectors are filled with `FALSE`, `0`, or `""`, depending on the type of
the vector. Here are some examples:
```{r}
logical(3)
numeric(4)
rep(c(1, 2), 2)
```
Let's create an empty numeric vector congruent to `numbers`:
```{r}
n = length(numbers)
result = numeric(n)
```
As with the input, you can use the induction variable and indexing to set the
output for each iteration.
Creating a vector or list in advance to store something, as we've just done, is
called _preallocation_. Preallocation is extremely important for efficiency in
loops. Avoid the temptation to use `c` or `append` to build up the output bit
by bit in each iteration.
Finally, write the loop, making sure to get the input and set the output.
Here's the loop for the squared sums example:
```{r}
for (i in index) {
prev = if (i > 1) result[i - 1] else 0
result[i] = (numbers[i] + prev)^2
}
result
```
### Break & Next
The `break` keyword causes a loop to immediately exit. It only makes sense to
use `break` inside of an if-statement.
For example, suppose we want to print each string in a vector, but stop at the
first missing value. We can do this with `break`:
```{r}
my_messages = c("Hi", "Hello", NA, "Goodbye")
for (msg in my_messages) {
if (is.na(msg))
break
message(msg)
}
```
The `next` keyword causes a loop to immediately go to the next iteration. As
with `break`, it only makes sense to use `next` inside of an if-statement.
Let's modify the previous example so that missing values are skipped, but don't
cause printing to stop. Here's the code:
```{r}
for (msg in my_messages) {
if (is.na(msg))
next
message(msg)
}
```
These keywords work with both for-loops and while-loops.
### Example: The Collatz Conjecture
The [Collatz Conjecture][collatz] is a conjecture in math that was introduced
in 1937 by Lothar Collatz and remains unproven today, despite being relatively
easy to explain. Here's a statement of the conjecture:
[collatz]:
> Start from any positive integer. If the integer is even, divide by 2. If the
> integer is odd, multiply by 3 and add 1.
>
> If the result is not 1, repeat using the result as the new starting value.
>
> The result will always reach 1 eventually, regardless of the starting value.
The sequences of numbers this process generates are called _Collatz sequences_.
For instance, the Collatz sequence starting from 2 is `2, 1`. The Collatz
sequence starting from 12 is `12, 6, 3, 10, 5, 16, 8, 4, 2, 1`.
As a final loop example, let's use a while-loop to compute Collatz sequences.
Here's the code:
```{r}
n = 5
i = 0
while (n != 1) {
i = i + 1
if (n %% 2 == 0) {
n = n / 2
} else {
n = 3 * n + 1
}
message(paste0(n, " "))
}
```
As of 2020, scientists have used computers to check the Collatz sequences for
every number up to approximately $2^{64}$. For more details about the Collatz
Conjecture, check out [this video][collatz-video].
[collatz-video]: https://www.youtube.com/watch?v=094y1Z2wpJg
Planning for Iteration
----------------------
At first it may seem difficult to decide if and what kind of iteration to use.
Start by thinking about whether you need to do something over and over. If you
don't, then you probably don't need to use iteration. If you do, then try
iteration strategies in this order:
1. vectorization
2. apply functions
* Try an apply function if iterations are independent.
3. for/while-loops
* Try a for-loop if some iterations depend on others.
* Try a while-loop if the number of iterations is unknown.
4. recursion (which isn't covered here)
* Convenient for naturally recursive problems (like Fibonacci),
but often there are faster solutions.
Start by writing the code for just one iteration. Make sure that code works;
it's easy to test code for one iteration.
When you have one iteration working, then try using the code with an iteration
strategy (you will have to make some small changes). If it doesn't work, try to
figure out which iteration is causing the problem. One way to do this is to use
`message` to print out information. Then try to write the code for the broken
iteration, get that iteration working, and repeat this whole process.
Exercises
---------
_These exercises are meant to challenge you, so they're quite difficult
compared to the previous ones. Don't get disheartened, and if you're able to
complete them, excellent work!_
### Exercise
Create a function `compute_day` which uses the [Doomsday algorithm][doomsday]
to compute the day of week for any given date in the 1900s. The function's
parameters should be `year`, `month`, and `day`. The function's return value
should be a day of week, as a string (for example, `"Saturday"`).
_Hint: the modulo operator is `%%` in R._
[doomsday]: https://en.wikipedia.org/wiki/Doomsday_rule