beyond-exception-handling.Rmd

---
title: Beyond exception handling
layout: default
---

This is a translation of [Beyond exception handling: conditions and restarts](http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html) by Peter Seibel, from Lisp to R. The original document is copyright (c) 2003-2005, Peter Seibel; translated with permission.

The majority of the translation involves changing Lisp syntax to R syntax.
There are few differences in the overall system.

Beyond Exception Handling: Conditions and Restarts
======================================================

One of R's great features is its *condition* system. It serves a
similar purpose to the exception handling systems in Java, Python, and
C++ but is more flexible. In fact, its flexibility extends beyond error
handling--conditions are more general than exceptions in that a
condition can represent any occurrence during a program's execution that
may be of interest to code at different levels on the call stack. For
example, in the section "Other Uses for Conditions," you'll see that
conditions can be used to emit warnings without disrupting execution of
the code that emits the warning while allowing code higher on the call
stack to control whether the warning message is printed. For the time
being, however, I'll focus on error handling.

The condition system is more flexible than exception systems because
instead of providing a two-part division between the code that signals
an error^1^ and the code that handles it,^2^ the condition system splits
the responsibilities into three parts--*signaling* a condition,
*handling* it, and *restarting*. In this chapter, I'll describe how you
could use conditions in part of a hypothetical application for analyzing
log files. You'll see how you could use the condition system to allow a
low-level function to detect a problem while parsing a log file and
signal an error, to allow mid-level code to provide several possible
ways of recovering from such an error, and to allow code at the highest
level of the application to define a policy for choosing which recovery
strategy to use.

To start, I'll introduce some terminology: *errors*, as I'll use the
term, are the consequences of Murphy's law. If something can go wrong,
it will: a file that your program needs to read will be missing, a disk
that you need to write to will be full, the server you're talking to
will crash, or the network will go down. If any of these things happen,
it may stop a piece of code from doing what you want. But there's no
bug; there's no place in the code that you can fix to make the
nonexistent file exist or the disk not be full. However, if the rest of
the program is depending on the actions that were going to be taken,
then you'd better deal with the error somehow or you *will* have
introduced a bug. So, errors aren't caused by bugs, but neglecting to
handle an error is almost certainly a bug.

So, what does it mean to handle an error? In a well-written program,
each function is a black box hiding its inner workings. Programs are
then built out of layers of functions: high-level functions are built on
top of the lower-level functions, and so on. This hierarchy of
functionality manifests itself at runtime in the form of the call stack:
if `high` calls `medium`, which calls `low`, when the flow of control is
in `low`, it's also still in `medium` and `high`, that is, they're still
on the call stack.

Because each function is a black box, function boundaries are an
excellent place to deal with errors. Each function--`low`, for
example--has a job to do. Its direct caller--`medium` in this case--is
counting on it to do its job. However, an error that prevents it from
doing its job puts all its callers at risk: `medium` called `low`
because it needs the work done that `low` does; if that work doesn't get
done, `medium` is in trouble. But this means that `medium`'s caller,
`high`, is also in trouble--and so on up the call stack to the very top
of the program. On the other hand, because each function is a black box,
if any of the functions in the call stack can somehow do their job
despite underlying errors, then none of the functions above it needs to
know there was a problem--all those functions care about is that the
function they called somehow did the work expected of it.

In most languages, errors are handled by returning from a failing
function and giving the caller the choice of either recovering or
failing itself. Some languages use the normal function return mechanism,
while languages with exceptions return control by *throwing* or
*raising* an exception. Exceptions are a vast improvement over using
normal function returns, but both schemes suffer from a common flaw:
while searching for a function that can recover, the stack unwinds,
which means code that might recover has to do so without the context of
what the lower-level code was trying to do when the error actually
occurred.

Consider the hypothetical call chain of `high`, `medium`, `low`. If
`low` fails and `medium` can't recover, the ball is in `high`'s court.
For `high` to handle the error, it must either do its job without any
help from `medium` or somehow change things so calling `medium` will
work and call it again. The first option is theoretically clean but
implies a lot of extra code--a whole extra implementation of whatever it
was `medium` was supposed to do. And the further the stack unwinds, the
more work that needs to be redone. The second option--patching things up
and retrying--is tricky; for `high` to be able to change the state of
the world so a second call into `medium` won't end up causing an error
in `low`, it'd need an unseemly knowledge of the inner workings of both
`medium` and `low`, contrary to the notion that each function is a black
box.

The R Way
------------

R's error handling system gives you a way out of this
conundrum by letting you separate the code that actually recovers from
an error from the code that decides how to recover. Thus, you can put
recovery code in low-level functions without committing to actually
using any particular recovery strategy, leaving that decision to code in
high-level functions.

To get a sense of how this works, let's suppose you're writing an
application that reads some sort of textual log file, such as a Web
server's log. Somewhere in your application you'll have a function to
parse the individual log entries. Let's assume you'll write a function,
`parse_log_entry()`, that will be passed a string containing the text of a
single log entry and that is supposed to return a `log_entry()` object
representing the entry. This function will be called from a function,
`parse_log_file()`, that reads a complete log file and returns a list of
objects representing all the entries in the file.

To keep things simple, the `parse_log_entry()` function will not be
required to parse incorrectly formatted entries. It will, however, be
able to detect when its input is malformed. But what should it do when
it detects bad input? In C you'd return a special value to indicate
there was a problem. In Java or Python you'd throw or raise an
exception. In R, you signal a condition.

Conditions
----------

A *condition* is an S3 object whose class indicates the general nature of
the condition and whose instance data carries information about the
details of the particular circumstances that lead to the condition being
signaled.^3^ In this hypothetical log analysis program, you might define
a condition class, `malformed_log_entry_error`, that `parse_log_entry()`
will signal if it's given data it can't parse.

Conditional classes are regular S3 classes, built up from a list with components `message` and `call`. There is no built in function to generate a new object of class condition, but we can add one:

```{r condition}
condition <- function(subclass, message, call = sys.call(-1), ...) {
  structure(
    class = c(subclass, "condition"),
    list(message = message, call = call, ...)
  )
}
```

When using the condition system for error handling, you should define
your conditions as subclasses of `error`, a subclass of `condition`.
Thus, you might define `malformed_log_entry_error`, with a slot to hold
the argument that was passed to `parse_log_entry()`, like this:

```{r malformed_log_entry_error}
malformed_log_entry_error <- function(text) {
  msg <- paste0("Malformed log entry: ", text)
  condition(c("malformed_log_entry_error", "error"),
    message = msg, 
    text = text
  )
}
```

Condition Handlers
------------------

In `parse_log_entry()` you'll signal a `malformed_log_entry_error` if you
can't parse the log entry. You signal errors with the function `stop()`. 
`stop()` is normally just called with a string, the error message, but you
can also call it with a condition object. Thus, you could write
`parse_log_entry()` like this, eliding the details of actually parsing a
log entry:

```{r}
parse_log_entry <- function(text) {
  if (!well_formed_log_entry(text)) {
    stop(malformed_log_entry_error(text))
  }
  
  new_log_entry(text)
}
```

What happens when the error is signaled depends on the code above
`parse_log_entry()` on the call stack. To avoid a top level error message,
you must establish a *condition handler* in one of the functions leading
to the call to `parse_log_entry`. When a condition is signaled, the
signaling machinery looks through a list of active condition handlers,
looking for a handler that can handle the condition being signaled based
on the condition's class. Each condition handler consists of a type
specifier indicating what types of conditions it can handle and a
function that takes a single argument, the condition. At any given
moment there can be many active condition handlers established at
various levels of the call stack. When a condition is signaled, the
signaling machinery finds the most recently established handler whose
type specifier is compatible with the condition being signaled and calls
its function, passing it the condition object.

The handler function can then choose whether to handle the condition.
The function can decline to handle the condition by simply returning
normally, in which case control returns to next most recently established
handler with a compatible type specifier. To handle the condition, 
the function must transfer control out of the signaller via a *nonlocal exit*.
In the next section, you'll see how a handler can choose where to transfer
control. However, many condition handlers simply want to unwind the stack 
to the place where they were established and then run some code. The function
`tryCatch()` establishes this kind of condition handler. The basic
form of a `tryCatch` is as follows:

```{r, eval = FALSE}
tryCatch(expression, 
  condition_class_1 = function(var) ...,
  condition_class_2 = function(var) ...,
  ...
)
```

If the *expression* returns normally, then its value is returned by the
`tryCatch()`. The body of a `tryCatch()` must be a single
expression; but you can always use `{` to combine several expressions into a
single form. If, however, the expression signals a condition that's an
instance of any of the *condition class*s
then the code in the appropriate error clause is
executed and its value returned by the `tryCatch()`. The *var*
is the name of the variable that will hold the condition
object when the handler code is executed. If the code doesn't need to
access the condition object, you can omit the variable name.

For instance, one way to handle the `malformed_log_entry_error` signaled
by `parse_log_entry()` in its caller, `parse_log_file()`, would be to skip
the malformed entry. In the following function, the `tryCatch()`
expression will either return the value returned by `parse_log_entry()` or
return `NULL` if a `malformed_log_entry_error` is signaled. 

```{r parse_file_try_catch}
parse_log_file <- function(file) {
  lines <- readLines(file)
  
  lapply(lines, function(text) {
    tryCatch(
      malformed_log_entry_error = function(e) NULL,
      parse_log_entry(text)
    )
  })
}
```

When `parse_log_entry()` returns normally, its value will be collected by the
`lapply()`. But if `parse_log_entry` signals a `malformed_log_entry_error`, 
then the error clause will return `NULL`.

This version of `parse_log_file()` has one serious deficiency: it's doing
too much. As its name suggests, the job of `parse_log_file()` is to parse
the file and produce a list of `log_entry` objects; if it can't, it's
not its place to decide what to do instead. What if you want to use
`parse_log_file()` in an application that wants to tell the user that the
log file is corrupted or one that wants to recover from malformed
entries by fixing them up and re-parsing them? Or maybe an application
is fine with skipping them but only until a certain number of corrupted
entries have been seen.

You could try to fix this problem by moving the `tryCatch()` to a
higher-level function. However, then you'd have no way to implement the
current policy of skipping individual entries--when the error was
signaled, the stack would be unwound all the way to the higher-level
function, abandoning the parsing of the log file altogether. What you
want is a way to provide the current recovery strategy without requiring
that it always be used.

### Java style exception handling

`tryCatch` is the nearest analog in R to Java- or
Python-style exception handling. Where you might write this in Java:

    try {
      doStuff();
      doMoreStuff();
    } catch (SomeException se) {
      recover(se);
    }

or this in Python:

    try:
      doStuff()
      doMoreStuff()
    except SomeException, se:
      recover(se)

in R you'd write this:

```{r, eval = FALSE}
tryCatch({
  doStuff()
  doMoreStuff()
}, some_exception = function(se) {
  recover(se)
})
```


Restarts
--------

The condition system lets you do this by splitting the error handling
code into two parts. You place code that actually recovers from errors
into *restarts*, and condition handlers can then handle a condition by
invoking an appropriate restart. You can place restart code in mid- or
low-level functions, such as `parse_log_file()` or `parse_log_entry()`,
while moving the condition handlers into the upper levels of the
application.

To change `parse_log_file()` so it establishes a restart instead of a
condition handler, you can change the `tryCatch()` to a
`withRestarts()`. The form of `withRestarts` is very similar to a
`tryCatch()`. In general, a restart name
should describe the action the restart takes. In `parse_log_file()`, you
can call the restart `skip_log_entry` since that's what it does. The new
version will look like this:

```{r parse_file_restart}
parse_log_file <- function(file) {
  lines <- readLines(file)
  
  lapply(lines, function(text) {
    withRestarts(
      parse_log_entry(text),
      skip_log_entry = function(e) NULL
    )
  })
}
```

If you invoke this version of `parse_log_file()` on a log file containing
corrupted entries, it won't handle the error directly; you'll end up in
the debugger. However, there among the various restarts listed by `findRestarts()`
will be one called `skip_log_entry`, which, if you choose it,
will cause `parse_log_file()` to continue on its way as before. To avoid
ending up in the debugger, you can establish a condition handler that
invokes the `skip_log_entry` restart automatically.

The advantage of establishing a restart rather than having
`parse_log_file()` handle the error directly is it makes `parse_log_file()`
usable in more situations. The higher-level code that invokes
`parse_log_file()` doesn't have to invoke the `skip_log_entry` restart. It
can choose to handle the error at a higher level. Or, as I'll show in
the next section, you can add restarts to `parse_log_entry()` to provide
other recovery strategies, and then condition handlers can choose which
strategy they want to use.

But before I can talk about that, you need to see how to set up a
condition handler that will invoke the `skip_log_entry` restart. You can
set up the handler anywhere in the chain of calls leading to
`parse_log_file()`. This may be quite high up in your application, not
necessarily in `parse_log_file()`'s direct caller. For instance, suppose
the main entry point to your application is a function, `log_analyzer()`,
that finds a bunch of logs and analyzes them with the function
`analyze_log()`, which eventually leads to a call to `parse_log_file()`.
Without any error handling, it might look like this:

```{r}
log_analyzer <- function() {
  logs <- find_all_logs()
  lapply(logs, analyze_log)
}
```

The job of `analyze_log()` is to call, directly or indirectly,
`parse_log_file()` and then do something with the list of log entries
returned. An extremely simple version might look like this:

```{r}
analyze_log <- function(log) {
  entries <- parse_log_file(log)
  lapply(entries, analyze_entry)
}
```

where the function `analyze_entry()` is presumably responsible for
extracting whatever information you care about from each log entry and
stashing it away somewhere.

Assuming you always want to skip malformed log entries, you could change
this function to establish a condition handler that invokes the
`skip_log_entry` restart for you. However, you can't use `tryCatch()`
to establish the condition handler because then the stack would be
unwound to the function where the `tryCatch()` appears. Instead, you
need to use the lower-level macro `withCallingHandlers()`. The basic form of
`withCallingHandlers()` is as follows:

```{r, eval = FALSE}
withCallingHandlers(
  expr,
  condition_1 = function() ...,
  condition_2 = function() ....
)
```

An important difference between
`tryCatch()` and `withCallingHandlers()` is that the handler function bound by
`withCallingHandlers()` will be run without unwinding the stack--the flow of
control will still be in the call to `parse_log_entry()` when this
function is called. A call to `invokeRestart()` will find and invoke
the most recently bound restart with the given name. So you can add a
handler to `log_analyzer()` that will invoke the `skip_log_entry` restart
established in `parse_log_file()` like this:^5^

```{r}
log_analyzer <- function() {
  logs <- find_all_logs()
  
  withCallingHandlers(
    malformed_log_entry_error = function(e) invokeRestart("skip_log_entry"),
    lapply(logs, analyze_log)
  )
}
```

In this `withCallingHandlers()`, the handler function is an anonymous function
that invokes the restart `skip_log_entry`. You could also define a named
function that does the same thing and bind it instead. In fact, a common
practice when defining a restart is to define a function, with the same
name and taking a single argument, the condition, that invokes the
eponymous restart. Such functions are called *restart functions*. You
could define a restart function for `skip_log_entry` like this:

```{r}
skip_log_entry <- function() invokeRestart("skip_log_entry")
```

Then you could change the definition of `log_analyzer()` to this:

```{r}
log_analyzer <- function() {
  logs <- find_all_logs()
  
  withCallingHandlers(
    malformed_log_entry_error = skip_log_entry,
    lapply(logs, analyze_log)
  )
}
```

As written, the `skip_log_entry` restart function assumes that a
`skip_log_entry` restart has been established. If a
`malformed_log_entry_error` is ever signaled by code called from
`log_analyzer()` without a `skip_log_entry` having been established, the
call to `invokeRestart()` will signal an error when it fails to
find the `skip_log_entry` restart. If you want to allow for the
possibility that a `malformed_log_entry_error` might be signaled from
code that doesn't have a `skip_log_entry` restart established, you could
change the `skip_log_entry` function to this:

```{r}
skip_log_entry <- function() {
  r <- findRestart("skip_log_entry") 
  if (is.null(r)) return()
  
  invokeRestart(r)
}
```

`findRestart` looks for a restart with a given name and returns an
object representing the restart if the restart is found and `NULL` if
not. You can invoke the restart by passing the restart object to
`invokeRestart()`. Thus, when `skip_log_entry` is bound with
`withCallingHandlers()`, it will handle the condition by invoking the
`skip_log_entry` restart if one is available and otherwise will return
normally, giving other condition handlers, bound higher on the stack, a
chance to handle the condition.

Providing Multiple Restarts
---------------------------

Since restarts must be explicitly invoked to have any effect, you can
define multiple restarts, each providing a different recovery strategy.
As I mentioned earlier, not all log-parsing applications will
necessarily want to skip malformed entries. Some applications might want
`parse_log_file()` to include a special kind of object representing
malformed entries in the list of `log-entry` objects; other applications
may have some way to repair a malformed entry and may want a way to pass
the fixed entry back to `parse_log_entry()`.

To allow more complex recovery protocols, restarts can take arbitrary
arguments, which are passed in the call to `invokeRestart()`. You can
provide support for both the recovery strategies I just mentioned by
adding two restarts to `parse_log_entry()`, each of which takes a single
argument. One simply returns the value it's passed as the return value
of `parse_log_entry()`, while the other tries to parse its argument in the
place of the original log entry.

```{r}
parse_log_entry <- function(text) {
  if (well_formed_log_entry(text)) {
    return(new_log_entry(text))
  }
  
  withRestarts(
    stop(malformed_log_entry_error(text)),
    use_value = function(x) x,
    reparse_entry = function(fixed_text) parse_log_entry(fixed_text)
  )
}
```

The name `use_value()` is a standard name for this kind of restart. You can 
define a restart function for `use_value` similar to the
`skip_log_entry` function you just defined. 

```{r}
use_value <- function(x) invokeRestart("use_value", x)
```

So, if you wanted to change
the policy on malformed entries to one that created an instance of
`malformed_log_entry`, you could change `log_analyzer()` to this (assuming
the existence of a `malformed_log_entry` constructor with a `text` parameter):

```{r}
log_analyzer <- function() {
  logs <- find_all_logs()
  
  withCallingHandlers(
    malformed_log_entry_error = function(text) {
      use_value(malformed_log_entry(text))
    },
    lapply(logs, analyze_log)
  )
}
```

You could also have put these new restarts into `parse_log_file()` instead
of `parse_log_entry()`. However, you generally want to put restarts in the
lowest-level code possible. It wouldn't, though, be appropriate to move
the `skip_log_entry` restart into `parse_log_entry()` since that would
cause `parse_log_entry()` to sometimes return normally with `NULL`, the
very thing you started out trying to avoid. And it'd be an equally bad
idea to remove the `skip_log_entry` restart on the theory that the
condition handler could get the same effect by invoking the `use-value`
restart with `NULL` as the argument; that would require the condition
handler to have intimate knowledge of how the `parse_log_file()` works. As
it stands, the `skip_log_entry` is a properly abstracted part of the
log-parsing API.

Other Uses for Conditions
-------------------------

While conditions are mainly used for error handling, they can be used
for other purposes--you can use conditions, condition handlers, and
restarts to build a variety of protocols between low- and high-level
code. The key to understanding the potential of conditions is to
understand that merely signaling a condition has no effect on the flow
of control.

The primitive signaling function `signalCondition()` implements the mechanism of
searching for an applicable condition handler and invoking its handler
function. The reason a handler can decline to handle a condition by
returning normally is because the call to the handler function is just a
regular function call--when the handler returns, control passes back to
`signalCondition()`, which then looks for another, less recently bound handler that
can handle the condition. If `signalCondition()` runs out of condition handlers
before the condition is handled, it also returns normally.

The `stop()` function you've been using calls `signalCondition()`. If the error is
handled by a condition handler that transfers control via `tryCatch()`
or by invoking a restart, then the call to `signalCondition()` never returns. 

Another condition signaling function, `warning()`, provides an example of a
different kind of protocol built on the condition system. Like `stop()`,
`warnings()` calls `signalCondition()` to signal a condition. But if `signalCondition()` returns,
`warning()` doesn't throw a top-level error--it prints the condition to
`stderr` and returns `NULL`, allowing its caller to proceed.
`warning()` also establishes a restart, `muffle_warning()`, around the call to
`signalCondition()` that can be used by a condition handler to make `warning()` return
without printing anything. Of course, a condition signaled with `warning()`
could also be handled in some other way--a condition handler could
"promote" a warning to an error by handling it as if it were an error.

For instance, in the log-parsing application, if there were ways a log
entry could be slightly malformed but still parsable, you could write
`parse_log_entry()` to go ahead and parse the slightly defective entries
but to signal a condition with `warning()` when it did. Then the larger
application could choose to let the warning print, to muffle the
warning, or to treat the warning like an error, recovering the same way
it would from a `malformed_log_entry_error`.

You can also build your own protocols on `signalCondition()`--whenever low-level
code needs to communicate information back up the call stack to
higher-level code, the condition mechanism is a reasonable mechanism to
use. But for most purposes, one of the standard error or warning
protocols should suffice.

Unfortunately, it's the fate of error
handling to always get short shrift in programming texts--proper error
handling, or lack thereof, is often the biggest difference between
illustrative code and hardened, production-quality code. The trick to
writing the latter has more to do with adopting a particularly rigorous
way of thinking about software than with the details of any particular
programming language constructs. That said, if your goal is to write
that kind of software, you'll find the R condition system is
an excellent tool for writing robust code and one that fits quite nicely
into R's incremental development style.

### Writing Robust Software

For information on writing robust software, you could do worse than to
start by finding a copy of *Software Reliability* (John Wiley & Sons,
1976) by Glenford J. Meyers. Bertrand Meyer's writings on Design By
Contract also provide a useful way of thinking about software
correctness. For instance, see Chapters 11 and 12 of his
*Object-Oriented Software Construction* (Prentice Hall, 1997). Keep in
mind, however, that Bertrand Meyer is the inventor of Eiffel, a
statically typed bondage and discipline language in the Algol/Ada
school. While he has a lot of smart things to say about object
orientation and software reliability, there's a fairly wide gap between
his view of programming and The R Way. Finally, for an excellent
overview of the larger issues surrounding building fault-tolerant
systems, see Chapter 3 of the classic *Transaction Processing: Concepts
and Techniques* (Morgan Kaufmann, 1993) by Jim Gray and Andreas Reuter.

^1^*Throws* or *raises* an exception in Java/Python terms

^2^*Catches* the exception in Java/Python terms

^3^In this respect, a condition is a lot like an exception in Java or
Python except not all conditions represent an error or *exceptional*
situation.

^5^The compiler may complain if the parameter is never used. You can
silence that warning by adding a declaration `(declare (ignore c))` as
the first expression in the `LAMBDA` body.