Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checking for exercise errors #403

Merged
merged 10 commits into from
Jul 29, 2020
Merged

Checking for exercise errors #403

merged 10 commits into from
Jul 29, 2020

Conversation

cpsievert
Copy link
Contributor

@cpsievert cpsievert commented Jul 21, 2020

Closes #356. Adds a new option exercise.error.checker for customizing the feedback provided when an exercise submission results in an evaluation error (for control over feedback when parsing user code results in error, see rstudio/gradethis#123). This option must be a function that anticipates the same arguments as exercise.checker and return a feedback object. A recommend default for this option is the new gradethis::grade_learnr_error function.

Testing notes

Install remotes::install_github("rstudio/learnr") and remotes::install_github("rstudio-education/gradethis") and run the following tutorial (rmarkdown::run("test.Rmd")). Verify that entering stop("boom") and clicking "Submit Exercise" returns in positive (green) feedback and that entering anything else results in sensible/informative negative (red) feedback.

---
title: "Checking errors"
output: learnr::tutorial
runtime: shiny_prerendered
---

```{r setup, include=FALSE}
library(learnr)	
library(gradethis)
tutorial_options(
  exercise.checker = grade_learnr, 
  exercise.error.checker = grade_learnr_error
)
```

* Throw an error message of `"boom"`:

```{r grade_error, exercise = TRUE}

```

```{r grade_error-solution}
stop("boom")
```

```{r grade_error-code-check}
grade_code("Nice error!")
```

@cpsievert cpsievert changed the title Carson/feature/on user error Checking for exercise errors Jul 21, 2020
@cpsievert cpsievert force-pushed the carson/feature/on_user_error branch from db9c955 to 1b49f8a Compare July 21, 2020 16:05
R/options.R Outdated Show resolved Hide resolved
…ng a checker function on exercise evaluation errors, closes #356
@cpsievert cpsievert force-pushed the carson/feature/on_user_error branch from befc8b8 to 7bac39a Compare July 22, 2020 16:43
R/exercise.R Outdated Show resolved Hide resolved
@cpsievert cpsievert requested a review from schloerke July 22, 2020 21:24
@cpsievert cpsievert marked this pull request as ready for review July 22, 2020 21:26
R/options.R Show resolved Hide resolved
Copy link
Collaborator

@schloerke schloerke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with suggestions

@cpsievert cpsievert merged commit 7164dde into master Jul 29, 2020
@cpsievert cpsievert deleted the carson/feature/on_user_error branch July 29, 2020 22:22
schloerke pushed a commit that referenced this pull request Jul 30, 2020
* empty_results()'s html_output needs to req()-able; otherwise, a value is never actually returned (follow up to #235)

* Refactor evaluate_exercise() and add exercise.error.checker for running a checker function on exercise evaluation errors, closes #356

* code review feedback

* knitr fig options were actually needed

* shorter tempfile pattern

* docs

* Provide an exercise.error.checker option

* Document new exercise.error.checker option

* update news
schloerke added a commit that referenced this pull request Aug 7, 2020
* master:
  Disable completion of R code inside quotes, closes #401 (#413)
  Bump version (#414)
  Save tutorial output to temp folder when tutorial folder does not have write permissions (#412)
  Throw an informative error if an exercise chunk is NULL (#411)
  Remove outdated exercise option
  Checking for exercise errors (#403)
schloerke added a commit that referenced this pull request Sep 2, 2020
* master:
  Add events to website (#423)
  Add a checker test with error as solution (#422)
  Fixes for learnr on external evaluator (#420)
  Add shinytest support (#407)
  Validate the ability to create an idb store (#417)
  Disable completion of R code inside quotes, closes #401 (#413)
  Bump version (#414)
  Save tutorial output to temp folder when tutorial folder does not have write permissions (#412)
  Throw an informative error if an exercise chunk is NULL (#411)
  Remove outdated exercise option
  Checking for exercise errors (#403)
  Add event handler system (#398)
  External evaluator fixes (#399)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable tutorial.error.checker to provide checks on error
2 participants