Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial support for multiple solutions #286

Merged
merged 25 commits into from
Feb 16, 2022
Merged

Conversation

gadenbuie
Copy link
Member

Overview

This PR adds initial support for multiple solutions. Solutions can be written in the -solution chunk, using # <name/description> ---- comments as separators. (In RStudio you can add a solution heading using the Insert a new code section action, or Ctrl + Shift + R.)

The solutions become available in the .solution_code_all list in the check environment. The name or description in the code section separator becomes the name of the list. We also give the solutions list an internal class with print methods.

.solution_code_all <- 
  gradethis_solutions(
    one = "runif(1)",
    two = "runif(2)",
    three = "runif(3)"
  )

.solution_code_all
# one --------------------------------------------------------------------------

runif(1)

# two --------------------------------------------------------------------------

runif(2)

# three ------------------------------------------------------------------------

runif(3)
str(.solution_code_all)
#> List of 3
#>  $ one  : chr "runif(1)"
#>  $ two  : chr "runif(2)"
#>  $ three: chr "runif(3)"
#>  - attr(*, "class")= chr "gradethis_solutions"

The .solution_code and .solution objects are still populated as previously using the last solution in the chunk, and mock_this_exercise() handles the solution code as expected.

ex <- mock_this_exercise("runif(4)", !!format(.solution_code_all))
ex$.solution_code
#> [1] "runif(3)"
ex$.solution_code_all[["three"]]
#> [1] "runif(3)"
ex$.solution
#> [1] 0.3866510 0.8096415 0.5790076

To make the above work and to consolidate logic, I refactored mock_this_exercise() to use prepare_check_env(), meaning that both gradethis_exercise_checker() and mock_this_exercise() follow a similar code path when preparing the check env. (There are a few things that are duplicated but different on either side, of course.)

Example exercise

Here's an example exercise borrowed from Allow several ways to respond with learnr by djourd1 on StackOverflow.

Reprex tutorial
---
title: "Tutorial"
output: learnr::tutorial
runtime: shiny_prerendered
---

```{r setup, include=FALSE}
library(learnr)
pkgload::load_all("~/work/gradethis")
knitr::opts_chunk$set(echo = FALSE)

library(gapminder)
gm <- data.frame(gapminder)
```


## Multiple Solutions

### Exercise 

This example is borrowed from
[Allow several ways to respond with learnr](https://stackoverflow.com/questions/71102380/allow-several-ways-to-respond-with-learnr)
by [djourd1](https://stackoverflow.com/users/2741402/djourd1) on StackOverflow.

The solution chunk contains the following code.

```{r ref.label="gdpzim-solution", echo=TRUE, eval=FALSE}

```

<details><summary>Grading Code</summary>

This grading code is written in a `-check` chunk but could also be written in a `-code-check` chunk.

```{r ref.label = "gdpzim-check", eval = FALSE, echo = TRUE}
```

</details>

```{r gdpzim, exercise=TRUE}
gm[____, ____]
```

```{r gdpzim-solution, exercise.reveal_solution = FALSE}
# country first ----
gm[gm$country == "Zimbabwe" & gm$year==1997,  "gdpPercap"]

# year first ----
gm[gm$year==1997 & gm$country == "Zimbabwe" ,  "gdpPercap"]
```

```{r gdpzim-check}
grade_this({
  mistakes <- c()
  
  # Evaluate all of the solutions, pass if no mistakes for any solution
  for (i in seq_along(.solution_code_all)) {
    feedback <- code_feedback(.user_code, .solution_code_all[[i]])
    if (is.null(feedback)) {
      soln_name <- names(.solution_code_all)[[i]]
      pass("Good job, you did it the **{soln_name}** way!")
    } else {
      mistakes <- c(mistakes, feedback)
    }
  }
  
  # Otherwise report mistakes made
  message <- paste(mistakes, collapse = "\n\nAlternatively: ")
  fail("{message}")
})
```

image

image

image

Other things

A few other incidental changes:

  1. I refactored call_standardise_formals() (mostly style changes) (129fe76)
  2. I imported more helper functions from rlang, a few is_*() functions (2524b27)
  3. I fixed a small bug with our use of list2env() where the parent of the created env should be the empty env, not the parent frame (very in the weeds)
  4. I added a small helper function env_rls(), an rlang-backed replacement for pryr::rls() (811df61)
  5. I fixed a small bug with wrong_value() when this is missing, i.e. when the value is missing in the user's call (81284ce)

@rossellhayes
Copy link
Contributor

@gadenbuie I'm curious about the incorrect answer in the reprex. It seems like we should only be including the alternative feedback, since the first feedback message is pointing away from a valid answer.

@gadenbuie
Copy link
Member Author

I'm curious about the incorrect answer in the reprex. It seems like we should only be including the alternative feedback, since the first feedback message is pointing away from a valid answer.

@rossellhayes It's not doing anything smart. At this stage, handling multiple solutions is entirely manual and the grading code in that example just shows all of the feedback available if the submission didn't pass.

The goal of this PR is just to make multiple solutions available in the grading code and to pre-process them so that they're manageable and consistent.

Copy link
Contributor

@rossellhayes rossellhayes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial support looks great to me! Should we include the code feedback logic in another PR, or add it here?

@gadenbuie gadenbuie changed the title Initial support for multiple solutions Add initial support for multiple solutions Feb 16, 2022
@gadenbuie gadenbuie merged commit 51517ee into main Feb 16, 2022
@gadenbuie gadenbuie deleted the multiple-solutions branch February 16, 2022 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants