Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial support for test cases #664

Merged
merged 14 commits into from
Feb 16, 2022
Merged

Add initial support for test cases #664

merged 14 commits into from
Feb 16, 2022

Conversation

gadenbuie
Copy link
Member

@gadenbuie gadenbuie commented Feb 15, 2022

Adds initial support for test cases, which can be added to a <exercise-label>-test-cases chunk. The test cases can be named and split using code sections, e.g. # first test ----. They are added to the exercise object under the test_cases item, which is either NULL or a list of test cases. If there are no code sections, the single test cases will be in an unnamed list, otherwise test cases are named with default names test_case_NN where NN is a left-zero-padded integer.

I've included a manual test in tests/manual/test-cases.Rmd

Example Rmd code
```{r addition-setup}
x <- 1 + 1
```

```{r addition, exercise = TRUE}

```

```{r addition-solution}
1 + 1
```

```{r addition-check}
check_this_exercise(user_code, solution_code)
```

```{r addition-test-cases}
1 + 1

# one plus two ----
1 + 2

# one plus three ----
1 + 3

# one equals three ----
1 = 3

# 2 minus one ----
2 - 1
```
pkgload::load_all()

x <- prepare_tutorial_cache_from_source("tests/manual/test-cases.Rmd")

str(x$items$data[[1]][c("solution", "check", "test_cases")])
#> List of 3
#>  $ solution  : chr "1 + 1"
#>   ..- attr(*, "chunk_opts")=List of 1
#>   .. ..$ label: chr "addition-solution"
#>  $ check     : chr "check_this_exercise(user_code, solution_code)"
#>   ..- attr(*, "chunk_opts")=List of 1
#>   .. ..$ label: chr "addition-check"
#>  $ test_cases:List of 5
#>   ..$ test_case_00    : chr "1 + 1"
#>   ..$ one plus two    : chr "1 + 2"
#>   ..$ one plus three  : chr "1 + 3"
#>   ..$ one equals three: chr "1 = 3"
#>   ..$ 2 minus one     : chr "2 - 1"

For now, this initial support only ensures that -test-cases chunks are removed from the tutorial presentation and are stored alongside the exercise object. In the future, we hope to expand test case support, but we want to gather some experience with this format and structure first.

@gadenbuie gadenbuie added this to In Progress in v0.11.0 Feb 15, 2022
Copy link
Contributor

@rossellhayes rossellhayes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great first step. Will test case code and expected results all be stored together in the single chunk? We'll have to think of an interface for including that

@gadenbuie
Copy link
Member Author

Will test case code and expected results all be stored together in the single chunk? We'll have to think of an interface for including that

The test cases can be arbitrary code, so for now it can be whatever we need it to be. I was originally thinking it would be examples of student submissions but it could be testing code or anything else really that would become part of the exercise data.

Now that I've written out the above, I think I'm now leaning more toward -tests for the chunk name, e.g. addition-tests, with the object name $tests and individual items $tests$test01, etc. -test-cases is a bit more specific than the feature might end up being.

@rossellhayes
Copy link
Contributor

rossellhayes commented Feb 16, 2022

Got it! Would it be implemented with a testthat-style interface (i.e. all tests must not produce an error)?

@gadenbuie
Copy link
Member Author

@rossellhayes I just changed the name from -test-cases to -tests and updated the structure accordingly. Because this overlaps considerably with rstudio/gradethis#286, I moved the main functions in this PR, plus a few others into inst/staticexports and started using staticimports. The plan is to statically import these functions in gradethis too once this PR merges. This sets up learnr as the parent location for these kinds of helper/utility functions that we want to use across packages.

@gadenbuie gadenbuie changed the title Initial support for test cases Add initial support for test cases Feb 16, 2022
@gadenbuie gadenbuie merged commit 214413a into main Feb 16, 2022
@gadenbuie gadenbuie deleted the support-test-chunk branch February 16, 2022 21:34
v0.11.0 automation moved this from In Progress to Done Feb 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

None yet

2 participants