Collect and report users's tutorial state for the current session in the Shiny app

[gadenbuie]

This PR addresses a common problem of tutorial app authors wanting to know the current state of the user's progress in the tutorial, for example to know which questions/exercises have received submissions, what answer was submitted and whether that answer was correct or incorrect.

To achieve this we now establish a reactive values list in session$userData$tutorial_state that is available to app authors in the shiny process. Authors can interact with this reactive list using context = "server" chunks but the reactive value is not accessible to submitted exercise code.

As implemented, the reactive list is a named list with names matching the question or exercise label and containing:

• type: one of "question" or "exercise"
• answer: the user's submitted answer as a character vector of possibly variable length
• correct: A logical TRUE or FALSE value of length 0 or 1. Logical NA values indicate the answer was not checked.
• timestamp: A character timestamp in UTC indicating when the event was recorded, e.g. "2021-08-10 14:56:38.834 UTC".

The tutorial_user_state.Rmd tutorial in the sandbox demonstrates the structure and behavior of the tutorial_state reactive. Here's a snapshot:

learnr::get_tutorial_state()
## List of 2
## $ two-plus-two:List of 3
##   ..$ type   : chr "exercise"
##   ..$ answer : chr "3 + 3\n\n"
##   ..$ correct: logi FALSE
##   ..# timestamp: chr "2021-08-10 14:56:38.834 UTC"
## $ quiz-1      :List of 3
##   ..$ type   : chr "question"
##   ..$ answer : chr "tools"
##   ..$ correct: logi FALSE
##   ..# timestamp: chr "2021-08-10 14:56:37.231 UTC"

The latest question submission is always reported, since landing on the correct answer is a terminal action. Exercise submissions are reported in the same way: submissions are always reported to the state reactive until the user reaches the correct answer. Thereafter, additional exercise submission events are ignored. (Technical note: if a student submits any code after the correct submission, the correct exercise submission event won't survive if the tutorial is reloaded in the browser.)

The final aspect of this PR is a minor change to scrub correct and feedback items from question and exercise submissions (respectively) for client storage only. For local storage, we keep these items. Regardless, the tutorial_state is reconstructed on tutorial start when restoring tutorial state from storage. This ensures that the correct flag in tutorial_state is accurate wrt. the current tutorial source in case the tutorial was updated after the users' state was stored. Note that when client storage is used, under certain conditions, it is possible that an exercise may not be re-checked on startup.

cc @rundel The goal here is partly to give learnrhash and others an easy-to-access and lightweight representation of the current tutorial state. Rather than extracting submissions from the state objects in the tutorial storage, learnrhash can simply hash/encode the current users state of the tutorial.

Related PRs and Issues

• Closes Question state - include correct flag #393 by giving authors/developers a consistent place to find question/exercise state, including correct
• Closes export three functions per feature request #464 and Export two functions: get_all_state_objects() and submissions_from_state_objects() #454 by providing an exported interface to tutorial state and info
• Closes Obtaining assessment data (pass/fail/trials) #14 by providing pass/fail for current user. (Obtaining assessment data (pass/fail/trials) #14 also request number of tries but it's unlikely we'll support that soon.)

[rundel]

Sounds great to me - I probably wont have a chance to play with it this week but I will take a look as soon as I can.

[schloerke]

Exciting!

Later we should talk about possibly exposing a reactive() value instead of a reactiveVal(), but the end user interface will not change.

Should we consider other information, like timestamp? Or should that be saved for the event logging.

[gadenbuie]

A final question @schloerke: I refactored the tutorial state setting into fairly minimal get_tutorial_state() and set_tutorial_state() helpers. They're currently internal, but we might want to export get_tutorial_state(). It's not strictly necessary since we're using session$userData which is publicly available in the Shiny app. OTOH, by exporting the function we'd give ourselves a little room for future API changes. wdyt?

[schloerke]

OTOH, by exporting the function we'd give ourselves a little room for future API changes. wdyt?

Agreed. Let's export get_tutorial_state() and steer authors to only use get_tutorial_state(). (We should only use session$userData$tutorial_state internally)

[schloerke]

@davidkane9 Would learnr::get_tutorial_state() (added in this PR) cover your needs from #464 ?

I believe the only functionality that isn't covered is read_request()

[schloerke]

Now get_tutorial_info() handles the standard read_request() information from #464

Should we close #464?