Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refine exercise completion logic when allow_skip = FALSE #633

Merged
merged 3 commits into from
Dec 30, 2021

Conversation

gadenbuie
Copy link
Member

Addresses #616 by augmenting the exercise_submission event to include a completed item. This boolean value indicates whether or not the submission should count toward the completion of the exercise and therefore the current section. This is primarily relevant when allow_skip: false is set in the tutorial YAML heading.

Previously, as described in #616, any exercise submission counted toward completion, including pressing "Run Code" on an empty exercise input. This will no longer be the case.

The new logic is set in the exercise_result event handler, which is where the exercise_submission event is emitted. There, we use the exercise cache to access the full exercise object and augment the exercise_submission event data with a new completed item. The full logic is to:

  1. Only set correct when submission doesn't require a check
  2. Add completed to the data. The flag is TRUE when there is submission code and
    • the exercise doesn't have any grading
    • OR the exercise has grading code (either $check or $code_check) and the exercise was checked
  3. The first event reporting data.completed causes the progress to move forward. (I haven't traced the logic, but testing indicates that the first completed event isn't erased by subsequent non-completion counting events).

The end result is that the user must make one non-empty "Run Code" submission for exercises without grading code or one "Submit Answer" submission for exercises with grading code in order to move forward. We do not, however, require that the answer be correct — in our experience limiting users in this way would cause more frustration than it's worth, since it's easy for mistakes in grading code to accidentally make "correctness" impossible.

One other limitation is that if the exercise has initial code, we don't impede progress if the user submits the initial code as it was provided. I think doing so would ultimately lead to issues down the line but I'd be open to revisiting that policy now that I understand the flow better.

learnr-improve-exercise-completion-demo.mp4

Copy link
Contributor

@rossellhayes rossellhayes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

R/events_default.R Outdated Show resolved Hide resolved
@gadenbuie gadenbuie merged commit 1e841da into main Dec 30, 2021
@gadenbuie gadenbuie deleted the improve-exercise-completion-logic branch December 30, 2021 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants