Refine exercise completion logic when allow_skip = FALSE
#633
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses #616 by augmenting the
exercise_submission
event to include acompleted
item. This boolean value indicates whether or not the submission should count toward the completion of the exercise and therefore the current section. This is primarily relevant whenallow_skip: false
is set in the tutorial YAML heading.Previously, as described in #616, any exercise submission counted toward completion, including pressing "Run Code" on an empty exercise input. This will no longer be the case.
The new logic is set in the
exercise_result
event handler, which is where theexercise_submission
event is emitted. There, we use the exercise cache to access the full exercise object and augment theexercise_submission
eventdata
with a newcompleted
item. The full logic is to:correct
when submission doesn't require a checkcompleted
to the data. The flag isTRUE
when there is submission code and$check
or$code_check
) and the exercise was checkeddata.completed
causes the progress to move forward. (I haven't traced the logic, but testing indicates that the first completed event isn't erased by subsequent non-completion counting events).The end result is that the user must make one non-empty "Run Code" submission for exercises without grading code or one "Submit Answer" submission for exercises with grading code in order to move forward. We do not, however, require that the answer be correct — in our experience limiting users in this way would cause more frustration than it's worth, since it's easy for mistakes in grading code to accidentally make "correctness" impossible.
One other limitation is that if the exercise has initial code, we don't impede progress if the user submits the initial code as it was provided. I think doing so would ultimately lead to issues down the line but I'd be open to revisiting that policy now that I understand the flow better.
learnr-improve-exercise-completion-demo.mp4