Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Answer evaluation in UnlockProtocol #101

Merged
merged 3 commits into from
Jun 29, 2015

Conversation

jathak
Copy link
Contributor

@jathak jathak commented Jun 29, 2015

This implements @papajohn's proposal in #59.

@albert12132
Copy link
Contributor

The way unlocking works is the following:

  1. Verify if the student's guess is correct (this is the part you modified).
  2. If it is deemed correct, we take the student input (literally just the string they input) to be the answer for the test case.
  3. When GradingProtocol runs, it will check if the repr of the prompt expression (>>> x + 4) is equal to the unlocked answer (again, literally just the string they input).

Suppose we have a prompt:

>>> x
?

If a student enters 2 + 3 and this is deemed correct, the unlocked test case will look like

>>> x
2 + 3

Then GradingProtocol will check if repr(x) (which is the string '5') is equal to the string '2 + 3', which it isn't.

Getting around this requires either rewriting interact or rewriting GradingProtocol; changing _verify won't be enough.

@jathak
Copy link
Contributor Author

jathak commented Jun 29, 2015

Okay. I get it now. I rewrote this to leave _verify as-is, but change interact to call _verify again with evaluated input if exact string matching fails. If evaluated input matches, it then uses that as the answer for the test case.

@albert12132
Copy link
Contributor

The code looks good to me. Can you write one or two tests for this in tests/protocols/unlock_test.py?

@jathak
Copy link
Contributor Author

jathak commented Jun 29, 2015

Done.

@albert12132
Copy link
Contributor

LGTM.

albert12132 added a commit that referenced this pull request Jun 29, 2015
@albert12132 albert12132 merged commit 795c41e into okpy:master Jun 29, 2015
@jathak jathak deleted the enhancement/jathak/eval-unlock branch June 29, 2015 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants