-
Notifications
You must be signed in to change notification settings - Fork 535
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add complete action verification (#845)
- Loading branch information
Showing
3 changed files
with
115 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
Based on the content of the screenshot and the elements on the page, determine whether the user goal has been successfully completed or not. | ||
|
||
The JSON object should be in this format: | ||
```json | ||
{ | ||
"reasoning": str, // Describe the state of the user goal and explain why it has been completed or not completed. | ||
"user_goal_achieved": bool // True if the user goal has been completed, False otherwise. | ||
} | ||
|
||
Make sure to ONLY return the JSON object, with no additional text before or after it. Do not make any assumptions based on the screenshot, return a response solely based on what you observe in the screenshot and nothing else. | ||
|
||
Examples: | ||
{ | ||
"reasoning": "The screenshot shows a success message for a file upload field. Since the user's goal is to upload a file, it has been successfully completed.", | ||
"user_goal_achieved": true | ||
} | ||
{ | ||
"reasoning": "The screenshot shows a job application form with fields. Since the user's goal is to submit a job application, it has not been successfully completed.", | ||
"user_goal_achieved": false | ||
} | ||
|
||
Elements on the page: | ||
{{ elements }} | ||
|
||
User Goal: | ||
{{ navigation_goal }} | ||
|
||
User Details: | ||
{{ navigation_payload }} |