This document is to help with the runnng of a user testing for the DAAT prototype.
Remember, you only need to test 5 users to get enough feedback. Doesn't feel like enough? Read this!
- Recruit some users to test with - ideally this will be people who are currently box collecting volunteers.
- Make sure you have a pad and pen to make notes, or you can also record the session with the candidate’s permission
- Pay particular attention to any strong emotional words, and any usage challenges/unexpected uses of the app (don’t show them how to use it, just watch what they do) - make sure you capture everything!
- Any user is better than no users, even if they aren't perfect!
Hi, Max. My name is Jen and I'll be guiding you through this session. Before we begin I'll just go over exactly what we'll be doing - to make sure I don't miss anything I'll be reading my notes, if that's OK?
*agrees *
You probably already have a good idea of why we've asked for your help today, but let me go over it again briefly.
We're asking people to try out our DAAT app that helps volunteers record their monthly volunteer return form. The purpose of our session is to try and see whether the app works as is intended.
One thing that is really important to remember is we're testing the site, not you! You can’t do anything wrong here. In fact, this is probably the one place today where you don’t have to worry about making mistakes.
This is only a prototype of the product so some of the functionality is not in place - so if something doesn't work, don't worry. But please let us know what you would expect to happen.
As you use the site, please talk out loud as much as you can - tell me what you're looking at, what you want to do and what you're thinking.
Please don't worry about hurting our feelings. We're doing this to improve the site, so we need to hear your honest reactions.
If you have any questions as we go along, just ask them. I may not be able to answer them right away, since we’re interested in how people use the site when they don’t have someone sitting next to them to help. But if you still have any questions when we’re done I’ll try to answer them then. And if you need to take a break at any point, just let me know.
One last thing - I'll be taking notes throughout the session to make sure I don't forget anything.
(Note: at this point you can tell the user that you have a microphone or other people watching from a different room etc. And if you want to record them it's best to have them sign a permission slip.)
Do you have any questions so far?
*asks questions or says to continue *
Great. Before we look at the site I would like to ask you a few questions.
(Note: These questions aren't really for your benefit but to make the person feel more comfortable. So make this as informal as possible and have a brief informal conversation if you like! If you do have questions you want ask it's best to do this at the end)
First, what's your occupation?
..
Now, roughly how many hours a week altogether—just a rough estimate— would you say you spend using the Internet, including Web browsing and email, at work and at home?
...
What kinds of technology do you use (smart phone, laptop, ipad)?
...
What kinds of sites are you looking at when you browse the Web?
...
Do you have any favorite Web sites?
..
Great, that's it for the questions, we can now open the app.
(Open the app on landing screen)
First, I would like you to look at this page and tell me what you make of it. What strikes you about it? What do you think? Please don't click on anything yet
...
Thanks, now I would like you to do a specific task. Again, please try and think out loud as you go along.
Task 1: Imagine you have just finished counted the money from 2 boxes. Please use the app to record the contents from the boxes.
You can make up any information you don't know.
...
Task 2: Imagine you have just come back from the banking these boxes. Please use the app to record this and submit your information to the DAAT finance team.
Note: If they ever go silent gently remind them to carry on thinking out loud! Throughout the interview, you can ask questions such as:
- What is this? What is it for?
- What do you think of that?
- What do you expect that will do?
- So, what goes through your mind as you look at this?
- What are you looking for?
- What would you do next? Why?
Thank you for that, this was really helpful for us.
(Note: At this point you can ask any follow up questions and see if anyone else observing has follow up questions)
Possible follow up questions:
- How does this compare to your expectations/prior experience with the paper form?
- What did you like about this app? What did you dislike?
- What would have been helpful to allow you to complete this task more easily?
- Is there anything that you would change?
- How would you describe this product to a friend?
- If you had three magic wishes to improve this product, what would they be?
- Ask them to email you with any additional thoughts that come to them afterwards - you often receive some great extra nuggets this way
- Ask if they could recommend anyone else to test with - this is crucial as it’s really time-consuming to recruit testers so you’ll need all the help you can get!
Do you have any questions for me now that we are done?
...
Goodbye! 👋
- If you can't keep up with what the user is saying it's fine to ask them to slow down! Though this is why it can be best to test in pairs, so one person can take notes while the other speaks to the user.
- If the user says 'hmmm' etc remind them to speak out loud so you can understand their thoughts
- Explore any issues they may raise
- You don't have to have all the answers! Throw any questions back onto the candidate to understand why they've asked it and why it's important to them. A common question candidates ask is ‘would it then do X…?’ - at this stage, you don’t know (you may have a plan but this is as yet untested!) so you want to find out whether doing X would be useful/helpful to them or not, and why they have asked about it
- Steve Krug user test script - this document is based on his script
- Example user test with Steve Krug