RP003 Tooltip Usability Test

The UXR team learned during previous rounds of testing (RP001 and RP002) that users didn't notice tooltips and were confused about TDM calculator content. The UXR and design teams attempted to create solutions to this issue between RP001 and RP002 rounds of testing; however, users still struggled with identifying tooltips. Therefore, the UXR team determined that more focused testing on tooltip discoverability was needed to determine why previous design iterations did not solve this issue.

Contents

1. Prerequisites
   1. Test Plan
2. Preparation
   1. Website Overview Page
   2. Target Users
   3. Sample Project Landing Pages
3. Recruitment
   1. End Users
4. Interview Logistics
   1. Interview Materials
   2. Tool Setup
   3. UXRs Training
5. Results
   1. Interview Records
   2. Data Analysis
   3. Findings
6. Additional Resources
   1. Guide to Usability Testing
   2. Referenced Resources
   3. Contact Us

1 - Prerequisites

i - Test Plan

• Created a test plan in response to DR: Enhanced tooltip discoverability testing needed to assess current design limitations.
  • TDM: RP003 Tooltip Discoverability Research Plan.
    • Referenced prior usability test RP002: Usability Test 2 to reuse and update relevant testing materials and tools.
    • Discussed recruitment with stakeholder. See Target Users and Recruitment sections below for details
    • Conducted RP003: Brainstorming Phase - TDM to inform research plan for this usability testing.

2 - Preparation

i - Website Overview Page

• Reused the overview webpage [webpage needs update], which presents the TDM Calculator project.
  • Serves as a current landing page for user interviews in usability testing.
  • Introduces LA TDM Calculator Project with details about:
    • TDM and TDM Calculator
    • TDM Program Update
    • Current status of TDM Calculator
    • Resources from LADOT

ii - Target Users

• Utilized HfLA community to represent users for usability testing.
  • DR: Determining who to recruit for the tooltip discoverability testing.

iii - Sample Project Landing Pages

• Created landing page assets.
  • Referenced landing page content from RP002 - See Instruction Page Template.
  • Referenced sample test cases to determine sample project information.
  • Reused landing page assets from RP002 to use during user interviews.
    • Text in both landing pages is the same; however, the green boxes with the “LA TDM Calculator Website” text on each page go to different versions of the calculator (UAT & Dev). Purpose of the two environments was to test the before and after with the same user, to see if the users from both groups had the same experience or if the changes led to easier discovery of the tooltips.
      • Landing Page - UAT.
        • Landing Page Images: Before RP003 Testing.
      • Landing Page - Dev.
        • Landing Page Images: After RP003 Testing.

3 - Recruitment

i - End Users

• Sent out Slack messages with a recruitment form on HfLA Slack channels for internal usability testing and received 11 responses from the HfLA community.
• Contacted participants directly on Slack to schedule and confirm testing session time.
• Organized scheduling and testing information on a RP003: TDM Tooltip Discoverability Participant Tracker - De-Identified.
  • Full participant tracker in the PII drive contains information about each participant’s name and email, interviewer, testing date and time, and confirmation status. Contact research lead for access if needed.

4 - Interview Logistics

i - Interview Materials

• Referenced prior usability testing RP002 - Usability Testing 2.
• Created RP003: Dogfooding Script.
  • Interview script for 1:1 interviews includes intro, site testing, and closing questions, which helps UXR to follow the same procedures for all interview sessions and make notes accordingly as interviews proceed. A copy of this template is created for each interview and the link placed into the Usability Testing Session Table.
• Used the TDM background image for each interview session.

ii - Tool Setup

• Used HfLA Slack channels for recruitment.
• Used the TDM-UX@Hackforla.org email address to send out calendar invites to participants via Google Calendar.
• Used HfLA Zoom account to conduct interviews.

iii - UXRs Training

• UXR leads reviewed RP002: TDM UXR Team Lead Responsibilities for Usability Testing to ensure consistency between previous testing procedures and the tooltip usability testing.
• No training was conducted specific to this round because training was smaller in scale, internal, and there were no new UXR volunteers.

5 - Results

i - Interview Records

• Saved interview records in the PII drive. Contact research lead for access if needed.

ii - Data Analysis

• Coded participant responses using RP003: Interview Coding & Analysis spreadsheet.

• Utilized Miro Board to analyze data:

  • Created affinity diagrams to further code interview responses for analysis purposes.
  • Created RP003: Tooltip Testing Miro Board Quick Impressions to synthesize and document initial findings from the coded participant responses.

• Created the RP003: Identified Issues spreadsheet to assess the impact level of the issues that arose during testing.

iii - Findings

• Created RP003: Tooltip Discoverability Findings Report to present to stakeholders.
• Created RP003: Tooltip Discoverability Proposed Solution to present to stakeholders.

6 - Additional Resources

i - Guide to Usability Testing

• Beginner's Guide to Usability Testing by Maze

ii - Referenced Resources

• GitHub Issue #1219 - user test for tooltip discoverability
• GitHub Wiki page for all prior TDM Calculator usability testing
• Usability testing Miro Board
• Decision Records
• TDM Calculator Participant Tracker Template

iii - Contact Us

• Our Slack channel: #tdm-ux.

Return to Contents