-
Notifications
You must be signed in to change notification settings - Fork 0
/
portal.html
338 lines (297 loc) · 29 KB
/
portal.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="In 3 weeks with a shoestring budget, I led a team unfamiliar with design, improvised lean research methods, and changed the fate of a service portal's redesign from downstream UI bandaids to upstream human-centered solutions.">
<meta name="author" content="Josh Kim">
<title>Rescuing a service portal through lean research | Josh Kim is a humanity centered accessibility strategist, educator, and design researcher</title>
<!-- Stylesheets -->
<link href="css/bootstrap.min.css" rel="stylesheet">
<link href="css/jk.css" rel="stylesheet">
<!-- Fonts from Google -->
<link rel="preconnect" href="https://fonts.gstatic.com">
<link href="https://fonts.googleapis.com/css2?family=Atkinson+Hyperlegible:ital,wght@0,400;0,700;1,400;1,700&family=Libre Baskerville:wght@400&display=swap" rel="stylesheet">
<!-- Favicon -->
<link rel="shortcut icon" href="favicon.ico" type="image/x-icon">
</head>
<body>
<!-- Skip link for keyboard users -->
<a class="skip-main" href="#main">Skip to main content</a>
<!--Navigation -->
<header class="header">
<div class="container">
<nav class="jnav" aria-label="main">
<a href="index.html">Josh 内藤</a>
<span class="divider"></span>
<ul>
<li><a href="work.html">Work</a></li>
<li><a href="speaking.html">Speaking</a></li>
<li><a href="about.html">About</a></li>
</ul>
</nav>
</div>
<!--end of container -->
</header>
<!-- Banner -->
<main role="main" id="main" tabindex="-1">
<article class="content">
<div class="container">
<!-- Intro -->
<div class="row ">
<div class="col-lg-9">
<h1><span class="h-desc">Deloitte Digital</span> Rescuing a service portal through lean research</h1>
<p>I saved an agency millions of dollars by shifting their approach towards design away from user interface bandaids and towards content centered strategy.</p>
</div>
</div><!-- row -->
<!-- Feature img -->
<div class="row">
<div class="col-lg-12">
<figure>
<img class="ft-img" src="img/portal/xyzportal.gif" style="width:100%;" alt="A service portal featuring tab navigation with a search bar, a section for popular services, and a section for recently used services." />
</figure>
</div>
</div><!-- row -->
<!-- Content -->
<div class="row ">
<div class="col-lg-9 col-xl-8">
<h2 id="situation">Challenge</h2>
<p>As a design consultant within the government space, I sometimes experience what Dr. Hartson calls a “priest in a parachute” assignment.</p>
<figure>
<blockquote>
<p>In the early days of usability it was often assumed that a usability practitioner was needed only in small doses and only at certain crossroads within the project… they played a secondary role, something like a ‘priest in a parachute’: the human factors engineer dropped down into the middle of a project and stayed just long enough to give it a blessing.</p>
</blockquote>
<figcaption>The UX Book, pg. 75</figcaption>
</figure>
<p>In this instance, <strong>I was assigned to inform minor UI design changes to an agency's service portal.</strong> I realized that the goals of this initiative were driven by stakeholder priorities to visually facelift the interface instead of centering real user needs.</p>
<p>In other words, I was brought in to give "blessings" towards a project already in flight.</p>
<h3 id="people">An analyst team</h3>
<p>To make things more complex, the project team developing the portal were analysts who believed UX design (beyond UI tweaks) was a subjective and unnecessary step.</p>
<figure>
<img src="img/portal/team.png" style="max-width:100%;" alt="It me, a small designer parachuting down upon a bunch of analysts in suits." />
<figcaption>A team of analysts, a disconnected client and manager, and a designer in a parachute. What could possibly go wrong.</figcaption>
</figure>
<h3 id="goal">Setting goals</h3>
<p>Despite being a priest in a parachute, I gave myself two goals:</p>
<ol>
<li>Meet the client's expectations by providing a list of prioritized UI recommendations.</li>
<li>Shift the focus from stakeholder-led priorities (UI design fixes) towards user-centered ones using user research.</li>
</ol>
<h2 id="heuristic"><span class="h-desc">Week 1</span> Heuristic Evaluation</h2>
<p>After an initial kick off meeting and several conversations with the client and my team members, I started with a heuristic evaluation to:</p>
<ul>
<li>Better understand the product ahead of user research.</li>
<li>Complete my contractual duty of a list of prioritized UI recommendations.</li>
<li>Establish team member trust in my competence within a short time frame.</li>
<li>Softly ease my team members into the language of design through the use of <a href="https://uitraps.com/">UI Tenets and Traps</a> (I find it's associated brand recognition, tactile cards, and plain language easy to share).</li>
</ul>
<p>By the end of the first week, I gave a presentation on my findings. By meeting the client's needs early, I was able to win buy in for conducting user interviews to "validate" my findings (ya boi was actually plotting sweet generative research).</p>
<figure>
<img src="img/portal/he.png" style="width:100%;" alt="A powerpoint slide identifying an issue with repetitive icons." />
</figure>
<h2 id="user-interviews"><span class="h-desc">Week 2</span> Conducting lean UX research</h2>
<p>Through my stakeholder interviews, I learned that there were two primary user groups of the portal:</p>
<ul>
<li>Requestors who are looking to request a service</li>
<li>Approvers who approve those requests</li>
</ul>
<p>Given the limited timeframe, I took a lean research approach to unravel <em>just enough</em> insights to reframe the focus of the project.</p>
<p>Specifically I:</p>
<ul>
<li>Used a snowball sampling strategy starting from a small list of names from my client to reduce time spent on recruitment and create flexibility to pivot if any unknown user groups emerged.</li>
<li>Planned for onsite interviews to collect rich data from their contextual surroundings.</li>
<li>Prepared back up methods (task-based usability testing and closed card sorting) to enable my sessions to be adaptable (should any method fail) and better triangulate my work.</li>
</ul>
<p>I also asked one of the analysts who was serving as a product manager to join me as a notetaker. I specifically reached out to him as his buy in would be invaluable for the future of the work.</p>
<figure>
<img src="img/portal/research_1.png" style="width:100%;" alt="An exmaple row in a spreadsheet featuring IDs representing interviewees, the question asked, and the participant's response."/>
<figcaption>Raw notes were transcribed from a recording by the analyst who participated in the interview within excel. IDs are made of the interview number and the interviewers' combined first initials e.g. I-JA-1... I (Interview)-J (Josh) A (Alan)-1 (Interview #).</figcaption>
</figure>
<p>Our first session together was a success. The PM analyst learned to trust research as a key step after he listened to our users.</p><p>With his support, we convinced the other analysts to join us in future sessions. And, by the end of our interviews, <strong>all of my team members were beginning to speak in the voice of the user.</strong></p>
<h3>Closed card sorting</h3>
<p>Preparing additional methods was also a successful bet.</p>
<p>For interviews where our team had extra time to work with the participant, I conducted closed card sorts to gauge their familiarity with the portal and to better understand their mental models.</p>
<p>I asked participants to organize a deck of notecards with page titles from the portal into a hierarchy that was most intuitive to them. From the limited amount of findings, I discovered that even power users struggled to recreate the original information architecture beyond a 20% accuracy.</p>
<p>These quantitative findings and physical artifacts would pay dividends in my reporting later on.</p>
<figure>
<img src="img/portal/card-sort.png" style="width:100%;" alt="A spreadsheet page featuring user stories derived research. They are all backed by quotes followed by participant IDs."/>
<figcaption>
Most participants generated denser hierarchies and failed to understand the meaning of some page titles.
</figcaption>
</figure>
<h3>A hidden population</h3>
<p>Halfway into the second week, I had an “aha” moment that we were only covering a small piece of the puzzle.</p>
<p>Our client had selected the initial interviewees based off of their frequency of use of the portal, and as a result, we ended up talking mostly to liaisons who would submit requests on behalf of other employees.</p>
<p><strong>Because the team had always relied on <em>quantitative data on usage of the portal</em> to identify who their end users were before, they had missed an entire population of users who abandoned using the portal due to its poor usability.</strong></p>
<p>I noticed this in our <em>qualitative research</em> because the liaisons often observed that many employees would opt to call the helpdesk instead of submitting their own tickets. Our snowball sampling confirmed this when we interviewed our first non-liasion employee.</p>
<p>Although it was satisfying my hypothesis snowball sampling could unveil an unknown population was validated, I was already running out of time by the end of the second week and was politely asked by the client to wrap up our work by the end of the third.</p>
<div class="jk-grey">
<p class="caption"><b>Area for improvement:</b> Had I known we had access to quantitative data earlier on portal usage, I would have sampled the interviews by extreme outliers. This would have netted the beginner users who quit the portal early into interviewing as opposed to later.</p>
</div>
<h2 id="analysis"><span class="h-desc">Week 3</span> Analysis and reporting</h2>
<h3 id="contextual-analysis">Contextual Analysis</h3>
<p>I opted to use a digital WAAD (work activity affinity diagram) adapted from Lextant which I have used several times in the past during time-crunch situations.</p>
<p>In an excel document, I converted the raw notes transcribed by our development team into work activity notes (WANs). I then started coding the work activity notes by key terms, topics, and behaviors on a separate column.</p>
<p>Upon completion, I used this column to infer contextual stories and provisional personas. These would directly inform the findings highlighted in my report.</p>
<figure>
<img src="img/portal/contextualstories.png" style="width:100%;" alt="A spreadsheet page featuring user stories derived research. They are all backed by quotes followed by participant IDs."/>
<figcaption class="row ">
I kept one master file to use as a source of truth for user interviews and analysis. All contextual stories were comprised of WANs (Work Activity Notes, ID'd as I-xx-x Wx ) which are cleaned up versions of the raw notes. Everything is linked to a qualitative source to ensure the voice of the user is retained and trackable.
</figcaption>
</figure>
<h3>Key insights</h3>
<p>I spotlighted three high level findings to my client, each with their own sub-groups of issues and salient quotes:</p>
<ul>
<li>Users want to be able to track tickets and know responsible parties to resolve lost or delayed requests.</li>
<li>Unintuitive onboarding and poorly designed forms detract from the usability of the portal.</li>
<li>Users want to be able to complete tasks quicker and more efficiently, but are limited by system constraints.</li>
</ul>
<figure>
<blockquote>
<p>I would like to have more control, I really feel out of control with this process. I keep being told that if I want to get something done faster just call the help desk. It's a very disturbing subject.</p>
</blockquote>
<blockquote>
<p>I wasn't even aware that you had to go to the portal to take names out or add in names. I called the help desk and they explained to me how to use the portal.</p>
</blockquote>
<blockquote>
<p>My biggest gripe is they administration just comes out with systems. They don't talk to people and end users to see how they feel about the system. With this maybe they just missed us.</p>
</blockquote>
<figcaption>Quotes like these from real users were able to capture my client's attention on issues he was previously unaware of.</figcaption>
</figure>
<h3>Provisional personas</h3>
<p>To make it easier for my client to visualize and understand his users, I also created provisional personas:</p>
<ul>
<li>the Middle Man (liaisons)</li>
<li>the Specialist (highly technical approvers)</li>
<li>the Uninformed (your day to day employee)</li>
</ul>
<p>They were provisional to clearly communicate to the client that there were not enough interviews to fully uncover the width and depth of user frustrations and goals.</p>
<p>By stressing the major discovery of the "Uninformed" provisional persona in just one week, I reasoned that with more research, we could discover and address even more unknown issues that could save the agency millions in training costs and unsatisfied users.</p>
<figure>
<img src="img/portal/personas.png" style="width:100%;" alt="Two pages from my personas report including The Uninformed and The Middle Man."/>
<figcaption>I opted to use tag lines for the provisional personas instead of real life pictures to emphasize the need for more research. Large quotes with trackable WAN IDs summarized major concerns.</figcaption>
</figure>
<h3>Recommendations and immediate tasks</h3>
<p>Finally, I packaged up a list of recommendations and immediate tasks for my client to take action on including, but not limited to:</p>
<ul>
<li><strong>Conduct more open-ended research.</strong> I stressed there were different behaviors associated with the usage of the portal amongst submitters-- most notably between the Middle Man (routine usage) and the Uninformed (uncommon, needs-based usage). I made it clear that due to our original research scope and late discovery of the Uninformed, we were likely missing a significant amount of issues.</li>
<li><strong>Improve the content and information architecture of the portal.</strong> I was able to present my client artifacts and quantitative data from my appended card sorts. He was shocked to see that even expert users struggled to make sense of the portal's navigation and page titles.</li>
<li><strong>Establish a style guide.</strong> Both the heuristic evaluation and usability tests unveiled that the in consistency of components was creating a reoccuring issue for users. I recommended him to start by defining a set of design principles and conducting an atomic UI inventory analysis.</li>
<li><strong>Create user feedback mechanisms.</strong> I stressed these issues may have gone undetected as users who wanted to report problems had no formal methods of doing so.</li>
<li><strong>Add ticket tracking.</strong> The most commonly cited complaint during user research was the inability to efficiently track submitted tickets. This caused confusion when tracking individuals responsible for delays, frustration from lost or forgotten tickets, and an overall lack of control.</li>
</ul>
<figure class="img-caption-sm "><img src="img/portal/report.png" style="width:100%;" alt="Two pages from my research report: Additional research needs and IA & Content Audit."/>
<figcaption>
I ensured all my artifacts had IDs I could easily reference to tie my work back to the raw data. This came in handy whenever the client asked to learn more about an insight. I was able to quickly respond with an exact context, scenario, and timestamp on the fly.
</figcaption>
</figure>
<h2 id="overtime"><span class="h-desc">Week 4</span> Prototyping content improvements</h2>
<p><strong>The report was successful in winning myself a small extension for research and shifting the project's plans entirely.</strong> Instead of chasing minor UI tweaks, the team was now looking to hire their first full-time UX designer to tackle the larger recommendations in my report.</p>
<p>In our last week, I worked with my PM analyst to immediately schedule interviews with new hires and infrequent users on the portal. We hoped that by targeting these groups, we could better flesh out our "Uninformed" provisional persona.</p>
<p>I also planned to append contextual interviews with experimental prototypes to begin testing some content design hypotheses for the homepage of the portal.</p>
<h3 id="prepare-insitu">Preparing for in-situ prototyping</h3>
<p>To add on to our interviews, I took inspiration from John Whalen’s book “Design for How People Think, where he recommends presenting several versions of a prototype during early tests. This way, users can articulate which parts of a prototype they like and dislike. It can also reveal further unmet needs or nuances.</p>
<p>I created three greyscale prototypes: the original portal design, a portal that the client liked (which was “pretty” but not necessarily functional), and a portal that was designed based off of findings in our initial research on content and IA.</p>
<p>In my first pass at the research-informed design, I reduced technical jargon, removed unnecessary or repetitive design elements, and added clearer labels.</p>
<figure>
<img src="img/portal/Before.png" style="width:100%;" alt="The old design has a navigation, a search bar, three tabs, and a list of popular service cards. The new design removed the tabs as they were repetitive to the navigation and introduced larger cards with more descriptive iconography."/>
<figcaption>
The old design featured a list of cards that contained "popular" services with confusing language.
</figcaption>
</figure>
<figure>
<img src="img/portal/After.png" style="width:100%;" alt="The new design removed the tabs as they were repetitive to the navigation and introduced larger cards with more descriptive iconography."/>
<figcaption>
My first redesign aimed to improve the content and minimize noise and redundancies like the tabular navigation and repetitive icons.
</figcaption>
</figure>
<h3 id="contextual-interviews">Early findings</h3>
<p>In the first couple of contextual interviews, we learned that users were still struggling to understand the language of the portal and how to navigate from the homepage.</p>
<p>In all versions of the prototypes, including my redesign, the content remained to be a barrier-- it still wasn't in the user's language.</p>
<p>It seems obvious in hindsight, but “ordering hardware” is not as clear as “order a mobile phone.”</p>
<figure>
<img src="img/portal/prototyping.png" style="width:100%;" alt="Four sticky notes with scribbled suggestions are placed above printed versions of the wireframes."/>
<figcaption>
I asked users to think aloud and use sticky notes to add wanted features. This acted as a catalyst for follow up "why" questions during in-situ prototyping activities and made for valuable artifacts to take back to the client.
</figcaption>
</figure>
<p>By the end of the week, I made further improvements to the prototype based on 4 additional contextual interviews of beginner users. I packaged these up into a Zeplin document which I passed on to the development team for future reference.</p>
<ol>
<li>I removed the icons from all of the cards and focused on improving content and categorization. This would allow the team to focus on the content first, and prioritize understanding the user's language.</li>
<li>I recommended introducing “recently used services” to anticipate the needs of more advanced users without the extra clicks.</li>
<li>I introduced “popular services for employees like you” due to mismatched expectations between services used most by all employees vs. services used by specific types of employees.</li>
</ol>
<figure >
<img src="img/portal/finalui.png" style="width:100%;" alt="Two pictures. The first is of my typographic scale and style guide. The second is a full high-fidelity mockup of the portal." />
<figcaption>
The final prototype I handed off was designed to nudge the team towards smarter, more accessible content choices like including plain language descriptions to services.
</figcaption>
</figure>
<h2 id="conclusion">Parting gifts and conclusion</h2>
<p>Although my time on the project had come to an end, I wanted to keep the team on track for its transition from stakeholder-driven decision-making to human-centered prioritization. Before officially leaving, I arranged some parting gifts.</p>
<ul>
<li>I provided a user interview crash course to the PM analyst and handed him a copy of Steve Portigal’s “Interviewing Users.”</li>
<li>I conducted an atomic UI inventory audit to create a basic style guide for the development team to reduce redundant elements, begin to standardize their work, and provide a launchpad for their new UX designer.</li>
<li>I wrote parting notes to each of the team’s members and provided a list of recommended readings. I requested the manager to purchase a copy of Adam Silver’s “Form Design Patterns” for the team.</li>
</ul>
<figure>
<img src="img/portal/library.png" style="width:100%;" alt="Two black bookshelves on top of a yellow rug with small grey chairs."/>
<figcaption>I made full use of the library I curate at work as an educational space. Here, I was able to introduce members of the team to design and provide them recommended reads to checkout on the spot.</figcaption>
</figure>
<h3 id="nice-emails">A kind email from a new UX ally</h3>
<figure>
<blockquote>
<p>We really appreciate all the help and dedication you brought to our team! Especially enabling us to be more self-sufficient in designing for the user in the future.</p>
</blockquote>
<figcaption>PM Analyst</figcaption>
</figure>
</div>
</div>
</div>
<!--Container-->
</article>
</main>
<!-- Footer -->
<footer class="footer">
<div class="container">
<h2 class="sr-only">Footer</h2>
<div class="row">
<div class="col-4 col-md-2">
<h3>SITEMAP</h3>
<ul role="list">
<li><a href="work.html">Work</a></li>
<li><a href="speaking.html">Speaking</a></li>
<li><a href="about.html">About</a></li>
</ul>
</div>
<div class="col-4 col-md-2">
<h3>CONTACT</h3>
<ul role="list">
<li><a href="resume.html">Resume</a></li>
<li><a href="mailto:joshkimux@gmail.com">Email</a></li>
<li><a href="https://www.linkedin.com/in/joshkimux/">LinkedIn</a></li>
</ul>
</div>
<div class="col-4 col-md-2">
<h3>A11Y</h3>
<ul role="list">
<li><a href="accessibility.html">Statement</a></li>
<li><a rel="external" href="https://www.a11yproject.com/spotlight/josh-kim/">Spotlight</a></li>
<li><a rel="external" referrerpolicy="strict-origin" href="https://a11y-webring.club/next">Webring</a></li>
</ul>
</div>
<div class="offset-0 col-sm-6 offset-md-1 col-md-5 offset-lg-2 col-lg-4">
<h3>HAIKU</h3>
<ul>
<li class="espresso">Need more espresso</li>
<li class="depresso">This site gives me depresso</li>
<li><a href="portfolio.html">Despite progresso</a></li>
</ul>
</div>
</div>
</div>
</footer>
<!-- js -->
<script src="js/jk.js"></script>
</body>
</html>