3 Methods, 6 Weeks
Background
In years past, research at SAP Concur typically consisted of in-depth moderated usability sessions. These sessions were conducted in-house with locally sourced participants and could take up to weeks or months of execution before results were ready to be shared out to the design teams. This gave the research team a reputation of being slow, which had deterred some teams from reaching out for support.
I challenged this status quo by executing on 3 different research methods within 6 weeks, providing actionable insights to the UX design team.
Summary
In August 2018, the mobile UX design team requested research on a reimagined design for the Travel Allowance product. The team was updating the overall mobile UI and wanted to understand whether the new design was an improvement upon their previous design or if it would introduce new confusion.
My study plan was to:
· Understand the product & user goals through SME interviews
· Gather baseline metrics to understand how users navigated the existing design
· Conduct usability studies using the new design to gather qualitative insights
SME Interviews
Time taken: 2 weeks
At SAP Concur, one of our greatest internal resources are our implementation specialists. These are the employees who set up and configure our product solutions for our customers. This means that these employees understand the many ways that our product can be utilized, what those use cases are, and the many workarounds that customers use when our product doesn’t support the complex use case they may need.
I kicked off my research with 7 SME interviews with recommended experts on the Travel Allowance product. These interviews were attended remotely by members of the mobile UX design team in the US and Germany. From the interviews, we were able to discern key themes and areas of focus, like reducing duplicate data entry and creating parity between the mobile and web products. These interviews gave us clues to the top areas where we could expect to see users struggle with our product, as well as insights into new areas that the design team should focus their efforts.
Unmoderated Baseline
Time taken: 1 week
The next task was to determine the benchmark for usability of the existing product. Our team utilized a prototyping analytics tool in addition to our unmoderated testing platform. This allowed us to collect raw qualitative data and analytical insights during the unmoderated study.
The lead designer created a click-through prototype of the existing system with screenshots of the live product. I setup the study details and recruitment criteria for our session.
Less than a day after launching the study we had 10 completed video recordings to synthesize.
With our prototype analytics tool, we were able to create heat maps that showed where our users were clicking on the screen during the task. In many cases, users skipped over data entry fields or failed to recognize the primary action button that would take them to the next screen.
Armed with the analytics and the unmoderated videos, I synthesized the key takeaways from the study and specific pain points that users encountered. The design team was able to use this information to inform their design before the next round of testing.
Moderated Study
Time taken: 2 weeks
(Following 1 week of design iteration)
While the designers focused on incorporating the findings from the SME interviews and unmoderated study into their new design, I began to develop the study plan for our moderated usability study.
I developed the:
Left: Baseline design with heat map showing action button confusion. User was expected to click the button at the top right of the screen.
Right: Modified design with a clear action button at the bottom of the screen.
Screener criteria
Pre-study interview questions
Task
Post-study questions
Reporting template
I conducted 9 remote moderated studies with users in North America, Europe, and Asia. The tested design showed significant improvements from the baseline study, especially in areas where users had previously struggled.
Results
Total Time: 6 weeks
Each round of testing produced actionable insights that the design team was able to incorporate into their final design. Once the final design was tested in the moderated usability studies, we saw that many of the changes resulted in noticeable improvements to the user experience.
In just 6 weeks, I coordinated and executed 3 unique research methods that built upon each other to inform the design direction. This design direction is being moved to production in 2019.