UX Research Team - Measuring Impact

Goals

Determine how Internal Design Research teams (also called User Research teams or UX Research teams), specifically those in the tech industry, track and document impact in their organizations. Identify the tools they use, what metrics they track, how they define impact, how they delegate the role of tracking impact, different strategies for short-term and long-term impact, and whether they use in-house or purchased solutions. This information will be used to track and demonstrate the impact of Design Research at Dropbox.

Early Findings

Metrics for Impact

  • To demonstrate impact, internal design research teams can show how X research insight led to Y decision and Z design change, which positively impacted a value metric.
  • Teams can also survey internal stakeholders to gather opinions on the perceived impact of a specific research outcome.
  • Other quantitative markers of success may include the number of beta tests to try out new features based on research, the number of usability tests run, the killing of features not needed by users, and the decrease in customer support questions about functionality.

Tools and Frameworks for Measuring Impact

  • The AARRR framework by Dave McClure tracks acquisition, activation, retention, referral, and revenue.
  • The RARRA framework by Gabor Papp tracks retention, activation, referral, revenue, and acquisition.
  • Hue is a UX measurement tool that aims to measure financial return, set design targets, and create what-if scenarios for the design and business teams.
  • Survey software like Qualaroo, Satismeter, Wootric, and Usabilla can track user perceptions of a product.
  • Software like Google Analytics, Mixpanel, CrazyEgg, and HotJar provide analytics on how users are using a website.

Roles in the Internal Design Research Team

Proposed next steps:

You need to be the project owner to select a next step.
Based on our preliminary hour of research, we found a diversity of information on how to measure the impact of internal design research teams in the tech industry. We recommend continuing our research to give an industry overview of the ways internal design research teams measure impact. We would dedicate one request to identifying what tools UX research teams use, what metrics they track, how they define impact. A second request would research how they delegate the role of tracking impact, different strategies for short-term and long-term impact, and whether they use in-house or purchased solutions. The scope is global.
Furthermore, we also recommend identifying 2-3 case studies of successful user experience research teams. For each case study, we would identify the company they worked in, the type of product they worked on, the roles within the team, how they measured impact, and metrics for their success.
Alternatively, we could identify 4-5 best practices used by user experience research teams to measure impact. For each best practice, we would give a description of the practice, explain why it is a best practice, identify at least one company using the practice, and provide any measure for success.