Testing Research Methods

Generative research focuses on discovering opportunities early in the design process, while evaluative research assesses an existing concept, design, or prototype to discover issues or opportunities to iterate.

Testing research methods, such as concept testing, usability testing, heuristic evaluation, A/B testing, and accessibility audits, are used to assess an existing concept or product through hands-on testing by either users or expert evaluators.

Exploratory research, also known as front-end evaluation or discovery and framing, happens before beginning to develop a product or design, and can help justify expending resources on a project. This can include evaluative methods such as concept testing.

Formative evaluation happens during the design process when there are already prototypes to assess. This process focuses on the development and refinement of a design.

Summative evaluation happens at the end of the design process, after the design or product is launched. When it comes to evaluative research, organizations tend to focus on this stage.

For example, we might run a usability test once the new website section has launched, or run A/B tests to decide which of two or more designs is most effective.

Testing and evaluation allow designers to:

• challenge assumptions
• make decisions based on actual data from users
• build empathy for users
• get a sense for both broader trends and for the variation among individual experiences

Mixed methods research, which combines a variety of research methods, offers more holistic insights than any one method alone. For a mixture of perspectives, designers should consider combining listening and testing methods, or different types of testing methods that provide different types of data, such as qualitative and quantitative, or attitudinal and behavioral.

Heuristic evaluation is a form of usability testing in which a group of expert evaluators classifies usability problems based on the ten usability heuristics and ranks issues based on severity.

This method can save time and money by honing a product using internal resources, but should not be seen as a substitute for completing testing with actual users.

Usability testing is a research method that directly observes users as they complete tasks on a website, app, digital prototype, or other interface. Usability testing offers a mixture of qualitative insights and a small sample size of behavioral, quantitative data.

Moderated usability testing can be conducted either remotely or in-person. With this method, a tester facilitates the user test synchronously, walking the participant through the tasks live.

In unmoderated usability testing, the testing team sets up the test in an existing online platform and a user records themselves speaking aloud as they run through a series of tasks. This form of testing is typically conducted remotely.

An accessibility audit is an audit of a website or design according to Web Content Accessibility Guidelines (WCAG) to ensure that the website is perceivable, operable, understandable, and robust for people with disabilities.

An accessibility audit should combine both automated tools and manual testing to uncover accessibility issues.

Concept testing is a front-end evaluation method that evaluates the feasibility, appeal, and potential success of a product before developing a design further or releasing it to the public.

This can be seen as a hybrid of user research and market research, allowing users and stakeholders to influence the direction of a design from the beginning.

A/B testing, aka split testing, is a testing method that compares web analytics from different variations of a design, typically focused on specific testable elements with a clear success metric to compare. This can be done for either a single variant (testing two options) or a multivariate test (multiple options).

Eye tracking is a technique that can be used alongside traditional user testing. It uses infrared technology to track a participant’s eye motion across an interface to show what they are drawn to and how long they linger on different areas of the design.