After defining a problem, brainstorming, sketching, and pinpointing our best solutions, we could dive into the “prototype” stage. While this approach can certainly be effective, an alternative is to run a concept test before moving forward with prototypes.
Concept testing is a user research method that’s mostly used in marketing and product design.
The goal of a concept test is to get user feedback before investing time and resources in developing an idea more fully. We can think of a concept test as a traffic light. Based on the test results, we might go forward with an idea, slow down to revise it, or stop pursuing the idea altogether.
Concept testing gives us the opportunity to directly connect with users at a pivotal point in the design process. We can empathize with users when they are not in the room with us, but it is also valuable to learn exactly what they are thinking. We can feel more confident in our decisions if we ask the users directly, rather than inferring what they want based on prior research.
Running a concept test can help us:
- Validate an idea or determine if it is not viable
- Save resources by gathering user feedback early on
- Address assumptions that were made in the solution or problem statement
- Spark data-backed iterations and uncover alternative solutions
- Clarify how the design can actually meet user needs
We should keep in mind that a concept test does not fulfill the same purpose as a usability test. After developing a prototype, we run a usability test to observe how users experience a design. By contrast, the concept test helps us understand which design ideas might be most valuable for users.
The “when” and “how” of concept testing will vary for each project, but we can always start by outlining our goals. Once we decide what we want to learn, we can determine how to run a concept test that will fetch the desired insights.
As a general rule, we don’t want to run a concept test too early or too late. If our design concept is underdeveloped or unclear, it won’t be worthwhile to elicit feedback. The design will probably change significantly, and the users’ comments may become irrelevant. On the other hand, if our concept is overdeveloped to the point of being inflexible, then we likely can’t implement the feedback in a meaningful way. Thus, we should test when we have a clear concept that we are willing and able to adapt.
We can run multiple concept tests during a project, but the tests involve time and resources so we want to be intentional. While concept testing may fit well at the end of the “ideate” stage, it can be implemented in other stages.
If you have studied user research, the methods for concept testing may be familiar. Common methods include surveys, interviews, focus groups, and A/B testing. In a concept test, we borrow familiar research structures and tailor them to fit our goals.
To select the best method for our test, we can consider the following questions:
- Do we need quantitative data, qualitative data, or both?
- What can users tell us that we can’t learn from other sources?
- How will we measure and evaluate users’ responses? Do we need a certain number to react positively in order to move forward with an idea?
Regardless of the method, we must be ready to explain the concept to people who are unfamiliar with it. One strategy for communicating the concept is to use artifacts. Sketches, diagrams, or a written explanation could be used to clarify what the concept is and how it will work.
Additionally, we want to be aware of our own biases. Whether we feel thrilled or discouraged about the concepts, we should avoid revealing these feelings to participants. Even a written question could be phrased in a way that influences participants’ responses. We should design a test that encourages participants to provide their honest reactions — the data should reflect their true opinions, not what they think we want to hear.
Concept Test Interview
While there are many methods for running a concept test, we’ll explore the interview method.
Interviews are an effective method for gathering qualitative data about a design concept. This is a moderated technique that can be facilitated in-person or over a phone or video call.
Here’s how we could set up our concept test interview:
Set a research goal. Describe what you want to learn from this concept test.
Create a research question. The research goal describes the intended outcome, and the research question gives you a starting point. You will achieve the research goal by seeking answers to the research question. (The research question can be broad. In the interview, you will ask specific questions based on this broad question.)
Create an interview script. Decide how you will explain the concept. Create or select artifacts that can help clarify the concept. Include enough context so that participants know how the concept will work.
Write open-ended interview questions. Each interview question will explore a specific aspect of the research question. Make sure that the questions don’t reflect your personal biases or lead the participants to answer in a specific way.
Get feedback. Test your script and artifact on a team member or two. This can help you work out any glaring issues before speaking to participants.
Identify relevant participants. The participants should be potential users of the design. If the participants have no use for the concept, it will be difficult for them to provide relevant feedback. Review previous user research to identify key characteristics of your users.
Schedule and facilitate the interviews. Be curious and open-minded — don’t approach the session with a set idea about how users will respond. Ask follow-up questions to investigate why participants feel the way that they do. If you’re testing multiple concepts, swap the order of the concepts for each session. This mitigates the possibility that participants favor a certain option simply because it was presented first.
After running a concept test, we should pause to reflect on the results. Once we have organized and interpreted our findings, we can share the results with our team members and decide what to do next. This could entail a bit of discussion and debate! Like other forms of user research, the results of our concept test will not definitively reveal our next steps.
After analyzing the results of a concept test, we could:
- Combine ideas that appealed to users for different reasons
- Iterate on an idea to better align it to user needs
- Create a prototype based on an idea that users responded positively to
- Set aside an idea that is misaligned with user needs
- Refine the problem statement to better match user needs
Think about the answer to the following question to check your understanding of concept testing.
What is the goal of concept testing?