As a researcher, you love data. As a researcher, you love developing data-driven narratives. As a researcher, you love challenging assumptions and making a positive impact on the business. You love almost everything about being a researcher, but you hate open ends.
All that nice, quantitative data ruined by slang, swears, memes and misspellings. That wealth of information, depth, and nuance locked behind the tedious, time-consuming (and/or expensive) task of coding. As a result, many insights organizations don’t fully utilize the responses from their open-ended questions. Perhaps they code just a sample, use them only for survey quality control or cherry-pick verbatim quotes for their reports (we won’t tell).
The biggest risk of this approach is missing critical insights that may be hidden in the unstructured text data of open ends. The last several years have demonstrated the importance of empathy for consumer-facing brands. Missing the emotional value expressed in open ends could make the difference between a good campaign and a great campaign. Open-ended questions are also one of the most effective ways of generating unfiltered, unbiased consumer feedback.
Let’s take a look at how open-ended questions are used through the lens of several research examples.
Consumer Behavior Study: Open-ended questions in consumer behavior surveys can be used to better understand consumer attitudes, particularly when it comes to new or quickly-evolving topics. Perhaps most importantly, open-ends are critical in capturing the emotions and emotional intensity consumers have towards a topic.
- Examples: How do you feel about cooking at home? What does sustainability mean to you? Describe your ideal vacation.
Product Feedback / Concept Testing: Open ends in product feedback surveys are essential for capturing the attributes consumers recognized and/or valued in a product, particularly when using a predefined set of attributes is not advisable. Open ends are more likely to return unexpected feedback.
- Examples: What did you like most about [PRODUCT X]? How would you describe the taste? What do you think is missing? What is the biggest benefit you would get by using this product?
Audience Feedback / Ad Tests: Audience feedback and ad test are all about the feels, aka emotional response. Analysis is likely to center on sentiment/emotion measurement (how did the viewer feel about the content?) as well as the elements, moments or characters that drove this reaction.
- Examples: What did you like/dislike the most? What was confusing/interesting? What do you remember? How would you describe the character?
Customer Experience (CX) Surveys: Perhaps no survey has become as ubiquitous as the Net Promoter Score (NPS) survey. Regardless of the specific customer satisfaction methodology used, almost all incorporate an open-ended question that aims to get at the “why” behind the rating a customer gave. When analyzed, particularly in conjunction with the rating cohorts, they can reveal powerful insights regarding the customer experience.
- Examples: What is the primary reason for your score? What is one thing we could do to make you happier?
Brand and Awareness Study: Open-ended questions are particularly important for unaided type questions (unaided awareness or unaided brand attributes). Additionally, open ends can be used to better understand the emotional resonance of brands with consumers.
- Examples: When you think of anvils, what brands come to mind? When you think of Acme, what comes to mind first? How do you feel about the Acme Corporation?
So you’re almost ready to love open ends again. But…what about the significant time and resources needed to decipher and code in addition to potential for human bias and error? New research technology like Canvs AI is giving insights professionals the power to automate or semi-automate the coding of open-ended responses, bringing quantitative scale and statistical significance to this form of qualitative feedback. This has the potential not only to accelerate discovery and enhance analytical confidence, but also to expand the potential for open-ended questions in research.
