Since its inception in 2003, Net Promoter Score (NPS) has long been considered the gold standard for measuring customer loyalty and perception.
Developed by Fred Reichheld and Bain & Company, it was created with both brand and consumer in mind with the premise that a single question could provide a leading indicator of customer loyalty (followed by an open-ended question to understand the ‘why’ behind the score).
The goal was to give brands a credible methodology for measuring loyalty and provide simplicity for consumers, avoiding the lengthy customer feedback surveys of the past. More importantly, Reichheld’s research showed a consistent correlation between relatively high net promoter scores and relative growth (relative among peer companies) across industries. Fast forward to 2022 and you’ll find NPS along with other market research metrics have grown in popularity and represent a growing percentage of survey types.
In this article, we’ll examine how brands can extract more value from NPS by focusing on the why (covering two fundamental ways to approach this) and analyzing open-ends to generate better insight.
What is NPS & how is it calculated?
As we’ve highlighted, NPS has become a vital metric for many organizations to indicate the health of their brands, products, services, and overall customer experience. The core metric is derived from a single question, most commonly:
“How likely are you to recommend this product or service to a friend or colleague?”
Respondents will then provide a score based on a scale of 0 to 10 (where 0 is not at all likely and 10 is extremely likely which are then broken into 3 categories (Promoters, Passives, and Detractors). Promoters are defined as those who answer 9 or 10 on the 0-10 scale, Passives are those who answer 7 or 8, and Detractors are those who answer 6 or below. In addition to this question, an NPS survey will include at least one follow-up question (often an open-end) asking for a reason behind the score. The score is then calculated by subtracting the percentage of detractors from the percentage of promoters (passives are not used in the formula). For example, if 20% of respondents are detractors, 10% are passives and 70% are promoters, your NPS score would be 70-20 = 50.
Many NPS surveys will now go beyond this two-part questionnaire, asking a series of follow-up questions (both open-ended and closed-ended) to uncover further insight surrounding particular areas of the product, service, and experience. In this way, NPS (and similar CX surveys) can be used as a diagnostic tool, identifying aspects of the customer experience that correlate with Promoters or Detractors. But only if the survey instrument captures the right contributing factors.
It’s useful to also acknowledge the active debate on how NPS is measured, specifically its application in measuring customer loyalty. That’s right, NPS isn’t without its own detractors. A 2019 HBR article ‘Where Net Promoter Score Goes Wrong’ by Christina Stahlkopf (Director of Research and Analytics at C Space) makes a case for the limitations of NPS methodology involving a more complex “jobs to be done” rationale for how consumers actually recommend product, services, and brands in the real world.
This particular snippet is taken from the research that C Space conducted illustrates the complexity behind customer loyalty:
Another consumer hated Walmart, giving the company a zero on the NPS scale. “The stores are filthy and overcrowded,” he noted. But, he continued, “I recommended Walmart to a friend because they had an inexpensive desk that would be perfect for the friend’s room.”
One might consider this customer to be both Detractor AND Promoter, something which the ultimate question and the pre-determined follow-up questions might be unable to uncover. One response to this challenge would be to better utilize NPS open-ends to uncover these nuances. However, even this can be difficult to capture in most NPS surveys, and more likely uncovered through qualitative research (including the survey open-ends, more on that to follow). Still, others argue (and personal experience confirms) that the length of NPS surveys can have an impact on response rates and potentially impact the statistical significance of the sample.
Despite the debate, companies have clearly voted with their actions. NPS and the like don’t seem to be going away anytime soon. So how should companies think about maximizing the value of their NPS surveys? By designing to best capture the “why” behind the rating, the survey is more likely to identify your CX “constraints” and the path to addressing them to improve overall CX (and the business performance). As we’ve pointed out (and not for the last time), open-ended questions are a key tool for uncovering these unstructured insights.
Turn the ‘why’ into your superpower
While NPS may be a valuable gauge of customer experience and the processes supporting it, uncovering why Promoters, Passives, and Detractors gave a particular response turns a metric into something that’s incredibly powerful – an actionable business diagnostic instrument.
There are two fundamental ways of uncovering the ‘why’ (both of which have their own pros and cons).
The first approach is to ask closed-end follow-up questions regarding specific aspects of the experience, service or product. This will give you quantitative data that’s quick to answer, easy to analyze and specific in nature. This approach is best suited to gathering feedback regarding specific areas of strategic focus or known areas of weakness. However, as with the nature of close-ended questions, responses are restricted to predetermined choices. Additionally, the desire for different parts of the business to capture feedback can add significant length to your survey, which may negatively impact response rates.
The second approach is to utilize open-ended questions and get customers to tell you about their thoughts/feelings/experiences in their own words. This gives additional freedom to the customer to tell you about their experience but requires a considered approach if you’re to analyze feedback with speed, efficiency, and clarity. To help, here are some considerations with your open-ends to yield the highest response rates and best experience.
One approach that many organizations choose to take is to customize the open-ended questions based on the initial “likely to recommend” response. For example, with Detractors you might want to pose a question such as “What could we do to improve the experience?” This immediately adds context and hints to the customer that you’re actually listening to their feedback. For Promoters, you might approach the follow-up alternatively such as this ‘We’re delighted you feel that way, we’d love to understand what made your experience?’
To give you some further options, we’ve also provided some sample NPS open-ends below that you might consider using in your survey:
General
What is the primary reason for your score?
Why did you provide that rating?
What is one thing we could do to make you happier?
What most influenced your recommendation?
Rating Specific
What was missing or disappointing in your experience?
What could we do to improve the product or service?
How do you benefit from using our product/service?
Depending on the goal of your NPS survey, consider what you’re looking to uncover and choose the appropriate open-end to help you achieve this.
Analyzing NPS survey open ends
As we recently discussed in our blog post on analyzing open ends, there are generally three ways open-ended questions are classified:
Topics: This is performing a general content analysis of what people are “talking about” in the open ends. The key here is distilling or classifying natural expressions into topics and/or themes.
Codes/Nets: This is the hierarchical categorization/classification of themes as expressed in the open ends generally utilizing a code frame.
Sentiment/Emotion Analysis: This is extracting general sentiment and/or nuanced emotional reaction from the text to understand how the respondent feels.
All of these are relevant in analyzing the open ends in CX surveys. In the context of root cause analysis, identifying common themes or topics that emerge may help you identify factors driving competitive differentiation and loyalty (or lack thereof). Emotion analysis can also be useful for capturing a more nuanced view of how customers are engaging with the brand. For example, understanding the context when an advocate may be expressing negative emotions, or (as in the case of our Walmart shopper) a Detractor is expressing positive emotions. Finally, the verbatims themselves are often useful as a way of putting a human “voice” behind the data in your reports.
However, one of the most challenging tasks is analyzing the unstructured text data. The difficulty is being able to efficiently “read” and interpret all of the open-ended feedback at scale despite the myriad ways people express themselves. Typical approaches to analyzing open ends have significant disadvantages when it comes to NPS surveys. First, the volume of comments in NPS programs is likely too overwhelming for a hand-coding approach. So organizations may be more likely to simply cherry-pick verbatim quotes. Automated approaches on the other hand typically result in superficial analysis and can miss nuance, slang, sarcasm, etc. In either instance, organizations are not generating statistically significant insights from these comments.
As a result, CX and insights professionals are likely to lack confidence in open ends as a source of actionable data. However, new research technology like Canvs AI is giving insights professionals the power to automate or semi-automate the analysis of open-ended responses, bringing quantitative scale and statistical significance to this form of qualitative feedback. Fully analyzing the open ends in NPS surveys allows organizations to quickly identify the issues driving positive and negative feedback.
To give you an example, a major gaming company we worked with identified that Detractors were mentioning specific gameplay elements at a higher rate than others. By modifying elements of the game they identified through Canvs, they significantly improved their NPS score, and most importantly, the impact on the bottom line was clear with the company seeing an increase in player retention, average spend, and average time spent in-game.
Net Promoter Score (NPS) surveys are an important arrow in the CX quiver, but their real power emerges when they can be used to actually improve the metric their measuring. Open-ended questions provide a critical channel for the customer’s voice to be heard and when analyzed in the context of the broader survey, can deliver powerful insights for root cause analysis.
Interested in learning more about analysis of open-ended questions? Check out our latest eBook: The Essential Guide to Insights with Open-Ended Analysis.
