What do gnomes, castles and NPS have in common? They were all critical to Big Fish Games improving player retention, playing time and player revenue through consumer insights. 

Last week, Big Fish Games presented alongside Canvs AI at The Market Research Event (TMRE) in Nashville, TN, one of the largest gatherings for consumer insights professionals. In their talk, Big Fish Games shared an amazing case study of how they were able to analyze unstructured text data from player NPS surveys to improve the experience in their new EverMerge game. 

As a bit of background, Big Fish Games is a mobile gaming pioneer with over 20 active titles available. To compete effectively in the hyper-competitive mobile gaming space, Big Fish Games has adopted a “Player Health Program” to optimize the gaming experience. They’ve collected over one million open end responses from net promoter score (NPS) and player feedback surveys to better understand what players love and don’t love so much to grow the “Player Health Program.”

During their presentation, Stephen Boutin, Consumer Insights Manager from Big Fish Games, outlined why player feedback is so important to the company: “We really aim to have the player’s voice in every part of our decision-making process and our strategy. And so for a good example, we have a green light process, where essentially when we’re looking at a game through the course of its development from early concept phase to beta play testing all the way to worldwide, we really want to make sure that we are really hammering the voice of the player home. So that at each stage of that development process we’re taking into consideration what the player wants and needs. We have a lot of open-end feedback, so it lends itself to a lot of work for us to do.”

They faced a significant challenge that’s familiar to so many market and customer insights teams: how to efficiently “read” and interpret all of those open-ended comments. With a focus on its new EverMerge game, the team at Big Fish Games was able to correlate player satisfaction (as measured by NPS) with specific game play elements mentioned in the open ends. With so many variables potentially impacting player satisfaction, including dozens of characters, the open ends were critical to helping the Big Fish Games team improve the experience for their new game. Here’s where things get fun!

From pre-launch beta through the worldwide launch and beyond, Big Fish Games used Canvs to identify that mentions of gnomes and castles correlated with negative sentiment. In other words, Big Fish Games was able to use the verbatim survey responses to conduct a root cause analysis of detractor scores. Then, as the Big Fish Games development team started to implement changes to the game, the consumer insights team was able to track both the positive change in NPS and declining mentions of gnomes and castles! 

Sarah Gurbach, Consumer Insights Manager from Big Fish Games, captured the challenge they faced during the presentation: “When you’re in the millions [of open ends], it’s really difficult to quantify and group all of those different characters into one category. [With Canvs] I can monitor it and go back to the game teams and say yeah, players are talking about it and the sentiments positive or negative.”

The result of their “player first” approach to developing EverMerge was a 44% increase in NPS from the worldwide launch into the first round of game development updates, which coincided with increased playing time and more frequent repeat players.

The Big Fish Games story is a great reminder of the power of verbatim responses to understand how customers (or players) are feeling and why. Unfortunately, so much of this feedback gets wasted due to the cost and time of organizing open ends. For Big Fish Games, Canvs provided the ability to include all responses in their analysis at scale. Truly an impressive team generating impressive results that more than prove the value of consumer insights for the business. 

Speaking of which, we’d like to extend a BIG thank you to Sarah Gurbach and Stephen Boutin from the Big Fish Games team for sharing their story at TMRE! We’re proud to work with such a smart, talented team and help contribute to their success.