Organizations no longer control the conversation about their brand. Users select the place, the time, the medium, and the way they interact with you.
From smartphones and streaming content to social media platforms like Facebook and Reddit, consumers are talking about and engaging with your brand...and you’re not even in the room. That’s why experience matters. Event marketing, in-store experiences, digital communications, your website or app are a few of the touchpoints you still control. The organizations that effectively leverage these direct touches and improve their customer experience are industry
leaders and business beacons.
However, there’s an industry-wide problem in how we measure experience. Many researchers are still using traditional KPIs like interactions and conversions to measure ROI. These KPIs add value and shouldn’t be ignored. But, they miss the emotional factors connected to experiences that are better predictors of future behavior. That’s where Key Experience Indicators™ connects the dots.
Read on to learn about Key Experience Indicators™, why we feel Net Promoter Score isn’t a best fit tool, and hear from EBG experts Tricia Houston and Walter Blotkamp for their POV on experiential research.
Why Tools Like NPS Fall Short
There are a few key tools researchers use to measure experiences. Net Promoter Score (NPS), over most others, gained traction over the last 20 years. NPS is a simple calculation with one question and one subtraction that delivers a metric that assesses and monitors a brand’s performance. The underlying metric, a consumer’s likelihood to recommend, attempts to capture or reflect changes on both ends of the customer spectrum. By measuring a consumer’s emotional reaction to a recent interaction through NPS, decision-makers have an easy-to-use number to compare performance against time and competition.
But here’s where NPS falls short.
Negative scores can have too much influence on NPS.
This is because NPS does not accurately reflect the net difference between Active Promoters and Active Detractors. Customers who score a 9/10 certainly have a strong, positive emotional experience or relationship with the brand. However, those who score 0-6 include a broad range of consumers who are or may likely be negative. This greatly overstates the negatives, which can make subtraction of Active Promoters and Detractors misleading.
NPS tends to be highly volatile.
The wide range that makes up the Negative Detractor profile leads to increased variance and volatility that reflects spikes and falls of 20 points or more between periods. Consumers who might have an average or slightly below average experience can swing wildly across the Negative Detractor spectrum.
NPS varies greatly depending on the level of a consumer’s interactions with the brand.
NPS scores drastically decline as you move through the brand funnel. Consumers who are only familiar or aware of the brand are going to negatively influence the Net Promoter Score. This is readily apparent for smaller brands that are still increasing their market share. Negative scores for these brands don’t reflect brand performance. Rather, they simply reflect a brand’s presence.
So, anything that replaces NPS has to also:
Be simple
Capture emotional elements of interaction
Better capture both ends of the spectrum
Permit comparability across brands, products, industries
Enter EBG 5Q™ and EBB Stickiness Score
In 2017, we launched our experience expert group, The ExperienceBuilt Group, to approach experiential design projects with our new way of thinking. Blending KEIs™ and KPIs, The ExperienceBuilt Group not only measures client experiences but identifies and resolves friction points throughout the experience. In collaboration with MMR Research, the The ExperienceBuilt Group developed the EBG 5Q™ to deliver engaging, point-in-time feedback for our clients. The result, the Experience Built Business Stickiness Score. This composite score shows where you are and aren’t delivering against your brand promise. If you’re looking to deepen user connections, create positive emotions, drive loyalty, and develop top-of-mind recall, then look to the EBB Stickiness Score to understand the effectiveness of your processes and business operations.
WHAT IF you could get credit for the transformations you create through experiences?
WHAT IF you could access emotion metrics that truly reflect what people feel as the result of a brand experience?
And, WHAT IF your after-action report focused on connections instead of counts?
We sat down with Tricia Houston, Founder of The ExperienceBuilt Group, and Walter Blotkamp, VP of Design & Analysis at MMR Research, to learn more about KEIs™, the EBG 5Q™, and the value of measuring experiences.
Q: Tricia, what inspired you to dive into measuring experiences?
Tricia Houston: When I started in the marketing industry, we were measuring programs and events. But a lot of it was counting things, ‘how many t-shirts did we giveaway?’, ‘how many people showed up?’, ‘how many days?’, ‘how many impressions do we think we got?’ We were using diagnostic metrics to measure success and designing experiences based on this success. Essentially, X event had Y impressions so we should do it again this way.
But, did we connect with these folks emotionally and give value for their time? Did we make them deeper fans of the brand? For instance, we want to make teens bigger fans of the brand, but does giving away 30,000 cans indicate increased fandom?
Experiences aren’t a digital banner ad, you shouldn’t count them by impressions and ‘clicks;’ you’re devaluing the effort because you’re measuring it wrong.
Q: Why is experience so important?
Tricia Houston: Positive experiences create emotional connections and emotional connection leads to loyalty. Views don’t lead to loyalty. The goal of a modern business is deeper relationships with customers, partners, vendors, and employees. Exposure is easy. Connecting with someone, “standing” right in front of them with something of value (enjoyment of time, tangible products, memories) is the highest order of connecting with stakeholders.
That’s why we founded EBG, to measure experiences and use that data to make experience worth repeating.
Q: What do you do with the measurements and how do you optimize?
Tricia Houston: I often tell folks to start small to go big. Measuring isn’t actually fixing anything; it’s diagnosing. It’s easy to over-measure! The real work is understanding what you find and developing a solution that you can put into action. It can be overwhelming if you try to take on too much too fast. You don’t have to start with a large-scale study, start by having conversations with users or go be your own customer.
See what gaps you uncover and let that be your roadmap to optimization. Eventually, you’ll be too close to see the gaps. That’s when a partner comes in handy…someone to help you see any broken brand promises or operational gaps.
Q: What do you get out of it? How does this approach affect customer loyalty?
Tricia Houston: It allows for personalization, which is essential for doing business in 2022. The EBG 5Q™ approach enables you to monitor an individual score for each customer. That alone is not novel. But the level of personalization you can create is significantly enhanced given the nuance the scoring system provides. Instead of three types of customers as with NPS, you have six groupings to action against. This allows for more targeted responses, offers, and engagement with your users. It’s not all about troubleshooting either, it can be about cultivating loyalty as well. We identify your best customers and advocates who can be used to your advantage. Perhaps form an advisory panel or invite them to exclusive experiences!
Q: Walter, what were important factors to consider when creating the KEI™ model?
Walter Blotkamp: Two things stand out – viability and simplicity. If the approach does not improve upon existing measures, then there isn’t much point in pursuing it. At the same time, the model could not be so complex that implementation and interpretation become difficult. It is very encouraging that brand managers intuitively understand this approach – to the point of having a kind of “why haven’t we done this all along” aha moment!
Many people adopted NPS in part due to the simplicity of a single metric. KEI™ also results in a single metric with the added component of an intensity measure. If everyone truly loved a brand with great intensity, a score of 100 is theoretically possible. Conversely, if everyone truly felt intensely disgusted by a brand, a score of 0 is possible. Of course, those numbers are not going to really occur. But with those as endpoints, users understand that KEI™ rises and falls based not only on brand reaction but on the intensity of that reaction as well.
Q: How do you feel KEIs™ add nuance to traditional KPIs and CLV metrics?
Walter Blotkamp: There is always some scale preference bias among those providing scores, but at the same time it is really possible to “love” an experience to the point of actively representing it positively to others. We know this by many names such as Active Positives, Brand Advocates or Raving Fans. By using a right-brain morphii approach with intensity level, we believe we get a more meaningful look at both recent experience and overall brand attachment.
There are two value-add components here – first the right brain approach of morphii and second a recognition that brand experience involves both embedded memory and recent memory – one question is not enough. By combining these two and giving more weight to the embedded memory while incorporating how a recent experience can reinforce or alter that embedded memory, we improve upon existing measurement techniques.
Q: Can you describe how KEIs™ compare to NPS scores?
Walter Blotkamp: Sure! As Tricia noted earlier, negative scores can have an outsize impact on NPS. NPS does not account for a comparison between Active Positives and Active Negatives – the subset of Positive and Negative. Also, as a single score derived from a single metric, NPS can be biased to a recent experience and thus bounce around over periods of measurement. Brand experience, however, is more nuanced than that.
Experiences build upon each other to form overall impressions that should be more stable over time. This is essentially why great brands can recover from negative encounters while weaker brands cannot. The KEI™ approach results in more stable measurement yet is sharp enough to detect when truly momentous experiences lift or hurt the brand to the point of changing that overall embedded memory image.
An exact number to number comparison with NPS isn’t appropriate because the approaches are different. However, NPS is a relative measure that can sometimes be hard to interpret. For example, I could have 60% positive and 20% negative for an NPS of 40. But I also get an NPS of 40 if I have 50% positive and 10% negative – which is better? There’s no way to tell with NPS. The KEI™ approach looks at this differently by considering an imagery rating and an intensity rating, with the final number being on a 0-100 scale. As ratings change, there will always be a change in KEI™, which isn’t the case with NPS.
KEIs™ That Drive Impact
Instead of counting eyeballs count heartbeats. Key Emotional Indicators™ provide nuance where traditional measurement tools can’t. Understanding how you forge an emotional bond with your consumer will enable you to optimize your experiences and create real business impact. Measuring business metrics like ROI is important and necessary. Often, experiential impact doesn’t have direct ROI, but, building user equity over time has huge value that can impact the bottom line in the future.
