Customer Experience Lab

Busted brackets: What data-driven decision-making feels like

By Ian Sherman on March 28, 2018

Bracket busted? Mine too. ¯\_(ツ)_/¯

Am I disappointed? A little. Surprised? Not really. Embarrassed? Not in the slightest.

But how can I not be embarrassed? I’ve spent the past two months writing about, speaking about, and getting shout outs about picking the perfect bracket. What was that all about, if not ensuring that I would win March Madness?

Ready to up your CX game? Start a free Centriam trial today!

At the risk of copping out, I never claimed that I would win March Madness. I claimed to maximize the probability of winning March Madness. If you don’t believe me, go back and check my previous blog post, or download my slides. I gave myself a one-in-twelve chance of winning, four times better than average, but far from a sure thing.

I bring this up because it serves to illustrate what data-driven decision-making can feel like.

If the result of a data-driven decision turns out to be negative, it can feel even worse than when the results of regular decisions turn out negative because there’s more preparation, more at stake. As the old saying goes, “The bigger they are, the harder they fall.”

Going into Final Four weekend, with a bracket full of red, struck-through picks, I could be depressed:

Data-driven decision making - busted bracket

But I’m not. I stand by my bracket. I’m not surprised to be wrong because I knew, and took seriously, something called the base rate. The base rate is simply the starting, “base” probability of a given outcome. In this case it’s just the probability that a given bracket is going to win a given pool, which in my case is one in 50 or 2%.

Base rates are very important and often misunderstood. They’re misunderstood so frequently that psychologists have a name for it: the base rate fallacy, or base rate neglect. Very briefly, base rate neglect applies when people care more about individual details of a situation than baseline probabilities when estimating the likelihood of a given outcome.

 "If the result of a data-driven decision turns out to be negative, it can feel even worse than when the results of regular decisions turn out negative because there’s more preparation, more at stake." [Click to Tweet!]

You (or at least I) can feel base rate neglect right now while pointing out the inconsistency between putting so much work into picking my bracket and losing my March Madness pool. Because I know all the steps I took and all the decisions I made, it’s natural to overestimate my chances by simply focusing on the details of the actions I took. It feels like if I put in as much work as I did, I should expect to win.

But that feeling neglects the base rate. Because I start at such a low probability of winning – 98% of brackets lose – even extremely effective bracket-picking techniques can only be expected to improve the odds by so much. By keeping in mind the base rate, even after improving my odds several-fold I still expect to lose more than 90% of the time.

Want to learn more about data-driven decision making and how organizations are turning insights into actionable and scalable customer experience initiatives? Download the Centriam CX Product Overview.

What does this mean for data-driven decision-making in a business context? It means we should all take seriously the base rates of the outcomes we expect to see. If a customer has a 99% chance of never making another transaction, even the best Customer Experience interventions might only decrease that chance to 95%. It doesn’t mean we shouldn’t try; we just need realistic expectations.

data driven decision making - final four

As for me and March Madness, I’ve started building an Artificial Intelligence model on top of the simulator to try to improve my odds even more. With the 2019 Final Four here in Minneapolis, come next spring you might just find yourself reading about a March Madness prediction robot.

And it won’t neglect the base rates.


Topics: Customer experience analytics

Author: Ian Sherman

Ian helps Centriam’s clients ground their decision making in data and analytics. Ian has over 10 years of experience in advanced analytic techniques to turn data into insight and action. Prior to Centriam, Ian led the Marketing Analytics function at G&K Services where he identified and quantified untapped opportunities in customer experience, product penetration, and pricing. Ian also has experience working with diverse organizations including travel & hospitality, retail, manufacturing, telecom, and healthcare. Ian holds a B.S. in physics from Washington University in St. Louis and an M.B.A. from the Carlson School of Management at the University of Minnesota.
Find me on:

Explore the latest customer experience insights from Centriam

We're on a mission to empower organizations to use data insights to improve customer experience and measure the business impact of customer programs. Through the Customer Experience Lab, we share our latest insights with you.

Subscribe to email updates

Recent posts