Most performance management systems have a hidden assumption baked into them.
That what you're measuring represents reality.
It doesn't.
It represents the part of reality that your systems were built to accept.
The Score Was Real. The Progress Was Not.
A retail chain had been improving its satisfaction score for two years.
It sat at 4.1 when the new leadership team arrived. By the end of the second year it was 4.4. Store teams had been trained. Signage refreshed. Checkout processes redesigned. The score appeared in the board pack every month. Progress was being made.
Then someone asked a different question. Not "what do customers who fill out the survey think?" but "who fills out the survey?"
The answer was clear once anyone looked. The survey fired at the point of purchase. It reached customers who had found what they came for, completed a transaction, and felt satisfied enough to scan the QR code on the receipt.
Customers who came in, couldn't find what they wanted, and left empty-handed never filled out anything. The people being lost were systematically absent from the data.
Customer lifetime value had been declining throughout both years. Repeat purchase rates were down. New customer acquisition was masking the signal in the headline numbers. Until it stopped.
The score had measured satisfaction among the people who had already succeeded. Everyone else was invisible.
This is not a data quality problem. The data was accurate. Every score was genuine. The system worked exactly as designed.
That was the problem.
An Old Name for a Modern Mistake
There is a name for what happened here.
Survivorship bias.
It has been understood since at least the second world war. During the war, analysts studied bullet hole patterns on aircraft returning from missions. The damage was distributed across the fuselage, wings, and tail. The recommendation was to reinforce those areas.
Abraham Wald pointed out the error. The planes being analyzed were the ones that came back. The missing data was not random. It was informative. Planes hit in the engines didn't return. The absence of engine damage in the dataset wasn't evidence that engines were safe. It was evidence that engine hits were fatal.
Reinforcing where the visible damage appeared would have been the wrong response to the wrong data.
The retail chain made the same mistake eighty years later in a quarterly board meeting. Analyzing the customers who showed up in the dataset. Drawing conclusions about all customers. Building strategy on a selected slice of reality and calling it performance.
The mechanism is identical. The stakes are lower. The mistake is just as common.
The Easy Metric Problem
Survivorship bias in business is not usually deliberate. It is gravity.
Organizations measure what their systems already produce. Transaction volumes. Call resolution times. Conversion rates. Utilization. Revenue by segment. These numbers are not on dashboards because someone chose them carefully. They are there because operations generate them automatically. Clean. Real time. Easy to visualize.
Available is not the same as important.
But in most organizations, available wins.
The alternative — measuring something that actually matters — requires deciding what matters first. That conversation is harder. It forces disagreement. It requires commitment to a definition of success that can be held against you later.
Defaulting to what the system produces is cheaper. So that is what happens.
The Measurement Trap is not about bad intentions. It is about what floats to the surface. The important things tend to stay at the bottom as unmeasured, untracked, and therefore unmanaged.
What Doesn't Fit the Dashboard
Consider what most organizations do not measure.
Decision quality.
Not the speed of decisions, or the number made. Whether the decisions were actually good. Whether the reasoning held. There is no KPI for that. No quarterly review. No accountability structure for revisiting past calls honestly.
Customer trust.
Not NPS — it is a proxy, and a contested one. Real trust. The kind that takes years to build and shows up as repurchase, forgiveness, and unsolicited recommendation. No single metric captures it. So most organizations don't try.
Learning speed.
How fast the organization updates its beliefs when new evidence arrives. A compounding advantage. Never on a dashboard.
These things are real. They have real consequences. They are just harder to turn into a number.
So they don't appear in board packs. They don't shape priorities. They don't get resourced or fixed or optimized.
And the organization calls the things that do appear "performance."
Measurement Is a Design Choice
The antidote is not to measure less.
It is to treat measurement as a deliberate decision rather than a default.
Every metric in use is the answer to a question. The question is usually: what data do we already have? The better question is: what decision do we need to make, and what information would genuinely improve it?
Most organizations run this backwards. They look at what the systems produce, build a dashboard, and call it strategy.
A useful test: for any metric in your current board pack, ask whether it has actually changed a decision in the last twelve months. Not decorated one. Changed one.
If the answer is no, the metric is performing its presence.
You are not managing the business with that number. You are managing the appearance of managing it.
The Measurement Trap is not that organizations are careless. It is that they are systematically attracted to what is countable, consistently unable to see what isn't.
The data you have is not a window onto reality. It is a portrait of the survivors.
