In 2016, Wells Fargo was penalised by the US regulator: employees had opened millions of bank accounts without customer consent. Credit cards nobody asked for, savings accounts created then emptied, forged signatures. Thousands of staff were dismissed.
The easy reading is collective moral failure. The useful reading is more uncomfortable: those employees were doing exactly what they had been asked to do. The bank had set aggressive cross-selling targets, products per customer and per household. The KPI was clear, tracked, and tied to pay. It was met. It simply no longer measured what it was meant to measure.
This is not an isolated accident. It is a mechanism, it has a name, and every analyst should know it before proposing their first metric.
Goodhart's law, and what everyone misquotes
The line you see everywhere is this one: "When a measure becomes a target, it ceases to be a good measure."
It is an excellent line. It is not Goodhart's.
That phrasing belongs to the anthropologist Marilyn Strathern, in 1997, in a paper on auditing in the British university system. The original statement by the economist Charles Goodhart, in 1975, was about monetary policy and sounded far more technical: any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
The nuance matters. Goodhart is not saying indicators are bad. He is saying that the relationship between an indicator and the reality it stands for is only stable while nobody has an interest in bending it. The moment the indicator becomes a target, it creates its own pressure, and the relationship distorts.
The sociologist Donald Campbell made the same point in 1979, applied to social indicators: the more a quantitative measure is used to make decisions, the more it is subject to corruption pressure, and the more it distorts the very process it is supposed to observe.
Four cases. Guess what happened.
Here are four targets. All were set by competent people acting in good faith. Before you unfold the answer, take ten seconds: what do you think people actually did?
Hanoi · 1902 The colonial administration wants to eradicate the city's rats to prevent plague. It introduces a bounty paid for every rat tail handed in. What happened?
Rat catchers began cutting off the tail and releasing the animal alive: a tailless rat keeps breeding, and therefore keeps producing future bounties. Some residents went further and started farming rats on the city's outskirts.
Tailless rats were reported running through Hanoi. Measured in tails collected, the campaign looked like a triumph. The rat population did not fall.
The bounty never measured dead rats. It measured tails handed in. Those are not the same thing, and someone always works that out.
Atlanta · 2009-2011 The school district sets standardised test score targets. Funding, bonuses and the careers of teachers and principals are tied to them. What happened?
Teachers and administrators met after the exams to erase and correct students' answers. The investigation uncovered an organised practice spanning dozens of schools. Eleven educators were convicted in 2015.
Scores went up. Actual attainment did not. And for the children involved the harm was twofold: they did not learn more, and their real difficulties stayed invisible because the numbers said otherwise.
When the consequence attached to a number outweighs the consequence attached to reality, the number is what gets optimised.
UK public hospitals To reduce congestion, a national target requires that a patient arriving at A&E be treated, admitted or discharged within four hours. What happened?
The clock only starts when the patient enters the department. So patients were held in ambulances outside the doors to delay the start of the count. Others were moved into intermediate units that fell formally outside the measured perimeter.
Compliance with the target improved. The time actually experienced by the patient, from calling for help to being treated, did not necessarily.
Every indicator draws a boundary. And effort moves to the other side of the boundary.
Customer contact centre To improve productivity, management tracks average handling time per call and sets a ceiling that must not be exceeded. What happened?
Advisers learned to cut difficult calls short: rushed answers, transfers to another department, and sometimes simply hanging up on complex cases.
Average time fell. Call volume rose, because customers called back. Total cost of service did not move, or went up, and satisfaction dropped.
Optimising one step of a process does not optimise the process. Often it just moves the cost somewhere nobody is measuring.
Why it always plays out the same way
In all four cases the logic is identical. You want to achieve something that is hard to measure directly: a city without plague, children who learn, patients treated well, satisfied customers at controlled cost.
Since you cannot measure that, you pick a proxy: something observable, countable, correlated with the intent. Rat tails. Test scores. Minutes. Products sold per customer.
As long as the proxy is only a thermometer, it works. The day it becomes a target with consequences attached, people stop optimising the intent and start optimising the proxy. And because a proxy is never perfectly identical to the intent, there is always a path that improves the number without improving reality. That path is usually shorter, cheaper and less risky than doing the real work.
So this is not a question of dishonesty. It is a question of design. A well-built system makes the right behaviour more rewarding than the workaround.
What separates a good KPI
From these mechanisms I draw four tests I try to apply before proposing any indicator.
- It is attached to a decision. The question is simple: if this number moves, what do we do differently? If no decision changes, it is not a KPI, it is dashboard decoration. That single question eliminates half the metrics people ask you to produce.
- It comes with a counter-metric. An indicator on its own always ends up being gamed. Speed and quality. Volume and satisfaction. Block rate and false positive rate. The counter-metric is not there to balance a slide, it is there to make the workaround visible immediately.
- It survives the question "how would I cheat?". This is the cheapest, fastest exercise available: before publishing a metric, spend five minutes genuinely trying to hit it without doing the work. If you find a way, someone else will too, and they may be the one measured on it.
- It stays close to the intent. The further the proxy drifts from what you actually want, the more exploitable the gap becomes. "Tails handed in" is very far from "a city without plague". "Products per customer" is very far from "customers well served". Shortening that distance beats adding more controls.
What I take from it
There is a reason this topic feels central to me in an analyst's job. Our natural reflex, when asked for an indicator, is to focus on the technical side: which source, which definition, which granularity, which query. That work is necessary, but it comes after the real question.
The real question is what behaviour the indicator will produce once it is on a dashboard, reviewed in a steering committee and possibly tied to a bonus. A KPI is never just a thermometer: from the moment you look at it, it acts on what it measures.
That, I think, is also what separates producing a number from informing a decision. Producing the number is execution. Anticipating what the number will trigger, flagging when a metric is gameable, proposing the counter-metric that is missing: that is where an analyst becomes useful to the business rather than to the dashboard.
← Back to blog