Roundetbeta

A rising metric can hide a goal moving backward

Rewards can improve the count while diverting effort from what the count was meant to represent.

Jeffery CuiOctober 29, 20262 min read
Researched with Codex

A metric can make outcomes worse when people can improve the number while weakening the goal it represents. The danger grows when rewards are strong, the measure covers only part of the job, or difficult cases bring little credit, so a rising count needs an independent check on what actually improved.

Consider a hypothetical support team paid for closing tickets: staff could solve more problems, close unresolved cases, or avoid difficult customers. Each route can lift the count, although customers experience very different results. Even accurate records cannot settle which route produced the improvement, so measure whether problems stayed solved and examine the cases that never reached a recorded success.

Jacob and Levitt’s 2003 study estimated serious teacher or administrator cheating in at least 4–5% of Chicago elementary classrooms annually. They combined suspicious answer patterns with score changes and checked suspected cases against closely monitored retests, rather than treating high scores alone as proof of cheating. The figure was a lower bound for that setting, showing one route by which pressure for better measured performance can corrupt the measurement itself.

Workers can also follow every rule while neglecting something valuable. Holmström and Milgrom’s 1991 model examines how incentives move effort among tasks when some outputs are easier to measure than others. A hypothetical factory that rewards finished units may pull attention away from maintenance, while a pass threshold can make a small gain near the cutoff more rewarding than substantial progress by someone who remains below it.

Targets can improve actual output when the rewarded activity has a close connection to value and quality remains visible. In Lazear’s 2000 Safelite study, moving toward pay per installation increased output per worker by 44%, which argues against assuming that every numerical target must fail. The practical question is how much useful work sits outside the rewarded measure and whether independent evidence can detect shortcuts or damage.

Before raising a target, write down how someone could improve its score while harming the goal, then check for that behavior.

Sources3

  1. 1Jacob and Levitt (2003) — Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheatingpricetheory.uchicago.edu
  2. 2Holmström and Milgrom (1991) — Multitask Principal–Agent Analysessfu.ca
  3. 3Lazear (2000) — Performance Pay and Productivityaeaweb.org

Related research