Base Rate Neglect, explained.
Base rate neglect is overlooking how common an event was before receiving new evidence. A reliable signal can still produce many false positives when its target is rare.
Why it happens
A detection rate starts with actual targets; a positive predictive value starts with flagged cases. These are different groups. Count true and false flags before interpreting the meaning of one flag.
A detection rate starts with actual spam and asks how much gets flagged. The chance that a flagged message is spam starts with all flagged messages. Swapping those denominators is an easy mistake.
In this experiment, only 1% of messages are spam initially. A 90% detection rate produces about 9 correct flags per 1,000 messages. A 5% false alarm rate among the other 990 messages produces about 49.5 false flags on average. The probability that a flag indicates spam is therefore about 15.4%, not 90%.
Read the result
Lower the spam prevalence while keeping detection and false alarms fixed. The fraction of flags that are genuine falls because there are many more non-spam messages available to generate false alarms.
A worked example
A spam filter flags a message
Among 1,000 messages, suppose 10 are spam and the filter catches 9 of them.
A 5% false alarm rate among 990 legitimate messages adds about 49.5 false flags in expectation.
About 9 out of 58.5 expected flags are spam: roughly 15.4%, despite a 90% detection rate.
OPTIONAL DEEPER DETAILGo deeper: inside the model
Inside this model
P(spam | flag) = p × d / [p × d + (1 − p) × f], where p is the base rate, d is the detection rate, and f is the false alarm rate.
The dot grid samples 1,000 independent messages. Sample counts fluctuate; the model probability is calculated exactly from the controls. A sample with no flags has no observed proportion.
Where this idea is useful
Evidence should update a prior belief, not erase it. Ask both how often a signal finds its target and how often it appears without that target.
A common misconception
“A 90% detection rate means a flag is 90% likely to be correct.”
That swaps the conditioning. You also need prevalence and the false alarm rate to interpret a flag.
What this explanation leaves out
- Fixed error rates and independent messages simplify a real classifier. Rates can differ across populations and change over time.
- This model illustrates reasoning about evidence; it does not measure how people actually make judgments.
Which base rate should I use?
Use the population from which the case was drawn. A broad average can be misleading if a specific subgroup has a different prevalence or if the signal behaves differently within it.
Before trusting a striking signal, which denominator would you count?
Associated thinkers
Associations marked provisional are awaiting source review.
Further reading
Explore conditional probability and Bayes’ rule in Brown University’s Seeing Theory.