THE BOOKLET OF STATISTICAL VICES
Julian L. Simon
William Bennett, author of the best-selling The Book of
Virtues, also has written The Index of Leading Cultural Indica-
tors, widely distributed as a short pamphlet. It should have
been called The Booklet of Statistical Vices, because it is an
awesome and educational display of misleading statistical
contrivances. (Recently it has also been issued in longer
form.)
"[W]e have experienced substantial social regression...over
the last 30 years," Bennett's introduction says. Scary. But
what's the evidence?
"Number of Crimes Committed" is the first topic. Bennett
presents one chart showing "Millions of Violent Crimes" and
another showing "Millions of Total Crimes." Both embody a clas-
sic obfuscation - no adjustment for the growth of the country's
population since 1960. Therefore crimes per capita - the rele-
vant measure - have not grown so fast as Bennett's charts show.
(Bennett notes this in his discussion, but that's like small
print in a contract. It's the dramatically rising curves in the
graphs that hit the reader over the head.)
Even more crucial is the type of data used, as John Stossell
showed in a recent ABC documentary. Bennett uses the FBI series,
which refer only to reported crimes. The rate of reporting has
increased greatly in recent decades, thereby biasing the FBI data
trends.
Since 1973 the U. S. Bureau of Justice Statistics has there-
fore conducted "victimization surveys." The results are mind-
boggling : no apparent growth in violent crime (maybe a
decline), with a large decline in non-violent larceny, burglary,
theft, and auto theft. But Bennett presents only the FBI series
that support his thesis.
Bennett's second topic is "Median Prison Sentence for All
Serious Crimes." His graph shows a decline from 25 days in 1954
to 5 days in 1974, and then a modest rise to 1990, suggesting
that the society has gotten soft on punishing crime.
Can five days in prison really be the "median sentence"? Of
course not. Reading the fine print you find out that the numbers
do not refer in any way to the "median sentence" that is in the
title, but rather are a computation called "expected punishment".
This includes the chances of being arrested, of going to trial,
of being convicted, and getting any prison sentence, as well as
the sentence itself. This concept is interesting and probably
useful, but has almost no connection with what Bennett claims to
discuss. This abuses statistics akin to the non-sequitur fallacy
in rhetoric.
Another common statistical misuse is wrongly combining
apples and oranges - the "fallacy of composition." Does "serious
crimes" in the "median sentence" graph have the same composition
of crimes over the decades? Certainly not. Homicide and rape
and theft have changed at vastly different rates, so lumping them
together produces confusion at best.
Why does Bennett start this graph in 1954 whereas the first
set of graphs (and most of Bennett's others) start in 1960?
Answer: Starting this one at 1960 would make a less dramatic
chart. Similarly, starting the crime graphs earlier than 1960
also would seem less dramatic.
Taken together, these graphs suggest that the presentation
of crime data is rigged to get the worrisome effect that the
author wants to achieve.
Bennett's third topic is "Juvenile Violent Crime Arrest
Rates". This graph Bennett at least puts on a "per 100,000"
basis. But he does not tell us what the "100,000" refers to.
Juveniles? Population? One cannot know what mischief may lurk
behind the undefined number. Vagueness of definition is one of
the most useful practices for statistical doubletalk.
The "Juvenile ..." graph contains all the fallacies in
the earlier chart of total violent crimes, plus a new one: The
vertical axis in the graph does not start at zero. This is a
real meat-ax of a crude statistical trick, grade-school stuff. If
the plot had started at zero the rise in the curve would have
been much less eye-catching.
Next we notice that the subject is arrests rather than
crimes, though the casual reader is not likely to notice the
shift. Is it possible that juveniles have been getting arrested
more frequently for given crimes than in the past? We don't
know. But if so, arrests would give a misleading impression of
crime. Shifting definitions is a most useful contrivance for
portraying a false statistical picture.
The fourth topic is "Children Relying on AFDC". Again
Bennett uses the now-familiar device of showing total numbers of
children without adjusting for population increase. And again
Bennett mistitles the graph as "Relying on", when the data refer
to receiving AFDC. Who knows how many children get AFDC who do
not rely on it at all?
Indeed, how many children received AFDC half a century ago?
How many children receive AFDC-type payments in Somalia? Mighty
few, even though the children were and are more needy in those
cases than in the U.S. now. The explanation, of course, is that
the graph probably shows more generous government programs rather
than greater need or reliance. (Indeed, this is suggested by the
rapid rise in AFDC receipt shown between 1960 and 1975, with
constancy since then.)
So: First four topics in the short version - five graphs,
all misleading. This is a bravura display of statistical
obfuscation, providing a wonderful case study for my elementary
statistics course.
Julian L. Simon teaches business administration at the
University of Maryland. In the 1960s he pioneered what has
become the revolutionary "new statistics" - the resampling
simulation method.
301-951-0922 fax 301-951-8468 110 Primrose St., Chevy
Chase, Md.
page 1 /article4 bennett/June 13, 1994