PREVIEW OF THE BOOK
Here is a preview of key points in the book: 1) One cannot
avoid making judgments; the process of statistical inference
cannot ever be perfectly routinized or objectified. Even in
science, "fitting of the model to experience" (close to Braith-
waite's phrase in his 1953 book) requires judgment. 2) In
statistical inference as in all sound thinking, one's purpose is
central. All judgments should be made relative to that purpose,
and to costs and benefits. (This is the spirit of the Neyman-
Pearson approach). 3) The best ways to infer are different in
different situations. The differences are especially great in
science versus decision-making, and among the different fields of
economics, psychology, history, business, medicine, engineering,
physics. 4) The logically and historically primary task in most
scientific situations is to decide whether two (or more) sets of
observations should be considered similar or different, while the
secondary task is to decide how big is a difference (or other
magnitude). Hypothesis tests are intended to serve the primary
task, and confidence intervals are intended to serve the
secondary task; this distinction helps sort out the roles of the
two main modes of inference, which in the past have been tangled.
5) The different tools should be used as situations call for them
- sequential vs fixed sampling, Neyman-Pearson vs. Fisher, and so
on. 6) In statistical inference, when data and procedures are
ambiguous, the best wisdom is not to argue about the proper
conclusion. Instead, whenever possible one should go back and
get more data, hence lessening the importance of the efficiency
of statistical tests. In some cases, however - especially in
biostatistics, taking as an example unusual cases of cancer - one
cannot easily get more data, or even conduct an experiment. And
with respect to the past one cannot produce more historical data.
But one can gather more and different kinds of data, e.g. as was
done in the history of the inquiry into the connection between
smoking and lung cancer.
The book does not deal with the valuational aspects of
decision-making - neither the expected values nor the allowances
for risk. My book on managerial economics (1975) discusses ways
of embedding the probabilistics element into the larger framework
of decision theory.
Though Chapter IV-2 discusses the relationship of
statistical correlations to causality, and the overall concept of
causality, the book also does not deal with the practical aspects
of determining causality such as the search for hidden third
factors. One may consult texts on research methods (e. g. Simon
and Burstein, 1985) for discussion of this topic.
PREFACE
With luck, the arguments in this book are written
sufficiently clearly and simply that anyone willing to make an
effort can understand them. And I hope you will find the overall
argument integrated and without any logical gaps from one part
of the discussion to another. But even if I have succeeded in
doign that, it does not imply that the subject is simple. . Just
the opposite: The ideas underlying statistical inference are, in
my opinion, as subtle and difficult as any set of fundamentals in
any field. This inherent difficulty explains why, after
centuries and millenia of struggling for understanding of these
matters, it was not until the past century or so that these ideas
have finally begun to be understood, and there still is massive
controversy about the foundations of the science. (See the
Introduction)
Writers such as Fisher, Neyman, Pearson, Jeffreys, and
others have made great discoveries by focusing laser beams into
distant uncharted regions. But they have left much of the
firmament unmapped. I hope that this book provides a broad map
to help one pass from one of those discovered areas to another.
The book also proposes some particular ways of addressing the
subject that are intended to promote this integrated
understanding, such as treating the concept of probability with
the concept of operational definition and hence side-stepping and
obviating the centuries-long struggle between the frequentists
and the subjectivists.
The greatest strength, from my point of view, of the book's
treatment of statistical inference also surely wil be considered
its greatest weakness by many others: its non-axiomatic non-
deductive pragmatic approach oriented toward operational
definitions rather than the property definitions that are
necessary for axiomatic treatment. This non-formulaic approach
is also at the heart of the resampling (Monte Carlo) method,
which is the second focus of the book. Both these matters are
illuminated by a a delightful and revealing scenario by John
Barrow wrote about what might happen if we receive a response
from Martians to our extra-terrestrial messages, which depend
heavily upon mathematics, on the assumption that that will be the
easiest for the Martians to decode. (The scenario is reproduced
in Chapter IV-3.) Barrow imagines that the arriving Martians
present stupendous and exciting new mathematical ideas which are
not found to be false, but which they arrive at by induction -
and therefore are rejected by earthly mathematicians because they
do not employ the earthly method of deductive proof.
Our earthling mathematician are disappointed, in Barrow's
story. That is accurate: Terrestrial mathematicians are not
excited by a method that simply offers answers or solutions. The
method must also meet esthetic tests to be acceptable. It is
here that resampling faces its greatest obstacle; it lacks the
esthetic appeal of proof-based mathematical findings.
Whether the approach to understanding inference presented
here will eventually be accepted, I do not know. But I think it
safe to predict that eventually resampling must win its way into
the center of statistical practice, because its practical
advantages are enormous. That future seemed evident to me in
1967 when I first began to practice, teach, and write about
resampling, and indeed, by now the concept is entirely accepted
theoretically by mathematical statisticians. But just how soon
resampling will become the standard tool of first recourse in
everyday work continues to be unclear.
The situation of resampling has something in common with the
famous book Calculus Made Easy by Sylvanus Thompson, whose front
motto is "What one fool can do, another can, too". A motto for
resampling might be "What one mathematical dolt can invent,
another mathematical dolt can understand".
Though Calculus Made Easy is damned by academic
mathematicians, after almost a century it is still available in
paperback and selling briskly in college bookstores - simply
because it makes quite clear, using a system of approximation,
what is extraordinarily difficult to comprehend - after all, it
required Newton and Leibniz to invent it - using the
mathematician's elegant method of limits. Resampling has come in
for the same damnation and neglect for the last quarter century.
But its chances of lasting forever are even better than
Thompson's method, because resampling is actually a better tool
than the conventional formulaic method, rather than just a better
pedagogical method.
One of the joys of writing this book is that it integrates
ideas that I have been chewing on for a quarter of a century.
Now the links between them appear - not only the links of
substance, but also the links of points of view such as the
emphasis on open rather than closed systems, and the need for
judgment in all our statistical and scientific work.
Much material concerning the actual practice of resampling
statistics is shared between this book and the text Resampling:
The New Statistics. I believe that the two treatments benefit
from there being both applied and philosophical points of view. I
hope that no no reader is troubled by this overlap.
Peter Bruce has greatly assisted all my work in this book in
a variety of ways. With respect to this chapter, his help in
clarifying these ideas by discussing them with me, along with
teaching them jointly with me, has been very great.