ection>
STAT-SPOTTING
STAT-SPOTTING
A FIELD GUIDE TO IDENTIFYING DUBIOUS DATA
UPDATED AND EXPANDED
JOEL BEST
University of California PressBerkeleyLos AngelesLondon
University of California Press, one of the most distinguished university presses in the United States, enriches lives around the world by advancing scholarship in the humanities, social sciences, and natural sciences. Its activities are supported by the UC Press Foundation and by philanthropic contributions from individuals and institutions. For more information, visit www.ucpress.edu.
University of California Press
Berkeley and Los Angeles, California
University of California Press, Ltd.
London, England
© 2008, 2013 by The Regents of the University of California
First paperback printing 2013
ISBN 978-0-520-27998-8
eISBN 9780520957077
“Crystal-Clear Risk” on p. 23 is from “Party, Play—and Pay: Multiple Partners, Unprotected Sex, and Crystal Meth,” Newsweek, February 28, 2005, 39. © 2005 Newsweek, Inc. All rights reserved. Used by permission and protected by the Copyright Laws of the United States. The printing, copying, redistribution, or retransmission of the Material without express written permission is prohibited.
Earlier discussions of some of the examples used in this book appeared in “Birds—Dead and Deadly: Why Numeracy Needs to Address Social Construction,” Numeracy 1, no. 1 (2008): article 6, and in the “Out of Context” feature in issues of Contexts magazine published between spring 2005 and fall 2007 (volumes 4–6).
The Library of Congress has catalogued an earlier edition as follows:
Library of Congress Cataloging-in-Publication Data
Best, Joel.
Stat-spotting : a field guide to identifying dubious data / Joel Best.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-520-25746-7 (cloth : alk. paper)
1. Sociology—Statistical methods. 2. Social problems—Statistical methods. I. Title.
HM535.B4772008
301.072’7—dc222008017175
Designer: Nola Burger
Text: 11/15 Granjon
Display: Akzidenz Grotesk and Akzidenz Grotesk Condensed
Manufactured in the United States of America
21 20 19 18 17 16 15 14 13
10 9 8 7 6 5 4 3 2 1
The paper used in this publication meets the minimum requirements of ANSI/NISO Z39.48-1992 (R 2002) (Permanence of Paper).
The statistics you don’t compile never lie.
– STEPHEN COLBERT
CONTENTS
PREFACE TO THE 2013 EDITION
PART 1GETTING STARTED
A.SPOTTING QUESTIONABLE NUMBERS
B.BACKGROUND
PART 2VARIETIES OF DUBIOUS DATA
C.BLUNDERS
D.SOURCES: WHO COUNTED–AND WHY?
D.3Shocking Claims
D.4Naming the Problem
E.DEFINITIONS: WHAT DID THEY COUNT?
E.1Broad Definitions
E.2Expanding Definitions
E.3Changing Definitions
E.4The Uncounted
F.MEASUREMENTS: HOW DID THEY COUNT?
F.1Creating Measures
F.2Odd Units of Analysis
F.3Loaded Questions
F.4Raising the Bar
F.5Technical Measures
G.PACKAGING: WHAT ARE THEY TELLING US?
G.1Impressive Formats
G.2Misleading Samples
G.3Convenient Time Frames
G.4Peculiar Percentages
G.5Selective Comparisons
G.6Statistical Milestones
G.7Averages
G.8Epidemics
G.9Correlations
