Super Crunchers (27 page)

MTO testing:
Jeffrey R. Kling et al., “Experimental Analysis of Neighborhood Effects,”
Econometrica
75.1 (2007).

Examples of researchers utilizing such tests to answer these questions:
Alan S. Gerber and Donald P. Green, “The Effects of Canvassing, Direct Mail, and Telephone Contact on Voter Turnout: A Field Experiment,” 94
Am. Pol. Sci. Rev.
653(2000); Yan Chen et al., “Online Fund-Raising Mechanisms: A Field Experiment,” 5
Contributions to Econ. Analysis and Pol'y
(2006), http://www.bepress.com/bejeap/ contributions/vol5/iss2/art4; Stephen Ansolabehere and Shanto Iyengar,
Going Negative: How Political Advertisements Shrink and Polarize the Electorate
(1995).

Testing college roommates:
Michael Kremer and Dan M. Levy, “Peer Effects and Alcohol Use Among College Students,” Nat'l Bureau of Econ. Research Working Paper No. 9876 (2003) (men who drank prior to college earned much lower grades if they were placed with a heavy drinker instead of a teetotaler).

Testing ballot order:
Daniel E. Ho and Kosuke Imai, “The Impact of Partisan Electoral Regulation: Ballot Effects from the California Alphabet Lottery, 1978–2002,” Princeton L. and Pub. Affairs Paper No. 04–001, Harvard Pub. L. Working Paper No. 89 (2004).

For more information on Joel's work, see the following smorgasbord of
Waldfogel empiricism:
Joseph Tracey and Joel Waldfogel, “The Best Business Schools: A Market-Based Approach,” 70
J. Bus
. 1 (1997); Joel Waldfogel, “The Deadweight Loss of Christmas: Reply,” 88
Am. Econ. Rev.
1358 (1998); Felix Oberholzer-Gee et al., “Social Learning and Coordination in High-Stakes Games: Evidence from Friend or Foe,” Nat'l Bureau of Econ. Research, Working Paper No. 9805 (2003), http://www.nber.org/papers/W9805; Joel Waldfogel, “Aggregate Inter-Judge Disparity in Federal Sentencing: Evidence from Three Districts,”
Fed. Sent'g Rep.
, Nov.–Dec. 1991.

Other market reactions:
Joel and I have also ranked judges' bail decisions based on how the bail bonds market reacted to the decisions. Ian Ayres and Joel Waldfogel, “A Market Test for Race Discrimination in Bail Setting,” 46
Stan. L. Rev.
987 (1994).

Post-conviction earnings:
Jeffrey R. Kling, “Incarceration Length, Employment, and Earnings,”
Am. Econ. Rev.
(2006).

Recidivism rates:
Danton Berube and Donald P. Green, “Do Harsher Sentences Reduce Recidivism? Evidence from a Natural Experiment” (2007), working paper.

The Poverty Action Lab:
J. R. Minkel, “Trials for the Poor: Rise of Randomized Trials to Study Antipoverty Programs,”
Sci. Am.
(Nov. 21, 2005), http://www. sciam.com/article.cfm?articleID=000ECBA5-A101-137B-A10183414B7F0000.

Female chiefs:
Raghabendra Chattopadhyay and Esther Duflo, “Women's Leadership and Policy Decisions: Evidence from a Nationwide Randomized Experiment in India,” Inst. Econ. Dev. Paper No. dp-114 (2001), http://www. bu.edu/econ/ied/dp/papers/chick3.pdf.

Testing teacher absenteeism:
Esther Duflo and Rema Hanna, “Monitoring Works: Getting Teachers to Come to School,” Nat'l Bureau of Econ. Research Working Paper No. 11880 (2005); Swaminathan S. A. Aiyar, “Camera Schools: The Way to Go,”
Times of India
, Mar. 11, 2006, http://timesofindia.india times.com/articleshow/1446353.cms.

Kenyan de-worming:
Michael Kremer and Edward Miguel, “Worms: Education and Health Externalities in Kenya,” Poverty Action Lab Paper No. 6(2001), http://www.povertyactionlab.org/papers/kremer_miguel.pdf.

Auditing in Indonesia:
Benjamin A. Olken, “Monitoring Corruption: Evidence from a Field Experiment in Indonesia,” Nat'l Bureau of Econ. Research Working Paper No. 11753 (2005); “Digging for Dirt,”
Economist
, Mar. 16, 2006.

Progresa's poverty program:
Paul Gertler, “Do Conditional Cash Transfers Improve Child Health?: Evidence from PROGRESA's Control Randomized Experiment,” 94
Am. Econ. Rev.
336 (2004).

Paying mothers vs. fathers:
Gertler stressed that targeting mothers may not be appropriate in other cultures. “In fact, in Yemen right now we are doing a conditional cash transfer where we are randomizing giving the money to the mother vs. giving the money to the father to see whether it makes a difference.” See Paul Gertler, “Do Conditional Cash Transfers Improve Child Health?: Evidence from PROGRESA's Control Randomized Experiment,” 94
Am. Econ. Rev.
336 (2004).

CHAPTER 4

The beginning of EBM:
Gordon Guyatt et al., “Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine,” 268
JAMA
2420 (1992); “Glossary of Terms in Evidence-Based Medicine,” Oxford Center for Evidence-Based Medicine, http://www.cebm.net/glossary.asp; Gordon Guyatt et al., “Users' Guide to the Medical Literature: Evidence-Based Medicine: Principles for Applying the Users' Guides to Patient Care,” 284
JAMA
1290 (2000).

Ignaz Semmelweis takes the next step in medical Super Crunching:
“The Cover: Ignaz Philipp Semmelweis (1818–65),” 7
Emerging Infectious Diseases
, cover page (Mar.–Apr. 2001), http://www.cdc.gov/ncidod/eid/vol7no2/cover.htm.

Pasteur also early supporter of hand-washing:
Louis Pasteur was, however, convinced by Semmelweis's results. In a speech before the Academy of Medicine, Pasteur opined: “If I had the honor of being a surgeon…not only would I use none but perfectly clean instruments, but I would clean my hands with the greatest care….” Theodore L. Brown,
Science and Authority
(book manuscript, 2006).

Don Berwick, the modern-day Semmelweis:
Tom Peters, “Wish I Hadn't Read This Today,”
Dispatches from the World of Work
, Dec. 7, 2005, http:// tompeters.com/entries.php?note=008407.php; Inst. of Med.,
To Err Is Human: Building a Safer Health System
(1999); Donald Goldmann, “System Failure Versus Personal Accountability—The Case for Clean Hands,” 355
N. Engl. J. Med.
121 (Jul. 13, 2006); Neil Swidey, “The Revolutionary,”
Boston Globe
, Jan. 4, 2004; Donald M. Berwick et al., “The 100,000 Lives Campaign: Setting a Goal and a Deadline for Improving Health Care Quality,” 295
JAMA
324 (2006); “To the Editor,”
N. Engl. J. Med.
(Nov. 10, 2005).

Reducing risk of central-line catheter infections:
S. M. Berenholtz, P. J. Pronovost, P. A. Lipsett, et al. “Eliminating Catheter-Related Bloodstream Infections in the Intensive Care Unit,”
Crit. Care Med.
32 (2004), pp. 2014–2020.

Medical myths and practices that won't die:
Gina Kolata, “Annual Physical Checkup May Be an Empty Ritual,”
N.Y. Times
, Aug. 12, 2003, p. F1; Douglas S. Paauw, “Did We Learn Evidence-Based Medicine in Medical School? Some Common Medical Mythology,” 12
J. Am. Bd. Family Practice
143 (1999); Robert J. Flaherty, “Medical Mythology,” http://www.montana.edu/wwwebm/ myths/home.htm.

Can alternative medicine be evidence-based?:
The resistance to statistical evidence is even more pronounced with regard to “alternative medicine,” a vast field that encompasses everything from herbal supplements to energy therapy to yoga. More than a third of American adults use some form of alternative medicine annually; they spend more than $40 billion a year on alternative treatments. Patricia M. Barnes, et al., “Complementary and Alternative Medicine Use Among Adults,” CDC Advance Data Report No. 343 (May 27, 2004), available at http://www.cdc.gov/nchs/data/ad/ad343.pdf.

Many adherents and advocates of alternative medicine reject not only Western treatments but the Westernized notion of statistical testing. They sometimes claim that their practices are too “individual” or “holistic” to study scientifically and instead rely on anecdotes and case studies without adequate controls or control groups for comparison. I'm agnostic about whether alternative medicine is effective. But it verges on idiocy to claim that the effectiveness cannot be tested. If it really is important, as alternative medicine advocates claim, to take into account a larger set of information about the patient (“what kind of person has the disease”), then providers who do so should produce better results. There's no reason why Super Crunching can't be used to test whether alternative medicine works. To support EBM doesn't mean that you're hostile to alternative or innovative—even seemingly wacky—therapies. When it comes to the back-end inquiry of finding out which treatments are effective, there is no East and West. I throw my lot with two past editors-in-chief of the
New England Journal of Medicine
, Marcia Angell and Jerome Kassirer:

It is time for the scientific community to stop giving alternative medicine a free ride. There cannot be two kinds of medicine—conventional and alternative. There is only medicine that has been adequately tested and medicine that has not, medicine that works and medicine that may or may not work.

M. Angell and J. P. Kassirer, “Alternative Medicine: The Risks of Untested and Unregulated Remedies,” 339
N. Engl. J. Med.
839 (1998). See also Kirstin Borgerson, “Evidence-Based Alternative Medicine?” 48
Perspectives in Bio. and Med.
502 (2005); W. B. Jonas, “Alternative Medicine: Learning from the Past, Examining the Present, Advancing to the Future,” 280
JAMA
1616 (1998); P. B. Fontanarosa and G. D. Lundberg, “Alternative Medicine Meets Science,” 280
JAMA
1618 (1998).

The Aristotelian approach:
Kevin Patterson, “What Doctors Don't Know (Almost Everything),”
N.Y. Times Magazine
, May 5, 2002; David Leonhardt, “Economix: What Money Doesn't Buy in Health Care,”
N.Y. Times
, Dec. 13, 2006.

Contemporary uses (or lack thereof) of EBM:
Brandi White, “Making Evidence-Based Medicine Doable in Everyday Practice,”
Family Practice Mgt.
(Feb.2004), http://www.aafp.org/fpm/20040200/51maki.htm; Jacqueline B. Persons and Aaron T. Beck, “Should Clinicians Rely on Expert Opinion or Empirical Findings?” 4
Am. J. Mgd Care
1051 (1998), http://www.ajmc.com/files/article files/AJMC1998JulPersons1051_1054.pdf; D. G. Covell et al., “Information Needs in Office Practice: Are They Being Met?” 103
Ann. Internal Med.
596 (1985); John W. Ely et al., “Analysis of Questions Asked by Family Doctors Regarding Patient Care,” 319
BMJ
358 (1999); “The Computer Will See You Now,”
Economist
, Dec. 8, 2005, http://www.microsoft.com/business/peopleready/news/economist/ computer.mspx.

Coffee and Heart Disease:
A. Z. LaCroix et al., “Coffee Consumption and the Incidence of Coronary Heart Disease,” 315
N. Engl. J. Med.
977 (1986); Int'l Food Info. Council Foundation, “Caffeine and Health: Clarifying the Controversies”(1993), www.familyhaven.com/health/ir-caffh.htm.

91: 17 years:
Institute of Medicine, Committee on Quality of Health Care in America,
Crossing the Quality Chasm: A New Health System for the 21st Century
, National Academy Press, Washington, D.C. (2001).

One funeral at a time:
Various formulations of this quotation have been attributed to Max Planck, Albert Einstein, and Paul Samuelson.

Few doctors rely on research to treat individual patients:
Brandi White, “Making Evidence-Based Medicine Doable in Everyday Practice,”
Family
Practice Mgt.
(Feb. 2004), http://www.aafp.org/fpm/20040200/51maki.htm; D. M. Windish and M. Diener-West, “A Clinician-Educator's Roadmap to Choosing and Interpreting Statistical Tests,” 21
J. Gen. Intern. Med
. 656 (2006); Kevin Patterson, “What Doctors Don't Know (Almost Everything),”
N.Y. Times
Magazine
, May 5, 2002.

Where is the library in the doctor's office?:
A law library is physically and intellectually at the heart of many law practices. A client comes to a lawyer for advice, and the lawyer hits the books to find out what the law is for that client's particular problem. Until evidence-based medicine came along, most doctors didn't routinely research the problems of individual patients. The physician might try to keep up in the field generally. He or she might subscribe to the
New
England Journal of Medicine
and
JAMA
. Yet rare would be the case where a physician would pick up a book or journal article to do patient-specific research. In sharp contrast to a law office, most doctors' offices never had a library. If a physician didn't know the answer, she might consult with a specialist, but neither the physician nor the specialist was very likely to pick up a book and read.

Information must be retrievable to be useful:
Gordon Guyatt et al., “Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine,” 268
JAMA
2420 (1992); “Center for Health Evidence, Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine” (2001), http://www.cche.net/usersguides/ebm.asp; Lisa Sanders, “Medicine's Progress, One Setback at a Time,”
N.Y. Times
, Mar. 16, 2003, p. 29; InfoPOEMs, https://www.infopoems.com.

Data-based diagnostic decisions:
“To the Editor,”
N. Engl. J. Med
., Nov. 10, 2005.

Diagnostic-decision support software:
The major softwares in this field include Isabel, QMR, Iliad, Dxplain, DiagnosisPro, and PKC.

Super Crunching away diagnostic errors:
Jeanette Borzo, “Software for Symptoms,”
Wall St. J. (Office Technology)
, May 23, 2005, p. R10; Jason Maude biography, The Beacon Charitable Trust, http://www.beaconfellowship.org.uk/ biography2003_11.asp; David Leonhardt, “Why Doctors So Often Get It Wrong,”
N.Y. Times
, Feb. 22, 2006.