A Field Guide to Lies: Critical Thinking in the Information Age (27 page)

Our prior hypothesis (
a priori
in Latin) is that the suspect is guilty with a probability of .02 (one of fifty people who had access). Now let’s suppose the perpetrator and the horse got in a scuffle, and human blood was found at the scene. Our forensics team tells us that the probability that the suspect’s blood matches the blood found at the scene is .85. We construct a fourfold table as before. We fill in the bottom row under the table first: The suspect has a one in fifty chance to be guilty (the
Guilty: Yes
column), and a forty-nine in fifty chance to be innocent. The lab told us that there’s a .85 probability of a blood match, so we enter that in the upper left: the probability that the suspect is guilty
and
the blood matches. That means the lower left cell has to be .15 (the probabilities have to add up to one). The .85 blood match means something else: that there’s a .15 chance the blood was left by someone else, not our suspect, which would absolve him and render him not guilty. There’s a .15 chance that one of the people in the right-hand column will match, so we multiply 49 × .15 to get 7.35 in the upper right cell. We subtract
that from the forty-nine in order to find the value for the bottom right cell.

Now we can calculate the information we want the judge and jury to evaluate.

P(Guilty | Match ) = .85/8.2 = .10
P(Innocent | Match) = 7.35/8.2 = .90

Given the evidence, it is about nine times more likely that our suspect is innocent than guilty. We started out with him having a .02 chance of being guilty, so the new information has increased his guilt by a factor of five, but it is still more likely that he is innocent.

Suppose, however, some new evidence comes in—horsehair found on the suspect’s coat—and the probability that the horsehair belongs to the drugged horse is .95 (only five chances in one hundred that the hair belongs to a different horse). We can chain our Bayesian probabilities together now, filling out a new table. In the bottom margin, we enter the values we just calculated, .10 and .90. (Statisticians sometimes say that yesterday’s posteriors are today’s priors.) If you’d rather think of these numbers as “one chance in ten” and “nine chances in ten,” go ahead and enter them as whole numbers.

We know from our forensics team that the probability of a match for the hair sample is .95. Multiplying that by one, we get the entry for the upper left, and subtracting that from one we get the entry for the lower left. If there is a .95 chance that the sample matches the victimized horse, that implies that there is a .05 chance that the sample matches a different animal (which would absolve the suspect) so the upper right-hand cell is the product of .05 and the marginal total of 9 = .45. Now when we perform our calculations, we see that

The new evidence shows us that it is about twice as likely that the suspect is guilty as that he is innocent, given the evidence. Many attorneys and judges do not know how to organize the evidence like this, but you can see how helpful it is. The problem of mistakenly thinking that P(Guilty | Evidence) = P(Evidence | Guilt) is so widespread it has been dubbed
the prosecutor’s fallacy
.

If you prefer, the application of Bayes’s rule can be done mathematically, rather than using the fourfold table, and this is shown in the appendix.

F
OUR
C
ASE
S
TUDIES

Science doesn’t present us with certainty, only probabilities. We don’t know for 100 percent sure that the sun will come up tomorrow, or that the magnet we pick up will attract steel, or that nothing travels faster than the speed of light. We think these things very likely, but science yields only the best Bayesian conclusions we can have, given what we know so far.

Bayesian reasoning asks us to consider probabilities in light of what we know about the state of the world. Crucial to this is engaging in the kind of critical thinking described in this field guide. Critical thinking is something that can be taught, and practiced, and honed as a skill. Rigorous study of particular cases is a standard approach because it allows us to practice what we’ve learned in new contexts—what learning theorists call
far transfer
. Far transfer is the most effective way we know to make knowledge stick.

There is an infinite variety of ways that faulty reasoning and misinformation can sneak up on us. Our brains weren’t built to excel at this. It’s always been a part of science to take a step back and engage in careful, systematic reasoning. Case studies are presented as stories, based on true incidents or composites of true incidents, and of course, we are a story-loving species. We remember the stories and the interesting way they loop back to the fundamental
concepts. Think of the following as problem sets we can all explore together.

Shadow the Wonder Dog Has Cancer (or Does He?)

We got our dog Shadow, a Pomeranian-Sheltie mix, from a rescue shelter when he was two years old. He got his name, we learned, because he would follow us from room to room around the house during the day, never far away. As often happens with pets, our rhythms synchronized—we would fall asleep and wake up at the same time, get hungry around the same time, feel like getting exercise at the same time. He traveled with us often on business trips to other cities, becoming acclimated to planes, trains, and automobiles.

When Shadow was thirteen, he began having trouble urinating, and one morning we found blood in his urine. Our vet conducted an ultrasound examination and found a growth on his bladder. The only way to tell whether it was cancerous was to perform two surgical procedures that the oncologist was urging: a cystoscopy, which would run a miniature camera through his urethra into the bladder, and a biopsy to sample the mass and study it under the microscope. The general practitioner cautioned against this because of the risks of general anesthesia in a dog Shadow’s age. If it did turn out to be cancerous, the oncologist would want to perform surgery and start chemotherapy. Without any further tests, the doctors were still pretty certain that this was bladder cancer, known as transitional cell carcinoma (TCC). On average, dogs live only six months following this diagnosis.

As my wife and I looked into Shadow’s eyes, we felt utterly helpless. We didn’t know if he was in pain, and if so, how much more he
was facing, either from the treatment or from the disease. His care was entirely in our hands. This made the decision particularly emotional, but that didn’t mean we threw rationality out the window. You can think critically even when the decision is emotional. Even when it’s your dog.

This is a typical medical scenario for people or pets: two doctors, two different opinions, many questions. What are the risks of surgery? What are the risks of the biopsy? How long is Shadow likely to live if we give him the operation and how long is he likely to live if we don’t?

In a biopsy, a small needle is used to collect a sample of tissue that is then sent to a pathologist, who reports on the likelihood that it is cancerous or not. (Pathology, like most science we’ve seen, does not deal in certainties, just likelihoods and the probability that the sample contains cancer, which is then applied to the probability that the unsampled parts of the organ might also contain cancer; if you’re looking for certainty, pathology is not the place to look.) Patients and pet owners almost never ask about the risk of biopsy. For humans, these statistics are well known, but they are less well tracked in veterinary medicine. Our vet estimated that there was a 5 percent chance of life-threatening infection, and a 10 percent chance that some cancerous material (if indeed the mass was cancerous) would be “shed” into the abdomen on the needle’s way out, seeding further cancer growth. An additional risk was that biopsies leave behind scar tissue that makes it more difficult to operate later if that’s what you decide to do. The anesthesia needed for the procedure could kill Shadow. In short, the diagnostic procedure could make him worse.

Our vet presented us with six options:

1. Biopsy through the abdominal wall in the hope of obtaining a more definitive diagnosis.
   
2. Diagnostic catheterization (using a catheter to traumatize a portion of the mass, allowing cells to exfoliate and then be examined).
   
3. Biopsy using the same cystoscopic camera they wanted to use anyway to better image the mass (through the urethra).
   
4. Major surgery right now to view the mass directly, and remove it if possible. The problem with this is that most bladder cancers return within twelve months because the surgeons are unable to remove every cancerous cell, and the ones left behind typically keep on growing at a rapid rate.
   
5. Do nothing.
   
6. Put Shadow to sleep right now, in recognition of the fact that it most probably is bladder cancer, and he doesn’t have long to live anyway.
   

We asked about what the treatment options were if it was found to be cancer, and what they might be if it was not cancer. Too often, patients focus on the immediate, upcoming procedure without regard for what the next steps might be.

If
the mass was cancerous, the big worry was that the tumor could grow and eventually block one of the tubes that brings urine into the bladder from the kidneys, or that allows urine to leave the bladder and end up on a lawn or fire hydrant of choice. If that blockage occurs, Shadow could experience great pain and die within a day. Along the way to this, there could be temporary blockages as a result of swelling. Because of the position of the bladder within the body, and the angle of ultrasound, it was difficult to tell how close the mass was to these tubes (the ureter and urethra).

So what about the six options presented above—how to decide which (if any) to choose? We ruled out two of them: putting Shadow to sleep and doing nothing. Recall that the oncologist was pushing for surgery because that is their gold standard, their protocol for such cases. We asked for some statistics and she said she’d have to do some research and get back to us. Later, she said that there was a 20 percent chance that the surgery would end badly, killing Shadow right away. So we ruled out the major surgery because we weren’t even sure yet if the mass was cancerous.

We asked for life-expectancy statistics on the various remaining scenarios. Unfortunately, most such statistics are not kept by the veterinary community, and in any case, those that are kept skew toward short life expectancy because many pet owners choose euthanasia. That is, many owners opt to put their pets down before the disease progresses because of concerns about either the animal’s quality of life or the owners’ quality of life: Dogs with TCC often experience incontinence (we had already noticed that Shadow was leaving us little surprises around the house). We didn’t have a definitive diagnosis yet, but based on the sparse statistics that existed, it looked as though Shadow would live three months
with or without treatment
. Three months if we do nothing, three months if we give him chemo, three months if we give him surgery. How could that be? Ten years ago, we found out, vets would recommend euthanasia on first diagnosis of TCC. And at the first sign of chronic incontinence, owners would put their dogs down. So owners were typically ending their dogs’ lives before the cancer did, and this made the statistics unreliable.

We did some research on our own, using “transitional cell carcinoma” and “dog
or
canine” as the search terms. We found out that there was a 30 percent chance Shadow could improve simply by taking a nonsteroidal anti-inflammatory called Piroxicam. Piroxicam has its own side effects, including stomach upset, vomiting, loss of appetite, and kidney and liver trouble. We asked the vet about it and she agreed that it made sense to start him on Piroxicam no matter what else we were doing.