Cracked (7 page)

Now, if your rating has gone
down
after treatment, it means you have improved (and thus the pill has worked). But if it has increased, it means you have worsened (and thus the pill hasn't worked). Once the researchers gather the ratings for all patients in the trial, they can then compare the two groups to assess how superior the antidepressant is to the sugar pill in alleviating depression.

Now imagine that the clinical trial you have just participated in contained about five hundred other patients, all going through the same process as you. This of course is a significant amount, but it's still only one trial. What Kirsch did, you'll remember, is pool the results of
all
the trials he surveyed—both published and unpublished. So in effect Kirsch's second meta-analysis collated the results of many thousands of patients, all of whom had been studied in trials like the one I have just described. And it was on the basis of this second analysis, as I mentioned a moment ago, that Kirsch reached the alarming conclusion that antidepressants work hardly better than placebos. Here is what his results looked like.

As in his first meta-analysis, which only looked at the published trials, Kirsch's second meta-analysis, which assessed both published and unpublished trials, revealed that
both
placebo and antidepressant groups got better. But his second meta-analysis also revealed that the difference in rates of improvement between the antidepressant group and the placebo group was insignificant.

And that's the important bit. “After surveying all the trials, we discovered that the antidepressant group only improved by 1.8 points on the Hamilton Scale over the placebo group,” stated Kirsch. “Now, this may not mean much to you. But what if I were to tell you that your score can be reduced by a full 6.0 points if you are merely sleeping better? Well, you'd rightly conclude that 1.8 is a tiny difference. And that's precisely why the NICE [National Institute for Clinical Excellence] has said there must be a difference of at least 3.0 points for the difference to be deemed clinically significant. Yet the difference we found was only 1.8 points—totally clinically insignificant.”

Indeed, a 1.8 difference on the Hamilton Scale is barely noticeable in terms of a person's actual experience. But what was also interesting for Kirsch was that even the tiny 1.8 difference between the antidepressant and placebo groups still didn't mean anti-depressants worked better than placebos. “Just remember how the clinical trials work,” Kirsch explained to me. “You are told by your doctor you'll be given one of two pills. You are also told that the antidepressant pill will produce side effects in most patients [like drowsiness, diarrhea, nausea, forgetfulness, dry mouth, and so on]. So what happens if, after taking the pill, you start to experience some of these side effects? Well, like most people you'll figure out that you're on the real drug. And believing you are on the real drug, you'll now expect to get better. And it's this increased
expectation
of recovery that actually helps you improve.”

What Kirsch had worked out was that strong side effects convince patients that they are on the real drugs—and being so convinced, the placebo effect becomes all the greater. In other words, side effects increase the placebo effect. And this is how Kirsch could then account for the tiny 1.8 improvement in the antidepressant over the placebo group.

So, in summary, Kirsch's second meta-analysis (which included both published and unpublished trials) was far more dramatic than the first. It concluded that the new wave of antidepressants heralded as wonder drugs—Prozac, Seroxat (Paxil in the United States), Lustral (Zoloft), Dutonin (Sezone), Cipramil (Celxa), and Effexor—worked no better than dummy pills for the vast majority of patients. Of course, there were about 10 percent to 15 percent of people, the very extremely depressed, for whom these pills worked in a very minor way (about four points better than placebos on the Hamilton Scale), but this meant, as Kirsch pointed out, that about “85 percent to 90 percent of people being prescribed antidepressants are not getting any clinically meaningful benefit from the drug itself.”

4

Once Kirsch published his second analysis showing that antidepressants worked no better than sugar pills for the vast majority of patients, it immediately became front page news in the most respected papers in the UK:
The Guardian
,
The Times
,
The Independent,
and the
Daily Telegraph
. It made its way into newspapers and television and radio news programs in the United States, Spain, Portugal, Germany, Italy, Britain, South Africa, Australia, Canada, China, and many other countries. It was reported and debated in countless leading medical and scientific journals, including the prestigious scientific journal
Science
. Basically, overnight it transformed Kirsch into a global media fascination.
41

All this attention also made many in the medical community sit up and take note of his research. Just three months after Kirsch's work was published, for instance, a survey was conducted on nearly five hundred British doctors asking them whether Kirsch's findings would affect how they'd prescribe antidepressants in the future. Almost half of the doctors, 44 percent, said they would change their prescribing habits and consider alternative treatments. But this, of course, still meant that over 50 percent intended to go on prescribing as usual. Most of these 50 percent justified their position by arguing that in their clinical experience antidepressants do work, no matter what Kirsch's research indicates.

I asked Kirsch what he made of this. “Well, the truth is, these doctors are correct,” he responded. “Our research also shows that antidepressants work, but again not for the reasons most people suppose. They work because of their placebo effects, not because of the chemical in the drug—and that's the point we were making.”

There were other criticisms of Kirsch's work. “One we heard again and again,” said Kirsch, “was that the studies we surveyed must have been flawed. Some said these studies were too short to show the real effects of antidepressants. Others said that the patients studied were not depressed enough, while others said the patients were too depressed. But all these objections, taken as a whole, are very curious indeed, because the studies we surveyed were also those assessed by the regulatory agencies in the UK and US to justify approving these drugs for public use.
42
So if there were something wrong with the studies, why had the regulatory agencies in Europe and the US used them as a basis for approving these drugs?”

A further criticism, perhaps even more quixotic, was that even if the drugs don't work, it was still wrong of Kirsch and his colleagues to have published their results. Patients should be protected from findings that could undermine their faith in treatment. Kirsch disagreed adamantly, saying: “Without accurate knowledge, patients and physicians cannot make informed treatment decisions, researchers will be asking the wrong questions, and policymakers will be implementing misinformed policies. If the antidepressant effect is largely a placebo effect,” continued Kirsch, “it is important that we know this so that improvement can be obtained without reliance on addictive drugs which have potentially serious side effects.”
43

A final suite of criticisms was aimed at the methods Kirsch used. ‘There were a number of papers criticizing our statistical methods and redoing them,” said Kirsch, “some in appropriate ways, some in inappropriate ways, and some just making careless mistakes. But no matter how these analyses were done, nobody ever passed the 3.0 point threshold for clinical significance, which only gave additional support to our own conclusions.”

So Kirsch's research withstood the criticisms. But were there any other studies that actually replicated his findings? I put this question to Walter Brown, professor of psychiatry at Brown University, who has co-authored two studies analyzing the same set of clinical trials that Kirsch surveyed. His answer was unequivocal: “We pretty much found the same thing as Kirsch. For a small minority of patients (the most severely depressed), our studies showed that antidepressants may have some minor benefits. But for mildly\moderately depressed patients,” said Brown earnestly, “our results confirm that antidepressants offer no advantage over placebos, alternative therapies, or even moderate exercise.”
44

In other words, Brown's research confirmed that the vast majority of people taking antidepressants do not receive any chemical benefit. “There is no question that these drugs are overhyped to the general public,” reiterated Brown. “The research shows they are not as good as the psychiatric establishment and the pharmaceutical industry claim they are.”

While Irving Kirsch and Walter Brown reached the same conclusions independently, so too did a major study of antidepressants that the NHS commissioned in the UK. This NHS study also declared that the difference between placebos and antidepressants is so modest, that for mild to moderate depression antidepressants were not worth having at all.

“Our results were again like Irving Kirsch's,” said Dr. Tim Kendall, lead author of the study published in the
Lancet
. “Our widespread comparative meta-analysis of antidepressants showed pretty clearly that the difference between the published and unpublished studies of antidepressants in children was that for the published trials all the drugs worked, while for the unpublished trials none of the drugs worked. And if you looked at the published and unpublished combined, you'd only probably recommend the use of only one drug for childhood depression.”

At the same time as conducting that research, Kendall was also responsible for helping draw up the treatment guidelines for depression throughout the NHS. “Once we had those figures,” said Kendall, “we asked the depression guideline group what they would conclude if they only saw the published trials, and what would they conclude if they saw the whole data set (the published and the unpublished). We found that seeing the whole data set changed their view completely. And that is the key bit: drug companies not publishing negative trials actually changes clinicians' minds.”

Given the results of studies as outlined above, why do the regulatory agencies that evaluate antidepressants continue to approve these drugs for public use? The key to answering this question is to realize that the regulatory agencies do not take into account the results of negative trials when deciding to approve an antidepressant or not.
For example, the Food and Drug Administration (FDA), in the United States, and the Medicines and Healthcare Products Regulation Agency (MHRA), in Britain, merely require that a company show in just
two
clinical trials that their drug is more effective than a placebo. This is the case even if there are five, ten, or fifteen clinical trials showing negative results. In other words, regulatory agencies
discard the negative trials
. And they do this no matter how many there are. All they require are two positive trials to give the green light for public use.
45

The Food and Drug Administration (FDA) in the United States has publically defended what may seem to you and me a dubious approval process. In 2012, for instance, Lesley Stahl of CBS News interviewed the director of the unit responsible for approving antidepressants, Dr. Tom Laughren. Stahl asked Laughren why the FDA requires merely two positive clinical trials to approve a drug, even though many negative trials may exist. Laughren's response was faltering and confusing.

Laughren: “We consider everything that we have, we look at those trials individually …”

Stahl: “But how are you knowing that the two positives deserve bigger strength in the decision?”

Laughren: “Getting that finding of a positive study by chance, if there isn't really an effect, is very low. That's basic statistics, and that's the way clinical trials are interpreted [by the FDA]. A separate question is whether or not the effect you are seeing is clinically relevant.”

Stahl: “Okay—is it clinically relevant?”

Laughren: “The data we have shows that the drugs are effective.”

Stahl: “But what about the
degree
of effectiveness?”

Laughren: “I think we all agree that the changes that you see in the short-term trials, the difference in improvement between drug and placebo is rather small.”

Stahl: “So it's a moderate difference?”

Laughren: “It is a … well, it's a small—it's a modest difference.”

When I spoke to Dr Tim Kendall about Laughren's admission, Kendall confessed that he'd actually seen this interview and that it had completely baffled him. “What I do remember, when listening to Laughren, was that it actually sounded to me a little like mumbo jumbo,” Kendall said frankly. “I couldn't make any sense of it at all. From our point of view, whenever you're doing a trial or a meta-analysis, it is all about probabilities—about the probability that one thing works better than another, and that probability depends upon the evidence. Now, if someone conceals some of that evidence, it simply skews the result. So the idea that regulatory bodies like the FDA will continue approving drugs on the basis of only two positive trials, and are not bothered by all the other negatives, strikes me as wholly indefensible.”

Regulatory agencies seem to think otherwise, and the consequence of this has been dramatic: it has allowed an inordinate yearly rise in antidepressant prescriptions. Just consider the facts:

Antidepressant usage has more than tripled in the United States since 1986, reaching a staggering 235 million prescriptions in 2010.
46
And in Europe the situation is the same—the figures have tripled in recent years. In 2010 approximately 10 percent of all middle-aged Europeans were taking antidepressants,
47
while in 2011 a baffling 47 million prescriptions of antidepressants where dispensed in Britain alone.
48