The Why Axis: Hidden Motives and the Undiscovered Economics of Everyday Life (13 page)

Reframing Achievement

Tom Amadio was impressed with these results, but he pressed a question on a different front, beyond keeping kids in school:
Could we increase the test scores of his students?
After all, test scores are important door-openers, and are tied to future outcomes like years of education and high-paying jobs. Test scores also determine how much money a school district receives from city and state governments. Unfortunately, at the present time, minority students just cannot seem to catch their white counterparts when it comes to test scores. The racial achievement test gap remains both considerable and stubborn, and many urban schools fail at their mission to close it.

To answer Amadio’s challenge, we decided to run another set of field experiments that involved over 7,000 students in a variety of elementary and high school settings in Chicago and Chicago Heights. These tests took place in the schools’ computer labs, where students took a standardized test three times a year.
⁷

As an introduction to our experimental premise, perhaps you remember the images of two young girls who were gymnasts at the 2008 Summer Olympics. Both were winners. As the girls stood on the podium, each of them was overcome with intense emotion. And no wonder: they had both trained for years for this moment, sacrificing normal lives as teenagers to reach the very apex of gymnastic
performance. The photos were taken after they received their respective medals. One was decorated with the silver medal, the other with the bronze. When their photos were published in the press, one was beaming, the other appeared to be holding back tears.

Which do you think won the silver and which the bronze?

We all know that silver is better than bronze, but context is everything. The silver medalist who missed out on the gold was devastated, and her face looked as if she were sucking on a sour lemon. But the bronze medalist who just barely made the podium was clearly ecstatic.
⁸

Over the past forty years two psychologists—Daniel Kahneman and Amos Tversky—have revolutionized our understanding of the importance of human emotions like sensitivity to context in everyday choices we make. One of the things these two “fathers” of behavioral economics have shown is that the way humans understand the world has to do with the way we interpret (or “frame”) phenomena. Depending on how you frame something when you speak, you influence someone’s behavior in various ways. A parent might say to a child, “If you don’t eat those peas, you won’t grow up big and strong.” (That’s what behaviorists call “loss framing”—it frames a statement as a loss or punishment.) Alternatively, the parent could phrase the same thing in a more positive light and say, “If you eat your peas, you will grow up big and strong.” (That’s called “gain framing”—it frames the statement as a benefit or reward.)

Imagine you are a thirteen-year-old boy coming into the computer lab to take a standardized test. It’s a nice fall day, and you are restless, a little hungry, and all you can think about is that last round of your favorite video game and the pretty girl sitting at the desk behind you. You wish you were anywhere but stuck in this stupid lab to take another stupid test.

In walks the school’s assessment coordinator, Mr. Belville, who asks for everyone’s attention. (Mr. Belville also happens to be the school’s reading coordinator and head of the school’s technology department; he is the sort of overqualified and overdedicated administrator that single-handedly makes a school run.) The process of just getting the students to stop talking takes a minute, but finally they quiet down.

“Today,” Mr. Belville announces, “you’re going to take the next level of the standardized tests that you took back in the spring. But this time we’re going to be doing something different. If you do better on today’s test than you did the last time you took it, you’ll receive a reward of $20.”

Your eyebrows shoot up. So do everyone else’s. “Awesome!” someone yells. Suddenly everyone starts chattering at once. Mr. Belville immediately quiets the room.

“Now, before we begin the test, I’m going to be handing each of you a $20 bill,” he continues. “I want each of you to fill out this receipt confirming you’ve received the money. On the receipt sheets, I want you to write a little bit about what you plan to do with the cash. You will keep the money in front of you on your desk while you take the test. Remember, you will get to keep the $20 if you improve on the test. But if you don’t improve you will lose the $20.” He passes around the receipt forms and the $20.

You dutifully fill out the form and think about what you want to do with the $20, which you would like to put toward a new skateboard. You write down your dream on the form, and then place the $20 to the right of the keyboard, just above the mouse. You smile as you look at it. “My wheels,” you think. You imagine walking into the skateboard store and plunking down your money.

Mr. Belville returns to the front of the room, interrupting your daydream. “We will begin the test in two minutes. Please sign in on the computer.”

You sign in, and the clock advances. You watch the second hand. You can’t wait to start.

“Ready? Begin!”

Now, when you’ve taken these tests in the past, you usually whiz through them because you really don’t care about them—you think they are pretty pointless and you leave many questions blank. But this time, with your $20 sitting in front of you, you take your time. Some questions stump you initially, but instead of guessing and moving on, you start to really think about what the best answer might be.

At the end of an hour, Mr. Belville announces that the test session is over. You’re the last student still working on the exam. You answer the last question and hit “submit.” Almost immediately, your score shows up on the teacher’s computer screen. Once the whole class finishes up you can see how well you did compared to last spring’s test.

So how did you do?

In this field experiment, we actually divided the schoolkids into one of five groups. As described above, kids in one group received a $20 bill and were told if they did not improve on their previous test score, we would take the money away. This is what we described above as the “loss” group: the kids had the $20, and stood to lose it by not achieving on the test.

Students in the comparison, “gain-framing” group were told that if they improved on their previous test scores we would give them $20 immediately after the test, but they did not receive the $20 beforehand. Because they didn’t have the $20 directly in front of them, they stood to gain.

Students in a third group were told that if they improved on their previous test score, we would give each of them $20, but not until a month after the test. A fourth group received a $3 trophy if their scores improved. And as is always the case in our experiments,
we had a control group. This group was offered no reward, though we encouraged them to try to improve their scores.

Our incentives had a huge impact. Overall scores improved between 5 and 10 percentile points on a 100-point scale, putting the students on a more even footing with wealthier suburban kids. This was an amazing improvement. Even though the students had no idea an incentive was coming until just before the exam started, they improved remarkably. It showed that an important part of the racial achievement gap was not due to knowledge or ability, but simply to the students’ motivation while taking the test.

This result highlighted the importance of understanding what motivates students: though they weren’t very interested in taking the test, their scores shot up in the face of financial incentives. (Think of what might have happened had we offered these incentives and also given them time to prepare and study.) The goal of this experiment wasn’t to design an incentive scheme to be used in other schools. What we were after was a diagnostic tool that could help us understand whether the score gap was due to differences in knowledge, or differences in effort on the test itself. The answer to this question could help us design relevant interventions to reduce the gap.

That said, the incentives worked differently for the different groups. We discovered that the older students, in particular, responded well to the money, whereas the younger ones liked the trophies we offered instead. Offering a second-, third-, or fourth-grade student a $3 trophy before the test improved her performance by 12 percentile points. These effects were large; indeed,
they were similar to the impact of reducing class size by one-third or considerably improving teacher quality
. This is an important point, as we discussed in
Chapter 1
. Incentives don’t have to come in the form of money. In some situations, and for some people, a trophy (or flowers, chocolate—you name it) can go a long way.

As we expected, giving students the rewards beforehand—and threatening to take them away if their scores didn’t improve—boosted scores much more than promising to give them the money later. In fact, the students who were promised to receive $20 a month later did not improve at all. Again, it seemed that framing something as “it’s yours to lose” worked better than “it’s yours to win, and you will win it later.” To understand this, put yourself in a student’s shoes. If you are offered the financial reward for improving, you score a lot higher if, before you even take the test, you are thinking about buying those new skateboard wheels. For young children and teens, the world is all about the present; our experiment helped us understand what really motivates them.

Obviously, all we had managed to do here was to convince students to try a little harder. But we were worried. What if incentives lost their effect on behavior over time? That is, we thought we could get students to try hard a few times, but we wondered if, eventually, the incentives would lose their impact on behavior. Alternatively, would kids only try when they were offered an incentive? Would they give up if a $20 bill wasn’t at stake?

We often hear concerns from educators, parents, and policy makers that even while financial incentives may produce short-term improvements, children could be hurt in the long run; they might stop putting in any effort without compensation.
⁹
In fact, we found no evidence of one-time rewards
hurting
test scores in the future. As we expected, the one-time incentives also did not lead to lasting learning gains. The simple short-term experiment did show, however, that the children were more able than one might have previously thought, based on the standard approach to testing.

The next step was, of course, to stretch our behavioral economics interventions even further. What if students were rewarded every week for an entire semester for something like independent reading? So we launched another study that offered students in seven schools $2 (or an equivalently priced nonfinancial incentive) for every book they read over the course of a semester. We kept track of their reading by letting them log onto an online program called Accelerated Reader that has short quizzes for just about every book available to students. The quizzes aren’t difficult, but it’s hard to score well if you didn’t read the book. We decided that if a student scored 80 percent or higher on a quiz for a book, they could receive the rewards, which we gave out each week. As in the test incentive study, we compared giving students the incentive either at the beginning of the week or at the end of the week. We found that the incentives in both cases increased reading by 37 percent, but the extra reading had no impact on test scores.

Can the Same Idea Work with Teachers?

Of course, students don’t learn in a vacuum. We needed to find out whether offering incentives to teachers might work, too. After all, it’s hard to run a classroom while your students are undisciplined, indifferent, scared, hungry, or absent. And it’s harder still to work so hard and face the fact that many of your ninth graders are reading at a fourth-grade level, and that fewer than half of the kids you’re trying so hard to teach will graduate.