Scorecasting (23 page)

But strikeouts and walks are not the right statistic to measure, because many strikes occur when a batter swings and misses or fouls off a ball. In such cases, there is no umpire discretion. A better metric to look at is
called
balls and strikes.
^*
In other words, look only at pitches that do not involve swinging by the batter. It turns out that home batters receive far fewer called strikes per called pitch than away batters do.

It’s even more apparent when we look at called strikes and balls at different points in the game. Certain situations have a much bigger impact on the game’s outcome than others. Fortunately for us, Sabermetrics, an analysis of baseball through objective evidence, provides another useful tool to gauge the importance of a particular situation. A stats wizard,
Tom Tango, devised a metric called the Leverage Index to measure the relative importance of any game situation. The idea is to take every game situation and consider every possible scenario that could occur in that situation, the likelihood of each scenario playing out from that point, and what effect each of those scenarios would have on the ultimate outcome of the game. Add up all these possibilities, their likelihood of occurring, and their potential impact on the game and you have a measure of how crucial the current situation is. A Leverage Index of 1 is the average situation; an index of 2 means
the situation is twice as crucial. Here are two extreme examples: Down by four runs with two outs and nobody on base in the bottom of the ninth, where the game isn’t in much doubt, translates into a Leverage Index of 0.1—the situation is one-tenth as crucial as the average situation. Down by one run in the bottom of the ninth with two outs and the bases loaded, where the game is on the line, gives a Leverage Index of 10.9. It is almost 11 times more crucial than the average situation.

Using the Leverage Index to examine called strike and ball counts in different situations, we found, just as with the soccer referees, that in low-leverage situations, when the game is not in much doubt, the home team advantage in receiving fewer called strikes and more balls goes away. But as the following chart shows, the called-strike advantage for home teams grows considerably as the game situation gets more and more important. In noncrucial and average situations, the home team receives about the same strike calls as, or even a few more strike calls than, the away team per called pitch, but that changes dramatically when the game is on the line. In crucial situations, the home team receives far fewer called strikes per called pitch than does the away team.

This makes sense. If the umpire is going to show favoritism to the home team, he or she will do it when it is most valuable—when the outcome of the game is affected the most. You might even contend that in noncrucial situations the umpire may be biased against the home team to maintain an overall appearance of fairness.

Think back to that Jake Fox pitch in the Cubs-Brewers game, on a 3–2 count with the bases loaded and a tie game on the line in the bottom of the tenth inning. It was an astronomically high-leverage situation. Knowing the statistics, you would have bet the house that the pitch wouldn’t have been called a strike. And it wasn’t.

Let’s look at other calls that fall under the domain of the umpires, in particular, close calls that typically elicit a home crowd reaction. Two good examples would be stolen bases and double plays. We found that home teams are more likely to be successful when stealing a base and when turning a double play, yet the distance between the bases is identical in every stadium—stolen base success can’t be driven by home field idiosyncrasies. In addition, the success rates of home teams in scoring from second base on a single or scoring from third base on an out—typically close plays at the plate—are much higher than they are for their visitors in high-leverage/crucial situations. Yet they are no different or even slightly less successful in noncrucial situations. (Third-base coaches, take note: If it’s a close game and you’re playing at home, windmill your arms and send the runner!)

But the most damning evidence of umpire bias might be a function of a tool that was employed for the specific purpose of eliminating umpire bias. Remember the Pitch f/x system that tracks the characteristics of each pitch, including location? Well, its predecessor—a digital technology called Umpire Information System (UIS) from QuesTec—was installed five years earlier by Major League Baseball for the specific purpose of monitoring the accuracy of umpires. According to Major League Baseball, QuesTec was implemented in six ballparks in the first year; by the time it
was discontinued in 2008, 11 ballparks had the technology.
^*
With two cameras positioned at field level and two in the upper deck, QuesTec combined the four images to track where the ball crosses the plate, and it was used by baseball executives to determine how closely an umpire’s perception of the strike zone mirrored reality.

We also used the presence of QuesTec to evaluate umpire accuracy, but in a different way. We asked whether the same umpire behaved differently when he
knew
the cameras were monitoring him. If the home field advantage in called strikes disappears when the umpires know they’re being watched—while everything else stays constant—it’s pretty clear that official bias underlies it. Imagine you own a coffee shop and put out a jar in which patrons can donate or take loose change. You notice at the end of each day that the jar is empty. You deduce that either some customers are taking advantage by depleting the jar or your employees are stealing the coins. You tell only your employees that you are installing a hidden video camera. If the change jar is full at the end of each day, you’re pretty darn sure it was your employees, not customers, who were to blame.

To test our theory, we first compared all pitches, about 5.5 million of them, from 2002 to 2008 made in stadiums using QuesTec versus those without it. For example, we looked at all called pitches when the Astros visited the Cardinals (at their non-QuesTec stadium) and when the Cardinals visited the Astros (at their QuesTec-equipped stadium).

What did we find? Called strikes and balls went the home team’s way,
but only
in stadiums without QuesTec, that is, ballparks where umpires were not being monitored. This is consistent with an umpire bias toward the home team causing the strike-ball discrepancy. We also found something surprising. Not only did umpires
not favor the home team on strike and ball calls when QuesTec was watching them, they actually gave
more
strikes and
fewer
balls to the home team. In short, when umpires
knew
they were being monitored, home field advantage on balls and strikes didn’t simply vanish; the advantage swung all the way over to the visiting team.

We then looked at the same pitch counts in low-leverage (not crucial) and high-leverage (crucial) points in the game. Again, when a plate appearance is expected to have little effect on the outcome of the game, there is no bias for or against the home team. Umpires call things evenly whether QuesTec is present or not. But when the at-bat can have an impact on the game, we found both biases to be even more extreme. That is, when the game is on the line, home teams in non-QuesTec stadiums get a big strike-ball call advantage and those in QuesTec stadiums get a huge strike-ball call
disadvantage
.

In practical terms, when the umpire is
not
being monitored by QuesTec, a home batter in crucial game situations will get a called strike only 32 percent of the time if he doesn’t swing. In the same situation, a batter from a visiting team gets a called strike 39 percent of the time. That’s a big difference. Now consider the same two situations when the umpire
is
being monitored by QuesTec. Here the home batter gets a called strike 43 percent of the time, and the away batter only 35 percent of the time.

If we were consultants to a team equipped with umpire-monitoring technology, our first piece of advice would be: Get rid of QuesTec; it’s wrecking your home field advantage. (How many teams would have agreed so readily to QuesTec if they knew these numbers?) Of course, if we were consulting for MLB, we might have encouraged them to install the technology in
all
the ballparks or at least tell the umpires that was the case. (Today, that’s essentially what MLB has done.)

We also found the same results for the QuesTec stadiums before and after the system was installed. The called strike-ball differences between home and away teams declined sharply after QuesTec installation, and the decline was particularly pronounced in crucial situations. Even the
same
umpire behaved differently
depending on whether QuesTec was present, calling more strikes and fewer balls on home batters when he was being monitored and doing the opposite when he wasn’t.

Why would the home team advantage for strike-ball calls, particularly in crucial situations, switch completely in the other direction when QuesTec is present? You’d think the advantage would just disappear, creating no bias, but in fact the bias goes in the opposite direction. We suspect that as with the referees in soccer, umpires have to balance the costs and benefits of any bias (conscious or not) they might exhibit. If you know you are being monitored, you want to eliminate any perception of bias. And when the game is on the line, you know that any perceived bias will be scrutinized even more closely. With the speed of the game and the uncertainty of whether a 95-mph fastball hit or missed the outside corner of the plate, umpires may become overly cautious. Worried about accusations of home-team favoritism, the umpire seems to err in the other direction, particularly in situations that will be monitored and analyzed heavily afterward.

What about the other potential umpire biases we found that might benefit the home team, such as stolen bases, double plays, and other close plays? They remain the same whether QuesTec is present or not, which makes sense. After all, QuesTec monitors only the strike zone. It affects no other part of the game. Calls remain in favor of the home team because there’s no “surveillance video” on those calls.

If QuesTec is our smoking gun in the case to prove umpire home team bias in Major League Baseball, Pitch f/x provides the ballistic support.
^*
Using the Pitch f/x location data of the millions of pitches we examined earlier, we asked a series of questions: How likely is it that when the pitch is actually out of the strike zone, an umpire will call a strike on the home team versus the away team? How often is a ball called on a pitch actually within the
strike zone when the home team is batting versus the away team, the situation that enraged the die-hard Brewers fan Jack Moore? What about pitches just in or just out of the strike zone? How do these calls change in critical situations?

The following chart graphs the difference in the percentage of called pitches that are called strikes on home versus away batters for pitches within three inches of the dead center of the strike zone, pitches way out of the strike zone (at least three inches), and pitches within 1.5 inches of the strike zone, for example, just on or off the corners. We report the numbers separately for two-strike, three-ball, and full counts.

Note two points: (1) The home-away differences are largest for two-strike and three-ball counts and especially for full counts. (2) For the most ambiguous pitches—the ones on the corners—the home-away called-strike discrepancy is largest, which makes sense. The umpire has less discretion over pitches that are less ambiguous. Umpires will be reluctant to make a biased call if the pitch
is obviously a strike. In fact, for pitches in the dead center of the strike zone, there is no bias at all. Umpires call these pitches correctly 99 percent of the time whether a home or a visiting batter is standing in front of them. For pitches way outside the strike zone, the umpire has a little more leeway and shows a slight bias in favor of home batters. The umpire has the most discretion for pitches on the corners, and there the home batter bias is largest.