The Idea Factory: Bell Labs and the Great Age of American Innovation (19 page)

Shannon wasn’t interested in helping with the complex implementation of PCM—that was a job for the development engineers at Bell Labs, and would end up taking them more than a decade. “I am very seldom interested in applications,” he later said. “I am more interested in the elegance of a problem. Is it a good problem, an interesting problem?”
²⁷
For him, PCM was a catalyst for a more general theory about how messages move—or in the future
could
move—from one place to another. What he’d been working on at home during the early 1940s had become a long, elegant manuscript by 1947, and one day soon after the press conference in lower Manhattan unveiling the invention of the transistor, in July 1948, the first part of Shannon’s manuscript was published as a paper in the
Bell System Technical Journal
; a second installment appeared in the
Journal
that October.
²⁸
“A Mathematical Theory of Communication”—“the magna carta of the information age,” as
Scientific American
later called it—wasn’t about one particular thing, but rather about general rules and unifying ideas. “He was always searching for deep and fundamental relations,” Shannon’s colleague Brock McMillan explains. And here he had found them. One of his paper’s underlying tenets, Shannon would later say, “is that information can be treated very much like a physical quantity, such as mass or energy.”
²⁹
To consider it on a more practical level, however, one might say that Shannon had laid out the essential answers to a question that had bedeviled Bell engineers from the beginning: How rapidly, and how accurately, can you send messages from one place to another?

“The fundamental problem of communication,” Shannon’s paper explained, “is that of reproducing at one point either exactly or approximately a message selected at another point.” Perhaps that seemed obvious, but Shannon went on to show why it was profound. If “universal connectivity” remained the goal at Bell Labs—if indeed the telecommunications systems of the future, as Kelly saw it, would be “more like the biological systems of man’s brain and nervous system”—then the realization of those dreams didn’t only depend on the hardware of new technologies,
such as the transistor. A mathematical guide for the system’s engineers, a blueprint for how to move data around with optimal efficiency, which was what Shannon offered, would be crucial, too. Shannon maintained that all communications systems could be thought of in the same way, regardless of whether they involved a lunchroom conversation, a postmarked letter, a phone call, or a radio or telephone transmission. Messages all followed the same fairly simple pattern:

All messages, as they traveled from the information source to the destination, faced the problem of noise. This could be the background clatter of a cafeteria, or it could be static (on the radio) or snow (on television). Noise interfered with the accurate delivery of the message. And every channel that carried a message was, to some extent, a
noisy channel.

To a non-engineer, Shannon’s drawing seemed sensible but didn’t necessarily explain anything. His larger point, however, as he proved in his mathematical proofs, was that there were ways to make sure messages got where they were supposed to, clearly and reliably so. The first place to start, Shannon suggested, was to think about the
information
within a message. The semantic aspects of communication were irrelevant to the engineering problem, he wrote. Or to say it another way: One shouldn’t necessarily think of information in terms of
meaning.
Rather, one might think of it in terms of its ability to resolve uncertainty. Information provided a recipient with something that was not previously known, was not predictable, was not redundant. “We take the essence of information as the irreducible, fundamental underlying uncertainty that is removed by
its receipt,” a Bell Labs executive named Bob Lucky explained some years later.
³⁰
If you send a message, you are merely choosing from a range of possible messages. The less the recipient knows about what part of the message comes next, the more information you are sending. Some language choices, Shannon’s research suggested, occur with certain probabilities, and some messages are more frequent than others; this fact could therefore lead to precise calculations of how much information these words or messages contained.
³¹
(Shannon’s favorite example was to explain that one might need to know that the word “quality” begins with
q
, for instance, but not that a
u
follows after. The
u
gives a recipient no information if they already have the
q
, since
u
always follows
q
; it can be filled in by the recipient.)

Shannon suggested it was most useful to calculate a message’s information content and rate in a term that he suggested engineers call “bits”—a word that had never before appeared in print with this meaning. Shannon had borrowed it from his Bell Labs math colleague John Tukey as an abbreviation of “binary digits.” The bit, Shannon explained, “corresponds to the information produced when a choice is made from two equally likely possibilities. If I toss a coin and tell you that it came down heads, I am giving you one bit of information about this event.” All of this could be summed up in a few points that might seem unsurprising to those living in the twenty-first century but were in fact startling—“a bolt from the blue,” as one of Shannon’s colleagues put it
³²
—to those just getting over the Second World War: (1) All communications could be thought of in terms of information; (2) all information could be measured in bits; (3) all the measurable bits of information could be thought of, and indeed should be thought of,
digitally
. This could mean dots or dashes, heads or tails, or the on/off pulses that comprised PCM. Or it could simply be a string of, say, five or six 1s and 0s, each grouping of numerical bits representing a letter or punctuation mark in the alphabet. For instance, in the American Standard Code for Information Interchange (ASCII), which was worked out several years after Shannon’s theory, the binary representation for FACT IS STRANGER THAN FICTION would be as follows:

Thus Shannon was suggesting that all information, at least from the view of someone trying to move it from place to place, was the same, whether it was a phone call or a microwave transmission or a television signal.

This was a philosophical argument, in many respects, and one that would only infiltrate the thinking of the country’s technologists slowly over the next few decades. To the engineers at the Labs, the practical mathematical arguments Shannon was also laying out made a more immediate impression. His calculations showed that the information content of a message could not exceed the capacity of the channel through which you were sending it. Much in the same way a pipe could only carry so many gallons of water per second and no more, a transmission channel could only carry so many bits of information at a certain rate and no more. Anything beyond that would reduce the quality of your transmission. The upshot was that by measuring the information capacity of your channel and by measuring the information content of your message you could know how fast, and how well, you could send your message. Engineers could now try to align the two—capacity and information content. For anyone who actually designed communications systems with wires or cables or microwave transmitters, Shannon had handed them not only an idea, but a new kind of yardstick.

S
HANNON’S PAPER
contained a claim so surprising that it seemed impossible to many at the time, and yet it would soon be proven true. He showed that any digital message could be sent with virtual perfection, even along the noisiest wire, as long as you included error-correcting codes—essentially extra bits of information, formulated as additional 1s
and 0s—with the original message. In his earlier paper on cryptography, Shannon had already shown that by reducing redundancy you could compress a message to transmit its content more efficiently. Now he was also demonstrating something like the opposite: that in some situations you could increase the redundancy of a message to transmit it more accurately. One might think of the simplest case: Sending the message FCTSSTRNGRTHNFCTN FCTSSTRNGRTHNFCTN FCTSSTRNGRTHNFCTN—an intentional tripling—to improve the chances that the message was clearly received amid a noisy channel. But Shannon showed that there were myriad and far more elegant ways to fashion these kinds of self-correcting codes. Some mathematicians at the Labs would spend their careers working out the details, which Shannon was largely uninterested in doing.
³³

All the error-correcting codes were meant to ensure that the information on the receiving end was exactly or very close to the message that left the transmission end. Even fifty years later, this idea would leave many engineers slack-jawed. “To make the chance of error as small as you wish?” Robert Fano, a friend and colleague of Shannon’s, later pointed out. “How he got that insight, how he even came to believe such a thing, I don’t know.” All modern communications engineering, from cell phone transmissions to compact discs and deep space communications, is based upon this insight.
³⁴

For the time being, knowledge of Shannon’s work largely remained confined to the mathematics and engineering community. But in 1949, a year after the communications paper was published, it was published in book form for a wider audience. Shortly thereafter communication theory became more commonly known as information theory. And eventually it would be recognized as the astounding achievement it was. Shannon’s Bell Labs colleagues came to describe it as “one of the great intellectual achievements of the twentieth century”; some years later, Shannon’s disciple Bob Lucky wrote, “I know of no greater work of genius in the annals of technological thought.”
³⁵
A few observers, in time, would place the work on a par with Einstein’s. Shannon himself refused to see the theory in such grand terms; he even urged a magazine journalist, in
a genial prepublication letter in the early 1950s, to strip any comparison of his theory to Einstein’s ideas on relativity in a story: “Much as I wish it were,” he lamented, “communication theory is certainly not in the same league with relativity and quantum mechanics.”
³⁶

In any case, Shannon was ready to move on. He later quipped that at this point in time he was becoming highly interested in “useless” things—games, robots, and small machines he called
automata
. To some observers, Shannon’s new interests seemed to suggest that rather than advance his extraordinary insights on communications, he preferred to squander his genius on silly gadgetry or pointless questions, such as whether playing tic-tac-toe on an infinite sheet of graph paper would reveal patterns about tie games. Perhaps this was missing a point. Some of Shannon’s more curious pursuits were far less useless than they appeared, but he was growing increasingly reluctant to explain himself. People could see what he was doing, then they could think whatever they wanted.

A
FTER THE PUBLICATION
of information theory, no one could dare tell Shannon what he should work on. Often he went into his office just like any other Bell Labs employee. But beginning in the early 1950s sometimes he didn’t go in at all, or he wandered in late and played chess the rest of the day, much to the consternation, and sometimes the envy, of his colleagues.
³⁷
When he was in the office at Murray Hill he would often work with his door closed, something virtually unheard of at Bell Labs. And yet that breach, too, was permissible for Claude Shannon. “He couldn’t have been in any other department successfully,” Brock McMillan recalls. “But then, there weren’t many other departments where people just sat and thought.”

Increasingly, Shannon began to arouse interest not only for the quality of his ideas but for his eccentricity—“for being a very odd man in so many ways,” as his math department colleague David Slepian puts it. “He was not an unfriendly person,” Slepian adds, “and he was very modest,” but those who knew him and extended a hand of friendship realized that he would inevitably fail to reciprocate. They had to seek him out.
And in calling on him (knocking on his door, writing, visiting) one had to penetrate his shyness or elusiveness or—in the case of expecting a reply to a letter—his intractable habits of procrastination and his unwillingness to do anything that bored him.

Shortly after the information paper was published, he did make one significant social connection, marrying a young woman named Betty Moore who worked in the Labs’ computational department. Sometimes he and Betty would go to Labs functions—dinner at Jim Fisk’s house, where Bill Shockley and his wife would also be guests. But mainly the math department, a social group both in and outside of the office, would extend invitations to Shannon and he would politely refuse.