Darwin's Dangerous Idea (82 page)

right angle? Is this triangle over here congruent to that triangle over there?"

—A

Often the answer was obvious: you could just
see
that the lines had to UN TURING 1946, p. 124

intersect at a right angle, that the triangles were congruent. But it was another The attempts over the years to use Godel's Theorem to prove something matter—in fact, a considerable amount of inspired drudgery—to prove it important about the nature of the human mind have an elusive atmosphere of from the axioms, formally, according to the strict rules. Did you ever wonder, romance. There is something strangely thrilling about the prospect of "using when the teacher put a new diagram on the blackboard, whether there might science" to such an effect. I think I can put my finger on it. The key text is be facts about plane geometry that you could
see
were true but couldn't not the Hans Christian Andersen tale about the Emperor's New Clothes, but prove, not in a million years? Or did it seem obvious to you that, if you the Arthurian romance of the Sword in the Stone. Somebody (our hero, of yourself were unable to devise a proof of some candidate geometric truth, course) has a special, perhaps even magical, property which is quite invisible this would just be a sign of your own personal frailty? Perhaps you thought: under most circumstances, but which can be made to reveal itself quite

"There has to
be
a proof, since it's
true,
even if I myself can never find it!"

unmistakably in special circumstances: if you can pull the sword from the That's an intensely plausible opinion, but what Gödel proved, beyond any stone, you have the property; if you can't, you don't. This is a feat or a failure doubt, is that when it comes to axiomatizing simple
arithmetic
( not plane that everyone can see; it doesn't require any special interpretation or special geometry), there are truths that "we can see" to be true but that can
never
be pleading on one's own behalf. Pull out the sword and you win, hands down.

formally proved to be true. Actually, this claim must be carefully hedged: for What Godel's Theorem promises the romantically inclined is a similarly any
particular
axiom system that is
consistent
(not subtly self-dramatic proof of the specialness of the human mind. Godel's Theorem contradictory—a disqualifying flaw), there must be a sentence of arithmetic, defines a deed, it seems, that a genuine human mind can perform but that no now known as the Godel sentence of that system, that is not provable within impostor, no mere algorithm-controlled robot, could perform. The technical the system but is true. (In fact, there must be many such true sentences, but details of Godel's proof itself need not concern us; no mathematician one is all we need to make the point.) We can change systems, and prove that 430 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Sword in the Stone
431

first Godel sentence in the next axiom system we choose, but it in turn will edly,
at least some part of such a human being cannot be a mere machine, spawn its own Godel sentence, if it is consistent, and so on forever. No single even a huge collection of gadgets. If hearts are pumping machines, and lungs consistent axiomatization of arithmetic can prove all the truths of arithmetic.

are air-exchanging machines, and brains are computing machines, then This might not seem to matter very much, since we seldom if ever want to mathematicians' minds cannot be their brains, Godel thought, since
prove
facts of arithmetic; we just take arithmetic for granted without proof.

mathematicians' minds can do something that no mere computing machine But it is possible to devise Euclid-type axiom systems for arithmetic—

can do.

Peano's axioms, for instance—and prove such simple truths as "2 + 2 = 4,"

What, exactly, can they do? This is the problem of defining the feat for the such obvious middle-level truths as "numbers evenly divisible by 10 are also big empirical test. It is tempting to think we have already seen an example: evenly divisible by 2," and such unobvious truths as "There is no largest they can do what you used to do when you looked at the blackboard in prime number." Before Godel devised his proof, the goal of deriving
all
geometry class—using something like "intuition" or "judgment" or "pure mathematical truth from a single set of axioms was widely regarded by understanding," they
can just see
that certain propositions of arithmetic are mathematicians and logicians as their great project, difficult but within reach, true. The idea would be that they don't need to rely on grubby algorithms to the moon landing or Human Genome Project of the mathematics of the day.

generate
their
mathematical knowledge, since they have a talent for grasping But it absolutely can't be done. That is what Gödel's Theorem establishes.

mathematical truth that transcends algorithmic processes alto-gether.

Now, what does this have to do with Artificial Intelligence or evolution?

Remember that an algorithm is a recipe that can be followed by servile Godel proved his theorem some years before the invention of the electronic dunces—or even machines; no understanding is required. Clever computer, but then Alan Turing came along and extended the implications of mathematicians seem, in contrast, to be able to
use
their understanding to go that abstract theorem by showing, in effect, that any formal proof procedure beyond what such mechanical dunces can do. But although this seems to be of the sort covered by Godel's Theorem is equivalent to a computer program.

what Gödel himself thought, and it certainly expresses the general popular Godel had devised a way of putting
all possible axiom systems
in understanding of what Gödel's Theorem shows, it is much harder to

"alphabetical order." In fact, they can all be lined up in the Library of Babel,
demonstrate
than first appears. How can we distinguish a case of somebody and Turing then showed that this set was a subset of another set in the (or something) "grasping the truth" of a mathematical sentence from a case Library of Babel: the set of
all possible computers.
It doesn't matter what of somebody (or something) just wildly guessing correctly, for instance? You material you make a computer out of; what matters is the algorithm it runs; could train a parrot to utter "true" and "false" when various symbols were and since every algorithm is finitely specifiable, it is possible to devise a written on the blackboard in front of it; how many correct guesses without an uniform language for uniquely describing each algorithm and putting all the error would the parrot have to make for us to be justified in believing that the specifications in "alphabetical order." Turing devised just such a system, and parrot had an immaterial mind after all (or perhaps was just a human in it every computer—from your laptop to the grandest parallel supercom-mathematician in a parrot costume) (Hofstadter 1979)?

puter that will ever be built—has a unique description as what we now call a This is the problem that has always given fits to those who want to use
Turing machine.
The Turing machines can each be given a unique iden-Gödel's Theorem to prove that our minds are skyhooks, not just boring old tification number—its Library of Babel Number, if you like. Gödel's Theo-cranes. It won't do to say that mathematicians, unlike machines, can
prove
rem can then be reinterpreted to say that each of those Turing machines that any truth of arithmetic, for, if what we mean by "prove" is what Gödel means happens to be a consistent algorithm for proving truths of arithmetic (and, not by "prove" in his proof, then Gödel shows that human beings—or angels, if surprisingly, these are a Vast but Vanishing subset of all the possible Turing such there be—cannot do it either (Dennett 1970);
there is no
formal proof of machines) has associated with
it
a Gödel sentence—a truth of arithmetic it a system's Gödel sentence within the system. A famous early attempt to cannot prove. So that is what Gödel, anchored by Turing to the world of harness Gödel's Theorem was by the philosopher J. R. Lucas (1961; see also computers, tells us: every computer that is a consistent truth-of-arithmetic-1970), who decided to define the crucial feat as "producing as true" a certain prover has an Achilles' heel, a truth it can never prove, even if it runs till sentence—some Gödel sentence or other. This definition runs into insoluble doomsday. But so what?

problems of interpretation, however, ruining the "sword-in-the stone"

Gödel himself thought that the implication of his theorem was that human definitiveness of the empirical side of the argument ( Dennett 1970, 1972; beings—at least the mathematicians among us—cannot, then, be just see also Hofstadter 1979). We can see more clearly what the problem is by machines, because they can do things no machine could do. More point-considering several related feats, real and imaginary.

Rene Descartes, in 1637, asked himself how one could tell a genuine 432 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Sword in the Stone
433

human being from any machine, and he came up with "two very certain

"innconceivable," and even if, as many today believe, no machine will ever means":

succeed in passing the Turing Test, almost no one today would claim that the very idea is inconceivable. Perhaps this sea-change in public opinion has The first is that they [the machines] could never use words, or put together been helped along by the comouter's progress on other feats, such as playing other signs, as we do in order to declare our thoughts to others. For we can checkers and chess. In an address in 1957, Herbert Simon (Simon and certainly conceive of a machine so constructed that it utters words, and Newell 1958) predicted that computer would be the world chess champion in even utters words which correspond to bodily actions causing a change in less than a decade, a classic case of overoptimism, as it turns out. A few its organs (e.g., if you touch it in one spot it asks what you want of it, if you years later, the philosopher Hubert Dreyfus (1965) predicted that no touch it in another it cries out that you are hurting it, and so on). But it is computer would ever be able to play good chess, since playing chess not conceivable that such a machine should produce different arrange-required "insight," but he himself was soon trounced at chess by a computer ments of words so as to give an appropriately meaningful answer to what-program, an occasion for much glee among AI researchers. Art Samuel's ever is said in its presence, as the dullest of men can do. Secondly, even checkers program has been followed by literally hundreds of chess-playing though such machines might do some things as well as we do them, or programs, which now compete in tournaments against both human and other perhaps even better, they would inevitably fail in others, which would computer contestants, and will
probably
soon be able to beat the best human reveal that they were acting not through understanding but only from the chess players in the

disposition of their organs. For whereas reason is a universal instrument world.

which can be used in all kinds of situations, these organs need some particular disposition for each particular action; hence it is for all practical But is chess-playing a suitable "sword-in-the-stone" test? Dreyfus may purposes impossible for a machine to have enough different organs to once have thought so, and he has a distinguished predecessor—Edgar Allan make it act in all the contingencies of life in the way in which our reason Poe, of all people—whose certainty on this score drove him to unmask one makes us act. [Descartes 1637, pt. 5.]

of the great hoaxes of the nineteenth century, von Kempelen's chess "automaton." In the eighteenth century, the great Vaucanson had made me-Alan Turing, in 1950, asked himself the same question, and came up with chanical marvels that entranced the nobility, and other paying customers, by just the same acid test—somewhat more rigorously described—what he exhibiting behaviors that even today inspire our skepticism. Could Vau-called the imitation game, and we now call the Turing Test. Put two con-canson's clockwork duck really do what it is reported to have done? "When testants—one human, one a computer—in boxes (in effect) and conduct corn was thrown down before it, the duck stretched out its neck to pick it up, conversations with each; if the computer can convince you it is the human swallowed, and digested it" (Poe 1836a, p. 1255). Other ingenious artificers being, it wins the imitation game. Turing's verdict, however, was strikingly and tricksters had followed in Vaucanson's wake, developing the art of different from Descartes's:

mechanical simulacra to such a high pitch that one of them, Baron von Kempelen, in 1769, could exploit public fascination with such devices by I believe that in about fifty years' time it will be possible to program creating a deliberate tease: a
purported
automaton that could play chess.

computers, with a storage capacity of about 109, to make them play the Von Kempelen's original machine passed into the hands of Johann Nep-imitation game so well that an average interrogator will not have more omuk Maelzel,1 who made some improvements and revisions, and then than a 70 percent chance of making the right identification after five min-caused quite a stir in the early nineteenth century by exhibiting Maelzel's utes of questioning. The original question, 'Can machines think?' I believe Chess-Machine, never quite guaranteeing that it was just a machine, and to be too meaningless to deserve discussion. Nevertheless I believe that at surrounding the whole performance (for which he charged a pretty penny ) the end of the century the use of words and general educated opinion will with enough of the standard magician's ostentations to arouse anybody's have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. [Turing 1950, p. 435.]

Turing has already been proven right about his last prophecy: "the use of 1. This same Maelzel is the inventor (or perfecter) of the metronome, and made the ear words and general educated opinion" has already "altered so much" that one tnumpet that Beethoven relied on for years, once he began to go deaf. Maelzel also
can
speak of machines thinking without expecting to be contradicted— "on created a mechanical orchestra, the Panharmonicon, for which Beethoven wrote
Well-general principles." Descartes found the notion of a thinking machine
ington's Victory,
but the two had a falling out over property rights to that composition—