Lady Churchill's Rosebud Wristlet No. 26 (6 page)

By the way, I’m not claiming that consciousness can only occur in organic tissue. I believe that Strong AI is possible, meaning that I think it’s entirely possible for computers to be self-aware in the same way that human beings are. I just think it’s going to be extraordinarily difficult to achieve. The misconception that brains are just like computers makes people think that Strong AI will be easy.

I should also acknowledge that I don’t think this misconception that the brain is like a computer is impeding real scientific progress. It’s only people who don’t work in neuroscience that have fallen for this misconception; actual neuroscientists know better. What I’m concerned about is the way this misconception is limiting our imaginations, particularly in science fiction.

Now I’m going to rant a little bit about some of the science-fiction tropes that bother me because of they way they equate the brain with computer technology. The first one is the trope of uploading a person into a computer just by scanning their brain. So many stories depict this as being a simple procedure, but I’d say it is next to impossible. Here’s why:

When engineers examine a sample of a material to measure its physical properties, they make a distinction between destructive testing and non-destructive testing. Destructive testing means that the sample is rendered unusable by the test; for example, you test a material’s tensile strength by actually pulling it apart. Non-destructive testing means you perform some kind of examination that doesn’t damage the sample. For example, sometimes you can x-ray a weld and look for cracks that way.

When it comes to medicine, we have some non-destructive tests, like x-rays or ultrasound or MRIs. But the vast majority of tests are actually destructive. When the doctors take a blood sample to run some tests, they use that blood up; it’s gone, you can’t get it back. And think about all the conditions that require a blood test instead of an x-ray. Doctors can’t even tell if your kidney have stopped working without taking a blood sample. How likely is it that we’ll be able to read a person’s memories with less intrusive technology? That implies that your memories lie closer to the surface of your body than the fact that you’re in renal failure. I would not say that renal failure is a subtle condition; it affects every drop of your blood, and yet we need destructive testing to diagnose it. Your memories are not printed on your skin like ink on a page, where they can be read by simple inspection. Your memories are extremely subtle, and deeply embedded in your brain, so the idea that we can read those with a non-destructive test is, I think, ludicrous. It might be possible one day to scan all of a person’s memories into a computer, but I’ll bet you that the process will be destructive. The person will be nothing but a cloud of pink mist by the time the scan is done.

Then there’s the idea that we can download new memories or skills, or even a whole new personality, into someone’s brain. We’ve seen this in movies like
The Matrix
and television series like
Dollhouse
, but there’s also plenty of written science fiction that assumes this is the next logical step after the ability to read memories is achieved. This drives me crazy. Even if scanning memories were easy, it doesn’t follow that you’d be able to write memories into the brain. This idea is purely a result of our familiarity with computers; as soon as you stop thinking in terms of computers, you realize how little sense it makes.

For example, let’s consider photographs. You can lay a photograph down on the glass plate of your Hewlett-Packard or Epson scanner, and it will read it down to a level of detail that is invisible to the naked eye. Photographs are just about the easiest things in the world to scan; everything that’s interesting about them lies on the surface, rather than on deeper layers. But do I expect that the next generation of scanner will let you change the imagery on the photos you laid down on the glass? Of course not, and it’s not because I lack faith in the ingenuity of Hewlett-Packard or Epson. It’s because rewritability depends on the material that the original photos were printed on, and that’s not something the makers of scanning machines have any control over. If we started printing photos on special paper using erasable ink, maybe we could have scanners that could rewrite those photos as easily as they scanned them. But our old Polaroids weren’t designed to be rewriteable, so your flatbed scanner is never going to be able to rewrite them.

If we move away from photos and back into the realm of biology, rewritability becomes even more difficult. Do you think we’ll be able to take an ultrasound of your failing kidneys, edit the image to one of a healthy kidney, and save the changes back into your body? Do you think we’ll have an x-ray machine that will let the doctor repair fractures using a mouse cursor, and then overwrite your broken bone with the intact bone? No, of course not. So why do we see so many stories in which it’s possible to do this to your brain? It’s because we’ve been seduced by the idea that the brain is a computer. We accept those stories because we are still engaging in folk biology.

Okay, that’s enough ranting. You may say, we don’t actually believe that what those stories describe is possible, we’re practicing suspension of disbelief when we read them. That’s true, although I don’t think most readers put memory uploading or downloading in the same category as, say, faster-than-light travel; I think a lot of readers imagine that some of those brain technologies are coming in the relatively near future. But the more general point is that science fiction doesn’t have to be scientifically accurate to be interesting or worthwhile, and I absolutely agree with that.

One of the ways a speculation can be interesting, even if it’s not scientifically accurate, is if it raises philosophical questions. And the idea that the brain is a computer can be interesting in that regard; it lets you examine questions about free will and the material basis of consciousness. In a similar vein, the idea that the mind can be treated like software offers an interesting metaphor for the ways workers are dehumanized and commodified by modern society. But a lot of that territory is well explored by now, and most current stories that treat brains as computers are not trying to break new ground in that regard.

Perhaps the biggest reason we enjoy speculations in science fiction is for their strangeness, the sense of wonder they provoke. This, I think, is where the idea of the brain as a computer really falls short. Because nowadays it is anything but strange; it is commonplace. So, in a way, my complaint about this fallacy is really just a specific version of a very common complaint about science fiction: that so much of it is familiar instead of mind-expanding. This is why it bothered me when I realized that thinking of the brain as a computer is a kind of folk biology. Because science fiction should challenge us, and folk biology is not challenging. Folk biology cannot be challenging, by its very nature, because it reflects our intuition. Science fiction should not reinforce naïve ideas about how the world works.

Now, I realize that this is subjective: not everyone agrees on what’s strange or challenging. An idea that seems strange to me might seem perfectly ordinary to you, and vice versa. So let me give you an example of a property of the brain that something I find strange.

There is a memorization technique known as the “memory palace.” This is a method for remembering things that’s been used in various forms for thousands of years; you may have read about it in John Crowley’s novels
Little, Big
and
Ægypt
. The idea is to think of a building that you know so well you can walk through it in your mind and picture the decorations in each and every room. To memorize facts like names or numbers, you construct scenes that represent those facts; the more outrageous the scene is, the better. One example I recently read is that, if you want to remember the number 124, you could picture a spear cutting a swan into four pieces, because the spear looks like the number one, the swan looks like the number two, and the four pieces represents the number four. You’d picture this gruesome scene within one room of your memory palace. Then, when you need to recall that number, you mentally walk to that room, see a swan being cut into four pieces, and remember the number 124.

I once asked John Crowley if he used the memory-palace technique himself, and he said, no, he couldn’t see how it could possibly work in practice. This was a relief to me, because I had the exact same reaction when I read about it. But the fact is that people have used it with enormous success. People used it in ancient times, when parchment was so expensive that you couldn’t readily write things down. Memory artists today use a version of this technique. So even though I find it a completely bizarre concept, I have to assume that it actually works.

There are a number of things about this that are worth noting. One is that this is absolutely nothing like the way a computer’s memory works. Thinking about the brain as a computer does not help us understand why the memory-palace technique is effective; neither does thinking about the brain as a telephone switchboard, or a steam engine, or a clock. If we ever come up with a metaphor for the brain that helps us understand why the memory-palace technique works, then I think we’ll really be getting somewhere.

Another thing worth noting is that the memory-palace technique was developed before we had mechanical aids to memory. Many people have observed that the widespread use of writing reduced our reliance on our memories, but I think it had another, subtler effect. It started changing the way we think about our minds. Once people had seen a library of books, they could imagine their own memory as being like a library of books. And once people started doing that, they had taken the first step toward thinking about the brain as a technological device. And that makes me wonder, if we were all using the memory-palace technique, would we have come up with different metaphors for understanding the brain?

Earlier I mentioned that early attempts at blood transfusion failed because we didn’t know about blood types. Has there been a similar mistake made because we used folk biology when thinking about the brain? Yes, I think there has been: the belief that repressed memories can be recovered via hypnosis. I don’t mean to deny that it’s possible for a person to repress memories, but I believe many people have been unjustly accused of crimes on the basis of so-called recovered memories. I think the implicit assumption underlying the idea of recovered memory is that our brain is some kind of mechanical recording device, like a camera or a tape recorder. Because if you think that brains act like cameras or tape recorders, you’re likely to believe that an objective account of events is recorded somewhere in your brain. Even if you can’t immediately access it, you imagine that an accurate memory is there.

However, it turns out that it’s easy to implant false memories. You don’t need hypnosis to do it; you can just ask some leading questions and make someone think they saw all sorts of things they didn’t see. Some psychologists believe eyewitness identifications shouldn’t be used in court because of their unreliability. Our memories are so inaccurate that they are pretty much the opposite of cameras or tape recorders. And I wonder, if we were all using the memory palace technique, would we have been more skeptical of memories recovered by hypnosis? This is purely speculation on my part, but I feel like the memory-palace technique makes it clearer that memories can be based on your imagination just as much as your sensory impressions.

Anyway, the reason I brought up the memory-palace technique is that I find it astonishing. Maybe it seems ordinary to you, but it feels much stranger to me than thinking about the brain as a computer, and I’d like to read a science-fiction story about the brain that has the same effect. Maybe that would require writers to come up with a new metaphor for the brain, but maybe it wouldn’t. I don’t ask that this story reflect the latest research in neuroscience; I just want it to talk about the brain in a way that I haven’t seen a hundred times before.

I think science fiction has reached the point where it’d actually be more interesting for a story to explore the ways that the brain
isn’t
like a computer. Even comparing the brain to a steam engine might be preferable, if it helps readers realize that the computer metaphor is just a product of our times.

As I said before, science fiction doesn’t have to be scientifically accurate. But there are some things I want science fiction to have in common with science. Science is about examining your assumptions and not simply relying on common sense about the way the universe works. I think science fiction should do the same. And that’s why I think we should be on the lookout for folk biology in science fiction, and avoid it when we can. Because folk biology confirms what you already think you know, and we should ask more of science fiction than that.

If you move, please remember to send us your Change of Address. We thank you!

Email: [email protected]

Paper mail: LCRW, 150 Pleasant St., #306,

Easthampton, MA 01027