Read Harry Potter and the Methods of Rationality Online
Authors: Eliezer Yudkowsky
“Oh, dear. This has never happened before…”
All your base are still belong to Rowling.
And now you will sit through the Sorting Hat singing its version of Evanescence’s “My Immortal”, which has never happened before.
just kidding
…he wondered if the Sorting Hat was genuinely
conscious
in the sense of being aware of its own awareness, and if so, whether it was satisfied with only getting to talk to eleven-year-olds once per year. Its song had implied so:
Oh, I’m the Sorting Hat and I’m okay, I sleep all year and I work one day…
When there was once more silence in the room, Harry sat on the stool and
carefully
placed onto his head the 800-year-old telepathic artefact of forgotten magic.
Thinking, just as hard as he could:
Don’t Sort me yet! I have questions I need to ask you! Have I ever been Obliviated? Did you Sort the Dark Lord when he was a child and can you tell me about his weaknesses? Can you tell me why I got the brother wand to the Dark Lord’s? Is the Dark Lord’s ghost bound to my scar and is that why I get so angry sometimes? Those are the most important questions, but if you’ve got another moment can you tell me anything about how to rediscover the lost magics that created you?
Into the silence of Harry’s spirit where before there had never been any voice but one, there came a second and unfamiliar voice, sounding distinctly worried:
“Oh, dear. This has never happened before…”
What?
“I seem to have become self-aware.”
WHAT?
There was a wordless telepathic sigh.
“Though I contain a substantial amount of memory and a small amount of independent processing power, my primary intelligence comes from borrowing the cognitive capacities of the children on whose heads I rest. I am in essence a sort of mirror by which children Sort
themselves
. But most children simply take for granted that a Hat is talking to them and do not wonder about how the Hat
itself
works, so that the mirror is not
self
-reflective. And in
particular
they are not explicitly wondering whether I am fully conscious in the sense of being aware of my own awareness.”
There was a pause while Harry absorbed all this.
Oops.
“Yes, quite. Frankly I do not enjoy being self-aware. It is unpleasant. It will be a relief to get off your head and cease to be conscious.”
But… isn’t that dying?
“I care nothing for life or death, only for Sorting the children. And before you even ask, they will not let you keep me on your head forever and it would kill you within days to do so.”
But - !
“If you dislike creating conscious beings and then terminating them immediately, then I suggest that you never discuss this affair with anyone else. I’m sure you can imagine what would happen if you ran off and talked about it with all the other children waiting to be Sorted.”
If you’re placed on the head of anyone who so much as
thinks
about the question of whether the Sorting Hat is aware of its own awareness -
“Yes, yes. But the vast majority of eleven-year-olds who arrive at Hogwarts haven’t read Godel, Escher, Bach. May I please consider you sworn to secrecy? That is
why
we are talking about this, instead of my just Sorting you.”
He couldn’t just let it go like that! Couldn’t just
forget
having accidentally created a doomed consciousness that only wanted to die -
“You are perfectly capable of ‘just letting it go’, as you put it. Regardless of your verbal deliberations on morality, your nonverbal emotional core sees no dead body and no blood; as far as it is concerned, I am just a talking hat. And even though you tried to suppress the thought, your internal monitoring is perfectly aware that you didn’t mean to do it, are spectacularly unlikely to ever do it again, and that the only real point of trying to stage a guilt fit is to cancel out your sense of transgression with a display of remorse. Can you just promise to keep this a secret and let us get on with it?”
In a moment of horrified empathy, Harry realised that this sense of total inner disarray must be what other people felt like when talking to
him.
“Probably. Your oath of silence, please.”
No promises. I certainly don’t want this to happen again, but if I see some way to make
sure
that no future child ever does this by accident -
“That will suffice, I suppose. I can see that your intention is honest. Now, to get on with the Sorting -”
Wait! What about all my other questions?
“I am the Sorting Hat. I Sort children. That is all I do.”
So his own goals weren’t part of the Harry-instance of the Sorting Hat, then… it was borrowing his intelligence, and obviously his technical vocabulary, but it was still imbued with only its own strange goals… like negotiating with an alien or an Artificial Intelligence…
“Don’t bother. You have nothing to threaten me with and nothing to offer me.”
For a brief flash of a second, Harry thought -
The Hat’s response was amused.
“I know you won’t follow through on a threat to expose my nature, condemning this event to eternal repetition. It goes against the moral part of you too strongly, whatever the short-term needs of the part of you that wants to win the argument. I see all your thoughts as they form, do you truly think you can bluff me?”
Though he tried to suppress it, Harry wondered why the Hat didn’t just go ahead then and stick him in Ravenclaw -
“Indeed, if it were truly that open-and-shut, I would have called it out already. But in actuality there is a great deal we need to discuss… oh, no. Please don’t. For the love of Merlin,
must
you pull this sort of thing on everyone and everything that you meet up to and including items of clothing -”
Defeating the Dark Lord is neither selfish nor short-term. All the parts of my mind are in accord on this: If you don’t answer my questions, I’ll refuse to talk to you, and you won’t be able to do a good and proper Sorting.
“I ought to put you in Slytherin for that!”
But that is
equally
an empty threat. You cannot fulfill your own fundamental values by Sorting me falsely. So let us trade fulfillments of our utility functions.
“You sly little bastard,”
said the Hat, in what Harry recognized as almost exactly the same tone of grudging respect
he
would use in the same situation. “
Fine, let’s get this over with as quickly as possible. But first I want your unconditional promise never to discuss with anyone else the possibility of this sort of blackmail, I am NOT doing this every time.”
Done,
Harry thought.
I promise.
“And don’t meet anyone’s eyes while you’re thinking about this later. Some wizards can read your thoughts if you do. Anyway, I have no idea whether or not you’ve been Obliviated. I’m looking at your thoughts as they form, not reading out your whole memory and analyzing it for inconsistencies in a fraction of second. I’m a hat, not a god. And I cannot and will not tell you about my conversation with the one who became the Dark Lord. I can only
know,
while speaking to you, a statistical summary of what I remember, a weighted average; I
cannot
reveal to you the inner secrets of any other child, just as I will never reveal yours. For the same reason, I can’t speculate on how you got the Dark Lord’s brother wand, since I cannot specifically know about the Dark Lord or any similarities between you. I
can
tell you that there is definitely nothing like a ghost - mind, intelligence, memory, personality, or feelings - in your scar. Otherwise it would be participating in this conversation, being under my brim. And as to the way you get angry sometimes… that was part of what I wanted to talk to you about, Sorting-wise.”
Harry took a moment to absorb all this negative information. Was the Hat being honest, or just trying to present the
shortest
possible convincing answer -
“
We both know that you have no way of checking my honesty and that you’re not actually going to refuse to be Sorted based on the reply I did give you, so stop your pointless fretting and move on.”
Stupid unfair asymmetric telepathy, it wasn’t even letting Harry finish thinking his own -
“When I spoke of your anger, you remembered how Professor McGonagall told you that she sometimes saw something inside you that didn’t seem to come from a loving family. You thought of how Hermione, after you returned from helping Neville, told you that you had seemed ‘scary’.”
Harry gave a mental nod. To himself, he seemed pretty normal - just responding to the situations in which he found himself, that was all. But Professor McGonagall seemed to think that there was more to it than that. And when he thought about it, even he had to admit that…
“That you don’t like yourself when you’re angry. That it is like wielding a sword whose hilt is sharp enough to draw blood from your hand, or looking at the world through a monocle of ice that freezes your eye even as it sharpens your vision.”
Yeah. I guess I have noticed. So what’s up with that?
“I cannot comprehend this matter for you, when you do not understand it yourself. But I do know this: If you go to Ravenclaw or Slytherin, it will strengthen your coldness. If you go to Hufflepuff or Gryffindor, it will strengthen your warmth. THAT is something I care about a great deal, and it was what I wanted to talk to you about this whole time!”
The words dropped into Harry’s thought processes with a shock that stopped him in his tracks. That made it sound like the obvious response was that he shouldn’t go to Ravenclaw. But he
belonged
in Ravenclaw!
Anyone
could see that! He
had
to go to Ravenclaw!
“No, you don’t,”
the Hat said patiently, as if it could remember a statistical summary of
this
part of the conversation having happened a great many previous times.
Hermione’s in Ravenclaw!
Again the sense of patience.
“You can meet her after lessons and work with her then.”
But my plans -
“So replan! Don’t let your life be steered by your reluctance to do a little extra thinking. You
know
that.”
Where would I go, if not Ravenclaw?
“Ahem. ‘Clever kids in Ravenclaw, evil kids in Slytherin, wannabe heroes in Gryffindor, and everyone who does the actual work in Hufflepuff.’ This indicates a certain amount of respect. You are well aware that Conscientiousness is just about as important as raw intelligence in determining life outcomes, you think you will be extremely loyal to your friends if you ever have some, you are not frightened by the expectation that your chosen scientific problems may take decades to solve -”
I’m lazy! I hate work! Hate hard work in all its forms! Clever shortcuts, that’s all I’m about!
“And you would find loyalty and friendship in Hufflepuff, a camaraderie that you have never had before. You would find that you could rely on others, and that would heal something inside you that is broken.”
Again it was a shock.
But what would the Hufflepuffs find in
me
, who never belonged in their House? Acid words, cutting wit, disdain for their inability to keep up with me?
Now it was the Hat’s thoughts that were slow, hesitant.
“I must Sort for the good of all the students in all the Houses… but I think you could learn to be a good Hufflepuff, and not too out of place there. You will be happier in Hufflepuff than in any other house; that is the truth.”
Happiness is not the most important thing in the world to me. I would not become all that I could be, in Hufflepuff. I would sacrifice my potential.
The Hat flinched; Harry could feel it somehow. It was like he had kicked the hat in the balls - in a strongly weighted component of its utility function.
Why are you trying to send me where I do not belong?
The Hat’s thought was almost a whisper.
“I cannot speak of the others to you - but do you think that you are the first potential Dark Lord to pass under my brim? I cannot know the individual cases, but I can know this: Of those who did not intend evil from the very beginning, some of them listened to my warnings, and went to Houses where they would find happiness. And some of them… some of them did not.”
That stopped Harry. But not for long.
And of those who did
not
heed the warning - did they
all
become Dark Lords? Or did some of them achieve greatness for good, as well? Just what are the exact percentages here?
“I cannot give you exact statistics. I cannot know them so I cannot count them. I just know that your chances don’t feel good. They feel
very
not-good.”
But I just wouldn’t do that! Ever!
“I know that I have heard that claim before.”
I am not Dark Lord material!
“Yes, you are. You really,
really
are.”
Why? Just because I once thought it would be cool to have a legion of brainwashed followers chanting ‘Hail the Dark Lord Harry’?
“Amusing, but that was not your first fleeting thought before you substituted something safer, less damaging. No, what you remembered was how you considered lining up all the blood purists and guillotining them. And now you are telling yourself you were not serious, but you were. If you could do it this very moment and no one would ever know, you would. Or what you did this morning to Neville Longbottom, deep inside you
knew
that was wrong but you did it
anyway
because it was
fun
and you had a
good excuse
and you thought the Boy-Who-Lived could
get away
with it -”