Spycatcher (30 page)
Using these common groups Gardner checked back on previous Russian radio traffic, and realized that there were duplications across some channels, indicating that the same onetime pads had been used. Slowly he "matched" the traffic which had been enciphered using the same pads, and began to try to break it. At first no one would believe him when he claimed to have broken into the Russian ciphers, and he was taken seriously only when he got a major breakthrough in the Washington-to- Moscow Ambassadorial channel. He decrypted the English phrase "Defense does not win wars!" which was a "Spell/Endspell" sequence. Gardner recognized it as a book on defense strategy published in the USA just before the date the message was sent. At this point, the Armed Forces Security Agency shared the secret with the British, who at that time were the world leaders in cryptanalysis, and together they began a joint effort to break the traffic, which lasted forty years.
Operation BRIDE (as it was first known) but later DRUG and VENONA, as it was known in Britain, made painfully slow progress. Finding matches among the mass of traffic available took time enough. But even then there was no certainty the messages on each side of the match could be broken. The codebook was incomplete, so the codebreakers used "collateral" intelligence. If, for instance, they found a match between the Washington-to-Moscow KGB channel and the New York-to-Moscow trade channel, it was possible to attack the trade channel by using "collateral," information gathered from shipping manifests, cargo records, departure and arrival times, tide tables, and so forth, for the date of the message. This information enabled the codebreakers to make estimates of what might be in the trade traffic. Once breaks were made in one side of a match, it provided more groups for the codebook, and helped make inroads on the other side.
The British and Americans developed a key device for expanding the VENONA breaks. It was called a "window index." Every time a word or phrase was broken out, it was indexed to everywhere else it appeared in the matched traffic. The British began to index these decrypts in a more advanced way. They placed two unsolved groups on each side of the decrypted word or phrase and after a period of time these window indexes led to repetitions, where different words which had been broken out were followed by the same unsolved group. The repetition often gave enough collateral to begin a successful attack on the group, thus widening the window indexes. Another technique was "dragging." Where a "Spell/Endspell" sequence or name came up, and the cryptanalysts did not know what the missing letters of the spelled sequence were, the groups were dragged, using a computer, across the rest of the channels, and out would come a list of all the repeats. Then the cryptanalysts would set to work on the reverse side of the repeat matches, and hope to attack the "Spell/Endspell" sequence that way.
It was an imperfect art, often moving forward only a word or two a month, and then suddenly spilling forward, like the time the Americans found the complete text of a recorded speech in the Washington Ambassadorial channel. Often terrible new difficulties were encountered: onetime pads were used in unorthodox ways, up and down, or folded, which made the process of finding matches infinitely more problematic. There were difficulties, too, with the codebooks.
Sometimes they changed, and whereas the Ambassadorial, GRU, and trade channels used a straightforward alphabetically listed codebook, rather like a dictionary, so that the codebreakers could guess from the group where in the codebook it appeared, the KGB used a special multivolume random codebook which made decrypting matched KGB channels a mindbending task. The effort involved in VENONA was enormous. For years both GCHQ and NSA and MI5 employed teams of researchers scouring the world searching for "collateral"; but despite the effort less than 1 percent of the 200,000 messages we held were ever broken into, and many of those were broken only to the extent of a few words.
But the effect of the VENONA material on British and American intelligence was immense, not just in terms of the counterintelligence received, but in terms of the effect it had on shaping attitudes in the secret world. By the late 1940s enough progress was made in the New York/Moscow and Washington/Moscow KGB channels to reveal the extent of massive Russian espionage activity in the USA throughout and immediately after the war. More than 1200 cryptonyms littered the traffic, which, because they were frequently part of "Spell/Endspell" sequences, were often the easiest things to isolate in the traffic, even if they could not be broken. Of those 1200, more than 800 were assessed as recruited Soviet agents. It is probable that the majority of these were the low-level contacts which are the staple currency of all intelligence networks. But some were of major importance. Fourteen agents appeared to be operating in or close to the OSS (the wartime forerunner of the CIA), five agents had access, to one degree or another, to the White House, including one who, according to the traffic, traveled in Ambassador Averill Harriman's private airplane back from Moscow to the USA. Most damaging of all, the Russians had a chain of agents inside the American atomic weapons development program, and another with access to almost every document of importance which passed between the British and U.S. governments in 1945, including private telegrams sent by Churchill to Presidents Roosevelt and Truman.
Using leads in the decrypted traffic, some of these cases were solved. Maclean was identified as one of the sources of the Churchill telegrams, and many others besides; Klaus Fuchs and the Rosenbergs were unmasked as some of the nuclear spies; while comparison of geographical clues in the decrypts with the movements of Alger Hiss, a senior U.S. State Department official, over a lengthy period made him the best suspect as the agent on Harriman's plane. But despite frenzied counterintelligence and cryptanalytical effort, most of the cryptonyms remain today unidentified.
In Britain the situation was equally grim, but with one major difference. Whereas the Americans had all the Soviet radio traffic passing to and from the USA during and after the war, in Britain Churchill ordered all anti-Soviet intelligence work to cease during the wartime alliance, and GCHQ did not begin taking the traffic again until the very end of the war. Consequently there was far less traffic, and only one break was made into it, for the week September 15 to September 22, 1945, in the Moscow-to-London KGB channel.
There was a series of messages sent to a KGB officer in the London Embassy, Boris Krotov, who specialized in running high-grade agents. The messages came at a time of some crisis for the Russian intelligence services in the West. A young GRU cipher clerk in the Russian Embassy in Canada, Igor Gouzenko, had just defected, taking a mass of material incriminating spies in Canada and the USA, and in Britain a nuclear spy, Alan Nunn May. Most of the messages to Krotov from Moscow Center concerned instructions on how to handle the various agents under his care. Eight cryptonyms were mentioned in all, three of which were referred to as the "valuable ARGENTURA [spy ring] of Stanley, Hicks, and Johnson," two who were routinely referred to together as David and Rosa, and three others. By the end of the week's traffic all contact with the eight spies had been put on ice, and reduced to meetings, except in special circumstances, of once a month.
When I was indoctrinated into VENONA, I remember my first sight of the GCHQ copies of the Moscow-to-London KGB channel. Every time GCHQ broke a few more words in a message, they circulated to the very few users drop copies of the new decrypt. The copies were stamped TOP SECRET UMBRA VENONA, and listed the addresser and addressee, the date and time of the message, the channel and direction (for instance, KGB Moscow/London), and the message priority (whether it was routine or urgent). Underneath would be something like this:
TEXT OF MESSAGE
YOUR COMMUNICATION OF 74689 AND 02985 47199 67789 88005 61971
CONCERNING SPELL H I C K S ENDSPELL 55557 81045 10835 68971 71129
EXTREME CAUTION AT PRESENT TIME 56690 12748 92640 00471 SPELL S T A N L E Y ENDSPELL 37106 72885 MONTHLY UNTIL FURTHER NOTICE. SIGNATURE OF MESSAGE
(This is not a verbatim decrypt; merely a very close approximation to the kind of challenge we were faced with.)
VENONA was the most terrible secret of all, it was incomplete. It was obvious from the decrypts that each of the eight cryptonyms was an important spy, both from the care the Russians were taking to protect them all in September 1945, and because we knew that Krotov specialized in that type of agent. But there was precious little evidence from the traffic which could help us identify them. GCHQ circulated only translations which they had verified, and included the verbatim unsolved groups where they occurred, but they often attached to the copy a separate page of notes giving possible translations of the odd group, which had not yet been verified. Often a message would be repeated several times, as more groups were got out, and it was re-circulated.
Stanley, we were sure, must be Philby. Golitsin had heard the code name Stanley, and associated it with KGB operations in the Middle East, but there was no proof of this in the traffic. Hicks, therefore, was almost certainly Burgess because of the reference to the ARGENTURA, and because of a veiled reference to Hicks' temperament. Johnson was probably Blunt, although again there was no proof of it in the traffic. But the identity of the five other spies remained a mystery. Maclean was obviously not one of these, since he was in Washington in September 1945. The consequences for the Mitchell investigation were obvious. Any one of the five unidentified cryptonyms could be the spy inside MI5. I remember wondering, as I read the tantalizing decrypts, how on earth anyone at the top of MI5 had slept at night in the dozen years since they were first decrypted.
Perhaps the most extraordinary thing in the whole VENONA story was the fact that it was closed down on both sides of the Atlantic in 1954.
After the initial surge of activity in the late 1940s and early 1950s, and the rash of prosecutions which followed, cryptanalytical progress slowed to a virtual halt. Hand matching had reached the limitations of the human brain, and computers were not then powerful enough to take the program much further. There was another reason too; in 1948 the Russians began to alter their code procedures worldwide, removing all duplicated pads. The last casualty of this was the Australian VENONA operation, which had been making so much progress that the British and Americans were virtually reading the Russian ciphers continuously as the messages were produced. The Australians were never told at the time but were brought into it some years later, although when the extent of Soviet espionage penetration, especially of the Department of External Affairs, became apparent, they were provided with the intelligence in bowdlerized form, and it led to the establishment of ASIO (Australian Security Intelligence Organization) with MI5's help.
The reason for the change in Soviet codes became apparent in the early 1950s. The secret of the break had been leaked to the Russians by a young Armed Services Security Agency clerk, William Weisband. In fact, Weisband did not know the extent of the Russian mistake and it was only when Philby was indoctrinated in 1949 that they knew the breadth of their disaster, although other people, such as Roger Hollis, were indoctrinated in 1948, when the match suddenly ceased in Australia after he returned from organizing the setting up of ASIO. Although the duplicate onetime pads were withdrawn, the Russians could do nothing to prevent the continuing work on the traffic they had already sent up until 1948. But thanks to Philby's posting to Washington in 1949, they were able to monitor the precise progress that was being made. Once the Russians knew the extent of the VENONA leak, and the technical difficulties of finding more matches multiplied, it was only a matter of time before priorities moved on. In 1954 most of the work was closed down.
Years later, I arranged for Meredith Gardner to visit Britain to help us on the British VENONA. He was a quiet, scholarly man, entirely unaware of the awe in which he was held by other cryptanalysts. He used to tell me how he worked on the matches in his office, and of how a young pipe-smoking Englishman named Philby used to regularly visit him and peer over his shoulder and admire the progress he was making.
Gardner was rather a sad figure by the late 1960s. He felt very keenly that the cryptanalytical break he had made possible was a thing of mathematical beauty, and he was depressed at the use to which it had been put.
"I never wanted it to get anyone into trouble," he used to say. He was appalled at the fact that his discovery had led, almost inevitably, to the electric chair, and felt (as I did) that the Rosenbergs, while guilty, ought to have been given clemency. In Gardner's mind, VENONA was almost an art form, and he did not want it sullied by crude McCarthyism. But the codebreak had a fundamental effect on Cold War attitudes among those few indoctrinated officers inside the British and American intelligence services. It became the wellspring for the new emphasis on counterespionage investigation which increasingly permeated Western intelligence in the decades after the first break was made.
More directly, it showed the worldwide scale of the Soviet espionage
attack, at a time when the Western political leadership was apparently pursuing a policy of alliance and extending the hand of friendship. In the British traffic, for instance, most of the KGB channel during that September week was taken up with messages from Moscow detailing arrangements for the return of Allied prisoners to the Soviet authorities, groups like the Cossacks and others who had fought against the Soviet Union. Many of the messages were just long lists of names and instructions that they should be apprehended as soon as possible.
By the time I read the messages they were all long since dead, but at the time many intelligence officers must have been struck by the sense that peace had not come in 1945; a German concentration camp had merely been exchanged for a Soviet Gulag.
In 1959, a new discovery was made which resuscitated VENONA again. GCHQ discovered that the Swedish Signals Intelligence Service had taken and stored a considerable amount of new wartime traffic, including some GRU radio messages sent to and from London during the early years of the war. GCHQ persuaded the Swedes to relinquish their neutrality, and pass the material over for analysis. The discovery of the Swedish HASP material was one of the main reasons for Arthur's return to D1. He was one of the few officers inside MI5 with direct experience of VENONA, having worked intimately with it during the Fuchs and Maclean investigations.