Intelligence in War: The Value--And Limitations--Of What the Military Can Learn About the Enemy (30 page)

Since the last sequence was the only one on which the army and navy could agree, it was adopted; but since it entailed provoking war with the United States from the outset, it also required the making of a subordinate plan about how to neutralise American naval power in the Pacific. Since early in the century, the Japanese navy had planned to defeat the American Pacific Fleet by drawing it into Japanese home waters, wearing down its strength meanwhile by attritional attacks as it made its long voyage across the ocean; the logic of the strategy was enhanced after 1919 by the acquisition of the Central Pacific barrier formed by the ex-German islands. An early war with the United States, however, demanded a quicker means of reducing American naval power.

Admiral Isoroku Yamamoto, Commander of the Combined Fleet, which included the main carrier force, had been considering the problem since early 1941. In principle he opposed making war on the United States, which he knew well as a former English-language student at Harvard and naval attaché in Washington; he did not believe that Japan’s small industrial base could ever effectively support a war against the United States’ vastly larger economy. His well-known views had made him unpopular both with nationalist politicians and their supporters and within the armed forces; he had been sent to sea in 1939 largely to save him from assassination. The threats were not hollow; in 1936, a group of super-nationalist army officers had killed several moderate politicians, including the finance minister and a former prime minister, occupied central Tokyo and been overcome only after three days of street fighting. Yamamoto undoubtedly had reason on his side, as other naval officers saw. Confronted, however, by the reality of the army-dominated government’s determination to solve Japan’s economic problems by aggressive measures, Yamamoto stifled his objections and proposed an alternative attack strategy. He suggested using the carrier force to destroy the American Pacific Fleet at its moorings in Pearl Harbor, Hawaii, its Central Pacific base.

Yamamoto’s thinking was greatly influenced by the results of the British attack on the Italian battle fleet in the harbour of Taranto on 11 November 1940. Aircraft launched from the carrier
Illustrious
had then sunk three Italian battleships with torpedoes, losing only two out of twenty-one aircraft engaged. While the Japanese naval air arm experimented with adapting torpedoes—the principal Japanese torpedo was faster, of longer range and fitted with a larger warhead than any in use in other navies—to run level in shallow water, Yamamoto and his staff considered how to bring the Combined Fleet undetected to within striking distance of Pearl Harbor. By October 1941 the planners—including Commander Minoru Genda, who would lead a major part of the First Air Fleet in the attack—had outlined a scheme which would take the Combined Fleet from the stormy waters of the northern Kurile Islands, above the isolated American possession of Midway Island and then due south, off frequented shipping lanes, to within 200 miles of Hawaii. The fleet would move in complete radio silence and, if possible, within the leading edge of one of the turbulent weather fronts common in the North Pacific, which impeded visual reconnaissance and interfered with radio transmissions. During October the liner
Taiyo Maru
was sailed down the chosen route without sighting a single ship.

The Japanese and American governments meanwhile continued reasoned diplomatic exchanges. Japan asked for a relaxation of American trade embargoes—a joint army–navy committee estimated in June 1941 that oil reserves were being depleted at a rate a third greater than they were being replaced, a disastrous situation, since it implied inexorably that Japan would run out of oil (stocks plus replacement 33 million barrels, consumption 41 million barrels) in 1942–43.
⁴
In return Japan would undertake to cease her military intrusion into Southeast Asia and eventually leave French Indo-China. The United States demurred, proposing instead a scheme to settle affairs all over mainland Asia, which would have required Japan to withdraw from China as well as Indo-China, and to leave Manchuria also. The Japanese, as was to be expected, rejected the proposal. On 4 December an imperial conference took the decision to go to war against the United States, beginning with an attack on Pearl Harbor, on the 7th. The Combined Fleet was already en route.

AMERICAN PENETRATION OF JAPANESE CIPHERS

The attack on Pearl Harbor on 7 December has given rise to one of the largest conspiracy theories in history. There are many versions, most alleging American foreknowledge, and scarcely one agrees with another. Allegations of culpable incompetence apart, which deserve attention, the two most important theories allege, first, that the British had foreknowledge of Japanese intentions but chose to conceal what they knew from the United States in order to bring America into the war; second, that President Roosevelt knew independently what the Japanese intended but took no preventative action, since he currently sought a pretext to bring his country into the war on Britain’s side. The two theories, in some of their versions, overlap.

The subject is so large that it has stimulated the production of a library of books. Almost the only matter on which they agree is that American cryptanalysts, like their British counterparts at Bletchley, were freely reading Japanese ciphers before December 1941. Exactly what was read and when, what the interpreters made of the decrypts, and how the decrypts influenced the decisions taken by President Roosevelt, his cabinet officers, the chiefs of staff and the commanders in the zone of operations form the substance of the great Pearl Harbor mystery story.

It does not connect with the story of Midway, which began to unroll only six months later. There is, however, this caveat: the cryptanalytic organisation which may have failed before Pearl Harbor was also the organisation that helped to deliver the victory of Midway. It worked in this way.

American cryptanalysis differed from its British equivalent, in organisation, recruitment and ethos. Bletchley was a joint-service civil-military body, in which little distinction of rank was observed, built up on a word-of-mouth basis in the period immediately before the outbreak of war in 1939 and recruited largely among young Oxford and Cambridge dons. Proven mathematical ability was the principal qualification. The atmosphere at Bletchley was creatively amateur, informal and high-spirited; women formed a high proportion of the staff, some in senior positions, and romance flourished. There were many Bletchley marriages.

The American cryptanalytic organisation, by contrast, was sharply divided into naval and military branches, which co-operated uneasily, was highly bureaucratic and almost completely male-dominated. Most of the cryptanalytic personnel were uniformed servicemen and the principal qualification for selection was language skill, particularly in Japanese. Unlike Bletchley, which had a high opinion of itself and cultivated a genial university common room atmosphere, the American intelligence branches were regarded by the rest of the army and navy as backwaters, staffed by officers unsuitable for operational appointments, an opinion of which their members were aware. It is remarkable, in the circumstances, how well they maintained their professional morale. A key indication of the difference between the British and American systems is that, while Bletchley has entered into British national legend and found a popular place in fiction and film, its American equivalents enjoy no such acclaim. Quite wrongly, for what the Americans achieved was equally remarkable, indeed perhaps more so, as the story of Midway indicates.

The origins of the American cryptanalytic service belong, as do those of the British, in the First World War. Major Joseph Mauborgne, head of the army’s cipher research section in 1918, was a cryptographer far ahead of his time: he perceived the concept of the random key—one not retrievable by frequency analysis or, indeed, any mathematical or linguistic logic—and devised the one-time pad, still the only intrinsically unbreakable cipher. He would eventually become a general and the U.S. Army Chief Signal Officer.
⁵
Of the same vintage but of even great importance to the American cryptanalytic effort was a civilian, William Friedman (who coined the term “cryptanalysis”). The son of a Russian Jewish immigrant, who entered the United States at the age of one, Friedman resembled in character both Dilly Knox and Alan Turing. As eccentric as either, he displayed a mathematical ability almost equivalent to Turing’s but unfortunately also Knox’s psychological fragility. He had suicidal tendencies and just before the outbreak of the Pacific War suffered a nervous collapse brought on by overwork.
⁶

Yet Friedman was largely responsible for the most important of America’s cryptanalytic successes, the breaking of Purple. In October 1940, the army and navy agreed on a division of labour, not in any spirit of fraternal co-operation but because each lacked the numbers to do much more than concentrate on a single task; in 1938 Friedman had a staff of only eight in the Secret Intelligence Service (SIS), in December 1940 the navy in Washington only thirty-six in its equivalent OP-20-G. There were other personnel at outstations in both the continental United States and the Pacific but most were intercept operators and technicians.
⁷
The arrangement was that the army would work on foreign diplomatic intercepts on even days of the month, the navy on odd. The navy was meanwhile, naturally, working on Japanese naval intercepts; the army was not particularly interested in foreign army intercepts, since they were too faint to yield text.

The Americans, few though they were in number, had had considerable success in breaking into both Japanese naval and diplomatic traffic in the 1930s, assisted by a succession of night-time burglaries of the Japanese consulate in New York. By 1933 the naval cryptanalysts had solved the main Japanese naval Blue Code, a book code with a cipher additive. When it was replaced in June 1939 by JN-25 (Japanese Naval Code 25, as the Americans denoted it), another book code with a more complex additive, the Americans took time to recover from the setback, but by December 1940, with the help of recently acquired IBM card-sorting machines, they had reconstructed the system of additives, the first thousand groups of the code and two of the keys used to work the system.

They anticipated cracking the system completely in 1941. By then, however, all spare cryptanalytic manpower in Washington had been diverted to a new task: decrypting the Japanese diplomatic traffic enciphered on a new ciphering machine, known to the Americans as Purple. The Purple machine (Type 97 to the Japanese) was designed to achieve the same effect as Enigma—the automatic production of an almost infinitely variable cipher—but differed from it in construction. It was less mechanical, having no rotors, but instead a set of telephonic switches, connected to two typewriters. The first was used to input the text, the second to print out the encipherment for transmission. In between, the switches moved the incoming electrical current to achieve alphabetic substitutions. Because Japanese is a syllabic, not alphabetic, language, however, all texts had first to be written in an alphabetic equivalent; and for an inexplicable reason, equivalent to the Germans’ double-encipherment of the operator’s chosen indicator at the beginning of a transmission, the Purple machine’s switches enciphered vowels and consonants separately; the number of vowel substitutions was considerably smaller than that of consonants, and once that was recognised, a way into Purple was found.
⁸

The breaking of Purple—its product was known as Magic, the equivalent of British Ultra—would eventually yield huge intelligence advantages to the Americans, principally through the decipherment of the messages sent by the Japanese ambassador in Berlin, Baron Oshima, to Tokyo throughout the war, which revealed intimate details of Hitler’s capabilities and intentions. At the critical moment before Japan’s surprise attack on America in December 1941, however, Purple revealed little, while JN-25, the relevant Japanese naval code, was not useful for two reasons: the first was that the Imperial Navy attempted to observe, as far as was possible, radio silence in the period preparatory to the descent on Pearl Harbor; the second was that OP-20-G lacked the staff necessary to deal with the volume of intercepted traffic. The U.S. Navy’s Historical Center has now compiled a list of significant messages intercepted—but not decrypted—in the weeks before Pearl Harbor that bear on the issue of foreknowledge. Some, if read in real time, must have alerted a wary admiral to the danger threatening his fleet; in practice, the messages were bundled up and not decrypted and translated until September 1945, a month after the Japanese war had ended.
⁹

Pearl Harbor devastated not only the U.S. Navy’s Pacific Fleet but also OP-20-G’s Pacific outstations. While its Hawaiian station (HYPO) continued to operate, its outpost in the Philippines (CAST) first withdrew into a tunnel in Corregidor, then was evacuated to Australia. American naval intelligence in the Pacific was thus reduced to HYPO, the joint station in Australia and a branch in the British Combined Bureau in Ceylon. The British had had some inter-war success in attacking Japanese naval codes but, in the climate then current, with both the American services and the public adamant about revenge against the Japanese, the prime responsibility for breaking back into Japanese naval traffic lay with OP-20-G. The Commander-in-Chief Pacific, Admiral Chester Nimitz, was on tenterhooks against a renewed Japanese naval offensive; he equally hoped to profit, perhaps from a Japanese mistake but preferably an intelligence coup, and inflict a defeat on the enemy.

THE COURSE OF JAPANESE CONQUEST

There was little sign of stemming the tide of Japanese conquest in the last weeks of December 1941 and the first three months of 1942. Its progress seemed inexorable. Attacks launched out of Thailand quickly collapsed the British defences of northern Malaya. The new battleship
Prince of Wales
and the battlecruiser
Repulse,
operating without air cover in an effort to intercept Japanese coastwise landings, were sunk by bombers launched from French Indo-China on 10 December. Singapore, the great trading city at the tip of the peninsula, was ignominiously surrendered to an inferior Japanese force on 15 February. The British island of Hong Kong and the American islands of Wake and Guam, all indefensible, were captured on 25 December, 23 December and 10 December; the garrisons of Hong Kong and Wake put up a heroic, hopeless resistance. Burma was invaded in January and conquered by May. The invasion of the Dutch East Indies began in January also and was completed by March; there were several attempts by a combined Australian-British-Dutch-American fleet (ABDA) to check the Japanese amphibious campaign, culminating in the Battle of the Java Sea, on 27 February; a miscellaneous collection of Allied ships, bravely commanded by the Dutch Admiral Karel Doorman, but outnumbered and unable to intercommunicate, was overwhelmed. Meanwhile, General Douglas MacArthur was conducting the defence of the Philippines, the attack on which began on 8 December with a devastatingly successful Japanese air raid. The American and Filipino defenders were forced to withdraw as soon as the Japanese landings began, but in January succeeded in establishing a fortified line across the Bataan peninsula. There for the next three months they sustained a heroic defence, inflicting on the Japanese the only setbacks suffered on land during their great campaign of conquest. Eventually, however, shortage of food and supplies forced a surrender in April, and in May of the offshore island of Corregidor.