Update

I’ve rewritten State Department’s strip cipher – reuse of alphabet strips and key lists using information from the report ‘JCS Ad hoc committee report on cryptographic security of government communications'.

Interesting WWII documentary

Best seen under the influence of hallucinogenic drugs.

Update

I’ve added pics from SRH-368 in WWII Myths - U-boat tankers and ULTRA intelligence.

In the news

1). Articles from Anatoly Klepov on the compromise of Soviet communications in WWII:

"Historical truth" Beria and Suvorov about cryptography and radiolocation performance - Part 3

"Historical truth" Beria and Suvorov about cryptography and radiolocation performance - Part 4

2). War Nerd: China, Vietnam and Naval Nerf Wars

3). Frontline documentary on the NSA and the Snowden affair

4). New information on the Soviet M-105 (AGAT) cipher machine from Cryptomuseum.

Naval Enigma compromise and the spy in the United States Department of the Navy

Signals intelligence and codebreaking played an important role in WWII. British and American codebreakers solved many important Axis crypto systems, such as the German Enigma machine and the Japanese Navy’s code JN25. 

The solution of the German military’s plugboard Enigma was mostly carried out at Bletchley Park with Hut 6 attacking Army and Airforce ‘keys’, while Hut 8 worked on the naval traffic. During the Battle of the Atlantic the German U-boats relied mostly on the Enigma for their communications with U-boat Command, so the solution of these messages was a top priority for the Allied codebreakers.

Unfortunately the fact that the Navy used stricter procedures than the Army and Airforce plus their introduction of a separate 4-rotor Enigma machine in 1942 meant that in the period March 1941- September ‘43 the successes of Bletchley Park were spasmodic and suffered from time lag. Things changed in September since the new 4-rotor ‘Bombes’ built by the Americans were introduced and they could solve the Enigma settings reliably and in a timely manner (at least until the introduction of individual Enigma keys for U-boats in late 1944).

According to several books and articles the Germans never suspected that their codes were being read and they thought that the Enigma was unbreakable. The reality was a slightly more complex than that as can be seen from the numerous security measures they implemented during the war. Would they have taken so many precautions if they considered Enigma unbreakable?

Since the Enigma was used in huge numbers it was accepted that during the war both cipher machines and valid keylists had fallen into Allied hands. Thus current traffic could be compromised but only until new settings were introduced.

Another question is whether the Germans considered the Enigma to be vulnerable to cryptanalysis. After all it wasn’t every day that Enigma machines and keylists would fall into enemy hands. Here the story diverges since the different codebreaking departments in the Army (Inspectorate 7/VI), Navy (B-Dienst), Airforce (Chi Stelle) and Armed Forces High Command (OKW/Chi) had different opinions on the matter. Unfortunately we don’t know the full story of their security investigations as the relevant files have not been studied in detail but it is clear that they constantly researched ways of solving the Enigma.

In this area the Army codebreakers proved more suspicious than their counterparts in other departments and they always worried about the possibility of solution of the machine by the enemy. During the war they not only studied the Enigma but also interrogated Polish personnel regarding their solution of the prewar Enigma, known to the Germans as case ‘Wicher’.

The analysts of the naval cipher security department on the other hand seems to have shown a lack of imagination when it came to researching the Enigma. For some reason they constantly downplayed the possibility of cryptanalytic solution and attributed enemy successes to captured cipher material.

This can be seen from one of their reports found in NARA-RG 457- Entry 9032- box 1279 – NR 3775 ‘German Navy U-boat logs’. In the summer of 1943 the military intelligence service Abwehr learned from a Swiss-American working in the US Department of the Navy that the operational orders to U-boats were decoded by the Allies and all messages read.

According to google translate:

At 10.8. received the following message via KO Switzerland :

‘‘For several months, deciphering German naval codes with regard to operating Uboat commands succeeded. All commands are read.

Additional: Source Swiss-American secretary in high position in the U.S. Navy Department.‘’

It is not clear if this person was an Abwehr agent or someone who revealed this information to Swiss diplomatic or intelligence authorities and they in turn shared this information with the Germans.

The response of the security department was that the continuous reading of their communications by the Allies was out of the question. There was however the possibility of losing cipher material in one of the sunken U-boats. For that reason they instituted a change of the Enigma settings using the keyword ‘Andromeda’.

The emergency key-‘Stichwort’, worked on the following system according to the study ‘Cryptographic History of Work on the German Naval Enigma’, p6:

"Stichwort".

19. The "Stichwort" was a device used when the enemy suspected that the keys had been captured, or wished to protect himself from the danger of this happening. An example will make its nature and method of use clear.

20. All holders of the machine (at the time of this example) held a sealed envelope labelled PERSEUS. An order was sent out "STICHWORTHEFEHL PERSEUS". The holders opened their envelopes and found the word DANZIG inside (the "Kennwort"). They then "applied" DANZIG to the key as follows (1) D = 4 was added to the number of each wheel in the W.O. (2) A, N, Z respectively were added to the three letters of the Ringstellung. (3) I = 9 was added to each letter in the Stecker. This change was applied to all keys until further notice.

21. Suppose for instance the key for the day had been W.O. 275, Rings. BYL, Stecker A/F, B/J, D/X.... . The modified key would then be W.O. 631, Ring. CML, Stecker J/O, K/S, M/G .... .

22. This is a security measure with some "nuisance value" but is clearly not equivalent to a new key. Given that a key has been captured (the assumption on which the Stichwort is based) then - once we have discovered what is happening - there are 8 W.O.'s and 26 sets of Stecker to try; this might certainly be tiresome but does not compare with the 336 W.O.'s and 140 million million stecker on a completely unknown key.

The naval codebreakers would change their minds regarding the theoretical solution of their 4-rotor Enigma in late 1944 when one of their analysts named Hans-Joachim Frowein showed that it could be solved on a ‘crib’ of 25 letters (suspected plaintext in the ciphertext).

 

This report is TICOM I-38 and is available from Ticom Archive.

Codes of the European Economic Community

Hmmm, maybe it’s time to have a look at the finding aid just in case I see something interesting that I can copy.

 

Ok, ‘Cryptographic codes and ciphers: European Economic Community EEC’ this should be interesting. I wonder what’s in those files.

What the….

The German intercept stations in Spain

In the course of WWII the German signal intelligence agencies intercepted radio traffic from several fixed and mobile stations established throughout Europe. Some of these stations were located in neutral countries and they operated clandestinely, so as not to attract attention from the Allies.  Although these stations operated in secrecy the local governments were informed of their existence and had given their tacit approval.

The Spanish government under General Francisco Franco had close ties to Germany, as would be expected considering the support that the Nationalists had received from Germany and Italy during the Spanish Civil War of 1936-39. Without support from Hitler and Mussolini the Nationalists would not have been able to defeat the Republican forces. Yet despite these close ties the position of the Spanish government during WWII was to remain neutral and avoid foreign entanglements.

Even though Spain was neutral the police and the intelligence service cooperated to some extent with the German intelligence services Abwehr and Sicherheitsdienst. In the field of signals intelligence the authorities allowed the establishment of a main radio-intercept station in Madrid and smaller outstations throughout the country. These first of these stations were controlled by the German High Command’s deciphering department – OKW/Chi.

OKW/Chi was not the only German agency with radio stations in Spain. In the course of the war a clandestine naval D/F station was added to the OKW/Chi Seville facility and a Luftwaffe intercept station was established in Barcelona. Also in the latter stages of the war the main station in Madrid added a separate section for the Radio Security Service of the Armed Forces- Funkabwehr.

Update

I have uploaded TICOM report DF-116-J ‘The German intercept station in Madrid’ – 1948. Available from my Scribd and Google Docs accounts.

In the news

Debate on NSA surveillance with Glenn Greenwald, Alexis Ohanian, Michael Hayden and Alan Dershowitz

Glenn Greenwald: how the NSA tampers with US-made internet routers

Articles from Anatoly Klepov on the compromise of Soviet communications in WWII:

"Historical truth" Beria and Suvorov about cryptography and radiolocation performance - Part 1

"Historical truth" Beria and Suvorov about cryptography and radiolocation performance - Part 2

Tanks, tanks, tanks

1). I’m going to write something on the US M4 Sherman tank and whether it was a deathtrap or a war-winner (or somewhere in between).

2). Wait what’s this? Another report on the Best tank of WWII, ehm I mean the Soviet T-34 tank? Hmmm I guess I’ll have to copy it. It should be easy as it’s only 456 pages…

Compromise of the State Department’s strip cipher in 1944

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1,B1,C1,D1) enciphered with substitution tables were available. Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until late 1944) and by reading these messages the Axis powers gained insights into global US policy.

The strip cipher was not a weak system cryptologically, even though it could not offer the security of cipher machines. The success of German and Finnish codebreakers was facilitated in many cases by the poor way that the system was used by the State Department.

M-138-A strip cipher

The M-138-A system consisted of an aluminum frame (or later wooden/plastic) with room for 25 or 30 paper strips. Each strip had a random alphabet. The daily key specified the strips to be inserted and the order that they were to be inserted in. The plaintext was written vertically at the first column by rearranging the strips. Then another column was selected to provide the ciphertext.

 

Each embassy or consulate had 100 alphabet strips, 50 ‘circular’ alphabet strips and 50 ‘specials’. The ‘circulars’ were used for communications between embassies and for messages from Washington to several embassies. The ‘specials’ were used for direct communications between Washington and a specific embassy.

The way the system worked was that each day 30 alphabet strips were chosen out of the available 50 (both for the ‘circulars’ and the ‘specials’). The strips used and the order that they were inserted in the metal frame was the ‘daily key’. The strip system did not have a separate ‘key’ for each day. Instead there were only 40 different rearrangements.

German efforts to solve the US diplomatic strip cipher

Three different agencies worked on the US diplomatic M-138-A strip cipher. The German High Command’s deciphering department – OKW/Chi, the Foreign Ministry’s deciphering deparment Pers Z and the Air Ministry’s Research Department - Reichsluftfahrtministerium Forschungsamt.

At the Forschungsamt some work was done on the strip but apart from the fact that they solved some traffic we don’t know any more details. 

At OKW/Chi an entire team worked on the strip, led by the mathematician Wolfgang Franz and they built a specialized cryptanalytic device called ‘Tower clock’ (Turmuhr). This device was a ‘statistical depth-increaser’ according to US reports.

At  Pers Z they devoted significant resources against the strip cipher. A team of mathematicians, led by Professor Hans Rohrbach made extensive use of IBM/Hollerith punch card equipment in their efforts to solve the alphabet strips and also built a special decoding device called ‘Automaton’.

Proof of OKW/Chi success in 1944

The information given by Wolfgang Franz who was interrogated in 1949 is limited. In his report DF-176 he said in pages 6-9:

‘Especially laborious and difficult work was connected with an American system which, judging by all indications was of great importance. This was the strip cipher system of the American diplomatic service which was subsequently solved in part.’

‘All told, some 28 circuits were solved at the Bureau under my guidance, likewise six numerical keys-some of them only in part.’

A matter of some controversy is the extent of success they had in 1944 against this system.  The head of the mathematical research department of OKW/Chi, Dr Erich Huettenhain said in TICOM I-2 ‘Interrogation of Dr. Huettenhain and Dr. Fricke at Flensburg, 21 May 1945’, p2:

‘Q. What work was done on British and American codes and ciphers?
A. Diplomatic - most of the American strip cipher was read, strip cipher was used by the military as well as by the diplomatic.’

However in TICOM I-145 ‘Report on the US strip system by Reg Rat Dr Huettenhain’ he stated:

‘Only a little of the material received could be read at once. Generally it was back traffic that was read. As, however, the different sets of strips were used at different times by other stations, it was possible, in isolated cases, to read one or the other of the special traffics currently. We are of opinion that of the total material received, at the most one fifth was read, inclusive of back traffic. None was read after the beginning of 1944.’

This seems to be at odds with the version given by the same person in an unpublished manuscript written in 1970 in which he said: 

‘Auf diese Weise wurden von 1942 bis September 1944 insgesamt 22 verschiedene Linien und alle cq-Sprüche mitgelesen’

Translation: In this way, were read by 1942 to September 1944, a total of 22 different links and all cq (call to quarters) messages. (note that cq messages means ‘circulars’)

Were the Germans able to solve the State Department’s high level messages in 1944? The answer is yes.

In the US National Archives, in collection RG 457 ‘Records of the National Security Agency’ - Entry 9032 - boxes 205-213 ‘German decrypts of US diplomatic messages 1944’ one can find many decoded messages from US embassies and consulates around the world. Many have a note on the lower right side identifying the cryptosystem used. The German code for the strip cipher was Am10. This is mentioned in TICOM I-145 which says ‘The American strip system Am10’ and in TICOM DF-176, p7: ‘the Am10-that was the designation of the strip cipher system’.

In these boxes there are a few messages with the tag Am10 sent in 1944 and decoded in that year. They prove that the Germans could solve the strip system even in 1944. Here are four of these messages:

From box 209 – Bern-London

 

From box 210 – Madrid-Algiers

 

From box 209 - Algiers

 

From box 212 – Madrid-Washington

Messages between embassies should have been on the ‘circular’ strips. Messages to or from Washington should have been sent on the ‘special’ strips. From the TICOM reports and the few messages found in boxes 205-213 it is clear that the German codebreakers were able to solve the strip cipher even as late as 1944 and that included both the ‘circular’ messages and at least some of the ‘specials’.

In addition there is in these boxes a list with the code L-1456 vol VIII that according to NARA ‘does not appear to be linked to the other documents’. It is possible that it has some connection to the M-138-A case.

Acknowledgements: I have to thank Randy Rezabek of TICOM Archive for collaborating with me on this research project and covering parts of the cost and also my researcher Mike Constandy of Westmorland Research for going though the boxes and finding needles in a haystack.

Update

I have added a decoded message from the Bern OSS station in Allen Dulles and the compromise of OSS codes in WWII. This confirms the German statements that they could read OSS communications during WWII.

The Japanese FUJI diplomatic cipher 1941-43

In order to protect its diplomatic communications Japan’s Foreign Ministry used several cryptologic systems during WWII. In 1939 the PURPLE cipher machine was introduced for the most important embassies, however not all stations had this equipment so hand systems continued to play an important role in the prewar period and during the war.

One of the main hand systems was the J-19 code, enciphered either with bigram substitution tables or with transposition using a stencil.

Historical overview

The fist Japanese diplomatic system identified by US codebreakers was introduced during WWI and it was a simple bigram code called ‘JA’. There were two code tables, one of vowel-consonant combinations and the other of consonant vowel. Similar systems, some with 4-letter code tables were introduced in the 1920’s.

These unenciphered codes were easy to solve simply by taking advantage of the repetitions of the codegroups of the most commonly used words and phrases. US codebreakers solved these codes and thus learned details of Japan’s foreign policy. During the Washington Naval Conference the codebreakers of Herbert Yardley’s Black Chamber  were able to solve the Japanese code and their success allowed the US diplomats to pressure the Japanese representatives to agree to a battleship ratio of 5-5-3 for USA-UK-Japan. However this success became public knowledge when in 1931 Yardley published ‘The American Black Chamber’, a summary of the codebreaking achievements of his group. The book became an international best seller and especially in Japan it led to the introduction of new, more secure cryptosystems.

In the 1930’s the Japanese Foreign Ministry upgraded the security of its communications by introducing the RED and PURPLE cipher machines and by enciphering their codes mainly with transposition systems.