New book on the Italian codebreakers of WWII

The book Ultra» la fine di un mito. La guerra dei codici tra gli inglesi e le marine italiane. 1934-1945 -‘Ultra’ the end of a myth. The war of the codes between the British and Italian navies. 1934-1945’, by Enrico Cernuschi has been published recently.

Cernuschi is an authority on the Italian codebreakers of WWII and he has written several books on the Italian Navy. He has also co-authored with Vincent O'Hara Dark Navy: The Italian Regia Marina and the Armistice of 8 September 1943 and the Naval War College Review article ‘The Other Ultra: Signal Intelligence and the Battle to Supply Rommel's Attack toward Suez’.

His new book presents the code war in the Mediterranean in a different light and points out mistakes and exaggerations in the previous accounts such as the official histories ‘British Intelligence in the Second World War’. There is information on the Italian codebreakers and their successes with Royal Navy codes as well as French, Greek, Yugoslav and USA communications. This played a role in the convoy battles as the Italians could reroute their ships once they had been sighted by enemy forces and were in danger of attack. On the other side of the hill the author points out mistakes in the standard accounts of the compromise of the German Enigma and Italian Hagelin C-38 cipher machines and their importance for the N.Africa campaign.

For this book the author has researched the British national archives and the ‘Archivio dell'Ufficio Storico della Marina Militare’ in Rome.  According to him the codebreaking story in the Med must be rewritten.

The online magazine analisi difesa has interviewed the author about his latest book.

To err is human

I’ve rewritten The British War Office Cypher using information from TICOM report I-51 and the War Diary of Inspectorate 7/VI. I had written that the Germans were able to read this system from early 1941 till summer ’42 which was not correct. It seems that they first solved messages in summer 1941 and this was back traffic from late 1940 and early 1941. Then in the period September ’41 to January 1942 they read current traffic from the Middle East, especially during the Crusader offensive.

I have also corrected this in other essays mentioning the WOC.

The War Nerd on the Crimea crisis

Everything you know about the Crimea is wrong (-er)

Interview with HuffPost Live (segment starts at 22:50)

All hail the War Nerd!

In the news

NSA can collect and store an entire country’s phone calls

NSA targets Chinese telecommunications company Huawei

NSA targets system administrators

New books on Soviet cryptology in WWII

The role of signals intelligence and codebreaking in WWII has received a lot of attention from historians, especially after the release of new information in the 1970’s (ULTRA story). Information on the successes of the Axis codebreakers is not as easy to find but at least we know the main cases (Fellers code, Naval Cypher No3, etc) fairly well. However the work and successes of the Soviet codebreakers are still shrouded in secrecy.

During Soviet times there was no direct acknowledgment of cryptanalysis of Axis codes.  Soviet histories either glossed over that part of the war or referred to ‘radio-electronic combat’ which was limited to D/F, traffic analysis and jamming. After the breakup of the SU some new books and articles have been published that have a lot of information on the prewar organization and successes of the Soviet codebreaking agencies but not as much regarding their wartime efforts. Recently two new books have been published with more information on Soviet WWII codebreaking: ‘History of cryptology’ by Grebennkov Vadim Viktorovich and ‘The cryptographic front’ by Butirsky, Larin and Shankin.

I have read a chapter from the first book (with the help of google translate) that the author was kind enough to send me. There is certainly new information presented such as the solution of the codes of Germany’s allies in the Eastern Front. As for the second book, according to the table of contents it has the following chapters:

Preface
Chapter 1. History of manual encryption devices
Chapter 2. The twentieth century - the era of rotor cipher machines
Chapter 3. History of telephone speech coders
Chapter 4. Soviet cryptographic service
Chapter 5. Cryptographic WWII. Soviet decryption service
Chapter 6. Exploration and guerrillas
Chapter 7. Worked as a counterintelligence
Chapter 8. The agent radio
Chapter 9. Steganographic transmission means covert messages
Chapter 10. Postwar period
Conclusion
References and Resources

Apparently there is information on the decryption of foreign codes but i don’t have the book and I can’t read Russian. Perhaps a Russian reader of this site can read this book and give an overview.

Some information from the same authors is presented in a series of essays published in site journal.ib-bank.ru :

Pioneers of domestic machine cryptography

Transformation in the fundamental science of cryptology

In the service of the motherland, mathematics and cryptography

Thoughts on a giant of radio engineering

These essays have very interesting information on Soviet cryptology and some of the early cipher machines in use. In ‘Transformation in the fundamental science of cryptology’ i noted these very interesting statements (through google translate) regarding codebreaking:

Meanwhile, this statement is fundamentally wrong. Many results have been achieved "clean" methods, using mathematics and computer science.

…………………………………………….

Meanwhile, many American ciphers were then opened by the combined use of cryptanalytic and engineering methods and powerful computing. However, for the opening of these ciphers such a way - the only possibility.

These statements refer to the Cryptologia article ‘Soviet comint in the Cold war’ by David Kahn, specifically the part in page 7: ‘First, the Soviet Union seems to have gained most of its communications intelligence, not from cryptanalysis, but from bugs and traitors’ and ‘High-level American cryptosystems appear not to have been solved by analysis, but some were read because traitors had sold them to the Soviet Union’.

Perhaps one day when the story of codebreaking during the Cold War is fully revealed it will be interesting to compare the NSA’s efforts with their Soviet counterparts. 

The US AN/GSQ-1 (SIGJIP) speech scrambler

During WWII all the major participants used a number of speech privacy systems in order to protect their confidential voice communications sent over landline or radio-telephone links.

The US authorities used up to mid 1943 the Bell Labs A-3 speech scrambler, a device that utilized band-splitting and inversion. The A-3 was not secure and in the period 1941-44 the Germans were able to decode the conversations in real-time. American cryptologists knew that the A-3 was insecure and they developed the SIGSALY, a device that was a quantum leap in terms of security. However SIGSALY weighed over 50 tons and thus could only be used at prepared sites.

For field communications a speech privacy system was urgently needed but at the time it was not possible to combine a high level of crypto-security with a small and compact design. According to the postwar history of the US Army’s SSA-Signal Security Agency, in page 45, a portable speech scrambler called AN/GSQ-1 - SIGJIP was developed by Bell Telephone Laboratories in collaboration with the Signal Corps and in 1943 the responsibility for its further development was transferred to the SSA. The device was tested and its security was found wanting, however the need for such a device was so great that in 1944 several units were sent to the European, Mediterranean and South Pacific theaters. 

According to TICOM reports the Germans were able to retrieve such a device from a downed Mustang fighter and although they solved the cryptologic system, in practice it was thought that finding the ‘key’ used for each mission would be difficult due to the time element.

From TICOM I-31 ‘Detailed interrogations of Dr. Hüttenhain, formerly head of research section of OKW/Chi, at Flensburg on 18-21 June 1945’

From TICOM I-58 ‘Interrogation of Dr. Otto Buggisch of OKW/Chi’

More information on the AN/GSQ-1 is available from the book ‘Information security: An elusive goal’ by George F. Jelen, pages II-17 and II-18. According to the author the AN/GSQ-1 divided the speech signal into 37 -1/2 millisecond segments and then rearranged them based on the ‘key’. In 1944-45 they were installed on P-51 reconnaissance planes operating in Europe and one of the first in service was lost over Germany. The AN/GSQ-1 was not a success and was retired after a brief period of time not only because it offered limited security but also because P-51 pilots preferred to remove the SIGJIP and in its place install a tail-warning radar.

For those who want more info on the AN/GSQ-1- SIGJIP in the US National Archives and Records Administration - RG 457 - Entry 9032 - box 792- there is the file NR 2228 CBLL24 6144A 19450927 ‘PERFORMANCE SPEECH EQUIPMENT AN/GSQ-1 AN/GSQ-1A SIGJIP-SIGMAR’.

United States cryptologic security failures in WWII

Signals intelligence and codebreaking played an important role in WWII. British and American codebreakers solved many important Axis crypto systems, such as the German Enigma machine and the Japanese Navy’s code JN25. 

Historians have not only acknowledged these Allied successes but they’ve probably exaggerated their importance in the actual campaigns of the war.

Unfortunately the work of the Axis codebreakers hasn’t received similar attention. As I’ve mentioned in my piece Acknowledging failures of crypto security all the participants suffered setbacks from weak/compromised codes and they all had some successes with enemy systems. 

Britain, the Soviet Union and the United States did not have impenetrable codes. In the course of WWII all three suffered setbacks from their compromised communications.

After having dealt with the British side let’s have a look at the Americans and their worst failures. 

First it’s time for a short history lesson on the organizations responsible for making and breaking codes.

Herbert Yardley, MI-8 and the American Black chamber

The first dedicated codebreaking unit of the US military was organized during WWI by Herbert Osborne Yardley. Yardley had worked as a telegrapher and then as a State Department code clerk. During WWI he demonstrated the insecurity of US diplomatic codes by solving, on his own, a message sent from Colonel House to President Woodrow Wilson. This attracted the attention of his superiors and Colonel Ralph H. Van Deman, head of the Military Intelligence Division, made him a first lieutenant and assigned him head of the MI-8 department, responsible for codes and ciphers.

The MI-8 unit solved several foreign codes and their success led the War Department and the State Department to jointly fund Yardley’s activities in the postwar era. The codebreaking department was moved to New York and called the ‘Black chamber’. Their main effort in the 1920’s was against Japanese diplomatic codes and in this area they were able to prove their worth. Yardley’s group not only solved regular Japanese diplomatic traffic but scored a great victory during the Washington Naval Conference by discovering Japan’s minimum acceptable battleship requirements. This allowed the US diplomats to get the Japanese representatives to agree to a battleship ratio of 5-5-3 for USA-UK-Japan.

The secrets of room 6527

Interesting files are available from site room6527.com

The project is described as:

The FBI has been hiding sensitive records of American eavesdropping operations from parliamentary scrutiny for decades. FBI director J. Edgar Hoover (right) gave orders in 1948 for tricky political papers to be stored away in Room 6527 – known as the Confidential File Room - at its Washington headquarters. The records did not show up in any index so that the FBI would be able to deny any knowledge of the relevant documents should a parliamentary control commission ever start to ask questions.
Along with records of US eavesdropping on friendly states, Hoover also stashed away documents about Eastern Block spies or reports about the unusual sexual practices of senior Communist officials and politicians. There were so many documents that they began to threaten the vast official building’s structural mechanics. An internal FBI memo from September 1961 notes that secret papers had to be immediately transferred to other rooms due to the weight of Room 6527’s 26 filing cabinets. Thanks to a Freedom of Information Act request, the SonntagsZeitung and Le Matin Dimanche have gained access to these historic and previously unpublished intercept records.

Regarding the available files I had a quick look and the pdf ‘diplomatic code Greek’, pages 5-23 has a summary of AFSA (predecessor of the NSA) efforts versus the codes of Eastern European countries in the 1950’s. Very interesting stuff!

Update

I added pics from the report E-Bericht 7/44 of FNAST 9 in Swedish Army codes and Aussenstelle Halden.

Update

I added pics in German exploitation of Soviet radio-facsimile traffic

Recommended reading on the Eastern Front

Every year countless books dealing with WWII are published worldwide. Many cover the fighting in the East between Nazi Germany and the Soviet Union. This conflict was the largest land campaign of the war and claimed the lives of millions of people. There is no shortage of great battles for authors to write about such as the battles of Moscow, Stalingrad, Kursk etc

However with so many books available there is the problem of quantity vs quality. How many books have accurate information from both sides? How many present new information?

Unfortunately many books that I’ve read have serious mistakes because they rely mostly on other books for their information and not on the relevant archives. This is understandable since researching the archives is very costly both in terms of time and money.

What i look for in a book is clearheaded analysis and lots of statistical data from both sides. Based on this what are the books that I can recommend?

First let’s have a look at the ones that I found lacking.

Two of the most popular Eastern Front histories are ‘When Titans Clashed: How the Red Army Stopped Hitler’ by David Glantz and Jonathan House and ‘Russia's War: A History of the Soviet Effort: 1941-1945’ by Richard Overy. Having read them I can say that they have serious mistakes since they rely on Soviet sources for information on German strength and loss statistics. This leads to exaggerations that could have been avoided had the authors used the reports available from the German archives.

In addition there are other mistakes scattered throughout both books. For example Overy writes in chapter ‘The Citadel: Kursk 1943’ that the T-34 got a 3-man turret in 1943. He’s off by a year. This might seem like a small mistake but it undermines his argument that the Soviet forces won the battle not through numerical superiority but because they upgraded their equipment and tactics.

‘When Titans clashed’ is guilty of perpetuating several WWII myths. For example the chapter ‘An army in disarray, 1937-1939’ exaggerates the effects of the purges on the Red Army.  Despite its flaws ‘Russia’s war’ points out in chapter ‘The darkness descends’ that during the period 1936-38 41.218 officers were dismissed, not executed (or even all arrested). By May 1940 11.596 officers had been reinstated. The book says ‘Of the 179.000 officers employed in 1938 only 3.7 per cent were still formally discharged by 1940’. It is true that the higher ranks suffered disproportionately but many of these officers (such as Marshall Tukhachevsky) were lackeys of the regime and had not gained their position by merit. In chapter ‘The Red Army’ the T-26 tank is called ‘aging’ and ‘obsolete’ even though its operational characteristics were similar or superior to the German tank types Pz I, Pz II, Pz 35, Pz 38 and Pz III. Why was this tank ‘obsolete’? Its main problem was the 2-man turret and by that standard the mythic T-34 was also ‘obsolete’ since it only got a 3-man turret in 1944.

Having said that let’s take a look at some very interesting books: