Sunday, July 29, 2018

TICOM DF-174A


The report has information on the Enigma cipher machine, the SG 39 cipher machine and the Enigma modification Lückenfüllerwalze.

Thursday, July 5, 2018

The Higgs memorandum - Compromise of State Department communications by the Finnish codebreakers in WWII

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1, B1, C1, D1) enciphered with substitution tables were available.

Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until mid/late 1944) and by reading these messages the Axis powers gained insights into global US policy.

Germans, Finns and Japanese cooperated on the solution of the strip cipher. In 1941 the Japanese gave to the Germans alphabet strips and numerical keys that they had copied from a US consulate in 1939 and these were passed on by the Germans to their Finnish allies in 1942. Then in 1943 the Finns started sharing their results with Japan. 

Finnish solution of State Department cryptosystems

During WWII the Finnish signal intelligence service worked mostly on Soviet military and NKVD cryptosystems however they did have a small diplomatic section located in Mikkeli. This department had about 38 analysts, with the majority working on US codes.
Head of the department was Mary Grashorn. Other important people were Pentti Aalto (effective head of the US section) and the experts on the M-138 strip cipher Karl Erik Henriksson and Kalevi Loimaranta.

Their main wartime success was the solution of the State Department’s M-138-A cipher. The solution of this high level system gave them access to important diplomatic messages from US embassies in Europe and around the world. 


Operation Stella Polaris

In September 1944 Finland signed an armistice with the Soviet Union. The people in charge of the Finnish signal intelligence service anticipated this move and fearing a Soviet takeover of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris (Polar Star).

In late September roughly 700 people, comprising members of the intelligence services and their families were transported by ship to Sweden. The Finns had come to an agreement with the Swedish intelligence service that their people would be allowed to stay and in return the Swedes would get the Finnish crypto archives and their radio equipment. At the same time colonel Hallamaa, head of the signals intelligence service, gathered funds for the Stella Polaris group by selling the solved codes in the Finnish archives to the Americans, British and Japanese. 

The Stella Polaris operation was dependent on secrecy. However the open market for Soviet codes made the Swedish government uneasy. In the end most of the Finnish personnel chose to return to Finland, since the feared Soviet takeover did not materialize. 

The Higgs memorandum

In September 1944 colonel Hallamaa met with L. Randolph Higgs, an official of the US embassy in Sweden and told him about their successes with US diplomatic codes and ciphers.

This information was summarized in a report prepared by Higgs, dated 30 September 1944.

The report can be found in the US National Archives - collection RG 84 ‘Records of the Foreign Service Posts of the Department of State’ - ‘US Legation/Embassy Stockholm, Sweden’ - ‘Top Secret General Records File: 1944’.







Higgs met with colonel Hallamaa on September 29 and the OSS officials Tikander and Cole were also present during their discussion.

Hallamaa stated that he was an administrator, not a cryptanalyst and about 10-12 of his men worked on US diplomatic codes.

His unit had solved the US codes Gray, Brown, M-138-A strip cipher and enciphered codebooks (probably the A1, B1, C1).

The high level M-138-A system had been solved mostly by taking advantage of operator mistakes such as sending strip cipher information on other systems that had already been broken or sending the same message in different strips one of which had been broken.

The strip cipher was considered a strong encryption system and had been adopted by the Finns for some of their traffic.

Important diplomatic messages from the US embassies in Switzerland, Sweden and Finland were read by the Finnish codebreakers.

Regarding Bern, Switzerland most of the messages dealt with intelligence matters:

Replying to my request for information regarding the contents of the messages from our Legation in Bern to the Department, Col. Hallamaa said the great bulk of them were intelligence messages dealing with conditions in Germany, France, Italy and the Balkans. He spoke in complimentary terms about ‘Harrison’s’ information service’.

Regarding Helsinki, Finland Hallamaa stated that thanks to the decoded diplomatic traffic they were always informed of current US policy initiatives:

Col. Hallamaa said that they always knew before McClintock arrived at the Foreign Office what he was coming to talk about’.

Hallamaa revealed a lot of confidential information to the Americans and volunteered to have some of his experts interviewed. 

The interview was conducted on friendly terms with Higgs stating; ‘Col. Hallamaa was most pleasant and seemed to be entirely frank and open regarding the matters discussed’.

Additional information: In November 1944 the US cryptanalysts Paavo Carlson of the Army’s Signal Security Agency and Paul E. Goldsberry of the State Department’s cipher unit interviewed Finnish officials regarding their work on US codes. Their report can be found here.

Friday, June 8, 2018

The Tanks of Operation Barbarossa

Boris Kavalerchik, tank expert and author of the Journal of Slavic Military Studies article ‘Once Again About the T-34’ has published a book on ‘The Tanks of Operation Barbarossa’.


Summary:

When the Germans invaded the Soviet Union in 1941 the Red Army had four times as many tanks as the Wehrmacht and their tanks were seemingly superior, yet the Wehrmacht won the border battles with extraordinary ease the Red Armys tank force was pushed aside and for the most part annihilated. How was this victory achieved, and were the Soviet tanks really as well designed as is often believed? These are the basic questions Boris Kavalerchik answers in this absorbing study of the tanks and the tank tactics of the two armies that confronted each other at the start of the war on the Eastern Front. Drawing on technical and operational documents from Russian archives, many of which were classified until recently and are unknown to Western readers, he compares the strengths and weakness of the tanks and the different ways in which they were used by the opposing armies. His work will be essential reading for military historians who are interested in the development of armoured warfare and in this aspect of the struggle on the Eastern Front.

Q&A with Boris Kavalerchik

The author was kind enough to answer some of my questions.

1) How did you become interested in WWII history and why did you decide to write a book on Soviet vs. German tanks during the 1941 campaign?

I've been reading and collecting books and magazines about all kinds of military hardware since I was 12 years old. In college, I had to go through military training and become a tank platoon's commander in reserve, so my knowledge of tanks became much deeper and more practical than before. After a while I started to realize that military hardware is nothing without the people who use it, and I began to pay much more attention to military history. 

The Great Patriotic War has always had special importance for people of the USSR, where I used to live. Many of my relatives, including my father, fought in that war, and some of them were KIA. Naturally, I have heard and read a lot about these historic events and become quite interested in them. Eventually, I co-authored a book about the Soviet Union and Germany's preparations for WWII, as well as that war's beginning. Tank warfare played a very important part in determining the outcome of these battles, so I decided to dedicate a separate study to this subject. That is how my book came to life.

2) What new information have you uncovered that differentiates your book from other similar studies?

In the USSR, only officially approved historians had access to the state archives. Moreover, their work had to go through government censorship and could only support the official point of view on history, which very often was far from reality. After the collapse of the Soviet Union all archives gradually became open to regular people who were interested in events from the past. More and more original archival materials began to be published and even became available online. As a result, I managed to find a lot of information which was classified until recently and had been generally unknown, especially to western readers. 

This information allowed me to reach quite different conclusions in comparison to widely held beliefs about Red Army's tanks during WWII based on old Soviet propaganda. As a mechanical engineer, I also analyzed and compared Soviet and German tanks from a purely technical standpoint, but from rarely used angles which as a rule got neglected. The results struck me as very unusual and I hope they will be of interest to my book's readers.

3) Do you think that WWII era armored vehicles truly played a decisive role in combat operations or has their contribution been exaggerated due to the ‘coolness’ factor?

I think that a very important role in WWII was played not by armored vehicles themselves but by armored forces which included not only tanks but motorized infantry, artillery, combat engineers, anti-tank and anti-aircraft units, and so on. Moreover, as a rule, armored forces fought successfully only in combination and cooperation with other services and branches of the armed forces. Tanks do have the ‘coolness’ factor, so many people mistakenly take them for wonder-weapons, capable of independently deciding the outcome of any battle. In reality, this is not the case. Nevertheless, tanks were a very significant part of the armed forces of all participants of WWII.

4) In your opinion what are the worst mistakes that popular history books make regarding German and Soviet armored vehicles and the Eastern Front in general?

In my opinion, some authors of popular history books mistakenly judge people of previous generations and their armored vehicles from today's point of view using modern criteria. Every tank, without exception, has both positive and negative aspects. In order to determine them it is necessary to know the purpose and objectives of these tanks, which were not the same in all countries or in all periods of time. So, before criticizing any tank from the past, one should determine why it was designed and built the way it was. After understanding all factors which influenced tank design in a particular time and country, we can judge these combat vehicles much more objectively.

Friday, June 1, 2018

Update

In The Japanese FUJI diplomatic cipher 1941-43 I’ve added the following information under the paragraph ‘Pers Z effort’:

More information is available from the TICOM report DF-31B ‘How J.B. 57 Japanese Letter System Was Solved’, written by the cryptanalysts Annalise Huenke and Hans Rohrbach

The first break into system JB 57 came through two messages that had the same indicator (meaning they used the same transposition key). Once these were solved the system was identified as a transposed code, using a stencil.


Solution of this indicator led to the decipherment of more messages and dr Kunze (head of the ‘Mathematical Cryptanalytic Subsection’ of Pers Z) was able to use the information recovered in order to solve more message indicators. The inroads made by the solution of indicator groups led to the eventual recovery of the underlying code by the linguistic group and the current exploitation of this traffic.


Friday, May 25, 2018

TICOM DF-196

The NSA FOIA/MDR office has declassified the TICOM report DF-196 ‘Report on Russian decryption in the former German Army’.

Sunday, May 13, 2018

Another correction

After the release of TICOM report D-83, in The British Typex cipher machine I’ve changed the paragraph

In the period 1940-41 the cipher research department of the German Army’s signal intelligence agency Inspectorate 7/VI had several talented mathematicians (Pietsch, Steinberg, Marquart, SchulzRinow) tasked with examining difficult foreign cryptosystems. The war diary of Inspectorate 7/VI shows that these individuals investigated the Typex device and by May ’41 had ascertained that it was mainly used by the RAF and was issued with 10 rotors. Their research on its internal cipher operation however was slow and had not led to any breakthrough. Things changed in May when they visited the facilities of the Signal Intelligence Agency of the Supreme Command of the Armed Forces - OKW/Chi and were able to examine a Typex machine captured at Dunkirk. The device worked according to the Enigma principle with the two rotors on the left remaining stationary and the wiring of the entry and reflector wheels could be recovered’  

into

In the period 1940-41 the cipher research department of the German Army’s signal intelligence agency Inspectorate 7/VI had several talented mathematicians (Pietsch, Steinberg, Marquart, SchulzRinow) tasked with examining difficult foreign cryptosystems. The war diary of Inspectorate 7/VI shows that these individuals investigated the Typex device and by May ’41 had ascertained that it was an Enigma type device with 5 multistep rotors, the last two of which did not move during encipherment. Their research was confirmed in May, when they visited the facilities  of the Signal Intelligence Agency of the Supreme Command of the Armed Forces - OKW/Chi and were able to examine a Typex machine captured at Dunkirk. The device worked according to the Enigma principle with the two rotors on the left remaining stationary and the wiring of the entry and reflector wheels could be recovered’. 

Monday, April 30, 2018

Article on Chinese codes and ciphers

Interesting article from the journal Cryptologia: ‘Chinese cryptography: The Chinese Nationalist Party and intelligence management, 1927–1949’ by Ulug Kuzuoglu.

ABSTRACT

This paper is the first scholarly attempt to examine the history of Chinese cryptography and the role it played in building the intelligence network of the Chinese Nationalist Party (KMT) from 1927 to 1949. Rather than investigating the institutional structure of intelligence, I focus on Chinese characters, the primary medium that made cryptology and intelligence possible. Given that the Chinese writing system is by nature nonalphabetic and thus noncipherable, how did cryptography work in Chinese? How did the state and its scientists reengineer Chinese characters for the purposes of secret communication? This paper argues that due to the Chinese writing system itself, Chinese cryptography was bound to the use of codebooks rather than ciphers; thus, “codebook management” was central to building intelligence networks in China.

Friday, April 27, 2018

Cipher systems of the German Foreign Ministry

The TICOM report IF-266 ‘DEPARTMENT OF STATE REPORTS ON THE GERMAN FOREIGN OFFICE’ has some information on the codebooks and cipher procedures used by the German Foreign Ministry during WWII:





Pages from the diplomatic codebook No4:



Use of cipher systems by embassy and consulate:





Wednesday, April 11, 2018

Another dead end?

Last month i posted the recently declassified Carlson-Goldsberry report. A memo included in the report had the following handwritten notes: RG 84 box 1 and an NND number that looks like 857570 or 857560.


This seems to lead to US National Archives collection RG 84 ‘Records of Foreign Service Posts of the Department of State’.

The NND number doesn’t seem to be 857570 because that code is associated with reports of the ‘Supreme Headquarters Allied Expeditionary Force. Office of the U.S. Political Officer. 2/13/1944-ca. 9/2/1944’.

Other combinations such as 851510, 857510, 851570 are not valid.

This leaves NND 857560. This code tracks to RG 84 ‘Sweden’ - Entry 3198 ‘Top Secret General Records, 1944 – 1952’.

This makes sense as that report was sent from the US embassy in Sweden.


One would expect to find the Carlson-Goldsberry report there.

So when I told my researcher to check this box I expected that I would be able to get the reports sent from Sweden in late 1944 summarizing the talks US officials had with the Finnish codebreakers.

Unfortunately both my researcher and the NARA FOIA office have confirmed that these reports are not there!

So why does the note say RG 84 box 1 NND 857560?

I’ve asked the NSA FOIA office if they can give me more information on where they got this document. If they respond maybe I will be able to track down similar reports in NARA.

Update: Unfortunately the response from the NSA FOIA office was ‘….we have no additional information to provide’.

Saturday, March 31, 2018

The IBM Codatype cipher machine

In the files of the NSA’s Friedman collection there is a report by William F. Friedman, dated September 1937, which deals with a cipher machine called Codatype (1). 




Apparently David Salmon, the State Department’s chief of the Division of Communications and Records wanted Friedman’s opinion on the security afforded by the Codatype machine.

Although the device appeared to be ‘highly reliable, speedy and efficient’ Friedman’s conclusion was that ‘the degree of cryptographic security afforded by the machine is relatively low, and certainly not sufficient for governmental confidential or secret messages’ and ‘It is doubtful whether anything can be done to eliminate the more or less fatal cryptographic weakness of this model and still retain a machine and cryptographic system which will be practical for the purpose for which intended’.

Thus the Codatype remained a prototype and was not acquired by the State Department.

The device was designed by the IBM engineer Austin Robert Noll, US patent 2,116,732 (2):







Notes:


Monday, March 26, 2018

The Carlson-Goldsberry report - Compromise of State Department communications by the Finnish codebreakers in WWII

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1, B1, C1, D1) enciphered with substitution tables were available.

Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until mid/late 1944) and by reading these messages the Axis powers gained insights into global US policy.

Germans, Finns and Japanese cooperated on the solution of the strip cipher. In 1941 the Japanese gave to the Germans alphabet strips and numerical keys that they had copied from a US consulate in 1939 and these were passed on by the Germans to their Finnish allies in 1942. Then in 1943 the Finns started sharing their results with Japan. 

Finnish solution of State Department cryptosystems

During WWII the Finnish signal intelligence service worked mostly on Soviet military and NKVD cryptosystems however they did have a small diplomatic section located in Mikkeli. This department had about 38 analysts, with the majority working on US codes.
Head of the department was Mary Grashorn. Other important people were Pentti Aalto (effective head of the US section) and the experts on the M-138 strip cipher Karl Erik Henriksson and Kalevi Loimaranta.

Their main wartime success was the solution of the State Department’s M-138-A cipher. The solution of this high level system gave them access to important diplomatic messages from US embassies in Europe and around the world. 


Operation Stella Polaris

In September 1944 Finland signed an armistice with the Soviet Union. The people in charge of the Finnish signal intelligence service anticipated this move and fearing a Soviet takeover of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris (Polar Star).

In late September roughly 700 people, comprising members of the intelligence services and their families were transported by ship to Sweden. The Finns had come to an agreement with the Swedish intelligence service that their people would be allowed to stay and in return the Swedes would get the Finnish crypto archives and their radio equipment. At the same time colonel Hallamaa, head of the signals intelligence service, gathered funds for the Stella Polaris group by selling the solved codes in the Finnish archives to the Americans, British and Japanese. 

The Stella Polaris operation was dependent on secrecy. However the open market for Soviet codes made the Swedish government uneasy. In the end most of the Finnish personnel chose to return to Finland, since the feared Soviet takeover did not materialize. 

The American reaction and the Carlson-Goldsberry report

According to the NSA study History of Venona (Ft. George G. Meade: Center for Cryptologic History, 1995), it was at that time that the Finns revealed to the US authorities that they had solved their diplomatic codes. On 29 September 1944 colonel Hallamaa met with L. Randolph Higgs of the US embassy in Stockholm and told him about their success.


In response two cryptanalysts were sent from the US to evaluate the compromise of US codes in more detail. They were Paavo Carlson of the Army’s Signal Security Agency and Paul E. Goldsberry of the State Department’s cipher unit. Their report dated 23 November 1944 had details on the solution of US systems.


The Carlson-Goldsberry report

Unfortunately locating this report proved to be quite a problem. Initially I searched for it in the US National Archives (both in the NSA and OSS collections) but without success.

Thankfully the NSA FOIA/MDR office has managed to locate this file and they have finally declassified it.






The 4-page report summarizes the information gathered by US officials from their interviews of Finnish codebreakers in 16, 18 and 21 November 1944.

From the Finnish side Erkki Pale (head of the department working on Soviet ciphers) and Kalevi Loimaranta (member of the department dealing with foreign diplomatic codes) gave a summary of their work on various cryptosystems.

The Finns admitted to solving US diplomatic systems, both codebooks and the strip cipher M-138-A. According to them an unenciphered codebook could be reconstructed in 6 months but an enciphered one was harder to solve.

Regarding the M-138-A cipher it was solved because the alphabet strips were used for long periods of time, the same strips were used by several users and the numerical keys were the same for all users. Stereotypical beginnings and endings were also exploited in assumed plaintext cryptanalytic attacks.

There was cooperation with the German codebreakers on US systems and the Finns received a lot of intercepts from them.

The Finnish codebreakers also used a number of IBM machines for statistical work.

Although the Finns stated that after the introduction of channel elimination in January 1944 they could no longer solve strip cipher traffic a memo included in the report says that their detailed knowledge of channel elimination procedures may indicate continued success with the M-138-A system.


Acknowledgments: I have to thank my friends in the US for requesting this file from the NSA FOIA/MDR office and getting it declassified.

Tuesday, March 6, 2018

Bye bye mr Hayashi

In 2017 I tried to locate a file in the US national archives called ‘Interrogation of mr Hayashi’. 

According to the NSA FOIA office it had been recently transferred to NARA as part of transfer group TR-457-2016-0009. The reference I was given by NARA pointed to 36 boxes that have not been indexed, so the file could not be located by my researcher. 

I tried to find the Hayashi file again this year by asking NARA’s research department if they could locate it but I was told that ‘We have carefully searched our holdings with a particular focus on Record Group 457 - Entry P4…….however, we were not able to locate the file’.

Thus it seems that this is the end of my quest for the Hayashi file.

Saturday, February 24, 2018

Monday, February 19, 2018

Candle in the Dark: COMINT and Soviet Industrial Secrets, 1946-1956

The study ‘Candle in the Dark: COMINT and Soviet Industrial Secrets, 1946-1956’ by Carol B. Davis is available for download from the NSA website.

This version (unlike the copy at the Cryptologic Museum’s Library) is complete!

Saturday, February 17, 2018

To err is human vol 4

In my essay Rommel’s microwave link I had mentioned that a German speech cipher system was used on a microwave link connecting the German high command with Rommel’s HQ in North Africa.  

According to ‘Spread Spectrum Communications Handbook’ vol1:

'In 1935, Telefunken engineers Paul Kotowski and Kurt Dannehl applied for a German patent on a device for masking voice signals by combining them with an equally broad-band noise signal produced by a rotating generator. The receiver in their system had a duplicate rotating generator, properly synchronized so that its locally produced noise replica could be used to uncover the voice signal. The U.S. version of this patent was issued in 1940, and was considered prior art in a later patent on DSSS communication systems. Certainly, the Kotowski-Dannehl patent exemplifies the transition from the use of key-stream generators for discrete data encryption to pseudorandom signal storage for voice or continuous signal encryption. Several elements of the SS concept are present in this patent, the obvious missing notion being that of purposeful bandwidth expansion.

The Germans used Kotowski’s concept as the starting point for developing a more sophisticated capability that was urgently needed in the early years of World War II. Gottfried Vogt, a Telefunken engineer under Kotowski, remembers testing a system for analog speech encryption in 1939. This employed a pair of irregularly slotted or sawtoothed disks turning at different speeds, for generating a noise-like signal at the transmitter, to be modulated/multiplied by the voice signal. The receiver’s matching disks were synchronized by means of two transmitted tones, one above and one below the encrypted voice band. 

This system was used on a wire link from Germany, through Yugoslavia and Greece, to a very- and/or ultra-high frequency (VHF/UHF) link across the Mediterranean to General Erwin Rommel’s forces in Derna, Libya.'

After having a look at google patents I saw that Vogt was credited with a patent and I thought that this was the speech cipher system but I was corrected by klausis krypto kolumne commenter ‘Thomas’, who linked the Kotowski-Dannehl patent.

Thus in Rommel’s microwave link I’ve added a link and pics to patent US2211132A.

Friday, February 2, 2018

Joint Chiefs of Staff evaluation of Office of War Information ciphers

During WWII the US Office of War Information engaged in intelligence gathering and propaganda activities against the Axis powers.

The representatives of the OWI used various cipher systems in order to protect their communications and these systems were examined by the Joint Chiefs of Staff.

A summary report was issued in July 1944 and it found problems with physical security, classification procedures, stereotyped messages and cipher reuse.

Regarding OWI ciphers it was noted that ‘Present double transposition keys have been in use since they were produced by the Signal Corps in late 1942 and early 1943’ and the recommendation was ‘That immediate supersession of these keys be accomplished and that provision be made for their more frequent supersession in the future’.












Source: US National Archives, collection RG 208, Office of Wartime Information: General Records of the Security Officer, Entry 9. Location:  350/71/17/6, Box 1. Folder Communications Survey OWI.

Acknowledgements: I have to thank Robert Hanyok for locating and copying the JCS evaluation.