Saturday, October 13, 2018

Entry 9032 finding aid

I have uploaded the finding aid to US National Archives - collection RG 457 (Records of the National Security Agency) - entry 9032.

Note that this is not the only entry in collection RG 457.

Monday, September 10, 2018

German solution of State Department A-1 Code in 1944

During WWII the US State Department used several codebooks for enciphering radio telegrams. These were the low level Gray and Brown codes and the high level A1, B1 and C1 codes.

The latter codebooks were used with substitution tables.

It is clear that the German codebreakers were able to solve the substitution tables used with the A1 and C1 codes till late 1943 because these were given to the Japanese and decoded by the Allies in late 1944 (1):


According to a message of the Japanese military attaché the C1 code continued to be used by the US embassy in Bern, Switzerland so those messages could be read in 1944 (2):


Were the Germans also able to read messages enciphered on the A1 codebook in 1944?

The book ‘Hitler, the Allies, and the Jews’ by Shlomo Aronson mentions a message solved by the codebreakers of OKW/Chi (German High Command’s deciphering department) (3):

At the same time, the OKW/Chi decrypts tell us in their way what the Allies were doing in various ways, including the hectic activities of WRB's operatives upon its inception. Thus, the following cable from Washington, dated February 9, 1944, from the State Department and signed by Secretary of State Cordell Hull but in fact sent by the WRB to the American Legation in Bern, dealt with funds made available to the International Red Cross (ICRC) in Geneva to help Jews in Rumania, Croatia, Hungary, Slovakia, and Theresienstadt by the Joint Distribution Committee (AJDC), as authorized by the Treasury Department’.


The original message can be found in the US National Archives (4) and the classification is SECRET.


The note on the first page says A-1 so I assume that it was sent using the A-1 codebook. 

Thus it seems that the Germans continued to read diplomatic traffic sent on the A-1 code even in 1944.

Sources:

(1). US National Archives - collection RG 457 - Entry 9032 - box 1.018 - NR3225 ‘JAT write up - selections from JMA traffic'

(2). UK National archives HW 40/132 ‘Decrypts relating to enemy exploitation of US State Department cyphers, with related correspondence’.


(4). US National Archives - Microfilm Publication M1284, roll 38, indexed to file ‘840.48 Refugees/5195’. 

Saturday, September 8, 2018

Remaining research projects

What files am I still trying to locate? Let’s see.

1). TICOM report I-40

I requested this file from the NSA FOIA office in 2015. It was quickly located and placed in the review queue. However it has not been declassified yet…

2).  NAAS 5 reports:

There are two German Army signal intelligence reports covering the work of the NAAS 5 unit for the second half of 1944:

E-Bericht 4/44 der NAAst 5 (Berichtszeit 1.7-30.9.44) dated 10.10.44 

E-Bericht der NAAst 5 (Berichtszeit 1.10.44-30.12.44) dated 14.1.45

According to the NSA FOIA office they are probably in transfer group TR-0457-2017-0010. 

These files have been sent to the US National archives so I have to wait for NARA to process these files and then I can ask them to locate the NAAS 5 reports (assuming they are really there…).

3). Henriksson report:

According to my information on 18 October 1944 there was a meeting in Sweden between the US officials Wilho Tikander and L. Randolph Higgs and the Finnish officials Reino Hallamaa and Karl-Erik Henriksson.

Henriksson was the Finnish expert on US codes and ciphers and in this meeting he gave the Americans detailed information on the compromise of their diplomatic communications.

My researcher and the NARA research department have checked the files in collection RG 84 ‘Records of the Foreign Service Posts of the Department of State’ - ‘US Legation/Embassy Stockholm, Sweden’ - ‘Top Secret General Records File: 1944’ but they could not locate this file.

Thus I have filed FOIA requests with NARA and the State Department regarding this file. Maybe I’ll get lucky.

Sunday, July 29, 2018

TICOM DF-174A


The report has information on the Enigma cipher machine, the SG 39 cipher machine and the Enigma modification Lückenfüllerwalze.

Thursday, July 5, 2018

The Higgs memorandum - Compromise of State Department communications by the Finnish codebreakers in WWII

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1, B1, C1, D1) enciphered with substitution tables were available.

Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until mid/late 1944) and by reading these messages the Axis powers gained insights into global US policy.

Germans, Finns and Japanese cooperated on the solution of the strip cipher. In 1941 the Japanese gave to the Germans alphabet strips and numerical keys that they had copied from a US consulate in 1939 and these were passed on by the Germans to their Finnish allies in 1942. Then in 1943 the Finns started sharing their results with Japan. 

Finnish solution of State Department cryptosystems

During WWII the Finnish signal intelligence service worked mostly on Soviet military and NKVD cryptosystems however they did have a small diplomatic section located in Mikkeli. This department had about 38 analysts, with the majority working on US codes.
Head of the department was Mary Grashorn. Other important people were Pentti Aalto (effective head of the US section) and the experts on the M-138 strip cipher Karl Erik Henriksson and Kalevi Loimaranta.

Their main wartime success was the solution of the State Department’s M-138-A cipher. The solution of this high level system gave them access to important diplomatic messages from US embassies in Europe and around the world. 


Operation Stella Polaris

In September 1944 Finland signed an armistice with the Soviet Union. The people in charge of the Finnish signal intelligence service anticipated this move and fearing a Soviet takeover of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris (Polar Star).

In late September roughly 700 people, comprising members of the intelligence services and their families were transported by ship to Sweden. The Finns had come to an agreement with the Swedish intelligence service that their people would be allowed to stay and in return the Swedes would get the Finnish crypto archives and their radio equipment. At the same time colonel Hallamaa, head of the signals intelligence service, gathered funds for the Stella Polaris group by selling the solved codes in the Finnish archives to the Americans, British and Japanese. 

The Stella Polaris operation was dependent on secrecy. However the open market for Soviet codes made the Swedish government uneasy. In the end most of the Finnish personnel chose to return to Finland, since the feared Soviet takeover did not materialize. 

The Higgs memorandum

In September 1944 colonel Hallamaa met with L. Randolph Higgs, an official of the US embassy in Sweden and told him about their successes with US diplomatic codes and ciphers.

This information was summarized in a report prepared by Higgs, dated 30 September 1944.

The report can be found in the US National Archives - collection RG 84 ‘Records of the Foreign Service Posts of the Department of State’ - ‘US Legation/Embassy Stockholm, Sweden’ - ‘Top Secret General Records File: 1944’.







Higgs met with colonel Hallamaa on September 29 and the OSS officials Tikander and Cole were also present during their discussion.

Hallamaa stated that he was an administrator, not a cryptanalyst and about 10-12 of his men worked on US diplomatic codes.

His unit had solved the US codes Gray, Brown, M-138-A strip cipher and enciphered codebooks (probably the A1, B1, C1).

The high level M-138-A system had been solved mostly by taking advantage of operator mistakes such as sending strip cipher information on other systems that had already been broken or sending the same message in different strips one of which had been broken.

The strip cipher was considered a strong encryption system and had been adopted by the Finns for some of their traffic.

Important diplomatic messages from the US embassies in Switzerland, Sweden and Finland were read by the Finnish codebreakers.

Regarding Bern, Switzerland most of the messages dealt with intelligence matters:

Replying to my request for information regarding the contents of the messages from our Legation in Bern to the Department, Col. Hallamaa said the great bulk of them were intelligence messages dealing with conditions in Germany, France, Italy and the Balkans. He spoke in complimentary terms about ‘Harrison’s’ information service’.

Regarding Helsinki, Finland Hallamaa stated that thanks to the decoded diplomatic traffic they were always informed of current US policy initiatives:

Col. Hallamaa said that they always knew before McClintock arrived at the Foreign Office what he was coming to talk about’.

Hallamaa revealed a lot of confidential information to the Americans and volunteered to have some of his experts interviewed. 

The interview was conducted on friendly terms with Higgs stating; ‘Col. Hallamaa was most pleasant and seemed to be entirely frank and open regarding the matters discussed’.

Additional information: In November 1944 the US cryptanalysts Paavo Carlson of the Army’s Signal Security Agency and Paul E. Goldsberry of the State Department’s cipher unit interviewed Finnish officials regarding their work on US codes. Their report can be found here.

Friday, June 8, 2018

The Tanks of Operation Barbarossa

Boris Kavalerchik, tank expert and author of the Journal of Slavic Military Studies article ‘Once Again About the T-34’ has published a book on ‘The Tanks of Operation Barbarossa’.


Summary:

When the Germans invaded the Soviet Union in 1941 the Red Army had four times as many tanks as the Wehrmacht and their tanks were seemingly superior, yet the Wehrmacht won the border battles with extraordinary ease the Red Armys tank force was pushed aside and for the most part annihilated. How was this victory achieved, and were the Soviet tanks really as well designed as is often believed? These are the basic questions Boris Kavalerchik answers in this absorbing study of the tanks and the tank tactics of the two armies that confronted each other at the start of the war on the Eastern Front. Drawing on technical and operational documents from Russian archives, many of which were classified until recently and are unknown to Western readers, he compares the strengths and weakness of the tanks and the different ways in which they were used by the opposing armies. His work will be essential reading for military historians who are interested in the development of armoured warfare and in this aspect of the struggle on the Eastern Front.

Q&A with Boris Kavalerchik

The author was kind enough to answer some of my questions.

1) How did you become interested in WWII history and why did you decide to write a book on Soviet vs. German tanks during the 1941 campaign?

I've been reading and collecting books and magazines about all kinds of military hardware since I was 12 years old. In college, I had to go through military training and become a tank platoon's commander in reserve, so my knowledge of tanks became much deeper and more practical than before. After a while I started to realize that military hardware is nothing without the people who use it, and I began to pay much more attention to military history. 

The Great Patriotic War has always had special importance for people of the USSR, where I used to live. Many of my relatives, including my father, fought in that war, and some of them were KIA. Naturally, I have heard and read a lot about these historic events and become quite interested in them. Eventually, I co-authored a book about the Soviet Union and Germany's preparations for WWII, as well as that war's beginning. Tank warfare played a very important part in determining the outcome of these battles, so I decided to dedicate a separate study to this subject. That is how my book came to life.

2) What new information have you uncovered that differentiates your book from other similar studies?

In the USSR, only officially approved historians had access to the state archives. Moreover, their work had to go through government censorship and could only support the official point of view on history, which very often was far from reality. After the collapse of the Soviet Union all archives gradually became open to regular people who were interested in events from the past. More and more original archival materials began to be published and even became available online. As a result, I managed to find a lot of information which was classified until recently and had been generally unknown, especially to western readers. 

This information allowed me to reach quite different conclusions in comparison to widely held beliefs about Red Army's tanks during WWII based on old Soviet propaganda. As a mechanical engineer, I also analyzed and compared Soviet and German tanks from a purely technical standpoint, but from rarely used angles which as a rule got neglected. The results struck me as very unusual and I hope they will be of interest to my book's readers.

3) Do you think that WWII era armored vehicles truly played a decisive role in combat operations or has their contribution been exaggerated due to the ‘coolness’ factor?

I think that a very important role in WWII was played not by armored vehicles themselves but by armored forces which included not only tanks but motorized infantry, artillery, combat engineers, anti-tank and anti-aircraft units, and so on. Moreover, as a rule, armored forces fought successfully only in combination and cooperation with other services and branches of the armed forces. Tanks do have the ‘coolness’ factor, so many people mistakenly take them for wonder-weapons, capable of independently deciding the outcome of any battle. In reality, this is not the case. Nevertheless, tanks were a very significant part of the armed forces of all participants of WWII.

4) In your opinion what are the worst mistakes that popular history books make regarding German and Soviet armored vehicles and the Eastern Front in general?

In my opinion, some authors of popular history books mistakenly judge people of previous generations and their armored vehicles from today's point of view using modern criteria. Every tank, without exception, has both positive and negative aspects. In order to determine them it is necessary to know the purpose and objectives of these tanks, which were not the same in all countries or in all periods of time. So, before criticizing any tank from the past, one should determine why it was designed and built the way it was. After understanding all factors which influenced tank design in a particular time and country, we can judge these combat vehicles much more objectively.

Friday, June 1, 2018

Update

In The Japanese FUJI diplomatic cipher 1941-43 I’ve added the following information under the paragraph ‘Pers Z effort’:

More information is available from the TICOM report DF-31B ‘How J.B. 57 Japanese Letter System Was Solved’, written by the cryptanalysts Annalise Huenke and Hans Rohrbach

The first break into system JB 57 came through two messages that had the same indicator (meaning they used the same transposition key). Once these were solved the system was identified as a transposed code, using a stencil.


Solution of this indicator led to the decipherment of more messages and dr Kunze (head of the ‘Mathematical Cryptanalytic Subsection’ of Pers Z) was able to use the information recovered in order to solve more message indicators. The inroads made by the solution of indicator groups led to the eventual recovery of the underlying code by the linguistic group and the current exploitation of this traffic.


Friday, May 25, 2018

TICOM DF-196

The NSA FOIA/MDR office has declassified the TICOM report DF-196 ‘Report on Russian decryption in the former German Army’.

Sunday, May 13, 2018

Another correction

After the release of TICOM report D-83, in The British Typex cipher machine I’ve changed the paragraph

In the period 1940-41 the cipher research department of the German Army’s signal intelligence agency Inspectorate 7/VI had several talented mathematicians (Pietsch, Steinberg, Marquart, SchulzRinow) tasked with examining difficult foreign cryptosystems. The war diary of Inspectorate 7/VI shows that these individuals investigated the Typex device and by May ’41 had ascertained that it was mainly used by the RAF and was issued with 10 rotors. Their research on its internal cipher operation however was slow and had not led to any breakthrough. Things changed in May when they visited the facilities of the Signal Intelligence Agency of the Supreme Command of the Armed Forces - OKW/Chi and were able to examine a Typex machine captured at Dunkirk. The device worked according to the Enigma principle with the two rotors on the left remaining stationary and the wiring of the entry and reflector wheels could be recovered’  

into

In the period 1940-41 the cipher research department of the German Army’s signal intelligence agency Inspectorate 7/VI had several talented mathematicians (Pietsch, Steinberg, Marquart, SchulzRinow) tasked with examining difficult foreign cryptosystems. The war diary of Inspectorate 7/VI shows that these individuals investigated the Typex device and by May ’41 had ascertained that it was an Enigma type device with 5 multistep rotors, the last two of which did not move during encipherment. Their research was confirmed in May, when they visited the facilities  of the Signal Intelligence Agency of the Supreme Command of the Armed Forces - OKW/Chi and were able to examine a Typex machine captured at Dunkirk. The device worked according to the Enigma principle with the two rotors on the left remaining stationary and the wiring of the entry and reflector wheels could be recovered’. 

Monday, April 30, 2018

Article on Chinese codes and ciphers

Interesting article from the journal Cryptologia: ‘Chinese cryptography: The Chinese Nationalist Party and intelligence management, 1927–1949’ by Ulug Kuzuoglu.

ABSTRACT

This paper is the first scholarly attempt to examine the history of Chinese cryptography and the role it played in building the intelligence network of the Chinese Nationalist Party (KMT) from 1927 to 1949. Rather than investigating the institutional structure of intelligence, I focus on Chinese characters, the primary medium that made cryptology and intelligence possible. Given that the Chinese writing system is by nature nonalphabetic and thus noncipherable, how did cryptography work in Chinese? How did the state and its scientists reengineer Chinese characters for the purposes of secret communication? This paper argues that due to the Chinese writing system itself, Chinese cryptography was bound to the use of codebooks rather than ciphers; thus, “codebook management” was central to building intelligence networks in China.

Friday, April 27, 2018

Cipher systems of the German Foreign Ministry

The TICOM report IF-266 ‘DEPARTMENT OF STATE REPORTS ON THE GERMAN FOREIGN OFFICE’ has some information on the codebooks and cipher procedures used by the German Foreign Ministry during WWII:





Pages from the diplomatic codebook No4:



Use of cipher systems by embassy and consulate:





Wednesday, April 11, 2018

Another dead end?

Last month i posted the recently declassified Carlson-Goldsberry report. A memo included in the report had the following handwritten notes: RG 84 box 1 and an NND number that looks like 857570 or 857560.


This seems to lead to US National Archives collection RG 84 ‘Records of Foreign Service Posts of the Department of State’.

The NND number doesn’t seem to be 857570 because that code is associated with reports of the ‘Supreme Headquarters Allied Expeditionary Force. Office of the U.S. Political Officer. 2/13/1944-ca. 9/2/1944’.

Other combinations such as 851510, 857510, 851570 are not valid.

This leaves NND 857560. This code tracks to RG 84 ‘Sweden’ - Entry 3198 ‘Top Secret General Records, 1944 – 1952’.

This makes sense as that report was sent from the US embassy in Sweden.


One would expect to find the Carlson-Goldsberry report there.

So when I told my researcher to check this box I expected that I would be able to get the reports sent from Sweden in late 1944 summarizing the talks US officials had with the Finnish codebreakers.

Unfortunately both my researcher and the NARA FOIA office have confirmed that these reports are not there!

So why does the note say RG 84 box 1 NND 857560?

I’ve asked the NSA FOIA office if they can give me more information on where they got this document. If they respond maybe I will be able to track down similar reports in NARA.

Update: Unfortunately the response from the NSA FOIA office was ‘….we have no additional information to provide’.

Saturday, March 31, 2018

The IBM Codatype cipher machine

In the files of the NSA’s Friedman collection there is a report by William F. Friedman, dated September 1937, which deals with a cipher machine called Codatype (1). 




Apparently David Salmon, the State Department’s chief of the Division of Communications and Records wanted Friedman’s opinion on the security afforded by the Codatype machine.

Although the device appeared to be ‘highly reliable, speedy and efficient’ Friedman’s conclusion was that ‘the degree of cryptographic security afforded by the machine is relatively low, and certainly not sufficient for governmental confidential or secret messages’ and ‘It is doubtful whether anything can be done to eliminate the more or less fatal cryptographic weakness of this model and still retain a machine and cryptographic system which will be practical for the purpose for which intended’.

Thus the Codatype remained a prototype and was not acquired by the State Department.

The device was designed by the IBM engineer Austin Robert Noll, US patent 2,116,732 (2):







Notes:


Monday, March 26, 2018

The Carlson-Goldsberry report - Compromise of State Department communications by the Finnish codebreakers in WWII

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1, B1, C1, D1) enciphered with substitution tables were available.

Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until mid/late 1944) and by reading these messages the Axis powers gained insights into global US policy.

Germans, Finns and Japanese cooperated on the solution of the strip cipher. In 1941 the Japanese gave to the Germans alphabet strips and numerical keys that they had copied from a US consulate in 1939 and these were passed on by the Germans to their Finnish allies in 1942. Then in 1943 the Finns started sharing their results with Japan. 

Finnish solution of State Department cryptosystems

During WWII the Finnish signal intelligence service worked mostly on Soviet military and NKVD cryptosystems however they did have a small diplomatic section located in Mikkeli. This department had about 38 analysts, with the majority working on US codes.
Head of the department was Mary Grashorn. Other important people were Pentti Aalto (effective head of the US section) and the experts on the M-138 strip cipher Karl Erik Henriksson and Kalevi Loimaranta.

Their main wartime success was the solution of the State Department’s M-138-A cipher. The solution of this high level system gave them access to important diplomatic messages from US embassies in Europe and around the world. 


Operation Stella Polaris

In September 1944 Finland signed an armistice with the Soviet Union. The people in charge of the Finnish signal intelligence service anticipated this move and fearing a Soviet takeover of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris (Polar Star).

In late September roughly 700 people, comprising members of the intelligence services and their families were transported by ship to Sweden. The Finns had come to an agreement with the Swedish intelligence service that their people would be allowed to stay and in return the Swedes would get the Finnish crypto archives and their radio equipment. At the same time colonel Hallamaa, head of the signals intelligence service, gathered funds for the Stella Polaris group by selling the solved codes in the Finnish archives to the Americans, British and Japanese. 

The Stella Polaris operation was dependent on secrecy. However the open market for Soviet codes made the Swedish government uneasy. In the end most of the Finnish personnel chose to return to Finland, since the feared Soviet takeover did not materialize. 

The American reaction and the Carlson-Goldsberry report

According to the NSA study History of Venona (Ft. George G. Meade: Center for Cryptologic History, 1995), it was at that time that the Finns revealed to the US authorities that they had solved their diplomatic codes. On 29 September 1944 colonel Hallamaa met with L. Randolph Higgs of the US embassy in Stockholm and told him about their success.


In response two cryptanalysts were sent from the US to evaluate the compromise of US codes in more detail. They were Paavo Carlson of the Army’s Signal Security Agency and Paul E. Goldsberry of the State Department’s cipher unit. Their report dated 23 November 1944 had details on the solution of US systems.


The Carlson-Goldsberry report

Unfortunately locating this report proved to be quite a problem. Initially I searched for it in the US National Archives (both in the NSA and OSS collections) but without success.

Thankfully the NSA FOIA/MDR office has managed to locate this file and they have finally declassified it.






The 4-page report summarizes the information gathered by US officials from their interviews of Finnish codebreakers in 16, 18 and 21 November 1944.

From the Finnish side Erkki Pale (head of the department working on Soviet ciphers) and Kalevi Loimaranta (member of the department dealing with foreign diplomatic codes) gave a summary of their work on various cryptosystems.

The Finns admitted to solving US diplomatic systems, both codebooks and the strip cipher M-138-A. According to them an unenciphered codebook could be reconstructed in 6 months but an enciphered one was harder to solve.

Regarding the M-138-A cipher it was solved because the alphabet strips were used for long periods of time, the same strips were used by several users and the numerical keys were the same for all users. Stereotypical beginnings and endings were also exploited in assumed plaintext cryptanalytic attacks.

There was cooperation with the German codebreakers on US systems and the Finns received a lot of intercepts from them.

The Finnish codebreakers also used a number of IBM machines for statistical work.

Although the Finns stated that after the introduction of channel elimination in January 1944 they could no longer solve strip cipher traffic a memo included in the report says that their detailed knowledge of channel elimination procedures may indicate continued success with the M-138-A system.


Acknowledgments: I have to thank my friends in the US for requesting this file from the NSA FOIA/MDR office and getting it declassified.