The epic quest for the Carlson-Goldsberry report

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1,B1,C1,D1) enciphered with substitution tables were available.

Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until late 1944) and by reading these messages the Axis powers gained insights into global US policy.

Germans, Finns and Japanese cooperated on the solution of the strip cipher. The Japanese gave to the Germans alphabet strips and numerical keys that they had copied from a US consulate and these were passed on by the Germans to their Finnish allies. Then in 1943 the Finns started sharing their results with Japan.

The German effort

Unfortunately the information we have today on the compromise of the State Department’s strips cipher is limited. One problem is that the archives of the agencies that worked on this system are not available to researchers. Three different German agencies worked on the US diplomatic M-138-A strip cipher. The German High Command’s deciphering department – OKW/Chi, the Foreign Ministry’s deciphering deparment Pers Z and the Air Ministry’s Research Department - Reichsluftfahrtministerium Forschungsamt.

I know that the NSA has some interesting reports on the codebreaking successes of the Forschungsamt but they have not been declassified yet. Regarding OKW/Chi I don’t know if their archives (or parts of them) survived the war. Finally the files of the Pers Z agency were captured by the Anglo-Americans at the end of the war but the reports I’ve seen from the National Archives and Records Administration are mostly administrative files.

This means that so far our sources on the strip compromise are mainly TICOM reports written postwar.

The Finnish codebreakers and the strip cipher

The Finnish codebreakers also worked on the strip cipher and solved several links in the period 1942-44. In this area there was cooperation with their German counterparts, not only in receiving copies of the Japanese cipher material but also exchanges of personnel and analysis of the strip system.

The fact that the Finns cooperated with the Germans against this cryptosystem means that we can find out more about the German operation through Finnish sources and thus circumvent the lack of German archival sources.

Operation Stella Polaris

In September 1944 Finland signed an armistice with the Soviet Union. The people in charge of the Finnish signal intelligence service anticipated this move and fearing a Soviet takeover of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris (Polar Star).

In late September roughly 700 people, comprising members of the intelligence services and their families were transported by ship to Sweden. The Finns had come to an agreement with the Swedish intelligence service that their people would be allowed to stay and in return the Swedes would get the Finnish crypto archives and their radio equipment. At the same time colonel Hallamaa, head of the signals intelligence service, gathered funds for the Stella Polaris group by selling the solved codes in the Finnish archives to the Americans, British and Japanese. The Stella Polaris operation was dependent on secrecy. However the open market for Soviet codes made the Swedish government uneasy. In the end most of the Finnish personnel chose to return to Finland, since the feared Soviet takeover did not materialize. 

The American reaction and the Carlson-Goldsberry report

According to the NSA study History of Venona (Ft. George G. Meade: Center for Cryptologic History, 1995) by Robert Louis Benson and Cecil J. Phillips, it was at that time that the Finns revealed to the US authorities that they had solved their diplomatic codes. On 29 September 1944 colonel Hallamaa met with L.Randolph Higgs of the US embassy in Stockholm and told him about their success.

In response two cryptanalysts were sent from the US to evaluate the compromise of US codes in more detail. They were Paavo Carlson of the Army’s Signal Security Agency-SSA and Paul E. Goldsberry of the State Department’s cipher unit. Their report dated 23 November 1944 had details on the solution of US systems.

Unfortunately finding this report has proven to be quite a problem!

Freedom of Information Act requests

After trying to find this report in the US archives i gave up and filed FOIA requests with the State Department, NSA and NARA. The results:

1).  The State Department told me that they no longer have these files as they have been sent to NARA so I should bother them.

2). NARA could not locate this file but they did send me a list of references that I should look up.

3). The NSA informed me that they had expended the free time allowed for research and if I wanted to continue I’d have to pay. I decided not to.

Assistant Secretary Shaw

Apart from the FOIA requests I tried to find information on the people responsible for evaluating the compromise of State Department codes during the war.  A name that came up in relevant reports was Assistant Secretary Shaw. This was Gardiner Howland Shaw, Assistant Secretary of State in the period 1941-44 and in charge of the State Department’s cipher unit. Unfortunately NARA does not have a separate body of records for G. Howland Shaw.

Another lead I followed was the Shaw foundation but their response was that ‘To our knowledge, he left no immediate family members and we have no record of any of his State Dept work.’

The messages from the embassy in Sweden

After failing to find anything either with the FOIA requests or the Shaw search I decided it would be best to try to track down the messages sent from the US embassy Sweden to Washington during the days mentoned in ‘History of Venona’. Unfortunately the State Department messages are indexed according to a complicated system and it is very difficult to find anything:

So I asked NARA again if they could locate the messages of the embassy in Sweden for these specific dates and their response was:

‘We searched the Source Cards, 1940-1944; General Records of the Department of State, Record Group 59 and located index cards which lead us to believe that no record of these sensitive meetings/topics were kept by the State Department.  It is possible, though, that further examination of this series may yield records which may be pertinent to your research.’

They also gave me a reference to an OSS report in the Director’s files but I had already checked that.

Now I have to give credit where credit is due and the NARA people really did some great work in this case! Unfortunately even after all these efforts the Carlson-Goldsberry report continues to elude us…

A small win?

Although I haven’t been able to find the actual report I think that a page found in NARA-RG 457-Entry 9032-box 214-‘M-138-A numerical keys/daily key table/alphabet strips’ is a part of that report or at least contains information from it.

As can be seen in that page it says Department of State- Assistant Secretary, which should be G. Howland Shaw and the date says 23/11/1944, which matches the ‘History of Venona’ date. The file shows the coupling of alphabet strips with a set of keylists. This implies that the State Department did not use separate sets for each embassy but instead had a limited number of strips and keylists that were rearranged during the war.

Update March 2018: The NSA FOIA/MDR office has declassified this report. (Google drive link)