Friday, April 11, 2014

Soviet cryptologic security failures in WWII – A sneak peak

I’ve already covered the cryptologic failures of the United States and Britain in WWII but I still haven’t covered the Soviet Union. According to Soviet/Russian sources their codes were impenetrable and the Germans were never able to compromise their high level communications links. Is that true?

Well I’m still researching this case and I haven’t copied all the available documents. Once I do I will write a detailed essay on Soviet codes.

For now here is a sneak peak:

Thursday, April 3, 2014

Some thoughts on Soviet tank reliability in WWII

The Eastern front was the largest land campaign of WWII and millions of soldiers fought and died there in the period 1941-45. Although infantry dominated the fighting both sides used a large number of tanks and armored vehicles and these played a big role in breakthrough operations. Most historians focus on the ‘paper’ characteristics of tanks and the production statistics however a very important aspect of complex weapon systems is their reliability and kill/loss ratio. In the East the Germans were always outnumbered but the exchange ratios were in their favor. I’ve often wondered of how much that has to do with poor reliability of Soviet equipment.

Here is something I read recently from ‘Moscow to Stalingrad: Decision in the East’ by Earl F. Ziemke, in page 363:
Active as it was, the Soviet armor was apparently not giving fully satisfactory performance at this stage, and in early August, it became the subject of the following Stalin order:

‘Our armored forces and their units frequently suffer greater losses through mechanical breakdowns than they do in battle. For example, at Stalingrad Front in six days twelve of our tank brigades lost 326 out of their 400 tanks. Of those about 260 owed to mechanical problems. Many of the tanks were abandoned on the battlefield. Similar instances can be observed on other fronts. Since such a high incidence of mechanical defects is implausible, the Supreme Headquarters sees in it covert sabotage and wrecking by certain elements in the tank crews who try to exploit small mechanical troubles to avoid battle.’
Henceforth, every tank leaving the battlefield for alleged mechanical reasons was to be gone over by technicians, and if sabotage was suspected, the crews were to be put into tank punishment companies or "degraded to the infantry" and put into infantry punishment companies.'"

Were the problems really caused by sabotage and wreckers? Apparently not, since captured T-34 tanks used by the Germans in summer 1944 had the following problems:
Regardless of our limited experience, it can be stated that the Russian tanks are not suitable for long road marches and high speeds. It has turned out that the highest speed that can be achieved is 10 to 12 km/hr. It is also necessary on marches to halt every half hour for at least 15 to 20 minutes to let the machine cool down. Difficulties and breakdowns of the steering clutches have occurred with all the new Beute-Panzer. In difficult terrain, on the march, and during the attack, in which the Panzer must be frequently steered and turned, within a short time the steering clutches overheat and are coated with oil. The result is that the clutches don't grip and the Panzer is no longer maneuverable. After they have cooled, the clutches must be rinsed with a lot of fuel.

Also T-34 tanks captured by the Americans in Korea (built in 1945) continued to suffer from the same issues. According to Zaloga’s ‘T-34-85 Medium Tank’, p21-22
An analysis of a T-34-85 captured in Korea by the American tank producer Chrysler, conducted in 1951, provides a good assessment of the T-34- 85……………………. The study, found the following negative features about the tank:…………………………………. Wholly inadequate engine intake air cleaners could be expected to allow early engine failure due to dust intake and the resulting abrasive wear. Several hundred miles in very dusty operation would probably be accompanied by severe engine power loss.' The report was also critical of the lack of a turret basket, poor fire fighting equipment, poor electrical weatherproofing, lack of an auxiliary generator to keep the batteries charged, and lack of a means to heat engine oil for cold weather starts. The report noted that although Soviet manufacturing techniques were adequate for the job, there were many instances where poor or unskilled workmanship undermined the design, and where overworked machines led to course feeds, severe chatter or tearing of machined surfaces, a consequence no doubt of the extreme pressures placed on plants to ensure maximum output. For example, in the tank inspected (manufactured in 1945) the soldering job on the radiator was so poor that it effectively lost half of its capacity.

It’s also worth noting that even in 1941 German reports on captured Soviet T-26 and BT tanks pointed out serious productions issues. For the T-26 tank: The Pz.Kpfw.Zug created by the division is no longer operational. One Panzer is completely burnt out due to an engine fire. Both of the other Panzers have engine and transmission problems. Repetitive repairs were unsuccessful. The Panzers always broke down after being driven several hundred meters on good roads. As reported by technical personnel, both of the engines in the Panzers are unusable because they were incorrectly run in.
And for the BT tank: ‘B. T. (Christi): The main cause of failure is a transmission that is too weak in combination with a strong engine that should provide the tank with high speed, but is over-stressed when driven off road where the lower gears must be used for longer periods. In addition, as in the T 26, problems continuously arise that are due to entire design and poor materials, such as failure of the electrical system, stoppages in fuel delivery, breaks in the oil circulation lines, etc.’

Finally there are the Aberdeen tests on a T-34 tank:
'On the T-34 the transmission is also very poor. When it was being operated, the cogs completely fell to pieces (on all the cogwheels). A chemical analysis of the cogs on the cogwheels showed that their thermal treatment is very poor and does not in any way meet American standards for such mechanisms.’

The deficiency of our diesels is the criminally poor air cleaners on the T-34. The Americans consider that only a saboteur could have constructed such a device
The reliability issues of Soviet tanks during WWII point to serious problems with Soviet industry. The only other explanation is that a huge Nazi/White Guard wrecker movement existed in Soviet factories…

I think that even comrade Stalin would find this idea implausible!

Wednesday, April 2, 2014

The epic quest for the Carlson-Goldsberry report

During WWII the US State Department used several cryptosystems in order to protect its radio communications from the Axis powers. For low level messages the unenciphered Gray and Brown codebooks were used.  For important messages four different codebooks (A1,B1,C1,D1) enciphered with substitution tables were available.

Their most modern and (in theory) secure system was the M-138-A strip cipher. Unfortunately for the Americans this system was compromised and diplomatic messages were read by the Germans, Finns, Japanese, Italians and Hungarians. The strip cipher carried the most important diplomatic traffic of the United States (at least until late 1944) and by reading these messages the Axis powers gained insights into global US policy.
Germans, Finns and Japanese cooperated on the solution of the strip cipher. The Japanese gave to the Germans alphabet strips and numerical keys that they had copied from a US consulate and these were passed on by the Germans to their Finnish allies. Then in 1943 the Finns started sharing their results with Japan.

The German effort
Unfortunately the information we have today on the compromise of the State Department’s strips cipher is limited. One problem is that the archives of the agencies that worked on this system are not available to researchers. Three different German agencies worked on the US diplomatic M-138-A strip cipher. The German High Command’s deciphering department – OKW/Chi, the Foreign Ministry’s deciphering deparment Pers Z and the Air Ministry’s Research Department - Reichsluftfahrtministerium Forschungsamt.

I know that the NSA has some interesting reports on the codebreaking successes of the Forschungsamt but they have not been declassified yet. Regarding OKW/Chi I don’t know if their archives (or parts of them) survived the war. Finally the files of the Pers Z agency were captured by the Anglo-Americans at the end of the war but the reports I’ve seen from the National Archives and Records Administration are mostly administrative files.
This means that so far our sources on the strip compromise are mainly TICOM reports written postwar.

The Finnish codebreakers and the strip cipher
The Finnish codebreakers also worked on the strip cipher and solved several links in the period 1942-44. In this area there was cooperation with their German counterparts, not only in receiving copies of the Japanese cipher material but also exchanges of personnel and analysis of the strip system.

The fact that the Finns cooperated with the Germans against this cryptosystem means that we can find out more about the German operation through Finnish sources and thus circumvent the lack of German archival sources.
Operation Stella Polaris

In September 1944 Finland signed an armistice with the Soviet Union. The people in charge of the Finnish signal intelligence service anticipated this move and fearing a Soviet takeover of the country had taken measures to relocate the radio service to Sweden. This operation was called Stella Polaris (Polar Star).
In late September roughly 700 people, comprising members of the intelligence services and their families were transported by ship to Sweden. The Finns had come to an agreement with the Swedish intelligence service that their people would be allowed to stay and in return the Swedes would get the Finnish crypto archives and their radio equipment. At the same time colonel Hallamaa, head of the signals intelligence service, gathered funds for the Stella Polaris group by selling the solved codes in the Finnish archives to the Americans, British and Japanese. The Stella Polaris operation was dependent on secrecy. However the open market for Soviet codes made the Swedish government uneasy. In the end most of the Finnish personnel chose to return to Finland, since the feared Soviet takeover did not materialize. 

The American reaction and the Carlson-Goldsberry report
According to the NSA study History of Venona (Ft. George G. Meade: Center for Cryptologic History, 1995) by Robert Louis Benson and Cecil J. Phillips, it was at that time that the Finns revealed to the US authorities that they had solved their diplomatic codes. On 29 September 1944 colonel Hallamaa met with L.Randolph Higgs of the US embassy in Stockholm and told him about their success.

In response two cryptanalysts were sent from the US to evaluate the compromise of US codes in more detail. They were Paavo Carlson of the Army’s Signal Security Agency-SSA and Paul E. Goldsberry of the State Department’s cipher unit. Their report dated 23 November 1944 had details on the solution of US systems.

Unfortunately finding this report has proven to be quite a problem!

Freedom of Information Act requests
After trying to find this report in the US archives i gave up and filed FOIA requests with the State Department, NSA and NARA. The results:

1).  The State Department told me that they no longer have these files as they have been sent to NARA so I should bother them.
2). NARA could not locate this file but they did send me a list of references that I should look up.

3). The NSA informed me that they had expended the free time allowed for research and if I wanted to continue I’d have to pay. I decided not to.
Assistant Secretary Shaw

Apart from the FOIA requests I tried to find information on the people responsible for evaluating the compromise of State Department codes during the war.  A name that came up in relevant reports was Assistant Secretary Shaw. This was Gardiner Howland Shaw, Assistant Secretary of State in the period 1941-44 and in charge of the State Department’s cipher unit. Unfortunately NARA does not have a separate body of records for G. Howland Shaw.
Another lead I followed was the Shaw foundation but their response was that ‘To our knowledge, he left no immediate family members and we have no record of any of his State Dept work.

The messages from the embassy in Sweden
After failing to find anything either with the FOIA requests or the Shaw search I decided it would be best to try to track down the messages sent from the US embassy Sweden to Washington during the days mentoned in ‘History of Venona’. Unfortunately the State Department messages are indexed according to a complicated system and it is very difficult to find anything:

So I asked NARA again if they could locate the messages of the embassy in Sweden for these specific dates and their response was:
We searched the Source Cards, 1940-1944; General Records of the Department of State, Record Group 59 and located index cards which lead us to believe that no record of these sensitive meetings/topics were kept by the State Department.  It is possible, though, that further examination of this series may yield records which may be pertinent to your research.

They also gave me a reference to an OSS report in the Director’s files but I had already checked that.

Now I have to give credit where credit is due and the NARA people really did some great work in this case! Unfortunately even after all these efforts the Carlson-Goldsberry report continues to elude us…
A small win?

Although I haven’t been able to find the actual report I think that a page found in NARA-RG 457-Entry 9032-box 214-‘M-138-A numerical keys/daily key table/alphabet strips’ is a part of that report or at least contains information from it.
As can be seen in that page it says Department of State- Assistant Secretary, which should be G. Howland Shaw and the date says 23/11/1944, which matches the ‘History of Venona’ date. The file shows the coupling of alphabet strips with a set of keylists. This implies that the State Department did not use separate sets for each embassy but instead had a limited number of strips and keylists that were rearranged during the war.