To err is human vol2

In ‘German special intelligence, the M-138 strip cipher and unrest in India’ I had said that the Western Allies were able to monitor the exchange of information on Allied codes and ciphers between Germany, Finland and Japan thanks to their solution of the Japanese Coral cipher machine. This was not correct.

Ralph Erskine pointed out to me that the Coral was mainly used by Japanese naval attaches and that the relevant histories of this system do not mention it being used for transmitting Allied cipher material. Further research at the US National Archives and Records Administration revealed that the code used for transmitting solved Allied cipher material was JAT, a letter code used together with a Gronsfeld square and a book containing random 4-figure groups.  

I’ve corrected the relevant passage in my essay and added scans from the document ‘JAT write up - selections from JMA traffic'.

Update

I added the paragraph ‘Fuel tanks in the fighting compartment’ in WWII Myths - T-34 Best Tank of the war.

Update

I’ve added telegram No 2.181 in Allen Dulles and the compromise of OSS codes in WWII.

The wartime adventures of Prince Max Hohenlohe

During WWII the military forces of the Allies and the Axis battled in Europe and Asia but behind the scenes there were efforts to negotiate some sort of compromise peace. These efforts however never amounted to much since both sides distrusted each other and the military situation made it clear that the Allies could win the war through military force alone.  

Since the 1930’s a segment of German society that opposed the National-Socialist regime had tried to establish contact with foreign countries in order to topple Hitler. During the war the same groups contacted US and British officials in neutral countries and tried to gain their support in order to remove the NS regime from power. The Western Allies were aware of these efforts but they did not offer material support to the members of the German resistance.

At the same time elements of the NS regime came to realize that the war was lost and thus made cautious attempts to contact Allied officials that could promote some sort of compromise peace. Heinrich Himmler was leader of the SS security service and thus one of the most powerful men in Nazi Germany. Yet by 1943 he was beginning to realize that hopes for a successful conclusion of the war were slim. His subordinate General Walter Schellenberg, head of the foreign intelligence department of the Sicherheitsdienst, had many talks with Himmler on the need for a compromise peace and in 1943 he was able to make the first attempts at contacting Allied officials.

The Germans knew that Allen Dulles was in charge of the OSS-Office of Strategic Services station in Bern, Switzerland and they chose to contact him through people associated with the German resistance.

In early 1943 Prince Max Hohenlohe (working on behalf of the Sicherheitsdienst) was given permission to travel to Switzerland and meet Dulles. Unfortunately it doesn’t seem like their meeting remained a secret for long. In the Finnish national archives one can find the decoded version of message No 2.181 of April 7, 1943, giving an overview of their discussion.

 

The original is available from the US National Archives and Records Administration - collection RG 59.

Both the German resistance (through Admiral Canaris) and the Sicherheitsdienst (through Schellenberg) had warned Dulles that his communications were compromised but it doesn’t seem like he acted on this information. These efforts for a compromise peace were probably doomed from the start (especially since the Germans seemed to have overestimated the influence of Dulles) but even so without secure communications the talks could not have remained secret for long.

Fake history - ‘The imitation game’ movie review

Alan Turing was a legendary mathematician and a pioneer in the field of computer science. During WWII he worked for the Government Code and Cypher School, analyzing and solving Axis codes. His main achievement at GCCS was the solution of the plugboard Enigma, used widely by the German armed forces. Turing was undeniably a genius and after the war he continued his research into computers but in 1952 he got in trouble with the authorities after a police investigation into the break-in of his apartment revealed that the culprit knew Turing and had a homosexual relationship with him. Since homosexuality was a crime both men were convicted of ‘gross indecency’ and Turing had to undergo hormonal treatment. His conviction affected not only his personal life but also his professional opportunities. In 1954 Turing was found dead in his apartment, apparently the victim of cyanide poisoning. There is speculation on whether this was an accident or a suicide.

From the information presented so far it is clear that Turing was a fascinating individual and a movie about him was long overdue but is ‘The imitation game’ up to the challenge?

 

What I expected to see

Since I’ve read quite a lot on WWII cryptology and signals intelligence there are some events that I expected to see in the movie, not only because they would add realism to the film but also because they prove that truth is stranger than fiction. What are they?

Polish codebreakers: In the 1930’s the British and French codebreakers, despite their best efforts, were unable to solve the plugboard Enigma. If countries with great resources and a long tradition in cryptanalysis could not solve this device one would expect that no one else could have succeeded, especially a smaller country with limited resources. Yet contrary to all expectations the Polish codebreakers had not only managed to figure out the operating principle of the Enigma but had succeeded in retrieving the rotor wrings and then solved the internal settings of several networks. They managed to keep this a secret not only from the Germans but also from their close allies! They only revealed their success to the French and British representatives in July 1939 and thus helped them immensely in their codebreaking work.

Gordon Welchman and the diagonal board: Welchman was a talented mathematician who worked on the Enigma with success, eventually running Hut 6, responsible for German Army and Airforce Enigma trafffic. Welchman’s great contribution was coming up with the idea of the diagonal board. Turing’s bombes were modified to add the diagonal board which made them much more efficient in their operation. As Gordon puts it in ‘The Hut Six story’, p304: ‘Turing, though initially incredulous, was quick to appreciate the importance of this new twist in Enigma theory, which greatly reduced the number of bombe runs that would be needed to ensure success in breaking an Enigma key by means of a crib’

Naval Enigma – Lofoten raid: By 1940 the British codebreakers were routinely solving current Enigma traffic, mainly from Luftwaffe networks. Against the naval Enigma however they had made little progress because the device was used in a more secure manner (additional rotors and use of enciphered message indicators). By March ’41 their only operational success had been the solution of the Enigma ‘key’ for 5 days of 1938 and 6 days in April 1940. In order to force this deadlock the Brits decided to put statistics on the side and use brute force instead! In March 1941 a commando raid was mounted against the German forces in the Norwegian Lofoten islands with the goal of capturing Enigma cipher material (monthly keylists and indicator tables). This operation was a success with material retrieved from the German armed trawler Krebs. This material allowed Hut 8 to decrypt the February traffic during March. Then thanks to the intelligence gained from this ‘break’ they were able to solve the April and May traffic cryptanalytically.

4-rotor naval Enigma/4-rotor US bombe: In February 1942 the U-boat command stopped using the 3-rotor Enigma and instead introduced a modified 4-rotor version. This was much more secure than the 3-rotor version and immediately put an end to the British success. British and American efforts to solve it failed again and again. By December 1942 only 3 days traffic had been broken. This failure had strained relations between British codebreakers and the US navy’s OP-20-G. It was obvious that new 4-rotor ‘bombes’ were needed but the British reassurance that these would be soon introduced failed to materialize. The Americans then decided to build their own ‘bombes’ at the National Cash Register Corporation under engineer Joseph Desch. It was a good thing they did because the British 4-rotor ‘bombe’ design turned out to be problematic.

Are these events mentioned in the actual movie? Let’s see.

‘The imitation game’

The movie starts with the police investigation in 1952 and then takes us back to 1939 when Turing first visited GCCS. There he had to solve the Enigma while facing the hostility of Commander Denniston and the other cryptanalysts. Everyone thinks that Turing is a failure but eventually his ‘wacky’ idea to build a machine in order to decode a machine finally works and immediately the British know of the location of every German U-boat in the Atlantic. However they decide not to sink them all because that would alert the Germans. In fact Turing stops them from notifying an Allied convoy of an impending attack even though the brother of one of his fellow cryptanalysts is on board. Turing with the help of Stewart Menzies keeps his success with the Enigma a secret from the military authorities and also from Commander Denniston because he fears that they will misuse it. Instead he decides to use statistical theory in order to find where the Enigma intelligence should be used to have the best effect on the war effort. Having won the war on his own Turing then goes back to teaching and the movie shows how much he suffered from the police investigation and the hormonal therapy, leading to his accident/suicide.

Yeah….

The movie is definitely entertaining with great actors, great sets, great cinematography etc. However the storyline isn’t just exaggerated in parts or simplified for the general audience. We’re talking about huge errors and strange conspiracy theories being shown to viewers who probably don’t know any better.  

Am I exaggerating? Someone can counter that it’s not a documentary, it’s a movie. Let’s have look at the failures of the film in more detail. Grab a beer, coffee, tea, whatever works for you because you’ll need it…

Alan Turing = part nutty professor part rain man

Turing definitely wasn’t an average person and obviously had his idiosyncrasies, however the film makes him look completely helpless in his interactions with other people. Maybe the producers thought that the average viewer would only appreciate how smart Turing was by making him autistic.

Bletchley Park = Four guys and a pub

I thought that Bletchley Park was a huge organization with thousands of people working on Axis codes. Apparently I was wrong. According to the movie Bletchley Park consisted of four cryptanalysts (Turing included) and a rather homely pub. Oh, there’s also a storehouse where they keep the bombe ehh i mean ‘Christopher’.

Turing = MacGyver

Some books claim that the bombe was built by Harold Keen, the chief engineer of the British Tabulating Machine Company based on Turing’s designs. These books are wrong (according to the movie). The bombe was built by Turing himself with no assistance from anyone else.

Where are the Poles?

I’ve said earlier that the first to succeed with the military Enigma were the Polish codebreakers. For some reason the movie doesn’t acknowledge their success. When at the beginning of the movie Turing meets Commander Denniston he is told that everyone considers the Enigma unbreakable. This is after September 1939 because in the first scenes we hear the declaration of war between Germany and UK. Yet in July 1939 the Polish had revealed their success to the Brits and French.

Later in the movie when MacGyver Turing is building his bombe he says that his machine was inspired by an old Polish machine but is infinitely more advanced. This is not explained further nor is any reference made to the Polish solution. In fact the Turing bombe was not necessarily infinitely more advanced from the Polish device, it basically worked on a different principle.

Where’s Gordon?

Apparently Gordon Welchman never existed. However the diagonal board is mentioned once without explaining how it works. In the movie it is discovered by Hugh Alexander.

Turing industries: Build first - figure out how it works later

In real life Turing came up with the idea of exploiting a ‘crib’ (suspected plaintext in the ciphertext) in the Enigma traffic and built a device around that idea. In the movie Turing starts building the bombe as soon as he arrives at Bletchley Park. Yet he only figures out cribbing much later thanks to Joan Clarke!!! What was he building all that time?

Where are the Americans?

The movie doesn’t make any reference to the new 4-rotor Enigma introduced in 1942 in the U-boat command. No mention is made of the technologically advanced US 4-rotor bombes.

The Enigma panopticon

According to the movie once they finally solved the settings for a day they easily found the locations of all the enemy submarines in the Atlantic. No mention is made of difficulties in interpreting messages, delays in decoding, lack of traffic or failure to decode. Yet decoding naval messages didn’t mean that someone could get their coordinates since they were enciphered with a manual system before being enciphered once more on the Enigma. As I’ve written in B-Dienst vs Bletchley Park - The invasion of Norway and the Battle of the Atlantic ‘Coordinates were taken from a grid table. From June ’41 coordinates were further disguised by using fixed reference points on the grid table. From November ’41 an Adressbuch was used to encipher the grid references’. The coordinates problem was only mastered by the Allies in June 1944 when they captured an Adressbuch from U-boat U-505.

Conspiracy theories

The biggest problem I have with the movie is that it repeats some strange theories that probably belong in a conspiracy forum rather than a serious film.

The first one is that the Allies only used Ultra intelligence when their statistical theory??? showed that it would have a big impact on the war situation. In the film they show a whole convoy being abandoned to the U-boats because had they changed its course the Germans would have found out about Bletchley Park. In real life Ultra intelligence was used on ALL fronts. Every measure was taken to ensure secrecy and only the top commanders were fully indoctrinated into the secret but the intelligence WAS used. For example what did the Brits do when (thanks to captured keylists) in summer 1941 they started solving U-boats messages with little time lag? They rerouted all their convoys around U-boat concentrations. Only 5 of 26 SC convoys, 2 of 31 HX convoys and 3 of 49 ON convoys were attacked…

The other weird theory promoted by the movie is that the Soviet spy John Cairncross was known all along to the British security services. In fact Menzies tells Turing that it was him that allowed Cairncross to come to Bletchley Park so that he could send valuable intelligence to their ally Stalin. I guess no one can touch those British superspies. They can never lose. Even if you manage to get a spy into their organizations it’s only because they know about it and they allow it….

More mistakes

From the first scenes it’s clear that Commander Denniston doesn’t like Turing. He really, really doesn’t like him. Perhaps Turing fought for the Starks…

When the Enigma device is first shown Denniston says that Polish intelligence smuggled one out of Berlin. Nope. The first Enigma was captured in February 1940 from U-boat U-33.

The movie shows Enigma being used by U-boats, which is correct but it also shows Luftwaffe bombers sending messages which is not correct. Aircraft used hand ciphers for sending messages.

A U-boat is shown attacking warships while underwater. Not accurate. U-boats usually approached on the surface and their main goal was to sink merchant ships not warships.

The movie says in the beginning that thanks to the U-boats Britain was starving. Bullshit.

When Turing decides not to use the Enigma intelligence in fear of alerting the Germans Keira Knightley says: ‘they’ll have changed the design of Enigma by the weekend’. In fact the Germans were planning to replace the Enigma with a new device during the war but they never managed it due to production problems.

Fake history - ‘The imitation game’ trailer review

It’s not often that movies on WWII cryptology and signals intelligence appear on the big screen, probably because the subject matter is too complex for the general audience. A new movie, called ‘The imitation game’ has been released and it has received a lot of favorable reviews in the media. The movie focuses on the legendary mathematician Alan Turing and his efforts to solve the German Enigma cipher machine.

First let’s have a look at the trailer

Hmmm the movie certainly looks good (sets, costumes etc) and the main actors are all well known (Cumberbatch, Knightley, Strong) but the problems start to add up…

1). At 0:21: ‘It’s the greatest encryption device in history and the Germans use it for all communications’.

Ehm, I don’t know what greatest encryption device means but the military Enigma was not revolutionary in any sense. It was simply a clever modification of the commercial version, sold to companies and countries around the world. In fact the Brits had their own Enigma version called Typex. As for the second part the Germans use it for all communications it’s also wrong. The Germans used hand ciphers for low level messages, the Enigma at regiment/division level and also by naval units and airbases and cipher teleprinters for communications between higher commands. So the Enigma did not cover all German communications.  

2). At 0:25 ‘everyone thinks Enigma is unbreakable’

The Poles didn’t think so did they?

3).  At 1:15 ‘I’m designing a machine that will allow us to break every message, every day, instantly’.

Bullshit. Sorry but there’s no other way to put it…I understand that some parts of Enigma theory need to be ‘dumbed’ down so that the general audience will enjoy the movie but come on! The bombe devices couldn’t break every message, every day and they certainly couldn’t do it instantly. Running ‘cribs’ took time and under the best conditions solutions could be achieved in several hours. Under the worst it could take days, weeks or possibly never…

Even when the Enigma settings were retrieved it wasn’t possible to read the message instantly. Someone had to type the message on the specially modified Typex devices and write down the deciphered text. Without good ‘cribs’ the bombes didn’t work. Simple as that.

3).  At 1:20: ‘They had to create the world’s first computer’

……….The bombes were not general purpose computing devices. Apparently the producers of the film were thinking of the ‘Collosus’ but this was a different device, built by different people, for the solution of a different cipher machine, NOT the Enigma.

4). At 1:29: Fake rivalry between Turing and Denniston.

Historically inaccurate but I guess they had to insert some kind of conflict in the story for the general audience.

5). At 1:50: ‘The Navy thinks that one of us is a Soviet spy’.

Again historically inaccurate. There was a Soviet spy at Bletchley Park, John Cairncross but he was not a cryptanalyst and he didn’t work with Turing.

So in only 2 and a half minutes we’ve seen some serious mistakes. On the other hand maybe I’m nitpicking here.  So how does the actual movie hold up? We’ll get to that in the next post.

Unanswered questions of WWII cryptology – progress report

Back in 2013 I wrote down several cases of WWII cryptologic history that require more research. Since then I’ve spent a lot of time and money trying to find new information. Has the effort paid off? Let’s have a look at each case:

1). US State Department strip cipher

I wrote: How bad was the compromise of the State Department’s high level system? That question is hard to answer because there is limited information available and it doesn’t seem like the Americans were really interested in learning the full extent of the compromise. Some documents that would shed more light on this affair are proving very hard to find…

I’ve been able to find more information on this case, including the way the M-138 strip cipher was used by the State Department, decoded US diplomatic traffic found in the US and Finnish national archives, the exploitation of US traffic by the Japanese in the period 1940-41 and the efforts of the German cryptanalyst Wolfgang Franz at OKW/Chi.

It is clear that Germans, Japanese and Finns were able to solve many alphabet strips both circular and special and thus read State Department messages from embassies in Europe and Asia. The most important intercepted messages seem to have been those from Bern, Switzerland and Chungking, China.

Unfortunately several important reports are still classified by the NSA and we have to wait for the declassification procedure. At the same time I haven’t been able to track down the Carlson-Goldsberry report, detailing the Finnish solution of the State Department strip cipher. This report was written in late 1944 by two US cryptanalysts after interviewing Finnish codebreakers in Sweden. 

Another aspect of this case concerns the messages from the OSS - Office of Strategic Services and OWI - Office of War Information stations in Bern that were also sent via diplomatic channels. It’s not clear why these messages were sent using State department codes and not through their own systems. In this area information is lacking, since the OSS organization doesn’t seem to have officially acknowledged the compromise of their communications during the war.

2). NKVD 5^th Department codebreakers

During WWII the Soviet Union invested significant resources in the interception and exploitation of enemy radio traffic. The internal security service NKVD and the Army’s general staff had codebreaking departments with the former recruiting many talented mathematicians. According to author Matthew Aid  ‘By the end of World War II, the 5th Directorate controlled the single largest concentration of mathematicians and linguists in the Soviet Union.’

So far very limited information is available regarding their war time efforts versus foreign codes (not only Axis but also US, UK and those of neutral countries).

 3). Referat Vauck success

In the period 1942-44 the German Army’s signal intelligence agency Inspectorate 7/VI had a separate deparment (Referat 12) assigned with the solution of the encoded messages of Allied spy groups operating in occupied Europe. Head of the department was dr Wilhelm Vauck, so his unit was also called Referat Vauck. In 1944 they were transferred to the OKW radio defense department so their reports can no longer be found in the files of Inspectorate 7/VI.

I had written about this case: How successful were they during the war? Unfortunately we do not know. The relevant file in the British national archives HW 40/76 ‘Enemy exploitation of SIS and SOE codes and cyphers’ says that postwar files have been retained and my request for the release of the interrogations of dr Vauck has been rejected by the archives staff

Thankfully I‘ve been able to track down the monthly reports of Referat 12 for the period April ’42-February ’44 and I will be writing an essay on them.

An interesting discovery, made while I was trying to find information on Referat Vauck, was that OKW/Chi was also solving Allied agents codes during the war (with significant success it seems). Not much is known about this aspect of OKW/Chi operations…

4). Forschungsamt information

Goering’s Forschungsamt was one of the main German codebreaking/intelligence agencies of the period 1939-45, yet a detailed history of that organization still eludes us. This is another case where it’s up to the NSA to declassify the relevant documents, written by Forschungsamt personnel in the 1950’s.

5). German Enigma investigations

Several authors claim that the Germans never suspected that their Enigma cipher machine was solved by the Allies and that they considered it to be unbreakable.

I had written: The Germans constantly evaluated the security of their Enigma cipher machine. There were many studies on whether the daily key or parts of it could be retrieved through cryptanalysis. Those studies are the TICOM DF-190 to DF-190AN files…..More research is needed to evaluate the German methods and the way they influenced their security measures.

Since then I’ve posted information on case ‘Wicher’ (Polish solution of the Enigma) showing that the Germans knew the device had been compromised in the prewar period and in 1943 they got information from the US regarding the solution of their naval version.  So far it’s clear that the German Navy’s codebreakers found a solution for their 4-rotor machine in late ’44 but we don’t know much about the similar work of the Army cryptanalysts. More research is needed in this case.

6). Japanese Purple and Coral cipher machines

Were the Germans able to solve the cipher machines used by the Japanese foreign ministry and by Japanese military attaches?

I wrote: PURPLE was solved by American and Soviet codebreakers. Did the Germans have any success with it? Until recently the answer was no. 

However it seems there is more to this story.

The Coral machine was used by military attaches and the Anglo-Americans solved it in 1944. In the same year dr Steinberg of the German Army’s signal intelligence agency was transferred to OKW/Chi where he worked on a cipher machine used by the Japanese attaché. Did he manage to solve it? 

TICOM report I-64 ‘Answers by Wm. Buggisch of OKH/Chi to Questions sent by TICOM’ says ‘B. thinks Steinberg (of 209 fame) solved some Jap machine traffic which was difficult but not so hard as Enigma. B. thinks it was traffic of the Jap Military Attache.’

There is scattered information that points towards the solution of an important Japanese code or cipher machine in the period 1943/44 but no conclusive evidence. Maybe more information will become available in the future.

7). Soviet diplomatic code

I wrote: The Soviet Union used a code enciphered with one time pads as its main diplomatic system during WWII. This system if used correctly is unbreakable. 

Were the Germans able to read parts of this traffic? There are some strange statements in Allied and German reports…

The recently declassified TICOM report DF-111 ‘Comments on various cryptologic matters’ by Adolf Paschke (head of the linguistic cryptanalysis group in the German foreign ministry’s decryption department) says that in the years 1927-30 parts of the Soviet diplomatic traffic could be read since the additive pads were sometimes used twice if the message was long enough. Paschke had also identified the use of the same additive tables more than once in some links. Regarding wartime traffic he says that they couldn’t solve any since there were no repetitions but in the report he also added cryptically that Russian material of the Forschungsamt and the High Command’s deciphering department OKW/Chi were destroyed in 1943 during a bombing attack on Berlin.

Although the Germans might have not solved any Soviet diplomatic traffic they did succeed in solving Comintern communications.

8). M-209 decoding device

I wrote: I’m surprised that no one has figured out how this machine worked!

I have to say I’m still surprised that this device has not received any attention from historians and/or the media!