Sunday, December 25, 2011

Bletchley Park vs Berlin – The North African Seesaw

 All told, Britain arguably lost the signals intelligence war in that theatre between January 1941 and May 1942, and certainly did worse and suffered more damage there than it ever did in the Atlantic.

                           Robert Ferris ,‘’ Intelligence and strategy: selected essays’’ chapter 4

In my posts so far I’ve only dealt with the good people of Bletchley Park in the Typex compromise story. Now it’s time to change that and take a closer look at the performance of German vs British  codebreakers in actual campaigns of WWII.

This part will be about the fighting in N.Africa. My goal is to look at the codesystems of both countries and see which side had the advantage and for how long. Obviously I’ll also have to mention the successes each side had with other codes ( Italian ,American, Free French),however my main focus is on German and British codes.

From all the information I’ve seen it’s clear that the German side had a major advantage in the period 1941 to summer 1942 both in high level and tactical systems. The Enigma keys used by Rommel proved to be too much for Bletchley Park and the keys of the German navy in the Med also resisted attack.

The Brits were successful with the Luftwaffe keys used in the Med (‘’Red’’,‘’Light blue’’,’’Scorpion’’ among others) plus the could read Italian codes. They only read Rommel’s key Chaffinch (Chaffinch I : General , Chaffinch II : Comm between Panzerarmee Afrika and higher authority , Chaffinch III : Administrative ) thanks to captured material  in 17 Sept -19 Oct '41 and from 2 Noc -6 Dec '41 ,with some regularity and  often a week or more late. The operational Enigma key of Panzerarmee Afrika Phoenix (for use between Army,Corps and Divisions) was captured during Crusader (18 Nov ’41) and traffic read up to 23 Nov ’41 , then not broken until summer ’42.

The Chaffinch key was broken again on 10 April 1942 when 50% was read with a delay of a week or more. For the rest of ’42 all three keys were broken but with differing degrees of success. On average half the days were ‘broken’ and half of the breaks were achieved within 48h after receiving the messages.

June was also the month that the Phoenix key was broken  but it was usually read after a delay of several days.

Here are the detailed statistics concerning British success with German army ‘keys’ in N.Africa in 1942:


German army 'keys' broken by Bletchley Park -N.Africa 1942
Month
Chaffinch I
Chaffinch II
Chaffinch III
Phoenix
Jun-42
2
11
20
16
Jul-42
6
16
13
7
Aug-42
8
15
13
16
Sep-42
14
19
14
25
Oct-42
18
22
21
27
Nov-42
17
28
23
18
Dec-42
12
14
19
12

 

It should however be mentioned that at the same time that Bletchley Park was finally able to decode Rommel’s Enigma ‘keys’ another means of communication had become available to him. This was a microwave link from Derna, Libya to Athens, Greece via Crete. From there communications could be routed to Rome and Berlin. As far as I know this link was secure from eavesdroppers.

Regarding naval Enigma the U-boats in the Med used a modified form of the Home waters key from 1st October 1941 .GCCS was able to read their traffic till the key was changed in February 1942.Then no success until December 1942.

Surface ships and shore authorities in the Med and the Black Sea used the Porpoise key, which resisted attack until August 1942. By September it was being read regularly.

During the campaign low level Italian military codes were read continuously but their high level codebooks proved more secure (almost completely in 1942). The main problem for the Italians was that their naval machine ciphers were decoded and resulted in the sinking of convoys carrying supplies to Rommel’s forces. The machines in question were the commercial Enigma and the Hagelin C-38 (the main culprit).

While the Brits unsuccessfully tried to read Rommel’s communications ,the German codebreakers were  decoding messages in British high level systems :

The Army’s War Office Cypher (Army universal high-grade codebook, carried traffic between Whitehall, commands, armies, corps and, later, divisions) was read in the Med area from summer 1941 to January 1942.It gave accurate information on  8th Army’s strength and order of battle. The information on British tank strength seen in German decoded messages was so accurate that the War office ‘’was very concerned’’. [Source: British intelligence in the Second World War vol2,p298]

The RAF Cypher (high-grade codebook) was read in the Med from early 1941 until November 1942.Most messages were solved ‘’within 5-10 days’’ according to Voegele chief cryptanalyst of the GAF.

The RN’s Code and Cypher plus low level codes were read. According to report ADM 1/27186 messages in Naval Code No1 were read in 1941.In May ’41 a copy of Naval Code No1 was captured from HMS York ,sunk in Suda Bay Crete. It’s successor Naval Code No2 was broken in 1942 and a high proportion of traffic recoded by Auxiliary Vessels Tables was read.

The more high level Naval Cypher No2 (in use from Aug’40 to Jan’42) and No4 (used from Jan’42 to June’43) were also compromised. A summary of B-Dienst’s success with them follows :

1.     Naval Cypher No2 :First read Sept '40.Oct' 40 setback (change in encyphering procedure) .From March '41  until Sept '41 limited ~10%, ,from then on  high.

2.     Naval Cypher No4: First read March '42.By Oct '42 reconstructed. Messages relating to convoy movements in the Pacific ,Indian ocean and Red sea were read.

The Interdepartmental Cypher (used by Foreign Office,Colonial,Dominions and India offices and the services. Also used by Admiralty for Naval Attaches,Consular Officers,Reporting Officers)  was read extensively. It allowed the Germans to keep track of negotiations between Turkey and Britain. [Source: HW 40/85]

As if all these compromises were not enough the Brits were the victim of one of the most embarrassing episodes of the war. The American military attaché in Cairo colonel Bonner Fellers continuously transmitted (in the Military Intelligence Code) British plans , appreciations and strength and loss reports.

The Free French were also guilty of using faulty codes but Fellers telegrams were so important that everything was organized to intercept them and decode them in the fastest way possible. From early 1942 until July he was unintentionally providing the Germans with invaluable information.

In the tactical field the Brits did even worse. Due to the lack of a machine cipher for division downwards they had to rely on hand ciphers of limited security. They also resorted often to radio telephone communications that offered no secrecy. Attempts to disguise their conversations by using code words did not usually hinder the German eavesdroppers.

Unfortunately these security lapses on behalf of the British forces meant that  Rommel’s intelligence unit NFAK 621 ,headed by the able Captain Seebohm, was much more successful than it ought to be.

By solving low level codes , overhearing British commanders talk on the radiotelephone and using direction-finding and traffic analysis they were able to provide Rommel with a more or less accurate Allied  order of battle. The effect of this unit was a major multiplier of German military strength.

Especially in conditions of mobile warfare ,when messages were exchanged quickly with little regard for security ,Seebohm’s men gave Rommel the edge.

Their loss in 10 July 1942 when the unit was overrun by the Australian 2/24th Battalion crippled Rommel’s signal intelligence capability at a moment he needed it most .

So how did the Brits manage to win that campaign? From what I’ve mentioned so far it seems they were seriously beaten in the intelligence field ( in the period 1941-summer 1942)

First of all both sides were able to get some information about each other’s strengths and dispositions from various sources ( aerial photo-reconnaissance ,spies , army recon units , low level codes ,traffic analysis and D/F ).  Also both sides made mistakes and miscalculations from the intelligence they got. But more importantly there were many more factors influencing victory and defeat than merely signals intelligence.

Rommel’s main problem was his inability to provide supplies for his forces at long distances from his supply ports.Even though the RN was able to sink a lot of Italian supply ships the problem was not lack of supplies but inability to transport them far from the ports. This fact is discussed in detail by Martin van Creveld in ‘’Supplying War: Logistics from Wallenstein to Patton’’ . Rommel’s trucks burned more gas than they were carrying to the front because of the long distances involved. No amount of codebreaking could change that….

Even though Rommel was defeated at El Alamein he only managed to get so far with the help of signals intelligence. His reckless style of command would have led to disaster without this ‘’hidden ace’’. Such successes however heavily depended on silly British mistakes.

After July 1942 things changed fast. Once the British forces captured Seebohm’s unit they realized the extent of their compromise and immediately changed their signal procedures and codes. From then on the Brits would be considered ,by the Germans ,to have the most secure signal communications of any  Allied nation.

In the second half of 1942 the Germans not only lost their access to the British code systems that I mentioned earlier but  practically all their codes were being read. By reading Rommel’s keys the Brits got the strength reports for his troops and vehicles. The biggest success of Bletchley Park was a decoded signal sent to Middle East Command on 17 August 1942.This was  ‘’perhaps the most important single item of information that the Enigma had yet contributed to the desert campaign’’ [Source: British intelligence in the Second World War vol2,p298]. The decrypt of a Panzer Army appreciation ,It contained an outline of Rommel’s intentions for the coming offensive.

However the location of his units was not revealed from the Enigma :  No Enigma decrypt giving a comprehensive account of the Panzer army’s dispositions was obtained after 30 April 1942 ; that of 30 April -a Chaffinch decrypt- showed that the bulk of the enemy’s armour was in the north, but mentioned that some  elements of the DAK were ‘at the moment’ operating at the south of the line ‘to secure the southern flank’. [Source: British intelligence in the Second World War vol2,p722]

How was Rommel able to save his forces from annihilation and retreat to Tynisia in good order ? The answer is the same as before .Signals intelligence although of great importance cannot win battles. The Germans still had well equipped ,well trained and well led troops and they did not hesitate to move new units to Tunisia to counter the Allied landings.

In Tunisia the field was leveled as the American forces were very careless with the use of radio. The Free French forces also continued to make mistakes and use insecure codes. Still after a few successes the Axis forces were defeated.

So what are the lessons to be learned from the North African campaign?

First of all it is more important to protect your own codes than it is to read the enemy ones. The Germans definitely did much better than they should have against British codes in N.Africa. If the Brits had a machine cipher in widespread use for forward units things would definitely have been different for Rommel .A simple machine like the Hagelin C-38 if used correctly would be beyond the capability of his mobile unit NFAK 621 and messages could only be solved at higher headquarters with significant delay if at all. A small number of speech scramblers would also have meant that his units would have been destroyed in June 1942.

It is also apparent that codebreaking is important but not decisive. Having information is of no use if you can’t exploit it to your benefit. For example by reading Italian naval codes the Brits were able to sink a lot of supply ships. However due to bad luck on 18 Dec ’41 their naval K force (2 cruisers plus 2 destroyers) operating out of Malta drifted into a minefield and out of 4 ships 2 were sunk and 2 heavily damaged. This disaster coupled with the Luftwaffe campaign against Malta meant that during the first half of ‘42 the Italian navy was able to transport supplies virtually unmolested despite the Allies knowledge of their routes and schedule.

Rommel’s successes can definitely be attributed in some part to his superiority in signals intelligence but that advantage would have meant nothing if his forces didn’t also have good equipment , communications , logistics and training.

In the end the German successes with British codes have to be attributed in part to poor security on the part of the Brits. They continued to use codebooks that they knew were in German hands , they used tactical codes that were easy to solve and for inexplicable reasons they used the radiotelephone as if they were back home talking to a friend. Once these silly errors were corrected and once Bletchley Park started solving German army and navy enigma keys the roles were reversed .In a month Rommel’s best source of intelligence dried up for good.

Authors and academics that present the Ultra story in triumphant terms should take a good look at the N.African campaign and the defeat of Allied codes in the period 1941-summer 1942. The people at Bletchley Park were not the only ones skilled in codebreaking.

Sources : Intelligence and strategy: selected essays , British intelligence in the Second World War vol2 , Ticom reports: I-112 , I-113 , I-51 , War Secrets in the Ether , Rommel's intelligence in the desert campaign, 1941-1943 , FMS P-038 German Radio Intelligence , CSDIC SIR 1704 , American Signal Intelligence in Northwest Africa and Western Europe , HW 40/85

Acknowledgments: I have to thank Ralph Erskine for the Chaffinch and Phoenix key statistics.

6 comments:

  1. I believe that there are a lot of things missing that would be required to fully understand and compare.

    (i) The actual operational impact of Axis intercepts. That most RAF messages could be deciphered in 5-10 days is of course good work. But did it matter, or was this too late? What kind of messages were intercepted? For example, were bases in Egypt wired in, and therefore interception safe?

    (ii) The successes (or otherwise) of the Commonwealth Y-Service. At the tactical level the Germans also transmitted in the clear - things work both ways.

    (iii) The total failure of German intelligence to foresee and then accept that CRUSADER was happening to them in November 1941 (and in fact its reliance on the German Foreign Office mission under von Neurath to provide them with proof of it being a major offensive), and the failure of Operation SOMMERNACHTSTRAUM. If the intercepts were so good, how could either of these two things happen?

    (iv) The role of radio deception.

    Just more food for thought.

    All the best

    Andreas

    ReplyDelete
    Replies
    1. If i had to mention every detail I’d write a book not a blog post. Let me answer quickly:
      i).I don’t know specific details about the airbases that were monitored through these codes because they are not mentioned in the Ticom reports I have. Perhaps a post war review of RAF signals security has that info. There is such a file but I don’t have it.
      What was the operational impact? The Luftwaffe’s signal intelligence units were able to keep track of British formations, their organization and their operations by reading the RAF cypher, the low level SYKO code and by intercepting plaintext radiotelephone transmissions.
      ii).The problem is that the Germans used the Enigma from regiment upwards. That limits the operations of these units only for low level traffic. On the other hand NFAK 621 could solve both British division level codes and the WOC used at army level. A difference of an order of magnitude.
      iii).Just decoding messages doesn’t mean that you know what your opponent is going to do .In chess does the enemy hide his pieces? No but you still don’t know what he is going to do. Furthermore most of the traffic did not go through radio. Only a part of the radio traffic could be intercepted, only a part of the intercepted traffic could be decoded, only a part of the decoded traffic had any good intel. Then there are perceptions and misconceptions of the enemy…
      Iv). Was used by both sides. If you want more details check Ferris, ‘Intelligence and strategy: selected essays’

      Delete
    2. i) There's still no discernible operational impact to me. Fact of the matter remains that the RAF/SAAF had almost complete air superiority from day one of the offensive. Also, don't overestimate signals - a lot of knowledge was based on good old-fashioned photo recce. Clearly they combined these two. See e.g. here: http://crusaderproject.wordpress.com/2011/03/23/luftwaffe-appreciation-of-raf-strength-in-north-africa-20-november-1941/

      ii) I seriously doubt that in the heat of battle every call was put through Enigma. Do you have evidence for that? Time critical messages were almost certainly broadcast in clear, in my view, with limited coding. There is evidence of interception that supports that view. See comment below this post: http://crusaderproject.wordpress.com/2010/11/01/some-days-in-the-war-of-101-special-wireless-section/ In any case, this still means that orders from Regiment to Batallion/Abteilung were in clear.

      iii) That's rather my point. :)

      iv) I know.

      All the best

      Andreas

      Delete
    3. Ok let's see:
      i). Building the enemy's OOB was the main function of intelligence units. If you are talking about an impact on daily operations it is mentioned in some reports that it was possible to defend against bombing attacks by reading the codes that gave away the target
      ii). Messages in all armies go through specific codes based on clasification. For example secret would go by Enigma, confidential by hand cipher etc.
      As you said in combat operations some messages would be sent by the fastest way possible,even in the clear. That doesn't change the fact that the Enigma was the basis of German communications. By the same logic the M-209 was sometimes used up to army level in 1943-5. Does that mean that the Germans can be said to have penetrated Allied top level communications in that period?
      iii). I think you missed this paragraph :
      'First of all both sides were able to get some information about each other’s strengths and dispositions from various sources ( aerial photo-reconnaissance ,spies , army recon units , low level codes ,traffic analysis and D/F ). Also both sides made mistakes and miscalculations from the intelligence they got. But more importantly there were many more factors influencing victory and defeat than merely signals intelligence.'

      Delete
    4. i) Agree - but I think looking in isolation at the success of sigint without consideration of the operational impact is a bit academic.
      ii) I think this is an important element of the story - and we can then circle back to the point I was making about the unknown impact of Commonwealth Y-Services. The presence and use of Enigma at regimental level and above does not imply that Y-Services were unable to read messages at regimental level and above. Which then brings us back to the question of what they read, and how good they were, and what impact that had. I know your view is that ULTRA is overhyped, and the Axis performance understated (correct me if I got this wrong). But could the reverse not be true for Seebohm and the Y-Service on the Commonwealth side?
      iii) But we are talking about analysis of German intel (as a whole, not just sigint) in advance of the battle here, not the success or failure of the battle itself. I think this is a pertinent question in relation to this post, and the reply that many factors affected victory and defeat does not answer it.

      All the best

      Andreas

      Delete
    5. i).Agreed but like i said before i can't research this subject for a decade and then write a book. My goal is in very little space to cover all the important facts. Especially those that challenge the accepted view of events.
      ii). The problem is that there were severe limits in what the forward Allied units could achieve on their own due to the widespread use of cipher machines by the Germans. On the other hand that does not mean that Seebohm's people were always correct. They were however able to achieve more due to poor British cipher security.
      iii). You are an expert on operation Crusader not me. The reports i have are all of a general nature they dont say on 4 January 41 we read 6 messages from 8th army HQ etc

      Delete