NSA impersonating Google?

The plot thickens…

NSA and GCHQ might have been able to spy on people by impersonating Google.

According to motherjones they used a "man-in-the-middle (MITM) attack". Think of that next time you’re googling something!

Book review – Luftwaffe test pilot

During WWII both the Allies and the Axis were able to capture equipment used by the other side. This equipment (tanks, planes, artillery, rifles etc) was thoroughly tested in order to ascertain its performance and weak points.

Captured Allied planes were tested by a special unit of the Luftwaffe at Rechlin airport.

One of the test pilots was Hans-Werner Lerche and in his memoirs ‘Luftwaffe Test Pilot: Flying Captured Allied Aircraft of World War 2’ he writes about the planes he tested and their operational characteristics.

Lerche became interested in aviation at a young age, reading aviation magazines and building model planes. He wanted to learn how to fly a glider and in 1931 was accepted to a gliding training school. In the same course he met ‘a pleasant young lady… her name was Hanna Reitsch’. Lerche got his A, B and C-certificates of flying and later became a glider instructor, while at the same time studying engineering.

 

During the 1930’s he learned how to fly powered aircraft and when he was inducted for military service in the late 30’s he got himself assigned to the Luftwaffe. After passing another pilots course he was promoted to non-commisioned officer and was transferred to the ‘German aviation experimental establishment’. At the Rechlin center German and foreign planes were thoroughly tested.

Overall Lerche flew 125 different aircraft types during his career!

Each chapter of the book deals with specific Allied aircraft. The book covers the British Lancaster, Spitfire and Tempest, the US P-39, P-47, P-51 fighters and the B-17,B-24 bombers, the Soviet planes La-5, Yak-3, the German Ju 290 and Do 335 and several Italian types like the SM.82, SM.91, G.55, Cant 1018, Cant 1007, Ca 133.

For WWII aviation enthusiasts the book has very interesting information regarding the performance and peculiarities of various well known aircraft. For example:

Lancaster bomber  

The Lancaster was the main 4-engine bomber of the RAF’s Bomber Command.

In August ’44 a Lancaster was test flown by the author who evaluated the flying characteristics. The plane was also used in tests of electronic equipment installed in German nightfighters.

Spitfire

The famous Spitfire is an icon of the RAF’s Fighter Command. During WWII it was the premier British fighter plane.

A Spitfire IIA (1.200hp engine) was tested by Lerche who considered it a dangerous opponent due to its armament and low wing loading. A negative point was that when taxiing ‘the field of vision was not as good as in the Bf 109 because of the ‘wide shoulders’ of the 12-cylinder upright-V engine’.

B-17 ‘Flying Fortress’

In October ’43 a USAAF B-17 landed intact in Denmark. The plane made a forced landing but had only superficial damage. After removing ‘dead weight’ like armor and ammunition it was possible to fly it to German territory. According to Lerche the most interesting part of the aircraft were its supercharged engines.

During one of the test flights the B-17 demonstrated its ability to fly home with one or more engines out of action! At roughly 9.000 feet one of the engines malfunctioned. The crew got ready to abandon the plane at the pilot’s command but this was not necessary in the end. Even with 3 engines the plane made a safe landing.

The B-17 was flown to airbases so that German pilots would familiarize themselves with this type and even mock attacks were flown.

The only negative remarks on the B-17 were that the ‘forces acting on the ailerons were relatively high’ and the rudder was very heavy.

B-24 ‘Liberator’

In spring ’43 a B-24D landed by mistake in Sicily and was then flown by an Italian crew to Rechlin so that the Germans could test it. In the fuselage it had the title ‘Blonde Bomber II’.

According to the author the B-24 ‘felt rather unstable longitudinally’ and ‘showed quite high control forces when gliding’. The nose wheel was also a weak point as it required a concrete runway and it collapsed during one of the landings at Rechlin.

P-47 ‘Thunderbolt’

In November ’43 a P-47 had landed intact near Caen, France.

Contrary to orders Lerche had to share this aircraft with a fighter ace of the nearby unit who also wanted to fly it. In the end they agreed that Lerche would fly close by in order to ensure that the plane was operational and then the ace would fly it from Caen to Cormeilles.

 

The P-47 was a very heavy plane and its performance at low level was poor at ~500km/h. However at high altitude its performance was impressive reaching roughly 640 km/h at 29.500 feet.

According to the author the P-47 was not a good plane for dogfighting or low level attacks but it was ‘excellent in higher altitudes, in diving attacks and flying at maximum boost’.

P-51 ‘Mustang’

The famous P-51 ‘Mustang’ is one of the most iconic WWII aircraft. It had extraordinary long range for a single engine aircraft and it successfully escorted US bombers in Axis controlled territory.

In June ’44 a P-51B landed at Cambrai-South airfield during the Normandy invasion. Getting so close to the front was dangerous for Lerche so everything had to be done fast. A Ju 188 fast bomber was used for transport and the P-51 was quickly set up and flown by him.

 

The Mustang was tested extensively and Lerche calls it a ‘truly unique aircraft’. It was very fast at all heights and the performance was very good in all aspects. At an altitude of 23.000 feet it had a speed of ~670km/h.

The negative aspects were: lack of longitudinal stability when the fuel tanks filled to a certain extent, at full throttle it stalled ‘even in a sharp turn’ and the engine required careful handling as ‘when revving up it reacted very sensitively to the correct operating temperatures and, if handled roughly, it countered with unsteady running.’

Lavochkin La-5

The Soviet Lavochkin La-5 fighter was introduced in large numbers in 1943 and it was substantially better than the previous fighter planes of the Red AF.

In September ’44 a La-FN was captured at Gross-Schimanen, East Prussia. This plane was tested and the author found it to be a dangerous opponent at low altitudes.

His final report said that the La-5 had noteworthy performance below 3.000m but top speed was below that of German fighters. Range was short with roughly 40min of flight at the rated power, less with supercharger engaged.

The main problems identified were that fumes from the engine entered the cockpit and the engine was excessively noisy: ‘by no means pleasant was the noisy running of the La-5 engine, which had deafened me by the evening. On later flights I always tried to remember bringing along some cotton wool to plug my ears’.

Yakovlev Yak-3

The Yak-3 was introduced in 1944 and due to its light weight had impressive performance characteristics.

In January ’45 a Yak landed intact at Gross-Schimanen airfield, the same one that Lerche had visited recently. After an inspection it was flown back to Rechlin.

The Yak-3 weighed roughly 2.500 (5.512) kg and had a power loading of 4.5 lb/hp. Speed and acceleration were very good but performance at altitude was poor.

This plane was used not only for testing but was also ordered to Oranienburg airport to take part in an exhibition for Field Marshal Goering. Lerche flew the plane there and had to stand attention during the presentation. Then Goering asked him questions on the performance of the Yak.  

The State Department’s strip cipher – no clear cut answers

The compromise of the State Department’s strip cipher by Axis codebreakers in WWII was one of the worst failures of Allied crypto security. However this case has not received the attention it deserves because there is limited information available.

This is a shame since by reading the US diplomatic traffic the Germans were able to monitor US foreign policy, counter efforts of minor Axis nations to exit the war and even keep an eye on the activities of the OSS station in Berne. It also seems that the Japanese got intelligence of great value by reading the messages of the US Chunking embassy.

The problem is twofold. On the one hand there is limited information available from postwar interrogations of Axis personnel and on the other hand it is very difficult to find out how this system was used by the State Department during the war.

Let’s take a look at these problems.

The efforts of Axis codebreakers

The State Department adopted the strip cipher as its high level system in the 1930’s and used it till late 1944. During that period several countries were able to exploit this system. Japan, Germany, Finland, Hungary and probably Italy were able to solve messages.

Unfortunately the information we have is limited and not always reliable. The fact that many different countries and organizations were involved in the solution of the strip means that the relevant information is fragmented.

In my opinion the following version of events is the most accurate. In late 1937 the Japanese were able to enter the US consulate in Kobe and copy the ‘intercommunication’ strip set 0-1. Using these they obviously read some of the traffic during the period 1937-41 but we do not know how much, with what time lag and whether they also managed to solve ‘special’ strips used by embassies for direct communications with Washington.

In 1941 the Japanese shared the 0-1 set with the Germans. There were three German agencies that worked on the diplomatic strip cipher, OKW/Chi, Pers Z and the Forschungsamt. It seems that they started their investigations of this system in 1941 or even earlier but were probably unable to solve actual traffic at that time. Both Erich Huettenhain (chief cryptanalyst of OKW/Chi) and Hans Rohrbach (Pers Z cryptanalyst) refered to the years 1942-44 when they described the work on the strip and a report of Huettenhain, dated November 1941, says that Pers Z was unable to read messages on a US diplomatic system despite working with a  large staff for two years.

It is not clear if the Italians were also able to read the strips in 1941. Fenner, head of the OKW/Chi cryptanalysis department said in DF-187G that he received strip information from them but no mention of this affair is available in other TICOM reports.

In 1942 things must have changed regarding the 0-1 set (used till August ’42) and the Germans were also able to solve some of the ‘special’ sets. We definitely know that they solved the Berne strips but we don’t have details on much else. It seems that there was an agreement between the German agencies whereby OKW/Chi would attack the ‘special’ strips and Pers Z the ‘circular’ set.

In the same year the Germans gave the 0-1 set plus the ‘special’ strips for Riga and Helsinki to the Finns. The Finnish codebreakers were very professional and they were able to break into several ‘special’ strips during the year (Helsinki, Moscow, Madrid, Berne, Ankara, Stockholm, Beirut, Casablanca and probably others). Although in 1944 the Finns claimed that they hardly cooperated with the Germans that was definitely not true regarding the strips. They obviously exchanged results and in 1943-44 there were visits of Finnish personnel to Berlin and of German codebreakers to Finland to clarify the solution of the strips.

In 1943 the Finns revealed their success to the Japanese and gave them several ‘special’ sets. This became known to the Anglo-Americans through signals intelligence. The decoded Japanese messages betrayed the Finnish success.

The American reaction was first to claim that the cryptosystem was not the strip cipher and later to attribute enemy success to physical compromise.

It is not clear what security measures were implemented to secure the strip system other than changing the compromised strips with a new set. It seems the Americans did not want to believe that the Axis countries could read their high level system…

The Germans were able to solve the Berne ‘special’ strip in 1943 and thus read not only diplomatic messages but also communications of the OSS. In late 1943 (or early according to another report) Pers Z solved the 0-2 ‘circular’ set used from August ’42 to March ’43. All messages were decoded with the help of a decoding machine called the ‘Automaton’.

Other ‘special’ and ‘circular’ sets must have been solved during the year but again we don’t have those details. The codebreakers of OKW/Chi built a special cryptanalytic device called the ‘Tower clock’ (called statistical depth increaser in a US report) for work against the strip.

During 1943 there seems to have been a change in policy by OKW/Chi regarding the assignment of work on the strips. It seems that the previous arrangement whereby the ‘circular’ (0 sets) strips would be worked on by Pers Z was cancelled. Instead OKW/Chi worked with the Finns, giving them strips 0-2, 0-3, 0-4 and 0-5. The Hungarians were also involved in strip work and communicated with the Germans and the Finns but again details are lacking.

In 1944 it seems that the US authorities finally took measures that made the work of the Germans harder. Their efforts were hindered but not defeated. According to Huettenhain ‘about 1.5 years ago (Jan 1944) the strip system was made more difficult so that only certain lines could be read, for instance Berne to London, near the end nothing could be read’. In an unpublished manuscript written in 1970 he said: ‘In this way, were read by 1942 to September 1944, a total of 22 different links and all cq (call to quarters) messages’. It seems to me that the US measures had something to do with the special strips, either changing them more often (each month?) or giving them a different ‘key’ for each day (the standard system had only 40 different arrangements for the strips).

In the summer of ’44 the Germans started to give the Japanese representatives some of the strips they had solved. It seems this was a decision taken at the top and was probably connected with the worsening situation at the front.

 

How successful was the German effort? Unfortunately the statements made by Huettenhain and Wolfgang Franz (OKW/Chi cryptanalyst) are full of generalities. It is obvious that they were withholding the full extent of their success. Huettenhain said to the Anglo-Americans that ‘we can no longer state how many different sets of strips were reconstructed; probably 10 to 20’ but postwar he wrote that 22 ‘specials’ and all ‘circulars’ were read. Franz who was in charge of the strip solution said in DF-176 that his agency intercepted 70 different ‘traffics’ and solved 28 ‘circuits’.

The Finns also warned the Americans about the strip cipher in September ’44. US official Randolph Higgs after meeting colonel Hallamaa (head of Finnish sigint) wrote:

‘They [the Finns] had been greatly aided in their work on breaking our strips by carelessness on our part in the preparation of messages; (for example) we were constantly putting information in ciphers they had already broken regarding messages in new ciphers, after which they could 'crack' the new ones.

His general confidence in their ability to decode any of our messages anytime they wanted to, suggests very strongly that they do just that.’

From these statements it is obvious that the Axis codebreakers were much more successful with the strip system than has been acknowledged so far. More research is needed to reveal the full extent of their success. In the case of Italy and Hungary we know practically nothing regarding their work on the strip.

State Department cipher policy

The second problem in researching the strip cipher is the lack of a US report detailing the way the strip was used at different time periods. We know that each embassy had 50 ‘special’ alphabet strips and 50 ‘circulars’. Out of these 30 were chosen each day.

How long was the period of validity and how were the daily keys selected? From German reports and Japanese messages it seems that after mid ’42 the ‘circulars’ were valid for roughly 6 months while the ‘specials’ were used without a specific system. Some were valid for 2 months others for 10.  Important embassies probably changed the ‘specials’ after 2-4 months.

The question of ‘keys’ is interesting. The State Department did not have a different arrangement of the 30 strips for each day. Instead there were only 40 ‘keys’ used during the period of validity. Perhaps this changed in 1944 but it’s not clear.

Another interesting fact is that the embassies were not given different keys but had all the same system. According to David Kahn in 'Finland's Codebreaking in World War II':

‘Each post had its own set of strips; the key changed daily but was the same for all posts. This cryptographic weakness was probably permitted for logistical reasons.’

This was a serious mistake. It was not the only one made by the State's cipher department. They also reused some of the ‘special’ strip sets.

 

These and other mistakes facilitated the Axis solution of the strips. At this time there are many unanswered questions but I’m optimistic about the future. With a bit of luck more details will come out and I will be able to write a detailed account of the strip case.

Sources: various TICOM reports, ‘The Codebreakers’, ‘In the Name of Intelligence: Essays in Honor of Walter Pforzheimer’, ‘History of Venona’, ‘Japanese Intelligence in World War II’, British archives HW 40/132, Cryptologia article: 'Report on the decipherment of the American strip cipher 0-2 by the German Foreign Office', SRH-366 ‘History of Army strip cipher devices’, ‘Swedish signals intelligence’, ‘From Information to Intrigue’

Acknowledgements: I have to thank Frode Weierud, Michael van der Muelen and Ralph Erskine for sharing some of the information presented in this essay.

NSA cryptanalytic breakthrough

The Guardian has published more information on the NSA global surveillance program. This time it’s about NSA’s ability to intercept and decode a big part of global internet traffic.

According to these documents ‘US and British intelligence agencies have successfully cracked much of the online encryption relied upon by hundreds of millions of people to protect the privacy of their personal data, online transactions and emails, according to top-secret documents revealed by former contractor Edward Snowden.’

The Guardian also reveals that the NSA has made some kind of cryptanalytic breakthrough

‘Strict guidelines were laid down at the GCHQ complex in Cheltenham, Gloucestershire, on how to discuss projects relating to decryption. Analysts were instructed: "Do not ask about or speculate on sources or methods underpinning Bullrun." This information was so closely guarded, according to one document, that even those with access to aspects of the program were warned: "There will be no 'need to know'.

Very interesting stuff!

Unanswered questions of WWII cryptology

So far i’ve covered many interesting cases of WWII signals intelligence and codebreaking but that doesn’t mean that there aren’t some mysteries that require more research.

What are they?

1). US State Department strip cipher

In the late 1930’s the US State Department adopted the M-138-A strip cipher as its high level crypto system. In 1937 the Japanese were able to copy the strip set 0-1 and they passed these to the Germans in 1941, who in turn shared them with the Finns in 1942. In the same period it seems that the Italians also got hold of some strips. How bad was the compromise of the State Department’s high level system?

That question is hard to answer because there is limited information available and it doesn’t seem like the Americans were really interested in learning the full extent of the compromise. Some documents that would shed more light on this affair are proving very hard to find…

The M-138 strip system was difficult to solve provided it was used properly. The State Department did not use it properly with the result that in 1943-44 most of the strip traffic could be read by the Germans and the Finns. I’ll write more about this in the future.

2). NKVD 5^th Department codebreakers

In 1941 the NKVD’s codebreaking department was redesignated as the 5th Department under the efficient administrator Major Ivan Grigoryevich Shevelev. According to Matt Aid ‘By the end of World War II, the 5th Directorate controlled the single largest concentration of mathematicians and linguists in the Soviet Union.

What did these people do during the war? They couldn’t have spent all their time solving German low level hand ciphers. How many Axis and other foreign cryptosystems did they attack? How many could they solve? Did the ‘break’ foreign cipher machines like the Enigma or the Hagelin systems?

We simply don’t know. However it seems that a new book on WWII signals intelligence has been published in Russia recently. Unfortunately I don’t speak/read Russian…

3). Referat Vauck success

In 1942 the Germans organized a group tasked with solving enemy agents codes. This was department Vauck, named after its head dr Wilhelm Vauck. During the war they definitely solved enemy codes, usually those that had been physically compromised when the agent was arrested. However they also had some successes through cryptanalysis.

How successful were they during the war? Unfortunately we do not know. The relevant file in the British national archives HW 40/76 ‘Enemy exploitation of SIS and SOE codes and cyphers’ says that postwar files have been retained and my request for the release of the interrogations of dr Vauck has been rejected by the archives staff…

4). Forschungsamt information

I have already pointed out that the Anglo-Americans were able to capture many of the Forschungsamt higher-ups in 1945. Where is the information from their interrogations? Why wasn’t it released to TICOM authorities?

5). German Enigma investigations

The Germans constantly evaluated the security of their Enigma cipher machine. There were many studies on whether the daily key or parts of it could be retrieved through cryptanalysis. Those studies are the TICOM DF-190 to DF-190AN files.

I don’t have these but recently I was given a summary from Randy Rezabek of Ticom Archive. This file shows that the Germans had investigated several methods of attack on the Enigma and in many cases had calculated the time needed for a small team to carry them out. Many were within practical limits.

More research is needed to evaluate the German methods and the way they influenced their security measures.

6). Japanese Purple and Coral cipher machines

In the 1930’s the Japanese Foreign Ministry started using the PURPLE cipher machine as its high level system. PURPLE was solved by American and Soviet codebreakers. Did the Germans have any success with it? Until recently the answer was no.

However it seems there is more to this story.

The Coral machine was used by military attaches and the Anglo-Americans solved it in 1944. In the same year dr Steinberg of the German Army’s signal intelligence agency was transferred to OKW/Chi where he worked on a cipher machine used by the Japanese attaché. Did he manage to solve it?

TICOM report I-64 ‘Answers by Wm. Buggisch of OKH/Chi to Questions sent by TICOM’ says ‘B. thinks Steinberg (of 209 fame) solved some Jap machine traffic which was difficult but not so hard as Enigma. B. thinks it was traffic of the Jap Military Attache.’

7). Soviet diplomatic code

The Soviet Union used a code enciphered with one time pads as its main diplomatic system during WWII. This system if used correctly is unbreakable.

Were the Germans able to read parts of this traffic? There are some strange statements in Allied and German reports…

8). M-209 decoding device

I’m surprised that no one has figured out how this machine worked!

Update

Added a file from TICOM D-69 ‘Correspondence between OKW/CHI and intercept stations’ in US military attaché codes of WWII.

US intelligence budget

The Washington post has a nice article on the US secret intelligence budget. I found it strange that humint gets more money than sigint. What happened to you NSA? You used to get all the money…

Update

I added information from FMS P-038 ‘German Radio Intelligence’ and ADM 223/505 ‘Cypher security and W/T (Wireless Telegraphy) deception’ in The Slidex code.

Added a summary of ‘British intelligence’ vol4 in ‘Book review – British Intelligence in the Second World War’.

Rewrote parts of Dienstelle Klatt – A case of Soviet deception using information from ‘Foreign intelligence literary scene’ article: The legend of Agent Max’ and ‘The Crown Jewels: The British Secrets at the Heart of the KGB Archives’