Sunday, August 5, 2012

Enigma security measures

The plugboard Enigma was used by the German armed forces prior and during WWII. It was a modification of the commercial version. Its plugboard gave it a huge security boost compared to the standard version.






The Germans used the Enigma extensively and they were always worried about the security of their main cipher system. Their cryptologic security departments, scattered across their cryptologic agencies, researched ways to break the Enigma and based on their observations changes were made in operating procedures.
Here I will only look into the main security measures they took prior and during the war. From changes in keying and indicator procedures to the mechanical modifications.

Almost all the information comes from ‘The History of Hut 6 vol1’ which is a British postwar report declassified in 2006. Of course the best source would be a German report detailing their security measures and the reasoning behind them but I don’t know if such a document existed or if it survived the war.

The plugboard Enigma and the German armed forces
In 1934 the Army and Navy agreed to use the plugboard Enigma as their main cipher system. In 1935 the Airforce followed. At the start of WWII Germany was the only country in the world to use a cipher machine for all its mid and high level traffic.

During WWII tens of thousands of Enigma machines were used by the Germans. It seems that no one knows for sure how many Enigma machines were built. A good estimate seems to be that more than 40.000 were constructed.
Enigma rotors and key settings

Initially only three rotors were used (I,II,III).  In December 1938 rotors IV and V were introduced. These were used by all three services throughout the war. All the standard rotors had one notch.
The Navy introduced three more rotors for its own networks in the period 1938-39. These had two notches.

Up to the end of 1935 the key (wheel order and stecker) changed every three months. From January 1936 every month and from October 1936 every day.
Kenngruppen

In order for the receiving party to identify the specific key used it was necessary to send a five letter group called Buchstabenkenngruppe (letter identification group) together with the message. The letter identification group was composed of one of the available 3-letter kenngruppen plus two random letters in order to create a 5-letter group (Enigma traffic was sent in 5-letter groups).
The navy used a different system. The kenngruppen was taken from a book (Kenngruppenbuch) and then enciphered with a bigram substitution table, before being sent with the message.

Indicator procedures
The Enigma was used in the following way. First the machine was set up according to the daily key which specified the wheelorder, ring settings and plugboard settings. Then the cipher clerk chose a different starting position of the wheels for each message.

In order to communicate this position to the receiving party he had to first encipher it. The starting position was initially enciphered at a standard setting called Grundstellung. Up to 1940 the practice was to encipher the indicator twice (called ‘throw-on’ by BP).
For example if the Grund setting is AAK and we enciphered our message with the wheels at OFW then we encode OFW at position AAK and get VNZ, then press OFW one more time and get BLD . The indicator at the beginning of the message will be VNZ BLD. The receiving party will set their machine at AAK and enter VNZBLD which will give them OFWOFW. They will then set the rotors at OFW and proceed to decipher the message.

From September 1938 the Grund setting was dropped and the cipher clerk chose a random starting position for enciphering the message and enciphered it at another random position. In our example it would mean that instead of AAK the cipher clerk chooses three random letters say TGM and enciphers OFW at that setting thus giving XYU and again HLS. The indicator with the new system will be TGM XYU HLS.
In May 1940 the double encipherment of the indicator was dropped.

The Navy initially followed the Army-Airforce procedure of random indicator selection. However this changed in May 1937 when they started to use an indicator book (kennbuch) to select the indicator which was then enciphered on the Grundstellung and the output was used as the message key. This was then communicated to the other party after first enciphering it with a substitution table . Both Kennbuch and substitution tables were changed several times during the war.
This procedure avoided operator mistakes and non random indicators. However Naval keys continued to use a Grund setting throughout the war. For some reason some Naval keys continued to use repeating indicators after 1940.

In April 1945 the Navy introduced a change in their procedure by having 228 Grund settings valid for one month instead of a different one for each day.
Plugboard connections

Initially 6-8 stecker connections were used. From January 1940 10 were used and this remained standard procedure till the end of the war. A few keys, for whatever reason, continued to use fewer steckers.
General, Officers and Staff keys

Naval keys had three settings: General, Officers, Staff. The General key (wheelorder, ring settings, stecker) was not changed every day but was ‘paired’ for two consecutive days. That meant that the second day had the same wheelorder and ring settings but differed in the stecker.
The Officers messages were enciphered first on the Officers key that had identical settings with the General key but differed in the stecker. Then it was enciphered once more on the General. Officers settings were changed every 10 days. The Officers key was much harder to break than the General key.

Staff keys had their own settings (wheelorder, ring settings, stecker) and were enciphered once more on the General key. With the technology of that time they were more or less unbreakable.
Army keys changed every day. Certain types of Army keys (Armee, Heeres, Wehrmacht) had several versions. The General M/S (Maschinenschlüssel) key for Geheim (secret) traffic, the Stab M/S key for Geheimkommandosache (top secret) and OKH or Officers for the highest level messages.

Airforce keys also changed every day. There was a general administrative key ‘Red’ that was used by all units on all fronts, a high level key ‘Pink’ plus of course different keys for different geographic areas and different levels of command.

Overview of security measures

Proliferation of keys
In 1942 all three services introduced more keys to cover different geographic areas (Eastern front, Med, West etc) and operational commands (Fliegerkorps, Luftflotte, etc)

The army ‘broke’ its Eastern key into several. The Luftwaffe introduced separate keys for higher levels of command and the Navy introduced a separate key for the Atlantic U-boats.
From then on the number of keys continued to increase.

Effects on breaking:

More keys meant more work for BP or more realistically that many keys were not attacked at all. On the other hand traffic between different networks often led to reencodements as the same message passed through different keys. This allowed BP to use reencodements from easy keys (like ‘Red’) in order to break into harder ones.
Also it seems that the need to construct more keys led the German cipher officers in charge of constructing Enigma keys to take a shortcut…

Parts of older keys were reused in order to create new ones. For example the wheel order from 22 May 1941 would be coupled with the stecker settings for Ferbruary 2nd 1939 in order to produce a ‘new’ key.
This German ‘trick’ mostly occurred in 1942 and in Luftwaffe networks. The Brits called it Parkerismus after the person who had discovered it.


Enigma Uhr
In 1944 the Luftwaffe introduced, on some nets, the Uhr device which changed the stecker combinations automatically by turning a knob.



One major effect of the device was that it got rid of the reciprocal encipherment of the plugboard (although the reciprocity of the Enigma as a whole was not affected).
A description of the Uhr device is given in TICOM report I-20Interrogation of Sonderfuehrer Dr. Fricke of the Signal Intelligence agency of the Supreme command Armed, Forces (OKW/Chi)’, p3

(a) Stecker Uhr. This was a small device to change the plugging. It gave 40 variations. They knew that the strength of the machine lay in the stecker and therefore wished to divide the traffic load per stecker by 40. The machine was used only by the Luftwaffe, which had only 1.000 or so machines for higher echelons.
Operational use:

It was first used in July 1944. Originally it was used on two keys, later it was extended to cover 15 Luftwaffe keys.
Some keys used Uhr and UKW-D.

Effects on breaking:
The ‘History of Hut 6’ says ‘Uhr alone had on the whole little influence on our breaking of keys though it did mean that there was sometimes a lot of technical work to be done after the basic key was broken. Serious complications arose only when it was necessary to break a key on an Uhr message or when Uhr was combined with D’.

Umkehrwalze D

UKW-D was a new rewirable reflector used on the military Enigma. The people of Bletchley Park called it Uncle D.



A week before it was introduced BP intercepted messages mentioning it. As the official history puts it ‘the stoutest hearts might have quailed in those last days of 1943 had it been possible for us to realize the hydraheadead nature of our veiled antagonist’.

The reflector D was used principally by the Luftwaffe and in 2 or 3 army keys. A naval version was built but not used.
Operational use:

From January 1944 the ‘Red’ key started to use reflector D for some of the traffic. The wiring was changed every 10 days. From August ’44 UKW-D was greatly expanded and some keys used it exclusively.

Western Europe
August 1st 1944 marked the first use of reflector D on Luftwaffe keys used by units in Germany. According to the ‘History of Hut 6’ this ‘radically affected their exploitation’. During the second half of ’44 the German air keys went over to reflector D.

The German army’s Home Administration (Wehrkreis) network Greenshank used reflector D since 1943 (date of introduction not known but according to the BP history probably in January 1943). This key used a different UKW-D wiring each day. Greenshank proved to be BP’s nemesis as it resisted attack for years. The final score shows only 13 days read during the war. There were two more Wehrkreis keys identified by BP that used reflector D and thus proved secure.

Italian front
In November 1944 Puma (army-airforce liaison key Italy) changed over to UKW-D and ‘the period of regular sustained breaking was at an end’. Puma also used the Uhr device.

Leopard (Luftwaffe-Italy) used UKW-D in 1945.

Eastern front
From September 1944 reflector D was used in some Luftwaffe networks. By January 1945 UKW-D use was extensive.

Effects on breaking:

Several D keys were solved because the Luftwaffe often used reflectors B and D on the same key (called ‘nearly D’ by BP).  New cryptanalytic equipment had to be developed and produced. These were the American Duenna and Autoscritcher and the British Giant. These devices were introduced in late 1944.
Even this effort did not prevent a drop in the success rate. For example the Puma key went from 100% to 35% read.

Uncle D proved to be Bletchley Park’s toughest opponent.

Lückenfüllerwalze
The Germans understood that one of the greatest weaknesses of the Enigma was the uniform movement of the rotors. In order to counter this they built a new rotor that had 26 notches that could be set in active or inactive position.




Details about the Lückenfüllerwalze are given in TICOM I-20 ‘Interrogation of Sonderfuehrer Dr. Fricke of the Signal Intelligence agency of the Supreme command Armed, Forces (OKW/Chi)’, p4
(b) Because of the uniform motion of the enigma, they considered that if messages of 600 or 700 letters were sent, they could be broken. If the instructions on maximum message length were followed, they knew everything would be all right, but they felt sure that their instructions were not followed. So they developed a new wheel with 26 notches which could be filled in as desired. These were called Lückenfüllerwalze. They wished to avoid certain numbers of notches per wheel, and particularly consecutive notches, for with the latter it was difficult to predict the cycle except in special cases. Consequently they ordered that wheels should be used with one, five, seven, or nine notches only and never with consecutive notches. Some of those wheels were actually built in Berlin by HEIMSOETH & RINCKE, who built the enigma. They were to be produced in numbers by this firm and by Siemens Halske, and were expected to be ready on 1 May 1945. They were not ready, however.

Operational use:
It was not introduced during the war. It would have probably defeated BP as it attacked the greatest weakness of the Enigma, the uniform movement of the rotors.

Enigma M4

The 4-rotor M4 machine was known to have been distributed in the Home Waters area in 1941. The Atlantic U-boats got the M4 in February 1942. Other commands took longer to introduce it.


Effects on breaking:
The M4 was an order of magnitude more secure than the 3-rotor version.

In order to solve the M4 keys a new 4-rotor bombe was needed.
Due to constant delays in the production of a British 4-rotor bombe the Americans decided to build their own version and this was operational in September 1943. By late 1943 95 bombes were used and in 1944 160.

Thanks to the new bombe the U-boat key was ‘broken’ usually within 24-48 hours. However during the period February 1942 to September 1943 solving the U-boat key with existing methods was a huge problem.
Keys of naval commands that used the M4 with repeating indicators were ‘broken’ without the need for a 4-rotor bombe.

Important M4 keys like those used by surface ships (Aegir, Neptun) resisted BP attack.

Sonderschlüssel
In the summer of 1944 U-boats started carrying individual Enigma keys.

Operational use:
They were first used in November 1944 and by February 1945 they carried practically all the operational traffic of U-boat Command.

Effects on breaking:
They were practically unbreakable. As the ‘Cryptographic history of work on the German Naval Enigma’ puts it: ‘The "Sonders" are about the only type of key which make one feel the need for a statistical attack on Enigma instead of the normal method of cribbing.

Notschluessel
Notschluessel were emergency keys used when there was no way to issue new keys to a specific unit or when compromise was suspected. A new Enigma key was generated from a keyword (schlüsselwort) and a discriminant (kenngruppe) from another word (kennwort).

Operational use:
The Luftwaffe first used NOT keys in August 1944.

Effects on breaking:
NOT’s were broken through knowledge of the cipher instructions, through reencodements and cryptanalytically.

The ‘History of Hut 6’ says ‘Those systems were only intended by the Germans for use in emergency. However, NOT-keys have actually certain advantages over keys made up in the normal way owing to their freedom from rules of keys which may help the enemy cryptographer. They would have however, the fatal objection for regular use that if the key is generated from one word the number of possible keys is limited so drastically that some kind of key index becomes possible - for instance, on the second system the number of keys is determined by the number of German words at least 12 letters long - which must surely be much less than 100,000.

Random indicators
After abolishing the Grundstellung system the Army and Airforce allowed the Enigma operator to choose both a random message key and a random setting with which to encipher it.

However the keys chosen by operators were not always random. This effect (called cillies) was used by BP on its attack on the Enigma traffic.

The Germans introduced a new procedure to counter this. This procedure was as follows:

1). The cipher clerk had to choose a random text either from a book, a poem, a song etc
2). Set his Enigma at wheel order I, II, III, ring settings 01, 01, 01 and connect 10 steckers at random.

3). Choose a random Grund setting.
4). Enter the text in the Enigma and write down the output, then divide it into 6 letter groups

Each 6 letter group comprises a Grund setting and message key. The operator then used the new keys by setting the Enigma at the first three letters and enciphering (on the daily key) the last three letters. The output would be used as a message key.

Operational use:
Hard to establish by Bletchley Park as there was no external indication of this new procedure (apart from a fall in cillies). It may have been used extensively by Army and SS keys.

According to the report ‘Änderungen beim Schlüsseln mit Maschinenschlüssel’ the new indicator procedure was to become effective in August ’44.
 
 


 

Effects on breaking:

As the official history puts it: ‘It is impossible to interpret "Random Indicators" as anything but an anti-cilli device - a far more radical one than CY. It does indeed kill cillies and it is clear that the Germans had at last become conscious of this possible danger. The answer they now found to the danger of cillies was as effective as anything that could have been devised - short of a complete change of the indicating system - and it did lose us Orange. The only possible criticism we can make of the German action is that (as so often) it was too late: cillies were dying when they were killed. The history of Hut 6 would have been different had the Germans in the full flush of their 1940 triumphs been able to spare a thought for the suppression of cillies.’

Wahlworts (nonsense words)
Instead of stereotyped beginnings random words were inserted at the beginning and end of the message as an anti-crib device. Wahlworts were from 4-14 letters long.

Operational use:
This was ordered in the N.Africa army keys in December 1942. Nonsense words were used extensively by N.African, Balkan and Eastern keys but not to such a degree in the West.

Luftwaffe units inside Germany also used wahlworts.
Effects on breaking:

Cribing on addresses was no longer possible as the official history admits ‘breaking on straight addresses was now out of the question’. The general effect on N.African keys was limited since at that time the Brits relied on reencodements.
Against Luftwaffe keys the problem was more serious but again reencodements were used.

As the official history puts it ‘It cannot be denied that in wahlworts the Germans hit on a simple and effective method of making cribbing more difficult. It would have been still more effective but for the eternal German blunder of "too little and too late", introduced in 1940 on a wholesale scale, wahlworts might have knocked out the infant Crib Room before it had got properly on its feet: but in fact the Germans did not use the system at all till halfway through the war and not until the last few months used it on anything approaching a universal scale.

Mosse code

The Mosse code was a commercial code (5-letter) adopted with modifications by the Luftwaffe.
Operational use:

Used by the Luftwaffe since early 1944.

Effects on breaking:
It became a problem as an anti-crib device in 1945. When long addresses were changed into a 5-letter code this stopped cribbing.  However in cases were several regularly occurring phrases were replaced by a codeword this provided a useful crib.

Overall the Brits rated it as ‘a sound security measure’.

Double encoding
On the Raven and Gadfly keys some messages were enciphered twice. First with the standard procedure and then once again by using the settings of the Enigma at the end of the message without resetting the rotors.

Effects on breaking:
As the BP history says ‘Double Encoding was used on too small a scale to haw any effect worth mentioning on Hut 6 breaking. On the scale on which it was used it must have been no less a nuisance to the German cipher clerks than it was to Hut 6.


Burying

Stereotyped beginnings were a security risk. In order to counter this burying was used. Burying meant that the cipher clerk inserted the beginning and end of the message in the middle of the text before encoding it.
Operational use:

This precaution was introduced in the Army in December 1942.
Effects on breaking:

It was an effective anti-crib method.

CY procedure
The movement of the standard Enigma rotors was predictable due to their having only one notch. The fast rotor moved with every character, the middle rotor moved once every 26 key depressions and the slow rotor (the left one) moved only once every 676 key strokes (26x26). Since messages were limited to 250 characters this meant that the slow rotor was stationary during encodement. In order to counter this effect the Germans had the cipher clerk move the rotor by hand in the middle of the message. After 70 to 130 letters the clerk chose a random letter and moved the slow rotor by hand to that position (the new position must be at least 5 stages removed from the original one). Then he wrote in the message CY followed by the letter that represented the new slow rotor position and the letter following it in the alphabet. For example if he chose J then he would type CYJK and continue the rest of the message with the slow rotor in the new position J.

Operational use:
Introduced in September 1944 in some Luftwaffe keys. By October used on all Army keys and the SS ones.

The report ‘Änderungen beim Schlüsseln mit Maschinenschlüssel’ says that the new procedure for resetting wheels within messages would become effective on 15 September ’44.



Effects on breaking:
CY stopped cillying and affected cribs and reencodements. On the other hand it occasionally provided a shortcut to the ring setting.

It did not stop BP success but it was a serious effort to counter the predictability of the Enigma rotor movement.

Zusatz stecker
In May 1944 the Luftwaffe introduced a change in the plugboard connections every 8 hours. The Germans did not introduce three different stecker keys. Only a few of the connections were changed each time.

This procedure created many problems and it was discontinued in mid June.

Effects on breaking:
Practically none. The official history calls it ‘the most silly and trivial of their security devices


Wheel-order permutations
From October 1941 Luftwaffe keys reversed their wheel-order at 12 noon.

The Army used a similar method from July 1942. Every 8 hours the rotors were moved forward one step. For example if the daily key specified rotors I,IV and III in that order then after 8 hours it would be III,I,IV and finally IV,III,I.
From September 1942 the Luftwaffe adopted the three wheel-order system.

Some Army keys (like Greenshank) used 6 permutations.
Effects on breaking:

This was an effective anti-depth measure but the Brits did not use depths in any large degree so they were not affected on this front. However the unintended consequence was that Banburismus was no longer practical.

Conclusion

Despite all the talk about the Germans having complete faith in the security of the Enigma the reality is that they introduced many new procedures in order to improve its security.
New rotors were introduced prewar.

The indicator procedure was changed by dropping the Grundstellung and stopping the double encipherment of the message key.
Wheel permutations were used to counter ‘depths’.

Random indicators were used as an anti-cilli device.
Nonsense words and burying were used against cribbing.

The uniform movement of the wheels was countered with the CY procedure.
Traffic was split into many different keys.

Even mechanical modifications were employed to upgrade the Enigma like the Uhr device, the UKW-D reflector and the 4-rotor M4.
These measures show that the Germans understood the main problems of the Enigma. Had they been taken together and on all networks they could have defeated Bletchley Park.

There were two main reasons why the Germans did not defeat BP through their security measures.
The first one was that due to the huge number of Enigma machines in use it was not logistically possible to take simple measures such as rewire the rotors or introduce new ones.

The other problem was the fragmentation of their cryptologic security departments. The Germans had 6 main cryptologic agencies.
Out of these 4 were military (Army, Navy, Airforce and OKW/Chi). Each had a crypto security department. It was the job of that department to prepare codes and ciphers for the service and make sure that these were secure. There were good cryptanalysts in these departments and thanks to their efforts the security of the Enigma was upgraded during the war. However they did not have close contact with their counterparts in other agencies and so could not share on their research and findings.

This led to each service having a different standard of security. For example the Navy continued to use the Grund setting even after the Army and Airforce had abandoned it and the same thing happened with repeating indicators. The Uhr device was used only by the Luftwaffe, the UKW-D on the same keys as the reflector B and so on and so forth…
It should also be mentioned that the security departments were hampered by the resistance of the services to changes in operational procedures. Their reasoning was that the fighting troops should not be overloaded with new codes and procedures that were of questionable value.

A unified cryptologic security department would have ensured that all three services followed the same procedures and instituted security changes at the same time.
Still that does not necessarily mean that a centralized department would have defeated BP. After all BP’s centralization did not save British high level codes in the period 1939-43.

Timelines


Year

Month

Measure

1936

January

wheel order and stecker changed every month

October

wheel order and stecker changed every day

1937

May

Navy introduces Kennbuch-bigram system

1938

Naval rotors VI and VII introduced

September

New army -airforce indicator procedure

December

Rotors IV and V introduced for Army-Airforce

1939

Naval rotor VIII  introduced

1940

May

Double encipherment of the indicator stopped

1941

October

  wheel order reversed by midday on Luftwaffe keys

1942

New keys for all three services

February

Enigma M4 used by Atlantic U-boats

July

3 daily wheel order changes

December

nonsense words and burying introduced

1943

UKW-D on Greenshank

September

Army drops use of Kenngruppen

November

Luftwaffe drops use of Kenngruppen

1944

Random indicators used by army and SS

Mosse code used by the Luftwaffe

January

UKW-D introduced on RED key

May

3 daily stecker changes

July

Uhr device

August

Notschluessel

September

CY procedure

November

Sonderschlüssel used by U-boats

1945

April

new indicator procedure by Navy



Sources: HW 43/70 ‘The History of Hut 6, Volume I’, ‘British intelligence in the Second World War’ vol2 and vol3 part2, ‘Decrypted secrets: methods and maxims of cryptology’, HW 25/1 ‘Cryptographic history of work on German Naval ENIGMA by C H O'D Alexander’, HW 25/2 ‘The History of Hut 8 by A P Mahon’ Cryptologia article: ‘Enigma Message Procedures Used by the Heer, Luftwaffe and Kriegsmarine’, Cryptologia article: ‘ENIGMA-UHR’, cryptomuseum(for the pics), Wikipedia, ‘European Axis Signal Intelligence in World War II vol2’, TICOM I-20 ‘Interrogation of Sonderfuehrer Dr. Fricke of the Signal Intelligence agency of the Supreme command Armed, Forces (OKW/Chi)’

Acknowledgments: I have to thank Frode Weierud and Ralph Erskine for answering my questions on aspects of the Enigma history.

For those of you who want to actually use the Enigma check this simulator

1 comment: