One of the
main hand systems was the J-19 code, enciphered either with bigram substitution
tables or with transposition using a stencil.
Historical
overview
The fist Japanese
diplomatic system identified by US codebreakers was introduced during WWI and
it was a simple bigram code called ‘JA’. There were two code tables, one of vowel-consonant
combinations and the other of consonant vowel. Similar systems, some with
4-letter code tables were introduced in the 1920’s.
These unenciphered
codes were easy to solve simply by taking advantage of the repetitions of the codegroups
of the most commonly used words and phrases. US codebreakers solved these codes
and thus learned details of Japan’s foreign policy. During the Washington
Naval Conference the codebreakers of Herbert Yardley’s Black Chamber were able to solve the Japanese code and their
success allowed the US diplomats to pressure the Japanese representatives to
agree to a battleship ratio of 5-5-3 for USA-UK-Japan. However this
success became public knowledge when in 1931 Yardley published ‘The American Black Chamber’, a summary of the codebreaking
achievements of his group. The book became an international best seller and
especially in Japan it led to the introduction of new, more secure
cryptosystems.
In the 1930’s the Japanese Foreign Ministry upgraded the security of
its communications by introducing the RED and PURPLE cipher machines and by enciphering their codes mainly with
transposition systems.
J-19 FUJI code
The J-19 code (also listed as system JAE in
US reports) had bigram and 4- letter code tables similar to the ones used previously by the
Japanese Foreign Ministry. According to the NSA study ‘West Wind Clear: Cryptology
and the Winds Message Controversy A Documentary History’ it was used from 21 June 1941 till 15
August 1943.
In terms of security the J-19
FUJI and the similar codes J-16 MATSU to J-18 SAKURA, that preceded it in the
period 1940-41, were much more sophisticated than the older Japanese diplomatic
systems. They had roughly double the number of code groups at ~1.600, these
included 676 bigram entries and in addition there was a 4-letter table with 900
entries for ‘common foreign words, usually of a technical nature, proper
names, geographic locations, months of the year, etc’.
Once the messages were encoded using the J-19 code table then they had to be enciphered. There were two cipher procedures used with the J-19, substitution and transposition.
CIFOL VEVAZ substitution
J-19 messages
with the indicators CIFOL or VEVAZ were enciphered
using bigram substitution tables. A random letter sequence was coupled with the
coded text and each pair of letters was substituted using the substitution
table. According to the NSA study this cipher procedure was rarely used.
Columnar
transposition using a stencil
The basic
cipher system used with the J-19 code was columnar transposition based on a
numerical key, with a stencil being used for additional security. The presence
of ‘blank’ cages in the box created irregular lengths for each column of the
text. Three different stencils were used each month with each being valid for
10 days. The numerical key changed daily. There were four different settings
for the J-19 system: General, Europe, America and Asia.
Examples of stencils and transposition keys from
‘West Wind Clear’:
Importance
of the J-19 FUJI code
The J-19 code was important enough for both the Allies and the Axis to
devote significant resources in solving it, even going so far as to build
special cryptanalytic equipment. The reason they went through all this trouble is
that during the war only a small number of Japanese embassies had the PURPLE
cipher machine so the rest had to rely on hand systems and one of the main
diplomatic codes was the J-19 FUJI. Intercepted
diplomatic traffic from around the world on this system carried economic,
political, military and secret intelligence information.
A special
case was the Moscow embassy (moved to Kuibyshev
during the war) and their use of the J-19 for communications with Tokyo.
It seems that this embassy was either not given a PURPLE machine or perhaps they had to dismantle it in 1941, so in 1941-43 they
relied on the J-19 for their most important messages. During WWII Japan fought
on the side of the Axis but was careful to avoid a confrontation with the
Soviet Union. War between the SU and Japan finally broke out in August 1945 but
during the period 1941-45 Japanese diplomats were free to collect and transmit important
information from the SU on military and political developments as well as their
discussions and negotiations with Soviet officials. These messages were a prime
target for the Allied and German codebreakers.
Allied
exploitation of the improved J series codes
When the new
code J-16 MATSU was introduced in
August 1940 it proved much harder to solve that the previous Japanese systems. A
team of cryptanalysts of the Army’s Signal Intelligence Service, led by Frank Rowlett analyzed this traffic and came to the
conclusion that it was a bigram code enciphered with a transposition key. Rowlett
realized that this system was similar to the German ADFGVX cipher of
WWI that they had already researched extensively as part of their training
program. Their effort to solve this Japanese code had not progressed much when
they were given copies of the code, the stencils and some of the numerical
keys. These had been copied by US Naval Intelligence from a Japanese embassy or
consulate in the US. Thanks to these specimens (J-16 codebooks, ten-day forms and auxiliary
transposition keys) Rowlett’s work became much
easier and messages could be read. Changes in the code could be followed by
taking advantage of operator mistakes.
From the 1974 interview of Frank Rowlett, pages 25-28:
It is interesting to note that before they solved this code the head of
the codebreaking department, William
Friedman was very pessimistic about the prospects of success.
When the new J-19 system was introduced the US codebreakers were already
familiar with the basic characteristics of the cipher and Rowlett quickly made
important discoveries regarding the underlying code. However solution of the
daily key settings was a difficult problem, especially since more resources
were put into the solution of the traffic sent on the PURPLE cipher machine.
During the war traffic on this system increased significantly and the solution of the daily changing settings became a problem for the small group working on Japanese transposed codes, so there was an effort to automate the process. The device built was an attachment for standard IBM punch card equipment called the ‘Electromechanagrammer’ or ‘Gee-Whizzer’.
During the war traffic on this system increased significantly and the solution of the daily changing settings became a problem for the small group working on Japanese transposed codes, so there was an effort to automate the process. The device built was an attachment for standard IBM punch card equipment called the ‘Electromechanagrammer’ or ‘Gee-Whizzer’.
According to
the NSA study ‘It Wasn’t All Magic: The Early Struggle to Automate Cryptanalysis,
1930s – 1960s’, p50-51:
‘The Gee Whizzer had been the first to
arrive. In its initial version it did not look impressive; it was just a box
containing relays and telephone system type rotary switches. But when it was
wired to one of the tabulating machines, it caused amazement and pride.
Although primitive and ugly, it worked and saved hundreds of hours of dreadful
labor needed to penetrate an important diplomatic target. It proved so useful
that a series of larger and more sophisticated "Whizzers" was
constructed during the war……………….When the Japanese made one of their diplomatic
"transposition" systems much more difficult to solve through hand anagramming
(reshuffling columns of code until they made "sense"), the American
army did not have the manpower needed to apply the traditional hand tests.
Friedman's response was to try to find
a way to further automate what had become a standard approach to mechanically testing
for meaningful decipherments……………………………………..Rosen and the IBM consultants realized
that not much could be done about the cards; there was no other viable memory medium.
But it was thought that it might be possible to eliminate all but significant
results from being printed. Rosen and his men, with the permission and help of
IBM, turned the idea into the first and very simple Gee Whizzer. The Whizzer's two
six-point, twenty-five-position rotary switches signalled the tabulator when
the old log values that were not approaching a criterion value should be
dropped from its counters. Then they instructed the tabulator to start building
up a new plain-language indicator value.
Simple, inexpensive, and quickly
implemented, the Gee Whizzer reinforced the belief among the cryptoengineers in
Washington that practical and evolutionary changes were the ones that should be
given support.’
Australian
effort
The American
codebreakers were not the only ones who were regularly reading this system. Their
British allies were also exploiting the J-19 and in Australia a small group
called Diplomatic Special Section
solved several Japanese diplomatic systems. In Australia the Diplomatic Special Section (D Special Section) of the Australian Military Forces HQ in Melbourne decrypted Japanese diplomatic ciphers. This unit was headed in the period 1942-44 by A.D. Trendall, Professor of Greek at Sydney University. Despite the small size of the unit considerable success was achieved in the solution of Japanese communications.
According to the book ‘Breaking Japanese Diplomatic Codes David Sissons and D Special Section during the Second World War’, p62 the J-19 was the main Japanese diplomatic code used in the period 1941-43.
Progress
in 1941 was slow and up to February 1942 the only keys solved were those for
messages whose content was known (for example messages reporting the departure
of ships). However in 1942 things progressed rapidly.
In
March ‘42 a member of the British Foreign Office from
Singapore who possessed an excellent knowledge of Japanese joined the section. At
the same time personnel of the unit developed elaborate cryptanalytic methods for
recovering the daily settings and by May ‘42 the section was able to read
virtually all FUJI traffic and ‘all
bigrams, except those of very rare occurrence, and most tetragrams had been
recovered’.
The solution of the traffic on the Kuibyshev–Tokyo
link was one of their main commitments and in page 38 it says:
‘The report alludes, very briefly, to the high intelligence value of the intercepts of the telegrams exchanged between the Japanese Foreign Ministry and its Ambassador in Russia, Sato Naotake.
It seems that in its coverage of the Kuibyshev–Tokyo–Kuibyshev circuit,
the Section was able to provide strategic intelligence of value.’
German exploitation
of J-19 system
Foreign
diplomatic codes were worked on by three different German agencies, the German
High Command’s deciphering department – OKW/Chi, the Foreign
Ministry’s deciphering deparment Pers Z and the Air Ministry’s
Research Department - Reichsluftfahrtministerium Forschungsamt.
Forschungsamt effortAt the Air Ministry’s Research Department Japanese systems were worked on by Abteilung 7 (USA, UK, Ireland, South America, Spain, Portugal, Turkey, Egypt, Far East). The department had about 60-70 workers.
At this time there is very limited information available on the work of the Forschungsamt. EASI vol7, p82 says that they worked on ‘a transposition with nulls over two and four letter code’. This was clearly either the J-19 or one of the similar J series systems.
At the Foreign Ministry’s deciphering department Pers Z Japanese systems were worked on by a group headed by Senior Specialist dr Rudolf Schauffler. This section successfully solved the Japanese diplomatic transposed codes.
According to paragraph 176 of TICOM report I-22 ‘Interrogation of German Cryptographers of Pers Z S Department of the Auswaertiges Amt’, FUJI was designated as Japanese Code 57 and it was read for about two years.
More
information is available from the TICOM report DF-31B ‘How J.B. 57 Japanese
Letter System Was Solved’, written by the cryptanalysts Annalise Huenke and
Hans Rohrbach.
The
first break into system JB 57 came through two messages that had the same
indicator (meaning they used the same transposition key). Once these were
solved the system was identified as a transposed code, using a stencil.
The Germans were very interested in the messages from the embassy in the Soviet Union and according to David Kahn’s ‘The codebreakers’, p444:
‘the subsequent solutions provided the
Germans with information about Russian war production and army activities’
OKW/Chi effortAt the High Command’s deciphering department - OKW/Chi, Japanese diplomatic systems were worked on by a subsection of Referat 13, headed by 1st Lieutenant dr Adler. About 15 people were employed by the unit.
The OKW/Chi designation for FUJI was system J-13/J2B4BCüRuW (Japanese 2-letter and 4-letter code with stencil and transposition – Raster und Würfel). FUJI messages were first solved thanks to a repeat message sent from Paris to Tokyo. The first message and the repeat had the same plaintext (with small variations) and they had both been enciphered with the same key. This mistake facilitated their solution and the basic characteristics of the system were identified.
The solution
of the daily transposition settings and the different stencils was taken over
by personnel of the mathematical research department, specifically by the
mathematician dr Werner Weber.
According to
Part 3 of the report I-181 ‘Homework by Dr Werner Weber of OKW/Chi’, Weber started
working on Japanese diplomatic messages in July ’41 and he identified the
system as a transposed code. The underlying code for some of the messages was
the previously solved LA code, thus they could be read. The rest of the
messages had a new code.
Solution of the new system and recovery of the code proceeded slowly in 1941. In September ’41 Weber was allocated a small staff to help him with the Japanese traffic and by February ’42 some material could be read. During the year the new system was solved and most of the circular and European/Middle East traffic could be read. In the period summer ’42 to summer ’43 the previous year’s indicators were reused and the old transposition keys and stencils were either repeated or were modified in a predictable manner (with some exceptions).
At OKW/Chi they not only solved this code but also built a specialized cryptanalytic device called the ‘Bigram search device’ (bigramm suchgerät) for recovering the daily settings. EASI vol3, p65 says:
‘FUJI, a transposition by means of a
transposition square with nulls applied to a two and four letter code. This
system was read until it ended in August, 1943. It was broken in a very short
time by the use of special apparatus designed by the research section and
operated by Weber. New traffic could be read in less than two hours with the
aid of this machine.’
The ‘Bigram search
device’ is called ‘digraph weight recorder’ in the US report ‘European Axis
Signal Intelligence in World War II’ volume 2. In pages 51-53 details are given
on the operation of this device:
‘The digraph "weight" recorder
consisted of: two teleprinter tape reading heads, a relay-bank interpreter circuit,
a plugboard ‘’weight’’ assignor and a recording pen and drum.
Each head read its tape
photoelectrically, at a speed of 75 positions per second.’
The machine
could find a solution in less than two hours and did the work of 20 people,
thus saving manpower.
How did the
German machine compare with the American ‘Gee-Whizzer’? European Axis Signal Intelligence in World War
II’ volume 2 points out the differences in their operation and the pros and
cons of each device:
‘If the sections of text chosen from any
given message were well chosen from a cryptanalytic viewpoint, the American
machine proved much faster than the German machine, because the print of totals was easier for the
cryptanalyst to analyze than just the individual values listed by the German
machine; but if the sections of text were not well chosen and no true columns
were included in the choice, then the German machine ,had the advantage since
it recorded all possible juxtapositions quickly, and all true matches were
included in the data.’
Sources: ‘European
Axis Signal Intelligence in World War II’ volumes 2,3,6,7 , TICOM reports I-22, I-31, I-25, I-37, I-90, I-118, I-124, I-150, I-181, DF-31B, DF-187B
, ‘The Codebreakers’, ‘Breaking Japanese
Diplomatic Codes David Sissons and D Special Section during the Second World
War’,
United States Cryptologic History
Series IV: World War II Volume X: ‘West Wind Clear: Cryptology and the Winds Message Controversy A Documentary History’, United States Cryptologic History,
Special Series, Volume 6, ‘It Wasn’t All Magic: The Early Struggle to Automate Cryptanalysis, 1930s – 1960s’, NSA interviews of Frank Rowlett 1974
Additional information: In summer 1943 J-19 FUJI was replaced by three new systems. The transposed codes TOKI and GEAM and the enciphered code ‘Cypher Book No1’.
Information on the TOKI cipher is available from The Japanese TOKI diplomatic cipher 1943-45.
Additional information: In summer 1943 J-19 FUJI was replaced by three new systems. The transposed codes TOKI and GEAM and the enciphered code ‘Cypher Book No1’.
TOKI and GEAM were used in the period 1943-45 and they
were similar to J-19 in that they were codes transposed on a stencil.
Information on the TOKI cipher is available from The Japanese TOKI diplomatic cipher 1943-45.
Very interesting, thanks Christos.
ReplyDelete