Tag Archives: bombe machine

Puzzle History: Codebreaking and the NSA

Posted on by
6

ohr577n

[Image courtesy of NSA.gov.]

The National Security Agency has been in the news a lot over the last few years.

It arguably all started with Edward Snowden and the revelations about government surveillance, both domestic and foreign, that emerged in his wake. Between President Trump’s intimations of Obama-era wiretapping (which also supposedly involved England’s GCHQ) and recent news stories about NSA contractor Reality Winner leaking information, the NSA continues to draw mainstream attention in the 24-hour news cycle.

When you factor in the declassification of codebreaking intel during and after World War II, we know more about the NSA’s inner workings than ever before.

You might be asking what the NSA has to do with puzzles. Well, everything. Because the NSA was born as a codecracking organization.

enigma

The NSA was founded in November of 1952, but its formative stages began during World War II, as codebreakers were recruited in the U.S. starting in 1943. Not only were they tasked with tackling the German ENIGMA code, but their secondary mission was to solve “the Russian problem.” This group was known as Signals Intelligence, or SIGINT.

William Friedman, one of the early figures in American codebreaking, described cryptanalysis as “a unique profession, demanding a peculiar kind of puzzle-solving mentality combined with patience. So staffing this new organization was a curious endeavor.”

Those who were recruited came from all walks of life:

Career officers and new draftees, young women math majors just out of Smith or Vassar, partners of white-shoe New York law firms, electrical engineers from MIT, the entire ship’s band from the battleship California after it was torpedoed by the Japanese in the attack on Pearl Harbor, winners of puzzle competitions, radio hobbyists, farm boys from Wisconsin, world-traveling ex-missionaries, and one of the World’s foremost experts on the cuneiform tablets of ancient Assyria.

04/11/14 imitation game - bletchley park, milton keynes

A large campus was built that echoed the style and efforts of Britain’s Bletchley Park, including Alan Turing’s calculating machines, the bombes. Efforts on both sides of the Atlantic centered on cracking ENIGMA, the German codes used in all sorts of high-level communications. The teams worked alongside the bombes to try to determine which of the 456, 976 possible codes was being used in a given piece of communication.

It was a truly Herculean effort.

But while nearly half the staff focused on the Germans, others focused on cracking Russian codebooks, where words were translated into four-digit codes. Often, decrypting these codes involved “brute force” efforts, poring through numerous messages to pair up messages that used similar numerical groups, meaning they used the same cipher.

This would only work if the Soviets were lazy in their production of so-called “one-time pads,” encryption devices that had a particular code, which would be used once and then thrown away. Brute force codebreaking revealed that some of the one-time pads had been used more than once, a lapse in Soviet security that could work to the advantage of U.S. intelligence.

That deduction led to another stunning discovery: cracking the system used in encrypted messages to tell agents which encryption was used in a given missive. You see, each encoded message contained within it a code that dictated the cipher necessary to decrypt the message.

The Russians would later complicate this work by employing multiplexers: devices that would transmit numerous messages at once, making it harder to separate one message from another in the same dispatch.

hqdefault6

[Image courtesy of Virtantiq.com.]

The Germans would unwittingly aid the US in their Russian codebreaking efforts when a POW camp in Bad Aibling, Germany, was captured by the US army, and they uncovered a German device designed to “de-multiplex” Russian messages. The device was called the HMFS, because Hartmehrfachfernschreiber, while a great deal of fun to type, is hard to say quickly.

After World War II ended, U.S. intelligence consolidated their efforts on “the Russian problem,” continuing their work unraveling the Russian codebooks. Slowly, the codemasters began determining which organizations in the Soviet government used which codes. Even if the codes weren’t broken yet, it helped the intelligence community organize and prioritize their efforts.

The problem? They had a very tight timeframe to work in. Those duplicated codebooks were produced during a very small window of time in 1942, and only issued to Soviet agents in the three years that followed. By 1947, SIGINT analysts knew the Soviets would soon run out of the duplicated pads. Once they did, those recurring patterns of encrypted numbers would stop, and the best chance for cracking the Soviet codes would be lost.

Still, there was reason to be encouraged. Some important code words had been identified. TYRE was New York City, SIDON was London, and CARTHAGE was Washington; ENORMOZ appeared often enough that they determined it referred to atomic bomb research in Los Alamos.

It would also be revealed, through careful analysis of decrypted intel, that Soviet agents were embedded in both the U.S. Justice Department and in England’s Bletchley Park campus. The Justice Department agent was identified and tried, but released after the court found insufficient evidence to place her under surveillance in the first place.

This was one consequence of the secrecy surrounding codebreaking: an unwillingness to reveal their codebreaking success by turning over evidence of it. (As for the Bletchley Park spies, one was identified in 1951 and confessed in 1964. The other was never identified.)

By this time, the Russians had gone over ENIGMA machines captured during the German retreat, and had unraveled not only how the devices worked, but how to improve upon them. This would lead to the next-generation Russian Fialka machine.

fialka

With ever-increasing complexity when it came to encryption, thanks to increased automation, codebreaking evolved into not just intelligence work, but intelligence analysis. After all, if you don’t know something is important, you don’t necessarily give it the attention it deserves. As researcher Stephen Budiansky put it, “The top translators at Bletchley were intelligence officers first, who sifted myriad pieces to assemble an insightful whole.”

It also led to bigger, faster machines, like Goldberg and Demon, two computation machines designed to more efficiently pore over the vast amount of encrypted information being intercepted by the various U.S. intelligence services.

In 1948, though, the game changed. It changed so dramatically that November 1, 1948, is still remembered in NSA circles as Black Friday.

I hope you’re enjoying this look at the early days of America’s codebreaking efforts. Part 2 will continue next week, with a look at the rise of the NSA, Cold War cryptography, and more!

[Quotes and certain photos were sourced from Code Warriors: NSA’s Codebreakers and the Secret Intelligence War Against the Soviet Union by Stephen Budiansky.]

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!

The Story Behind Bletchley Park

Posted on by
6

It’s amazing when you consider the impact one of the most common forms of puzzle solving, cryptography, has had on world history. From the Revolutionary War to World War II, codebreaking was a battlefront as crucial and as exhausting as any contested piece of land on the map.

This was brought into stark clarity recently when I read The Secret Lives of Codebreakers by Sinclair McKay, which chronicles the work and lives of the team members at Bletchley Park who dedicated themselves to cracking the German Enigma code. It’s been said that Bletchley Park’s achievements shortened the war by two or three years. That’s no small feat.

(You know the decryption work done at Bletchley Park was good if it inspired parts of two of Ian Fleming’s James Bond novels. The film The Imitation Game barely scratched the surface.)

From The Secret Lives of Codebreakers:

Here, in these grounds fifty miles to the north of London, they would be introduced to the gravest secret of the war. Every intercepted enemy message — every signal from every captain, commander, military division, battleship, U-boat — all these encrypted communications, jumbled up into seemingly random letters in groups of four and five, and transmitted by radio, were gathered by the many listening posts around the British coastline.

…

From the Battle of Britain to the Blitz, from Cape Matapan to El-Alamein, from Kursk to the V-1 flying bombs, to D-Day and Japan, the work of Bletchley Park was completely invisible, yet right at the heart of the conflict. It was a key player whose presence, at all times, had to be kept utterly hidden from the enemy.

Imagine working these incredibly intensive sessions, obsessively looking over bits of code for hours at a time, knowing that the fate of the world rested on your shoulders, and a simple transcription mistake could cost lives. It’s a mind-boggling concept.

And the book shed a great deal of light on the decryption work itself. For instance, I had no idea how many variations of the Enigma code they were expected to crack.

The German rule was that no message should be more than 250 letters in length; if it was necessary to send a longer message, it should be split into multiple parts. This was designed to make life more difficult for codebreakers: the longer the message, the easier it might be for such a person to see patterns of letters forming among the apparent chaos.

Thanks to preambles in each Enigma message, codebreakers could at least be organized under different color keys: yellow, green, red, and blue. When they ran out of colors, they named keys after marine life, then birds, then elephants, then insects.

This was a war, and organization was key, no pun intended.

The first big break is known as the Herivel Tip, when one of the Bletchley Park crew deduced that some Enigma machine operators might choose the new day’s settings based on the letters used the previous day. This became a valuable jumping-off point for daily decryption attempts with their own coding device, the bombe machine.

In 1941, the first major Enigma-style code, the Abwehr code, was broken by Bletchley Park. In June of 1941, they broke the Vulture key, which revealed German activity on the Eastern front.

But the Germans were constantly adapting and refining their codes. German paranoia led to submarines using a different code than surface naval vessels. So “Dolphin,” the surface naval code, was not the only code troubling Bletchley Park. “Shark,” the submarine key, was a new concern. Thankfully, when U-559 sank in 1942 and its crew abandoned ship, its Enigma machine and a book of current “Shark” keys was salvaged by Allied forces without German knowledge.

In 1943, they broke “Porcupine,” giving them access to all German air force messages for a few weeks.

Around this time, the German High Command began using another method to transmit encrypted messages, and this became yet another focus of Bletchley Park. The “Fish” or “Tunny” communications were between generals and the Fuhrer himself.

This led to the development of the big brother of the bombe machine, Colossus, which combined the logic engines of a Turing machine with electronic valves that allowed it to read 5000 characters a second, five times faster than the previous top machine.

(While this was going on, Turing himself was developing a new speech encipherment system, Delilah, so named because she was a deceiver of men.)

In January 1945, efforts were still going strong, as Bletchley Park not only cracked the “Puffin” and “Falcon” keys of the German army, but effectively countered attempts by the Luftwaffe to implement new encryptions. By this point, dozens of variations of Enigma had been unraveled by the team.

Of course, decryption wasn’t the only thing accomplished at Bletchley Park.

The grand deception that led to the Normandy invasion was also managed there. They gradually fed false data to German Intelligence about military groups like the First United States Army Group preparing to enter France via the Pas de Calais and the Twelfth British Army into Scandinavia and Turkey. This allowed for D-Day to proceed, as German attention was diverted.

But The Secret Lives of Codebreakers goes beyond their wartime victories. McKay takes us behind the scenes of Bletchley Park to share not only the hard-won achievements of the cryptographers there, but also what daily life was like on the isolated estate. From living conditions and several romances to rivalries and petty feuds caused by high tensions, the book catalogues the realities of such stressful work in richly detailed fashion.

I think my favorite discoveries were related to the off-time of the recruits:

Oliver and Sheila Lawn have especially fond memories of the way that Bletchley-ites contrived to use their leisure time: “There was music,” says Mr. Lawn, “Play readings. And play actings. Quite a bit of amateur dramatics. And concerts of all kinds.”

Highland dancing, madrigals, creating palindromes…they were offered all sorts of activities to help keep spirits up after grueling decryption sessions. Certain musical artists were even invited up to perform there!

When you consider how important their work was and how many years they were sworn to silence about Bletchley Park, these revelations become all the more stunning. This isn’t just a fascinating work of puzzle history, this is history itself.

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!