Tag Archives: NSA

Elizebeth Smith Friedman, Codebreaker and Scourge of Nazi Spymasters

Posted on by
11

[William and Elizebeth Smith Friedman, hard at work.
Image courtesy of National Geographic.]

Last year, I rather ambitiously attempted to summarize the early history of American codebreaking and the NSA in a series of blog posts spanning World War II through the modern day. One of the names I cited in that series, William Friedman, is synonymous with American cryptography, thanks to his contributions to the cracking of the German ENIGMA code and his efforts to establish the National Security Agency.

Unfortunately, there is a gaping hole in the narrative I constructed. Because none of my sources made any reference to another crucial Friedman: Elizebeth Smith Friedman, William’s wife and partner in code-cracking.

Yes, she was name-dropped in my post about the book Code Girls: The Untold Story of the American Women Code Breakers of World War II, but she had to share those pages with a host of underappreciated women who were codebreaking geniuses.

[Image courtesy of HarperCollins Publishers.]

As The Woman Who Smashed Codes explains, she wasn’t just a talented codebreaker. She literally wrote the book on it. Eight of them, in fact. The Riverbank Publications — although often credited to her husband — covered new codebreaking techniques in rich detail, and they are still referred to today as part of the foundation of modern cryptography.

She also started the first and only American codebreaking unit ever run by a woman, serving as Cryptanalyst-in-Charge while jointly working for both the Treasury and the Coast Guard during and after World War II.

A history of American codebreaking without Elizebeth Smith Friedman is woefully incomplete, and in today’s post, I hope to rectify that oversight.

[Image courtesy of the Marshall Foundation.]

Elizebeth’s work with codes started in a most peculiar way. While seeking a job as a librarian after college, she was recruited by eccentric millionaire George Fabyan to live and work at Riverbank, his palatial estate that doubled as a self-funded research center for all sorts of scientific endeavors.

Elizebeth’s deep knowledge of Shakespeare was put to work attempting to prove Fabyan’s theory that there were secret messages encoded in the writings of Shakespeare. Although her work failed to uncover any hidden pattern in Shakespeare’s words or font choices, it did lead to two unexpected developments: a career in codebreaking and a budding romance with fellow Riverbank recruit William Friedman, whose own interest in codebreaking was sparked by the works of Edgar Allan Poe.

Thanks to the proliferation of radio, there was a seismic shift in how information was being passed between military units, governments, and other organizations, so the ability to listen in on one’s enemies (and allies) was not only a new strategic opportunity, but it was a relatively new science.

In short, America needed codebreakers who could crack the secret messages being transmitted (and intercepted). The military didn’t have them. The government didn’t have them.

But Riverbank did. And for the first eight months of World War I, the small group of William, Elizebeth, and those they trained handled ALL of the codebreaking for every part of the US government, from the State Department to the Army to the DOJ. William and Elizebeth began running a codebreaking school out of Riverbank, even embedding a secret message in a photo of the class taken on the last day of the course.

[Images courtesy of Elonka.com.]

In the aftermath of the First World War, codebreaking had become so important that countries were turning to machines to help develop uncrackable codes. And yet, at this point, American cryptography as a whole consisted of about 50 people. William went to work for the government, establishing the American version of Bletchley Park — Arlington Hall — and setting the stage for the creation of the NSA.

Elizebeth, on the other hand, cracked codes from home. And she did so for both the Treasury Department and the Coast Guard, who would send her sealed packages of intercepted encrypted intel and communications. In her first three months hunting down rum-runners during Prohibition, she solved two years’ worth of backlogged messages.

During World War II, Elizebeth’s Coast Guard Cryptography Unit turned their attention from smuggling (which waned during wartime) to cracking German codes. Under her tutelage, they would crack three different variations on the Enigma codes, each more complex than the last. (The British also cracked ENIGMA, independently of American efforts.)

Sadly, in the aftermath of the Pearl Harbor attack, the US military didn’t want civilians in charge of sensitive operations, so Elizebeth was demoted. Yes, she was no longer in charge of the group she started, trained, and cultivated, instead answering to a new boss of dubious cryptographic talents.

(Of course, the sexist dimwits making decisions like this had to grin and bear it when numerous other organizations and agencies continued to asked for Elizebeth’s assistance by name.)

And stealing Elizebeth’s credit was practically a cottage industry over at the FBI. We have them to thank for erasing Elizebeth’s role in particular — and the Coast Guard’s role in general — in hunting down, exposing, and compromising Nazi spy networks in South America, even though the FBI’s hamfisted blundering actually served to expose codebreaking operations in the past, forcing Elizebeth to crack new codes in order to regain the advantage the FBI had squandered.

Oh, yeah, did I mention that both during AND after World War II, Elizebeth continued to hound the Nazi forces in South America who sought to destabilize the region?

As one historian put it, referring to the thousands of pages of decryptions Elizebeth produced:

These pieces of paper saved lives. They almost certainly stopped coups. They put fascist spies in prison. They drove wedges between Germany and other nations that were trying to sustain and prolong Nazi terror. By any measure, Elizebeth was a great heroine of the Second World War.

The British knew it. The navy knew it. The FBI knew it. But the American public never did, because Elizebeth wasn’t allowed to speak.

[Image courtesy of Find a Grave.]

Even in their retirement, the Friedmans continued to contribute to the world of cryptography. They returned to the subject of Shakespeare with The Shakespeare Ciphers Examined, thoroughly debunking the whole idea of hidden codes in the Bard’s works.

When William died, Elizebeth even hid a secret message on his tombstone, for those who knew how to look. (It was Bacon’s cipher, something they both studied extensively during their time at Riverbank.) What a touching tribute to how she met her partner and husband.

And although the accolades and appreciation for Elizebeth’s incredible contributions have been slow in coming, they are trickling in. In the 1990s, the NSA renamed its auditorium from the William F. Friedman Memorial Auditorium to the William F. Friedman and Elizebeth S. Friedman Memorial Auditorium. A Justice Department building also has an auditorium bearing her name.

More information about the massive expansion of codebreaking worldwide is coming to light with every passing year. Hopefully that will mean greater attention for minds like Elizebeth, who used her puzzly mind to protect the world. That’s someone worth celebrating.

[Much of the information in this post comes from a wonderful book on Elizebeth, The Woman Who Smashed Codes by Jason Fagone, and it’s well-worth your time to check out her story in full.]

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!

The PN Blog 2017 Countdown!

Posted on by
Reply

It’s one of the final blog posts of the year, so what do you say we revisit all of 2017 with a countdown of my ten favorite blog posts from the past year!

#10 Farewell, David

I don’t mean to start off this countdown on a sad note by mentioning the loss of fellow puzzler and Penny Dell colleague David Lindsey. But it was an incredibly rewarding experience to talk to those who knew him better than I did, and put together a memorial piece in his honor. I learned so much, and it was a valuable part of the healing process for all of us. I had two different opportunities to get to know David, and that’s a rare gift.

#9 The Puzzle of the Bard

Puzzle history, codes, and wordplay are three common topics around here. So when I found a story that neatly covers all three, I simply couldn’t resist. Although this one is more conspiracy theory than verifiable puzzle history, it was great fun to do a deep dive into the ongoing debate surrounding Shakespeare’s identity and put a puzzly spin on the subject. The research alone made this one worthwhile.

#8 Holiday Puzzly Gift Guide

Every year, one of my favorite activities is putting together our Holiday Puzzly Gift Guide. I get to include the best products sent to me for review by top puzzle and game companies, mix in some of my own favorites, and draw attention to terrific constructors, game designers, and friends of the blog, all in the hopes of introducing solvers (and families of solvers) to quality puzzles and games.

#7 Cultural Sensitivity and Crosswords

Sadly, crosswords in general, and New York Times crosswords in particular, have a reputation for being stodgy, steeped in arcane vocabulary, obscure facts, and antiquated cultural references. As part of the ever-evolving narrative surrounding cultural sensitivity — not just ethnic, but in terms of gender and sexuality as well — it’s important to do more than acknowledge the debate. You have to participate in it.

#6 Puzzles in Unexpected Places

One was tucked away on a university website. Another sat in plain sight on a tombstone. The third came to light in a music fan’s collection. What did all three have in common? They represented a simple fact: puzzles are everywhere, a part of the cultural fabric in innumerable ways. I’m cheating a bit by mentioning three posts here, but they all fit the pattern. And it’s so much fun to discover puzzles in unexpected places.

#5 Puzzles for Pets

April Fools Day pranks are an Internet tradition at this point. Some websites go all out in celebrating the holiday. (Heck, ThinkGeek has started using the holiday to tease the public’s interest level in “fake” products, going on to actually release some of those April Fools pranks as real items later in the year!)

So when the idea was floated for PuzzleNation to get in on the pranking fun, I couldn’t resist. The result — Puzzles for Pets — was as layered as it was silly, complete with fake quotes, splash pages, and more. I even got my own dog, Bailey, in on the gag.

#4 Design Your Own Escape Room

Bringing a puzzle-solving mindset into other social activities has always been a passion of mine. I’ve written in previous blog posts about using my puzzly experiences in designing murder mystery dinners and other events. This year, I had the opportunity to try my hand at designing an escape room-style experience for a friend’s birthday, and sharing some of what I learned with you was a genuine treat.

#3 ACPT, New York Toy Fair, and more

There are few things better than spending time with fellow puzzlers and gamers, and we got to do a lot of that this year. Whether it was supporting new creators and exploring established companies at New York Toy Fair or cheering on my fellow puzzlers at the American Crossword Puzzle Tournament, getting out and talking shop with other creators is invigorating and encouraging. It really helps solidify the spirit of community that comes with being puzzly.

#2 Puzzle History

I mentioned puzzle history as a frequent blog post topic in #9, but recent revelations by government agencies in both the United States and Great Britain have allowed puzzlers greater access than ever before to the history of codebreaking over the last century or so.

In fact, so much information has come to light that I was able to do a three-part series on the history of the NSA and American codebreaking post-World War II. This was a labor of love that took weeks to put together, and I think it’s some of the best work I’ve ever done for the blog.

#1 Daily POP Crosswords

There’s nothing more exciting than getting to announce the launch of a product that has been months or years in the making, so picking #1 was a no-brainer for me. It had to be the announcement of Daily POP Crosswords.

But it’s not just the app, it’s everything behind the app. I’ve had the opportunity to introduce you to several of the terrific constructors we’ve recruited to make the puzzles as fresh and engaging as they can possibly be, and you’ll get to meet a few more in the weeks to come.

It may sound self-serving or schlocky to talk about our flagship products as #1 in the countdown, but it’s something that we’re all extremely proud of, something that we’re constantly working to improve, because we want to make our apps the absolute best they can be for the PuzzleNation audience. That’s what you deserve.

And it’s part of the evolution of PuzzleNation and PN Blog. Even as we work to ensure our current products are the best they can be, we’re always looking ahead to what’s next, what’s on the horizon, what’s to come.

Thanks for spending 2017 with us, through puzzle scandals and proposals, through forts and festivities, through doomsday prepping and daily delights. We’ll see you in 2018.

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!

Puzzles Endanger, Then Save, a Nation: The Spy Who Couldn’t Spell

Posted on by
1

While working on our multi-part series of posts about the history of codebreaking in America during the 20th century, I mentioned that some of the recent revelations about the National Security Agency were the result of Edward Snowden’s actions during his time as a government contractor.

What you might not know is that he has not been the only contractor to sneak information off of government computers in that fashion: a decade before Edward Snowden, there was Brian Patrick Regan.

Regan was a career soldier in the Air Force who eventually reached the rank of Master Sergeant and worked in signals intelligence.

Buried under hundreds of thousands of dollars in credit card debt, Regan decided his only way out of financial ruin was to try to sell US government secrets to a foreign government. He copied page after page of sensitive documents from national defense systems and snuck them out of his office, eventually amassing more than 15,000 pages, CD-ROMs, and other material in his home.

He would later bury bundles of these documents in various locations, including state parks, concealing the GPS coordinates of these valuables caches through a complicated series of encryptions where letters and numbers became three-digit sets.

You see, Regan had spent a fair amount of time studying cryptography, and fancied himself a top-shelf codemaster.

Regan used another set of encryptions of lesser complexity when he attempted to contact agents of the Libyan, Iraqi, and Chinese governments in order to sell off the treasure trove of secrets he’d amassed during his time at the National Reconnaissance Office.

One of these packets — a collection of three parcels intended for Libya — ended up in the hands of an FBI agent named Steven Carr.

From The Spy Who Couldn’t Spell by Yudhijit Bhattacharjee:

In the first envelope was a four-page letter with 149 lines of typed text consisting of alphabets and numbers. The second envelope included instructions on how to decode the letter. The third envelope included two sets of code sheets.

One set contained a list of ciphers. The other, running to six pages, listed dozens of words along with their encoded abbreviations: a system commonly known as brevity codes. Together, the two sets were meant to serve as the key for the decryption.

Some of the document had already been decrypted by FBI agents, and it revealed a member of the US intelligence community — claiming to be CIA, which was unverified, but definitely someone with top secret access — was trying to sell government secrets.

And this person had terrible spelling.

Brian Patrick Regan suffered from severe dyslexia. And, despite concerted efforts to perfect both his encryptions and his plan to net millions by selling government secrets, that dyslexia would be one of the clues that led Steven Carr to Regan’s doorstep.

It took Carr six months to connect Regan to the Libyan package, but once he did, surveillance on Regan began immediately.

When Regan attempted to board a plane to Zurich in 2001 — intending to meet with Iraqi and Libyan embassy officials — he was nabbed by the FBI and taken into custody.

Again, excerpted from The Spy Who Couldn’t Spell:

On searching Regan, officials found a piece of paper tucked between the inner and outer soles of his right shoe, on which were written addresses of Iraqi and Chinese embassies in Europe. The other materials they found on him and in his belongings were more mystifying. In a trouser pocket, Regan was carrying a spiral pad containing a page with 13 words that didn’t add up to anything: like tricycle, rocket and glove.

He had another 26 random words scribbled on an index card. Among the contents of Regan’s wallet was a piece of paper with a string of letters and numbers that read “5-6-N-V-O-A-I …” And in a folder he was carrying in his duffel bag were four sheets with handwritten lines of three-digit numbers.

FBI cryptanalyst Daniel Olson decoded some of the messages found on Regan when he was captured, but he had failed to unravel the multi-stage encryptions that concealed where Regan had buried his secret parcels. The government knew which state parks, but with acres and acres of possible hiding places, they needed more precise information.

And those parcels were the key, because they weren’t just packages to be sold to the highest bidder. No, those parcels doubled as a ransom in order to secure a better deal for himself with the US government. He wanted to blackmail the government for a reduced sentence.

They were his insurance plan.

As Thomas G. West said in Seeing What Others Cannot See, a book about visual thinking and dyslexia, “It’s not hard for a dyslexic to think ‘out of the box’ because they have never been in the box.”

Thankfully, Regan eventually realized that cooperation was in his best interest, and he revealed that each of the elaborate three-digit codes concealed a backdoor key built into the code itself.

Regan designed them this way so that, if he forgot the actual details of the encryption, all he would need is the starter word, a spark that would unlock the built-in key and help him decode the entire message.

This backdoor key system worked in a similar fashion to the Vigenere cipher, where a keyword or key phrase served as the entry point for a longer string of encrypted text. The trouble is… you need to know the cipher word or source in order to crack the code.

For example, during World War II, German agents in Europe used Daphne du Maurier’s Rebecca as the basis of a code for transmitting intelligence from Cairo to support a campaign by the Axis powers against the Allies in North Africa.

The discovery of the book among the possessions of two German radio operators who didn’t read English ultimately led to the breaking of the code, which in turn led to the capture of the German spies in Cairo.

Regan revealed the cipher words for the various hiding spots in state parks — which used cipher words from sources as peculiar as Regan’s own high school yearbook — and soon, the FBI recovered all but one of the buried parcels.

But Regan couldn’t remember the cipher word for the last one.

Daniel Olson would then step in, having learned some of Regan’s techniques as they uncovered the other parcels, and partially decrypting the remaining message enough to spark Regan’s memory. Regan finally came up with the last cipher key, and the final parcel was recovered.

Yes, once again, puzzly perseverance had saved the day!

Regan was found guilty on two counts of attempted espionage and one of gathering national defense information, and sentenced to life imprisonment with parole. Which, quite honestly, is getting off easy, considering that prosecutors were seeking the death penalty for his treasonous acts. (If prosecutors had gotten their way, he would’ve been the first person executed for espionage since the Rosenbergs in the ’50s.)

For the full story, including more in-depth explanations of Regan’s elaborate encryptions, check out The Spy Who Couldn’t Spell by Yudhijit Bhattacharjee.

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!

Puzzle History: Codebreaking and the NSA, part 3

Posted on by
Reply

View post on imgur.com

[Image courtesy of NSA’s official Twitter account.]

At the end of part 2 in our series, we left off during the early days of the NSA, as American cryptographers continued to labor under the shadow of the Black Friday change in Russian codes.

You may have noticed that part 2 got a little farther from puzzly topics than part 1, and there’s a reason for that. As the NSA evolved and grew, codebreaking was downplayed in favor of data acquisition. The reasons for this were twofold:

1. Context. You need to understand why given encrypted information is important in order to put it toward the best possible use. As Budiansky stated in part 1, “The top translators at Bletchley were intelligence officers first, who sifted myriad pieces to
assemble an insightful whole.”

2. Russian surveillance and bugging continued to grow more clever and sophisticated, pushing attention away from codebreaking. After all, what good is breaking codes or developing new ones if they can just steal unencrypted intel firsthand by monitoring
agents in the field?

Moving forward, the NSA would continue to pursue all manner of data mining, eventually leaving behind much of the codebreaking and analysis that originally formed the backbone of the organization. But that was in years to come. Cryptography was still a major player in NSA operations from the ’50s and onward.

View post on imgur.com

[The progression of “secret” and “top secret” code words.
Image courtesy of NSA’s official Twitter account.]

In May 1956, NSA cryptanalytic veterans pushed a proposal titled “Recommendations for a Full-Scale Attack on the Russian High-Level Systems,” believing that specially designed computers from IBM could provide the key for cracking the impenetrable Russian cryptography wall. Some cryptographers believed that ever-increasing processor speeds would eventually outpace even sophisticated codes.

By 1960, the NSA had spent $100 million on computers and analytical tools.

The problem? The NSA was collecting so much information that their increasingly small team of cryptoanalysts couldn’t dream of processing even a tiny portion of it.

But the quest for data access would only grow more ambitious.

In the wake of Sputnik’s launch in October of 1957, US signals intelligence would go where no man had gone before. The satellite GRAB, launched alongside Transit II-A in June of 1960, was supposedly meant to study cosmic radiation. (GRAB stood for Galactic Radiation and Background.)

View post on imgur.com

[Image courtesy of NSA’s official Twitter account.]

But it was actually intended to collect radar signals from two Soviet air-defense systems. This was the next step of ELINT, electronic intelligence work. (The younger brother of SIGINT.)

The NSA would later find a huge supporter in President Lyndon Johnson, as the president was heavily invested in SIGINT, ELINT, and any other INTs he could access. This did little to quell the intelligence-gathering rivalry growing between the CIA and NSA.

Of course, that’s not to say that the NSA ceased to do any worthwhile work in codebreaking. Far from it, actually.

During the Vietnam War, NSA analysts pored over North Vietnamese signals, trying to uncover how enemy pilots managed to scramble and respond so quickly to many of the US’s airstrikes conducted during Operation Rolling Thunder.

Careful analysis revealed an aberrant character (in Morse code) in messages that appeared in North Vietnamese transmissions before 90 percent of the Rolling Thunder airstrikes. By identifying when the enemy used that aberrant character, the analysts
were able to warn US pilots whether they were heading toward a prepared enemy or an unsuspecting one during a given sortie.

Other NSA teams worked to protect US communications by playing the role of an enemy analyst. They would try to break US message encryptions and see how much they could learn from intercepted US signals. Identifying flaws in their own procedures — as well as members of the military who were cutting corners when it came to secured communications — helped to make US communications more secure.

View post on imgur.com

[Image courtesy of NSA.gov.]

In 1979, Jack Gurin, the NSA’s Chief of Language Research, wrote an article in the NSA’s in-house publication Cryptolog, entitled “Let’s Not Forget Our Cryptologic Mission.” He believed much of the work done at the agency, and many of the people
hired, had strayed from the organization’s core mission.

The continued push for data acquisition over codebreaking analysis in the NSA led to other organizations picking up the slack. The FBI used (and continues to use) codebreakers and forensic accountants when dealing with encrypted logs from criminal organizations covering up money laundering, embezzlement, and other illegal activities.

And groups outside the government also made impressive gains in the field of encryption, among them IBM’s Thomas J. Watson Research Center, the Center for International Security and Arms Control, and even graduate student programs at universities like MIT and Stanford.

For instance, cryptographer Whitfield Diffie developed the concept of the asymmetric cipher. Joichi Ito explains it well in Whiplash:

Unlike any previously known code, asymmetric ciphers do not require the sender and receiver to have the same key. Instead, the sender (Alice) gives her public key to Bob, and Bob uses it to encrypt a message to Alice. She decrypts it using her private key. It no longer matters if Eve (who’s eavesdropping on their conversation) also has Alice’s public key, because the only thing she’ll be able to do with it is encrypt a message that only Alice can read.

This would lead to a team at MIT developing RSA, a technique that implemented Diffie’s asymmetric cipher concept. (It’s worth noting that RSA encryption is still used to this day.)

View post on imgur.com

[Image courtesy of Campus Safety Magazine.com.]

The last big sea change in encryption came when the government and military realized they no longer had a monopoly on codebreaking technology. Increased reliance and awareness of the importance of computer programming, greater access to computers with impressive processing power, and a groundswell of support for privacy from prying government eyes, led to dual arms races: encryption and acquisition.

And this brings us to the modern day. The revelations wrought by Edward Snowden’s leak of NSA information revealed the incredible depth of government data mining and acquistion, leading some pundits to claim that the NSA is “the only part of government that actually listens.”

Whatever your feelings on Snowden’s actions or government surveillance, there is no doubt that the National Security Agency has grown and changed a great deal since the days of cracking the ENIGMA code or working with the crew at Bletchley Park.

Where will American codebreaking go next? Who knows? Perhaps quantum computing will bring codes so complicated they’ll be impenetrable.

All I know is… it’s part of puzzle history.

I hope you enjoyed this multi-part series on the history of 20th-century codebreaking in America. If you’d like to learn more, you can check out some of the valuable sources I consulted while working on these posts:

Code Warriors: NSA’s Codebreakers and the Secret Intelligence War Against the Soviet Union by Stephen Budiansky

Whiplash: How to Survive Our Faster Future by Joichi Ito

The Secret Lives of Codebreakers by Sinclair McKay

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!

Puzzle History: Codebreaking and the NSA, part 2

Posted on by
1

[Image courtesy of NSA’s official Twitter account.]

At the end of part 1 of our look at the history of the NSA and American codebreaking, we left off with the pivotal Black Friday event.

On November 1, 1948, all intel coming from monitored Soviet signals went quiet. All traffic on military, naval, and police radio links was replaced with dummy messages. It was such an unprecedented and alarming event that London and Washington briefly considered that it might’ve been the first indication of preparations for war.

According to Code Warriors author Stephen Budiansky:

The full extent of the disaster only became apparent the following spring when real traffic started reappearing on the radio nets, now employing greatly improved — and completely unbreakable — technical and security procedures. The keying errors or other mistakes that had allowed most of the Soviets’ machine-enciphered military traffic to be routinely read by US and British codebreakers for the last several years had been corrected, and the much more disciplined systems that now replaced them slammed the cryptanalytic door shut.

Even the one-time pads that had offered some hope to attentive American codebreakers were updated, eliminating the ability to sort messages by which organization they originated from.

Codemakers had suddenly outpaced codebreakers.

View post on imgur.com

[The Kryptos sculpture outside CIA Headquarters. The NSA cracked
several of its codes before the CIA did. Image courtesy of Slate.com.]

The Office of Naval Intelligence wanted to take over from Signals Intelligence (SIGINT), demanding to see “everything” so they could do the job. They claimed SIGINT should limit their work to message translation, leaving interpretation to “the real experts.” This sort of territorial gamesmanship would continue to hamper government organizations for decades to come.

And that demand to see everything? That probably sounds familiar, in light of the revelations about government data collection and the PRISM program that were revealed in Edward Snowden’s leaks.

Black Friday was the start of all that, a shift from codecracking to the massive data collection and sifting operation that characterized the NSA for decades to come.

More amazingly, there was SO MUCH information collected during World War II that SIGINT was still poring over it all in 1949, decrypting what they could to reveal Soviet agents in the U.S. and England.

The fact that a high-ranking member of British Intelligence at the time, Kim Philby, was actually a Soviet double agent complicated things. After a decade under suspicion, Philby would flee to the Soviet Union in 1963, stunning many friends and colleagues who had believed in his innocence.

View post on imgur.com

[The spy and defector, honored with a Soviet stamp.
Image courtesy of Britannica.com.]

Although the Russians had flummoxed SIGINT, other countries weren’t so lucky. The East German police continued to use ENIGMA codes as late as 1956. Many of the early successes in the Korean War were tied to important decryption and analysis work by SIGINT. Those successes slowed in July of 1951, when North Korea began mimicking Russia’s radio procedures, making it much harder to gain access to North Korean intel.

Finally, the chaotic scramble for control over signal-based data gathering and codebreaking between the government and the military resulted in the birth of the National Security Agency on November 4, 1952, by order of President Truman.

One of the first things the NSA did? Reclassify all sorts of material involving historical codebreaking, including books and papers dating back to the Civil War and even the American Revolution.

View post on imgur.com

[The actual report that recommended the creation of the NSA.
Image courtesy of NSA’s official Twitter account.]

The creation of the NSA had finally, for a time at least, settled the issue of who was running the codebreaking and signals intelligence operation for the United States. And they were doing fine work refining the art of encryption, thanks to the work of minds like mathematician and cryptographer Claude Shannon.

One of Shannon’s insights was the inherent redundancy that is built into written language. Think of the rules of spelling, of syntax, of logical sentence progression. Those rules define the ways that letters are combined to form words (and those words form sentences, and those sentences form paragraphs, and so on).

The result? Well, if you know the end goal of the encoded string of characters is a functioning sentence in a given language, that helps narrow down the amount of possible information contained in that string. For instance, a pair of characters can’t be ANYTHING, because letter combinations like TD, ED, LY, OU, and ING are common, while combos like XR, QA, and BG are rare or impossible.

By programming codecracking computers to recognize some of these rules, analysts were developing the next generation of codebreakers.

Unfortunately, the Russian line was holding. The NSA’s failure to read much, if any, Soviet encrypted traffic since Black Friday was obviously becoming more than just a temporary setback.

Something fundamental had changed in the nature of the Russian cryptographic systems, and in the eyes of some scientific experts called in to assess the situation, the NSA had failed to keep up with the times.

I hope you’re enjoying this look at the early days of America’s 20th-century codebreaking efforts. Part 3 will continue next week, with the sea change from active codebreaking to data mining, plus Vietnam, the space race, and more!

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!

Puzzle History: Codebreaking and the NSA

Posted on by
6

ohr577n

[Image courtesy of NSA.gov.]

The National Security Agency has been in the news a lot over the last few years.

It arguably all started with Edward Snowden and the revelations about government surveillance, both domestic and foreign, that emerged in his wake. Between President Trump’s intimations of Obama-era wiretapping (which also supposedly involved England’s GCHQ) and recent news stories about NSA contractor Reality Winner leaking information, the NSA continues to draw mainstream attention in the 24-hour news cycle.

When you factor in the declassification of codebreaking intel during and after World War II, we know more about the NSA’s inner workings than ever before.

You might be asking what the NSA has to do with puzzles. Well, everything. Because the NSA was born as a codecracking organization.

enigma

The NSA was founded in November of 1952, but its formative stages began during World War II, as codebreakers were recruited in the U.S. starting in 1943. Not only were they tasked with tackling the German ENIGMA code, but their secondary mission was to solve “the Russian problem.” This group was known as Signals Intelligence, or SIGINT.

William Friedman, one of the early figures in American codebreaking, described cryptanalysis as “a unique profession, demanding a peculiar kind of puzzle-solving mentality combined with patience. So staffing this new organization was a curious endeavor.”

Those who were recruited came from all walks of life:

Career officers and new draftees, young women math majors just out of Smith or Vassar, partners of white-shoe New York law firms, electrical engineers from MIT, the entire ship’s band from the battleship California after it was torpedoed by the Japanese in the attack on Pearl Harbor, winners of puzzle competitions, radio hobbyists, farm boys from Wisconsin, world-traveling ex-missionaries, and one of the World’s foremost experts on the cuneiform tablets of ancient Assyria.

04/11/14 imitation game - bletchley park, milton keynes

A large campus was built that echoed the style and efforts of Britain’s Bletchley Park, including Alan Turing’s calculating machines, the bombes. Efforts on both sides of the Atlantic centered on cracking ENIGMA, the German codes used in all sorts of high-level communications. The teams worked alongside the bombes to try to determine which of the 456, 976 possible codes was being used in a given piece of communication.

It was a truly Herculean effort.

But while nearly half the staff focused on the Germans, others focused on cracking Russian codebooks, where words were translated into four-digit codes. Often, decrypting these codes involved “brute force” efforts, poring through numerous messages to pair up messages that used similar numerical groups, meaning they used the same cipher.

This would only work if the Soviets were lazy in their production of so-called “one-time pads,” encryption devices that had a particular code, which would be used once and then thrown away. Brute force codebreaking revealed that some of the one-time pads had been used more than once, a lapse in Soviet security that could work to the advantage of U.S. intelligence.

That deduction led to another stunning discovery: cracking the system used in encrypted messages to tell agents which encryption was used in a given missive. You see, each encoded message contained within it a code that dictated the cipher necessary to decrypt the message.

The Russians would later complicate this work by employing multiplexers: devices that would transmit numerous messages at once, making it harder to separate one message from another in the same dispatch.

hqdefault6

[Image courtesy of Virtantiq.com.]

The Germans would unwittingly aid the US in their Russian codebreaking efforts when a POW camp in Bad Aibling, Germany, was captured by the US army, and they uncovered a German device designed to “de-multiplex” Russian messages. The device was called the HMFS, because Hartmehrfachfernschreiber, while a great deal of fun to type, is hard to say quickly.

After World War II ended, U.S. intelligence consolidated their efforts on “the Russian problem,” continuing their work unraveling the Russian codebooks. Slowly, the codemasters began determining which organizations in the Soviet government used which codes. Even if the codes weren’t broken yet, it helped the intelligence community organize and prioritize their efforts.

The problem? They had a very tight timeframe to work in. Those duplicated codebooks were produced during a very small window of time in 1942, and only issued to Soviet agents in the three years that followed. By 1947, SIGINT analysts knew the Soviets would soon run out of the duplicated pads. Once they did, those recurring patterns of encrypted numbers would stop, and the best chance for cracking the Soviet codes would be lost.

Still, there was reason to be encouraged. Some important code words had been identified. TYRE was New York City, SIDON was London, and CARTHAGE was Washington; ENORMOZ appeared often enough that they determined it referred to atomic bomb research in Los Alamos.

It would also be revealed, through careful analysis of decrypted intel, that Soviet agents were embedded in both the U.S. Justice Department and in England’s Bletchley Park campus. The Justice Department agent was identified and tried, but released after the court found insufficient evidence to place her under surveillance in the first place.

This was one consequence of the secrecy surrounding codebreaking: an unwillingness to reveal their codebreaking success by turning over evidence of it. (As for the Bletchley Park spies, one was identified in 1951 and confessed in 1964. The other was never identified.)

By this time, the Russians had gone over ENIGMA machines captured during the German retreat, and had unraveled not only how the devices worked, but how to improve upon them. This would lead to the next-generation Russian Fialka machine.

fialka

With ever-increasing complexity when it came to encryption, thanks to increased automation, codebreaking evolved into not just intelligence work, but intelligence analysis. After all, if you don’t know something is important, you don’t necessarily give it the attention it deserves. As researcher Stephen Budiansky put it, “The top translators at Bletchley were intelligence officers first, who sifted myriad pieces to assemble an insightful whole.”

It also led to bigger, faster machines, like Goldberg and Demon, two computation machines designed to more efficiently pore over the vast amount of encrypted information being intercepted by the various U.S. intelligence services.

In 1948, though, the game changed. It changed so dramatically that November 1, 1948, is still remembered in NSA circles as Black Friday.

I hope you’re enjoying this look at the early days of America’s codebreaking efforts. Part 2 will continue next week, with a look at the rise of the NSA, Cold War cryptography, and more!

[Quotes and certain photos were sourced from Code Warriors: NSA’s Codebreakers and the Secret Intelligence War Against the Soviet Union by Stephen Budiansky.]

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!