Behind the scenes at Sony as hacking cisis unfolded

Studio Chief Michael Lynton Talks About the Realization of Extent of Damage.
http://l2.yimg.com/bt/api/res/1.2/ZCbjtE75X9i.hGGK.Sxq1g--/YXBwaWQ9eW5ld3M7Zmk9ZmlsbDtoPTUyMztpbD1wbGFuZTtweW9mZj0wO3E9NzU7dz04MDA-/http://globalfinance.zenfs.com/images/US_AHTTP_REUTERS_OLUSTECH_WRAPPER_H_LIVE_NEW/2014-12-16T005513Z_1_LYNXMPEABF00S_RTROPTP_3_SONY-OUTLOOK_original.jpg

The day after Sony Pictures employees discovered that company email was unusable following a cyberattack, senior executives came up with an old-style communication network: a phone tree, in which updates on the hack were relayed from person to person.

With computers and landline phones down during Thanksgiving week, the Sony Corp. (SNE) studio’s 6,000 employees were forced to improvise, with cellphones, Gmail accounts and notepads. The payroll department dug up an old machine to cut paychecks manually. Before long, the studio unearthed a cache of BlackBerrys, which still worked because they send and receive email via their own servers.

Sony Entertainment Chief Executive Michael Lynton told a meeting of senior executives that hackers hadn’t simply stolen data. They had erased it, rendering the entire computer system unusable.

“It took me 24 or 36 hours to fully understand this was not something we were going to be able to recover from in the next week or two,” Mr. Lynton recalled in an interview.
The next several weeks would make clear that Sony’s film and television studio was the victim of one of the most malicious cyberattacks in history—one that would result in the leak of hundreds of thousands of sensitive documents and embarrassing emails, the worsening of tensions between the U.S. and North Korea, a flip-flop on the release of a politically sensitive movie and damage to the company’s relationships with stars and theater owners.

[More from The Wall Street Journal: Pressure Builds for Calls on Flights in the U.S.]

The cascade of crises threw a spotlight onto the analytical and reserved Mr. Lynton, who has long had a lower profile in Hollywood than his top deputy, motion picture chief Amy Pascal, who works out of a larger office suite than he. Mr. Lynton, a 54-year-old former publishing, movie and Internet executive, has run Sony Pictures since 2004 but was known as a largely hands-off manager until the past month.

While the studio has been closed for the holidays, Mr. Lynton has been personally pursuing deals for wider distribution of “The Interview” in theaters and online, negotiations that normally fall to executives several levels beneath him. “I have tried to make sure all the decision-making related to this incident comes back to me so that, as much as possible, the operating groups are not distracted from the normal business they have to do,” he said.

Over Thanksgiving weekend, the IT department scrambled to get basic systems like email back online. Agents from the Federal Bureau of Investigation worked nearby, as did investigators from FireEye Inc., a cybersecurity company that deploys Ghostbusters-like teams to companies that have been hacked.

Kevin Mandia, FireEye’s chief operating officer, called the confluence of stolen credentials, erased hard drives, and leaked documents at Sony unprecedented in the history of corporate cyberhacks.

Though the hackers ordered Sony in their initial message to “obey us,” they never identified themselves or issued specific demands. Instead, they created maximum chaos. The week after the hack, the perpetrators leaked five Sony movies onto the Internet, along with thousands of internal documents and the Social Security numbers and other personal information of more than 47,000 people, including current and former employees, freelancers and a handful of movie stars.

FireEye’s investigators searched for clues on who had broken into the systems and when. But so much data had been destroyed that they have had trouble retracing the hackers’ steps. They still can’t confirm that the hackers have been eradicated from Sony’s systems, two people familiar with the investigation said.

Within a week, investigators had begun to suspect that North Korea had a hand in the breach, based on some hints in the attack code. And at one point, the malware appeared to ping one of the few Internet addresses linked to North Korea, investigators said.

North Korea has denied responsibility for the attack, but it hasn’t minced words in its anger over a movie that Sony was producing: “The Interview,” a lewd comedy about an attempt to assassinate North Korean leader Kim Jong Un.

During the movie’s production, Sony executives had consulted with government officials and experts at think tanks to discuss the film’s possible political implications, and made tweaks such as using the Columbia Pictures label instead of the Sony name. But Sony executives said they never considered the potential for direct retribution.

People involved in the investigation say North Korea remains the leading suspect. While a theory has been floated attributing the hack to an unknown disgruntled former employee, the FBI on Tuesday said “there is no credible information to indicate that any other individual is responsible for this cyber incident.”

Sony Corp. has been hacked before, so questions have been raised about its defenses. In 2011, hackers stole personal-account data for more than 100 million users of Sony’s PlayStation game system, a public-relations disaster. In the following years, Sony Corp. increased staff at its 24-hour security operations center near Washington, D.C., that worked for all of its U.S. units, as well as directly at the studio, according to former employees. Sony Pictures employed 42 firewalls—specialized computers designed to keep out hackers.

In the fall of 2013, Sony switched monitoring of its cybersecurity equipment from an outside company to an in-house team, according to an audit report from September 2014 included in leaked emails. It appeared that monitoring of one firewall and 148 other pieces of computer gear was lost in the shuffle.

It isn’t known if any of those lapses played a role in the breach, but the audit report had said, “Security incidents impacting these network or infrastructure devices may not be detected or resolved [in a timely fashion].”

By Dec. 16, Sony’s isolation increased even further when, along with a trove of Mr. Lynton’s emails, the hackers posted a message threatening violence against any theaters that showed “The Interview,” scheduled to open on Dec. 25, warning readers to “Remember the 11th of September 2001.”

The Department of Homeland Security said there was “no credible evidence” of an active plot against movie theaters, but cinema operators and even executives at other studios with movies opening around Christmas began urging Sony to cancel the film’s release, fearing the public would stay away from multiplexes. Sony resisted, and told the theaters it was their call—a move that angered cinema chains.

On Wednesday, Dec. 17, movie chains including Regal Entertainment Group , AMC Entertainment Holdings Inc. and Cinemark Holdings Inc. joined an industry conference call and indicated they wouldn’t screen the picture.

[More from The Wall Street Journal: These Streakers Resolve to Run Every Day of the Year]

Hours later, Sony canceled the Dec. 25 release altogether, prompting criticism that it was giving in to terrorism—exactly the reaction it had been trying all along to avoid. By Dec. 19, President Obama joined the chorus, saying Sony had made a mistake.

Even as Mr. Lynton defended the cancellation on CNN, the company was hunting for cable or digital companies willing to release the film online. Most were reluctant, but by the next week Google Inc. and Microsoft Corp. were on board, along with more than 300 independent theaters—enough to put together a piecemeal release simultaneously on big and small screens.

With more than $18 million in digital and box-office revenue so far, “The Interview,” which had a $44 million budget, isn’t a total write-off for Sony and its release would appear to put to rest the biggest question since the hack started: Whether the film would be seen by the public in any form.

If the company’s systems stay secure, Sony Pictures’ network is expected to be fully operating again within eight weeks. Further disclosures of emails and confidential documents may just be part of life for the studio, however: The hackers so far have released only a minuscule fraction of the 100 terabytes of data they claim to have stolen.

Mr. Lynton must still massage relations with partners worried about security when doing business with Sony and creative types who now have more insight than the company might like into how senior executives talk in private about them and their work.

Relations with exhibitors remain particularly tense. Only one cinema CEO contacted Mr. Lynton directly to say he wouldn’t play “The Interview.” Mr. Lynton ended up calling the heads of major chains personally to try to smooth things over after saying on CNN that theaters were to blame for the movie not being released.

The discussions were cordial, according to a person with knowledge of the calls, but one exhibition executive said the industry still holds a grudge against Sony for its handling of the matter.

Mr. Lynton said that amid the chaos and conflicting demands of the past month, his top priority was to make important decisions quickly: “You can’t be caught in the headlights doing nothing.” LINK