Losing the Signal: The Spectacular Rise and Fall of BlackBerry
Read Losing the Signal: The Spectacular Rise and Fall of BlackBerry Online
Authors: Jacquie McNish,Sean Silcoff
THE SPECTACULAR
RISE AND FALL
OF BLACKBERRY
JACQUIE McNISH / / / SEAN SILCOFF
Jim Balsillie fidgeted with his phone as his Dassault Falcon jet touched down at Dubai’s international airport. It had been several hours since the co-chief executive and relentless global pitchman for Research In Motion Ltd. (RIM) powered off his BlackBerry when the plane left the company’s Canadian hometown in Waterloo, Ontario. The man who convinced the world it could not operate without BlackBerry phones was itching to read his e-mails.
A restless forty-nine-year-old who kept in shape biking more than one hundred miles a week, Balsillie faced an uphill cycle in Dubai. After peaking two years earlier in 2009 as the world’s biggest seller of smartphones and falling just shy of $20 billion in revenues, BlackBerry’s sales were tumbling; its stock price was down more than 50 percent. Rivals Apple and Samsung had moved into the lead with a new generation of smartphones that expanded demand for wireless devices beyond the professional classes to fun-seeking consumers. The once addictive lure of BlackBerry’s miniature keyboard and secure e-mail and message services was now being eclipsed by touch-screen iPhones and Androids that put Angry Birds games and YouTube videos into people’s palms.
Nearly 1 billion people—a third of the world’s population—owned smartphones, devices that hadn’t existed a decade earlier. Not since the advent of network television in the late 1940s had a new technology been embraced so quickly by consumers. The race for market dominance was rapid and brutish. A few industry sovereigns, Motorola and Nokia, had already toppled. Some figured RIM would be the next casualty. The latest sign was a botched
launch of a PlayBook tablet to compete with Apple’s spectacularly successful iPad. The PlayBook had a tiny fraction of the apps that made the iPad popular and, in its rush to market, RIM had introduced the tablet without its trademark e-mail service.
Balsillie was in Dubai to deliver the keynote speech at the Persian Gulf’s premier consumer electronics trade show. He knew he had to project confidence if he was going to reshape the industry narrative that RIM had lost its way. Offstage he faced an even bigger challenge. He had days of back-to-back meetings lined up with RIM’s top local customers: ultrawealthy Middle East clients and telecom executives obsessed with BlackBerry’s secure instant messaging service, BBM. He had to convince them that RIM’s planned launch of the second generation of BlackBerry smartphones, already delayed by several months, would be better than Steve Jobs’ iPhones.
As his jet taxied along the steaming tarmac, passing large patches of bleached sand and desert scrub, a bead of ticklish sweat traced down Balsillie’s temple. He stared with disbelief at the device that revolutionized communication. Seconds earlier, tiny white bars formed a digital staircase in the upper right-hand screen of his BlackBerry Bold, indicating the phone was receiving a strong signal from a local wireless carrier. Good, good. But where were the four white dots indicating a connection to the BlackBerry network? Or the red flashing light announcing the arrival of e-mails in his in-box? Balsillie clicked the envelope icon on his screen to see if his in-box was filling up.
Nothing.
No new e-mails? After so many hours? Impossible. He closed his eyes. Shortly before the most important sales pitch of the year, the turnaround moment the company needed to bounce back, the product he was championing wasn’t working.
Slough, a town twenty miles west of London, England, has been little more than a historical footnote since its founding. It first enjoyed celebrity with British aristocracy in the 1660s as a resting place on the road to the therapeutic waters in Bath. In the 1930s the landscape was so fouled by smokestack factories that poet John Betjeman called for bombs to blow the town to smithereens. The BBC satire
The Office
gave the town another image problem in the early 2000s, choosing Slough (rhymes with “now”) as the setting for a fictional series about the comically dysfunctional branch of a paper merchant.
On that October day in 2011, Slough would become the location of another corporate mishap. The problem began inside a nondescript gray office building. The squat facility was one of a handful of RIM’s network operating centers that directed e-mail and BBM traffic for 70 million BlackBerry users. The nerve center of the Slough operation was a windowless room that was kept refrigerator-cold. Inside this brightly lit room, rows of gray steel cabinets hummed, firing millions of electronic messages every minute for customers in Europe, the Middle East, and Africa. A sister center in Waterloo, Ontario, handled the Western Hemisphere. No other phone maker operated its own in-house network, a rapid, encrypted, and tightly sealed system ensuring reliable high-speed traffic. That’s why President Barack Obama refused a Secret Service missive to yield his BlackBerry after he was elected in 2008. Kings and queens, sheiks and business chieftains, were equally addicted to their “CrackBerrys.”
Hours before Balsillie landed, the world’s safest, most dependable wireless network responded to what should have been a minor problem. A server crashed. Normally, crashes aren’t a big issue. The machines, computerized systems that briefly warehouse large packets of data such as e-mails, are so crucial to BlackBerry’s network that reserve servers are continually powered up for emergencies. Known as hot backups, these stand-in machines are a primed, well-fed pony express, ready and waiting to spring forth.
RIM’s recovery protocol was going according to plan within seconds of the Slough crash. The disabled server was shut off and its stalled messages began switching over to the backup. Unfortunately one component failed to do its job. A single router refused to route. The size of a suitcase, routers are computerized Rolodexes that store a database of Internet Protocol addresses to identify where incoming e-mails should be sent. When a new server takes over, routers are
supposed
to send waiting e-mails. Instead of sending addresses to the new server, the router, infected by an undetected software bug, instructed the disabled computer to restart—again and again and again. Before RIM’s engineers knew what was happening, the two powerful machines were locked in a dangerous shoving match. The faulty router kept hammering the restart button on the disabled server, while the healthy machine fought back, attempting to regain control. When the slugfest was over minutes later, the servers had both been kayoed. The router was now lifeless: not only had it stopped working, but its memory was wiped clean.
It was two in the morning when Mike Lazaridis got the call. Research In Motion’s founder was a quick draw on his BlackBerry, even when startled awake. This call was from an engineer from the control room of the Waterloo Network Operating Centre, known internally as the NOC (sounds like “knock”). After a short conversation, Lazaridis pulled himself from bed, explaining to his half-awake wife, Ophelia: “The NOC is down.”
Lazaridis had been through enough outages to know the last place he needed to be was the NOC control room, located on the top floor of a low-rise RIM building. He knew from experience that stress levels rose the minute he stepped into the war room. The company’s founder was an intimidating presence, and attending engineers had enough to worry about trying to resuscitate the damaged network. In his office he called into “the dial-up,” an emergency conference call that customarily takes place during an outage. At the other end, Robin Bienfait, chief information officer, tried to explain what was happening. Something unusual, something bad, was going on in Slough. Nobody could explain why, but millions of e-mails had stopped moving through Slough’s servers. Its systems had collapsed. Standard responses were making things worse. Stalled e-mails from Europe, the Middle East, and parts of Asia were rerouted to North America, but the cascading traffic was more than Waterloo could manage. North America was likely going down. If the situation didn’t change, it looked like the entire global network would be knocked out.
Aware of the consequences, Lazaridis tapped out a two-word text message on his BlackBerry to Balsillie: “Call me.” An e-mail devotee, Lazaridis rarely bothered with text messages. But with the network outage, short text messages, relayed outside RIM’s network through local carriers’ wireless signals, were his only option. When his phone rang later, Lazaridis was prepared.
“Jim, everything is okay, but you should know there is a problem. Our network is down. Nothing is getting through. It is a complete outage.”
“Down? How is that possible?” Balsillie asked. He had stepped off the jet in Dubai and was still fiddling with his unresponsive BlackBerry.
“We don’t know why. Everyone is trying to fix it. We will fix it. We just don’t know when we are going to get it back up.”
As he hung up, Balsillie’s instinct was to jet back home to help manage the crisis. But fleeing Dubai would make the situation worse, he realized. BlackBerry
users everywhere were going through angry withdrawal. While Lazaridis’s team raced to revive the network, he had to reassure powerful corporate and government customers that a fix was on the way. He would have to tap what little reserves of energy he himself had after months of corporate defeats and this new, potentially ruinous network outage.
Between meetings he juggled calls from powerful wireless carriers and major RIM users, Fortune 500 companies, and powerful governments. Unless their service was back up soon, customers complained, their faith in the company would be irreparably damaged.
“Don’t worry,” Balsillie reassured everyone. “It’s almost fixed, another hour or two, tops, and we’ll be back up.”
Hours later, he spoke to a full house at the Gulf Information Technology Exhibition, but nobody listened to his pitch on the latest Bold phone features. Everyone wanted to know how much longer they would be without e-mail and BBM messages. His private meetings were equally grueling. The region’s powerful business leaders were desperate to get back on line. “When will my BlackBerry start working again?” they begged.
Balsillie told everyone the same thing:
Hang on, any minute now.
To further placate powerful, needy clients, he doled out special editions of the company’s Bold phones. To the most powerful sheiks, he presented custom-made gold or white phones. The recipients were delighted, especially when Balsillie assured them they were one of two honorary recipients of the exclusive prize. The other phone, he confided, belonged to Mohammed bin Rashid Al Maktoum, constitutional monarch of Dubai and prime minister of the United Arab Emirates. When Balsillie left Dubai on October 13 he had handed out dozens of the “exclusive” phones. By then, however, the flashy devices were little more than paperweights, a gaudy symbol of his company’s missteps in recent years. RIM’s global network was still knocked out.
Until that October day, BlackBerry was an improbable success story, the winner of one of the wildest, most disruptive technology races of the past century. A pint-sized company from a small city in Mennonite country, led by two men with little in common but their outsized ambition, had changed the way the world communicated. Just as desktop computers unseated mainframes by simplifying and accelerating once laborious tasks, BlackBerrys cut through computer wires and phone lines to free workers from desks. E-mails and documents could be fired off from cars or sent before the check arrived at restaurants—all without anyone knowing the sender was out of the office.
BlackBerry changed more than the workplace. We were liberated from offices and homes. Employers, clients, family and friends too, could reach us wherever wireless radio signals traveled. Work was no longer nine to five; it never ended. The same was true of socializing—we never had to be alone with our thoughts. BlackBerrys made us fast and efficient, but a little neurotic. The handsets transformed legions of users into addicts. For three days in October 2011, RIM customers were forced to go cold turkey. No BlackBerry. Where did everybody go? Life seemed impossible. When the outage ended, users were as committed as ever to mobile messaging. For Research In Motion, however, it was a different story. RIM was losing the signal to the market it created.