I. Running Amok

It was just a friendly little argument about the fate of humanity. DemisHassabis, a leading creator of advanced artificial intelligence, waschatting with Elon Musk, a leading doomsayer, about the perils ofartificial intelligence.

They are two of the most consequential and intriguing men in SiliconValley who dont live there. Hassabis, a co-founder of the mysteriousLondon laboratory DeepMind, had come to Musks SpaceX rocket factory,outside Los Angeles, a few years ago. They were in the canteen, talking,as a massive rocket part traversed overhead. Musk explained that hisultimate goal at SpaceX was the most important project in the world:interplanetary colonization.

Hassabis replied that, in fact, he was working on the most importantproject in the world: developing artificial super-intelligence. Muskcountered that this was one reason we needed to colonize Marsso thatwell have a bolt-hole if A.I. goes rogue and turns on humanity. Amused,Hassabis said that A.I. would simply follow humans to Mars.

This did nothing to soothe Musks anxieties (even though he says thereare scenarios where A.I. wouldnt follow).

An unassuming but competitive 40-year-old, Hassabis is regarded as theMerlin who will likely help conjure our A.I. children. The field of A.I.is rapidly developing but still far from the powerful, self-evolvingsoftware that haunts Musk. Facebook uses A.I. for targeted advertising,photo tagging, and curated news feeds. Microsoft and Apple use A.I. topower their digital assistants, Cortana and Siri. Googles search enginefrom the beginning has been dependent on A.I. All of these smalladvances are part of the chase to eventually create flexible,self-teaching A.I. that will mirror human learning.

WITHOUT OVERSIGHT, MUSK BELIEVES, A.I. COULD BE AN EXISTENTIAL THREAT:WE ARE SUMMONING THE DEMON.

Some in Silicon Valley were intrigued to learn that Hassabis, a skilledchess player and former video-game designer, once came up with a gamecalled Evil Genius, featuring a malevolent scientist who creates adoomsday device to achieve world domination. Peter Thiel, thebillionaire venture capitalist and Donald Trump adviser who co-foundedPayPal with Musk and othersand who in December helped gatherskeptical Silicon Valley titans, including Musk, for a meeting with thepresident-electtold me a story about an investor in DeepMind whojoked as he left a meeting that he ought to shoot Hassabis on the spot,because it was the last chance to save the human race.

Elon Musk began warning about the possibility of A.I. running amok threeyears ago. It probably hadnt eased his mind when one of Hassabisspartners in DeepMind, Shane Legg, stated flatly, I think humanextinction will probably occur, and technology will likely play a partin this.

Before DeepMind was gobbled up by Google, in 2014, as part of its A.I.shopping spree, Musk had been an investor in the company. He told methat his involvement was not about a return on his money but rather tokeep a wary eye on the arc of A.I.: It gave me more visibility intothe rate at which things were improving, and I think theyre reallyimproving at an accelerating rate, far faster than people realize.Mostly because in everyday life you dont see robots walking around.Maybe your Roomba or something. But Roombas arent going to take overthe world.

In a startling public reproach to his friends and fellow techies, Muskwarned that they could be creating the means of their own destruction.He told Bloombergs Ashlee Vance, the author of the biography Elon Musk,that he was afraid that his friend Larry Page, a co-founder of Googleand now the C.E.O. of its parent company, Alphabet, could have perfectlygood intentions but still produce something evil byaccidentincluding, possibly, a fleet of artificialintelligence-enhanced robots capable of destroying mankind.

At the World Government Summit in Dubai, in February, Musk again cuedthe scary organ music, evoking the plots of classic horror stories whenhe noted that sometimes what will happen is a scientist will get soengrossed in their work that they dont really realize the ramificationsof what theyre doing. He said that the way to escape humanobsolescence, in the end, may be by having some sort of merger ofbiological intelligence and machine intelligence. This Vulcanmind-meld could involve something called a neural lacean injectablemesh that would literally hardwire your brain to communicate directlywith computers. Were already cyborgs, Musk told me in February.Your phone and your computer are extensions of you, but the interfaceis through finger movements or speech, which are very slow. With aneural lace inside your skull you would flash data from your brain,wirelessly, to your digital devices or to virtually unlimited computingpower in the cloud. For a meaningful partial-brain interface, I thinkwere roughly four or five years away.

Musks alarming views on the dangers of A.I. first went viral after hespoke at M.I.T. in 2014speculating (pre-Trump) that A.I. was probablyhumanitys biggest existential threat. He added that he wasincreasingly inclined to think there should be some national orinternational regulatory oversightanathema to Silicon Valleytomake sure that we dont do something very foolish. He went on: Withartificial intelligence, we are summoning the demon. You know all thosestories where theres the guy with the pentagram and the holy water andhes like, yeah, hes sure he can control the demon? Doesnt work out.Some A.I. engineers found Musks theatricality so absurdly amusing thatthey began echoing it. When they would return to the lab after a break,theyd say, O.K., lets get back to work summoning.

Musk wasnt laughing. Elons crusade (as one of his friends andfellow tech big shots calls it) against unfettered A.I. had begun.

Elon Musk smiled when I mentioned to him that he comes across assomething of an Ayn Rand-ian hero. I have heard that before, he saidin his slight South African accent. She obviously has a fairly extremeset of views, but she has some good points in there.

But Ayn Rand would do some re-writes on Elon Musk. She would make hiseyes gray and his face more gaunt. She would refashion his publicdemeanor to be less droll, and she would not countenance his goofygiggle. She would certainly get rid of all his nonsense about thecollective good. She would find great material in the 45-year-oldscomplicated personal life: his first wife, the fantasy writer JustineMusk, and their five sons (one set of twins, one of triplets), and hismuch younger second wife, the British actress Talulah Riley, who playedthe boring Bennet sister in the Keira Knightley version of Pride &Prejudice. Riley and Musk were married, divorced, and then re-married.They are now divorced again. Last fall, Musk tweeted that Talulah doesa great job playing a deadly sexbot on HBOs Westworld, adding asmiley-face emoticon. Its hard for mere mortal women to maintain arelationship with someone as insanely obsessed with work as Musk.

How much time does a woman want a week? he asked Ashlee Vance.Maybe ten hours? Thats kind of the minimum?

Mostly, Rand would savor Musk, a hyper-logical, risk-lovingindustrialist. He enjoys costume parties, wing-walking, and Japanesesteampunk extravaganzas. Robert Downey Jr. used Musk as a model for IronMan. Marc Mathieu, the chief marketing officer of Samsung USA, who hasgone fly-fishing in Iceland with Musk, calls him a cross between SteveJobs and Jules Verne.As they danced at their wedding reception,Justine later recalled, Musk informed her, I am the alpha in thisrelationship.

Photographs by Anders Lindn/Agent Bauer (Tegmark); by Jeff Chiu/A.P. Images (Page, Wozniak); by Simon Dawson/Bloomberg (Hassabis), Michael Gottschalk/Photothek (Gates), Niklas Hallen/AFP (Hawking), Saul Loeb/AFP (Thiel), Juan Mabromata/AFP (Russell), David Paul Morris/Bloomberg (Altman), Tom Pilston/The Washington Post (Bostrom), David Ramos (Zuckerberg), all from Getty Images; by Frederic Neema/Polaris/Newscom (Kurzwell); by Denis Allard/Agence Ra/Redux (LeCun); Ariel Zambelich/ Wired (Ng); Bobby Yip/Reuters/Zuma Press (Musk).

In a tech universe full of skinny guys in hoodieswhipping up botsthat will chat with you and apps that can study a photo of a dog andtell you what breed it isMusk is a throwback to Henry Ford and HankRearden. In Atlas Shrugged, Rearden gives his wife a bracelet made fromthe first batch of his revolutionary metal, as though it were made ofdiamonds. Musk has a chunk of one of his rockets mounted on the wall ofhis Bel Air house, like a work of art.

Musk shoots for the moonliterally. He launches cost-efficient rocketsinto space and hopes to eventually inhabit the Red Planet. In Februaryhe announced plans to send two space tourists on a flight around themoon as early as next year. He creates sleek batteries that could leadto a world powered by cheap solar energy. He forges gleaming steel intosensuous Tesla electric cars with such elegant lines that even thenitpicking Steve Jobs would have been hard-pressed to find fault. Hewants to save time as well as humanity: he dreamed up the Hyperloop, anelectromagnetic bullet train in a tube, which may one day whooshtravelers between L.A. and San Francisco at 700 miles per hour. WhenMusk visited secretary of defense Ashton Carter last summer, hemischievously tweeted that he was at the Pentagon to talk aboutdesigning a Tony Stark-style flying metal suit. Sitting in trafficin L.A. in December, getting bored and frustrated, he tweeted aboutcreating the Boring Company to dig tunnels under the city to rescue thepopulace from soul-destroying traffic. By January, according toBloomberg Businessweek, Musk had assigned a senior SpaceX engineer tooversee the plan and had started digging his first test hole. Hissometimes quixotic efforts to save the world have inspired a parodytwitter account, Bored Elon Musk, where a faux Musk spouts off wackyideas such as Oxford commas as a service and bunches of bananasgenetically engineered so that the bananas ripen one at a time.

Of course, big dreamers have big stumbles. Some SpaceX rockets haveblown up, and last May a driver was killed in a self-driving Teslawhose sensors failed to notice the tractor-trailer crossing its path.(An investigation by the National Highway Traffic Safety Administrationfound that Teslas Autopilot system was not to blame.)

Musk is stoic about setbacks but all too conscious of nightmarescenarios. His views reflect a dictum from Atlas Shrugged: Man has thepower to act as his own destroyerand that is the way he has actedthrough most of his history. As he told me, we are the first speciescapable of self-annihilation.

Heres the nagging thought you cant escape as you drive around fromglass box to glass box in Silicon Valley: the Lords of the Cloud love toyammer about turning the world into a better place as they churn out newalgorithms, apps, and inventions that, it is claimed, will make ourlives easier, healthier, funnier, closer, cooler, longer, and kinder tothe planet. And yet theres a creepy feeling underneath it all, a sensethat were the mice in their experiments, that they regard us humans asBetamaxes or eight-tracks, old technology that will soon be discarded sothat they can get on to enjoying their sleek new world. Many peoplethere have accepted this future: well live to be 150 years old, butwell have machine overlords.

Maybe we already have overlords. As Musk slyly told Recodes annual CodeConference last year in Rancho Palos Verdes, California, we couldalready be playthings in a simulated-reality world run by an advanced civilization. Reportedly, two Silicon Valley billionaires are working onan algorithm to break us out of the Matrix.

Among the engineers lured by the sweetness of solving the next problem,the prevailing attitude is that empires fall, societies change, and weare marching toward the inevitable phase ahead. They argue not aboutwhether but rather about how close we are to replicating, andimproving on, ourselves. Sam Altman, the 31-year-old president of YCombinator, the Valleys top start-up accelerator, believes humanity ison the brink of such invention.

The hard part of standing on an exponential curve is: when you lookbackwards, it looks flat, and when you look forward, it looksvertical, he told me. And its very hard to calibrate how much youare moving because it always looks the same.

Youd think that anytime Musk, Stephen Hawking, and Bill Gates are allraising the same warning about A.I.as all of them areit would be a10-alarm fire. But, for a long time, the fog of fatalism over the BayArea was thick. Musks crusade was viewed as Sisyphean at best andLuddite at worst. The paradox is this: Many tech oligarchs seeeverything they are doing to help us, and all their benevolentmanifestos, as streetlamps on the road to a future where, as SteveWozniak says, humans are the family pets.

But Musk is not going gently. He plans on fighting this with every fiberof his carbon-based being. Musk and Altman have founded OpenAI, abillion-dollar nonprofit company, to work for safer artificialintelligence. I sat down with the two men when their new venture hadonly a handful of young engineers and a makeshift office, an apartmentin San Franciscos Mission District that belongs to Greg Brockman,OpenAIs 28-year-old co-founder and chief technology officer. When Iwent back recently, to talk with Brockman and Ilya Sutskever, thecompanys 30-year-old research director (and also a co-founder), OpenAIhad moved into an airy office nearby with a robot, the usual complementof snacks, and 50 full-time employees. (Another 10 to 30 are on theway.)

Altman, in gray T-shirt and jeans, is all wiry, pale intensity. Musksfervor is masked by his diffident manner and rosy countenance. His eyesare green or blue, depending on the light, and his lips are plum red. Hehas an aura of command while retaining a trace of the gawky, lonelySouth African teenager who immigrated to Canada by himself at the age of17.

In Silicon Valley, a lunchtime meeting does not necessarily involve thatmundane fuel known as food. Younger coders are too absorbed inalgorithms to linger over meals. Some just chug Soylent. Older ones areso obsessed with immortality that sometimes theyre just washing downhealth pills with almond milk.

At first blush, OpenAI seemed like a bantamweight vanity project, abunch of brainy kids in a walkup apartment taking on themulti-billion-dollar efforts at Google, Facebook, and other companieswhich employ the worlds leading A.I. experts. But then, playing awell-heeled David to Goliath is Musks specialty, and he always does itwith styleand some useful sensationalism.

Let others in Silicon Valley focus on their I.P.O. price and ridding SanFrancisco of what they regard as its unsightly homeless population. Muskhas larger aims, like ending global warming and dying on Mars (just not,he says, on impact).

Musk began to see mans fate in the galaxy as his personal obligationthree decades ago, when as a teenager he had a full-blown existentialcrisis. Musk told me that The Hitchhikers Guide to the Galaxy, byDouglas Adams, was a turning point for him. The book is about aliensdestroying the earth to make way for a hyperspace highway and featuresMarvin the Paranoid Android and a supercomputer designed to answer allthe mysteries of the universe. (Musk slipped at least one reference tothe book into the software of the Tesla Model S.) As a teenager, Vancewrites in his biography, Musk formulated a mission statement forhimself: The only thing that makes sense to do is strive for greatercollective enlightenment.

OpenAI got under way with a vague mandatewhich isnt surprising,given that people in the field are still arguing over what form A.I.will take, what it will be able to do, and what can be done about it. Sofar, public policy on A.I. is strangely undetermined and software islargely unregulated. The Federal Aviation Administration overseesdrones, the Securities and Exchange Commission oversees automatedfinancial trading, and the Department of Transportation has begun tooversee self-driving cars.

Musk believes that it is better to try to get super-A.I. first anddistribute the technology to the world than to allow the algorithms tobe concealed and concentrated in the hands of tech or governmenteliteseven when the tech elites happen to be his own friends, peoplesuch as Google founders Larry Page and Sergey Brin. Ive had manyconversations with Larry about A.I. and roboticsmany, many, Musktold me. And some of them have gotten quite heated. You know, I thinkits not just Larry, but there are many futurists who feel a certaininevitability or fatalism about robots, where wed have some sort ofperipheral role. The phrase used is We are the biological boot-loaderfor digital super-intelligence. (A boot loader is the small programthat launches the operating system when you first turn on yourcomputer.) Matter cant organize itself into a chip, Musk explained.But it can organize itself into a biological entity that getsincreasingly sophisticated and ultimately can create the chip.

Musk has no intention of being a boot loader. Page and Brin seethemselves as forces for good, but Musk says the issue goes far beyondthe motivations of a handful of Silicon Valley executives.

Its great when the emperor is Marcus Aurelius, he says. Its notso great when the emperor is Caligula.

After the so-called A.I. winterthe broad, commercial failure in thelate 80s of an early A.I. technology that wasnt up tosnuffartificial intelligence got a reputation as snake oil. Now itsthe hot thing again in this go-go era in the Valley. Greg Brockman, ofOpenAI, believes the next decade will be all about A.I., with everyonethrowing money at the small number of wizards who know the A.I.incantations. Guys who got rich writing code to solve banal problemslike how to pay a stranger for stuff online now contemplate avertiginous world where they are the creators of a new reality andperhaps a new species.

Microsofts Jaron Lanier, the dreadlocked computer scientist known asthe father of virtual reality, gave me his view as to why the digeratifind the science-fiction fantasy of A.I. so tantalizing: Itssaying, Oh, you digital techy people, youre like gods; youre creatinglife; youre transforming reality. Theres a tremendous narcissism init that were the people who can do it. No one else. The Pope cant doit. The president cant do it. No one else can do it. We are the mastersof it . . . . The software were building is our immortality. Thiskind of God-like ambition isnt new, he adds. I read about it once ina story about a golden calf. He shook his head. Dont get high onyour own supply, you know?

Google has gobbled up almost every interesting robotics andmachine-learning company over the last few years. It bought DeepMind for$650 million, reportedly beating out Facebook, and built the GoogleBrain team to work on A.I. It hired Geoffrey Hinton, a British pioneerin artificial neural networks; and Ray Kurzweil, the eccentric futuristwho has predicted that we are only 28 years away from the Rapture-likeSingularitythe moment when the spiraling capabilities ofself-improving artificial super-intelligence will far exceed humanintelligence, and human beings will merge with A.I. to create thegod-like hybrid beings of the future.

Its in Larry Pages blood and Googles DNA to believe that A.I. is thecompanys inevitable destinythink of that destiny as you will. (Ifevil A.I. lights up, Ashlee Vance told me, it will light up first atGoogle.) If Google could get computers to master search when searchwas the most important problem in the world, then presumably it can getcomputers to do everything else. In March of last year, Silicon Valleygulped when a fabled South Korean player of the worlds most complexboard game, Go, was beaten in Seoul by DeepMinds AlphaGo. Hassabis, whohas said he is running an Apollo program for A.I., called it ahistoric moment and admitted that even he was surprised it happenedso quickly. Ive always hoped that A.I. could help us discovercompletely new ideas in complex scientific domains, Hassabis told mein February. This might be one of the first glimpses of that kind ofcreativity. More recently, AlphaGo played 60 games online against topGo players in China, Japan, and Koreaand emerged with a record of60--0. In January, in another shock to the system, an A.I. programshowed that it could bluff. Libratus, built by two Carnegie Mellonresearchers, was able to crush top poker players at Texas Hold Em.

Peter Thiel told me about a friend of his who says that the only reasonpeople tolerate Silicon Valley is that no one there seems to be havingany sex or any fun. But there are reports of sex robots on the way thatcome with apps that can control their moods and even have a pulse. TheValley is skittish when it comes to female sex robotsan obsession inJapanbecause of its notoriously male-dominated culture and itsmuch-publicized issues with sexual harassment and discrimination. Butwhen I asked Musk about this, he replied matter-of-factly, Sex robots?I think those are quite likely.

Whether sincere or a shrewd P.R. move, Hassabis made it a condition ofthe Google acquisition that Google and DeepMind establish a joint A.I.ethics board. At the time, three years ago, forming an ethics board wasseen as a precocious move, as if to imply that Hassabis was on the vergeof achieving true A.I. Now, not so much. Last June, a researcher atDeepMind co-authored a paper outlining a way to design a big redbutton that could be used as a kill switch to stop A.I. frominflicting harm.

Google executives say Larry Pages view on A.I. is shaped by hisfrustration about how many systems are sub-optimalfrom systems thatbook trips to systems that price crops. He believes that A.I. willimprove peoples lives and has said that, when human needs are moreeasily met, people will have more time with their family or to pursuetheir own interests. Especially when a robot throws them out of work.

Musk is a friend of Pages. He attended Pages wedding and sometimesstays at his house when hes in the San Francisco area. Its not worthhaving a house for one or two nights a week, the 99th-richest man inthe world explained to me. At times, Musk has expressed concern thatPage may be nave about how A.I. could play out. If Page is inclinedtoward the philosophy that machines are only as good or bad as thepeople creating them, Musk firmly disagrees. Some at Googleperhapsannoyed that Musk is, in essence, pointing a finger at them for rushingahead willy-nillydismiss his dystopic take as a cinematic clich.Eric Schmidt, the executive chairman of Googles parent company, put itthis way: Robots are invented. Countries arm them. An evil dictatorturns the robots on humans, and all humans will be killed. Sounds like amovie to me.

Some in Silicon Valley argue that Musk is interested less in saving theworld than in buffing his brand, and that he is exploiting a deeplyrooted conflict: the one between man and machine, and our fear that thecreation will turn against us. They gripe that his epic good-versus-evilstory line is about luring talent at discount rates and incubating hisown A.I. software for cars and rockets. Its certainly true that the BayArea has always had a healthy respect for making a buck. As Sam Spadesaid in The Maltese Falcon, Most things in San Francisco can bebought, or taken.

Musk is without doubt a dazzling salesman. Who better than a guardian ofhuman welfare to sell you your new, self-driving Tesla? Andrew Ngthechief scientist at Baidu, known as Chinas Googlebased in Sunnyvale,California, writes off Musks Manichaean throwdown as marketinggenius. At the height of the recession, he persuaded the U.S.government to help him build an electric sports car, Ng recalled,incredulous. The Stanford professor is married to a robotics expert,issued a robot-themed engagement announcement, and keeps a Trust theRobot black jacket hanging on the back of his chair. He thinks peoplewho worry about A.I. going rogue are distracted by phantoms, andregards getting alarmed now as akin to worrying about overpopulation onMars before we populate it. And I think its fascinating, he saidabout Musk in particular, that in a rather short period of time hesinserted himself into the conversation on A.I. I think he seesaccurately that A.I. is going to create tremendous amounts of value.

Although he once called Musk a sci-fi version of P. T. Barnum,Ashlee Vance thinks that Musks concern about A.I. is genuine, even ifwhat he can actually do about it is unclear. His wife, Talulah, toldme they had late-night conversations about A.I. at home, Vance noted.Elon is brutally logical. The way he tackles everything is like movingchess pieces around. When he plays this scenario out in his head, itdoesnt end well for people.

Eliezer Yudkowsky, a co-founder of the Machine Intelligence ResearchInstitute, in Berkeley, agrees: Hes Elon-freaking-Musk. He doesntneed to touch the third rail of the artificial-intelligence controversyif he wants to be sexy. He can just talk about Mars colonization.

Some sniff that Musk is not truly part of the whiteboard culture andthat his scary scenarios miss the fact that we are living in a worldwhere its hard to get your printer to work. Others chalk up OpenAI, inpart, to a case of FOMO: Musk sees his friend Page building new-wavesoftware in a hot field and craves a competing army of coders. As Vancesees it, Elon wants all the toys that Larry has. Theyre like thesetwo superpowers. Theyre friends, but theres a lot of tension in theirrelationship. A rivalry of this kind might be best summed up by a linefrom the vainglorious head of the fictional tech behemoth Hooli, onHBOs Silicon Valley: I dont want to live in a world where someoneelse makes the world a better place better than we do.

Musks disagreement with Page over the potential dangers of A.I. didaffect our friendship for a while, Musk says, but that has sincepassed. We are on good terms these days.

Musk never had as close a personal connection with 32-year-old MarkZuckerberg, who has become an unlikely lifestyle guru, setting a newchallenge for himself every year. These have included wearing a tieevery day, reading a book every two weeks, learning Mandarin, and eatingmeat only from animals he killed with his own hands. In 2016, it wasA.I.s turn.

Zuckerberg has moved his A.I. experts to desks near his own. Three weeksafter Musk and Altman announced their venture to make the world safefrom malicious A.I., Zuckerberg posted on Facebook that his project forthe year was building a helpful A.I. to assist him in managing hishomeeverything from recognizing his friends and letting them insideto keeping an eye on the nursery. You can think of it kind of likeJarvis in Iron Man, he wrote.

One Facebooker cautioned Zuckerberg not to accidentally createSkynet, the military supercomputer that turns against human beings inthe Terminator movies. I think we can build A.I. so it works for usand helps us, Zuckerberg replied. And clearly throwing shade at Musk,he continued: Some people fear-monger about how A.I. is a huge danger,but that seems far-fetched to me and much less likely than disasters dueto widespread disease, violence, etc. Or, as he described hisphilosophy at a Facebook developers conference last April, in a clearrejection of warnings from Musk and others he believes to be alarmists:Choose hope over fear.

In the November issue of Wired, guest-edited by Barack Obama, Zuckerbergwrote that there is little basis beyond science fiction to worry aboutdoomsday scenarios: If we slow down progress in deference to unfoundedconcerns, we stand in the way of real gains. He compared A.I. jittersto early fears about airplanes, noting, We didnt rush to put rules inplace about how airplanes should work before we figured out how theydfly in the first place.

Zuckerberg introduced his A.I. butler, Jarvis, right before Christmas.With the soothing voice of Morgan Freeman, it was able to help withmusic, lights, and even making toast. I asked the real-life Iron Man,Musk, about Zuckerbergs Jarvis, when it was in its earliest stages. Iwouldnt call it A.I. to have your household functions automated, Musksaid. Its really not A.I. to turn the lights on, set thetemperature.

Zuckerberg can be just as dismissive. Asked in Germany whether Musksapocalyptic forebodings were hysterical or valid, Zuckerbergreplied hysterical. And when Musks SpaceX rocket blew up on thelaunch pad in September, destroying a satellite Facebook was leasing,Zuckerberg coldly posted that he was deeply disappointed.

Musk and others who have raised a warning flag on A.I. have sometimesbeen treated like drama queens. In January 2016, Musk won the annualLuddite Award, bestowed by a Washington tech-policy think tank. Still,hes got some pretty good wingmen. Stephen Hawking told the BBC, Ithink the development of full artificial intelligence could spell theend of the human race. Bill Gates told Charlie Rose that A.I. waspotentially more dangerous than a nuclear catastrophe. Nick Bostrom, a43-year-old Oxford philosophy professor, warned in his 2014 book,Superintelligence, that once unfriendly superintelligence exists, itwould prevent us from replacing it or changing its preferences. Our fatewould be sealed. And, last year, Henry Kissinger jumped on the perilbandwagon, holding a confidential meeting with top A.I. experts at theBrook, a private club in Manhattan, to discuss his concern over howsmart robots could cause a rupture in history and unravel the waycivilization works.

In January 2015, Musk, Bostrom, and a Whos Who of A.I., representingboth sides of the split, assembled in Puerto Rico for a conferencehosted by Max Tegmark, a 49-year-old physics professor at M.I.T. whoruns the Future of Life Institute, in Boston.

Do you own a house?, Tegmark asked me. Do you own fire insurance?The consensus in Puerto Rico was that we needed fire insurance. When wegot fire and messed up with it, we invented the fire extinguisher. Whenwe got cars and messed up, we invented the seat belt, air bag, andtraffic light. But with nuclear weapons and A.I., we dont want to learnfrom our mistakes. We want to plan ahead. (Musk reminded Tegmark thata precaution as sensible as seat belts had provoked fierce oppositionfrom the automobile industry.)

Musk, who has kick-started the funding of research into avoiding A.I.spitfalls, said he would give the Future of Life Institute 10 millionreasons to pursue the subject, donating $10 million. Tegmark promptlygave $1.5 million to Bostroms group in Oxford, the Future of HumanityInstitute. Explaining at the time why it was crucial to be proactiveand not reactive, Musk said it was certainly possible to constructscenarios where the recovery of human civilization does not occur.

Six months after the Puerto Rico conference, Musk, Hawking, DemisHassabis, Apple co-founder Steve Wozniak, and Stuart Russell, acomputer-science professor at Berkeley who co-authored the standardtextbook on artificial intelligence, along with 1,000 other prominentfigures, signed a letter calling for a ban on offensive autonomousweapons. In 50 years, this 18-month period were in now will be seenas being crucial for the future of the A.I. community, Russell toldme. Its when the A.I. community finally woke up and took itselfseriously and thought about what to do to make the future better. LastSeptember, the countrys biggest tech companies created the Partnershipon Artificial Intelligence to explore the full range of issues arisingfrom A.I., including the ethical ones. (Musks OpenAI quickly joinedthis effort.) Meanwhile, the European Union has been looking into legalissues arising from the advent of robots and A.I.such as whetherrobots have personhood or (as one Financial Times contributorwondered) should be considered more like slaves in Roman law.

At Tegmarks second A.I. safety conference, last January at the Asilomarcenter, in Californiachosen because thats where scientists gatheredback in 1975 and agreed to limit genetic experimentationthe topic wasnot so contentious. Larry Page, who was not at the Puerto Ricoconference, was at Asilomar, and Musk noted that their conversationwas no longer heated.

But while it may have been a coming-out party for A.I. safety, asone attendee put itpart of a sea change in the last year or so,as Musk saystheres still a long way to go. Theres no questionthat the top technologists in Silicon Valley now take A.I. far moreseriouslythat they do acknowledge it as a risk, he observes. Imnot sure that they yet appreciate the significance of the risk.

Steve Wozniak has wondered publicly whether he is destined to be afamily pet for robot overlords. We started feeding our dog filet, hetold me about his own pet, over lunch with his wife, Janet, at theOriginal Hickry Pit, in Walnut Creek. Once you start thinking youcould be one, thats how you want them treated.

He has developed a policy of appeasement toward robots and any A.I.masters. Why do we want to set ourselves up as the enemy when theymight overpower us someday? he said. It should be a jointpartnership. All we can do is seed them with a strong culture where theysee humans as their friends.

When I went to Peter Thiels elegant San Francisco office, dominated bytwo giant chessboards, Thiel, one of the original donors to OpenAI and acommitted contrarian, said he worried that Musks resistance couldactually be accelerating A.I. research because his end-of-the-worldwarnings are increasing interest in the field.

Full-on A.I. is on the order of magnitude of extraterrestrialslanding, Thiel said. There are some very deeply tricky questionsaround this . . . . If you really push on how do we make A.I. safe, Idont think people have any clue. We dont even know what A.I. is. Itsvery hard to know how it would be controllable.

He went on: Theres some sense in which the A.I. question encapsulatesall of peoples hopes and fears about the computer age. I think peoplesintuitions do just really break down when theyre pushed to these limitsbecause weve never dealt with entities that are smarter than humans onthis planet.

Trying to puzzle out who is right on A.I., I drove to San Mateo to meetRay Kurzweil for coffee at the restaurant Three. Kurzweil is the authorof The Singularity Is Near, a Utopian vision of what an A.I. futureholds. (When I mentioned to Andrew Ng that I was going to be talking toKurzweil, he rolled his eyes. Whenever I read Kurzweils Singularity,my eyes just naturally do that, he said.) Kurzweil arrived with aWhole Foods bag for me, brimming with his books and two documentariesabout him. He was wearing khakis, a green-and-red plaid shirt, andseveral rings, including onemade with a 3-D printerthat has an Sfor his Singularity University.

Computers are already doing many attributes of thinking, Kurzweiltold me. Just a few years ago, A.I. couldnt even tell the differencebetween a dog and cat. Now it can. Kurzweil has a keen interest incats and keeps a collection of 300 cat figurines in his NorthernCalifornia home. At the restaurant, he asked for almond milk butcouldnt get any. The 69-year-old eats strange health concoctions andtakes 90 pills a day, eager to achieve immortalityor indefiniteextensions to the existence of our mind filewhich means mergingwith machines. He has such an urge to merge that he sometimes uses theword we when talking about super-intelligent future beingsa farcry from Musks more ominous they.

I mentioned that Musk had told me he was bewildered that Kurzweildoesnt seem to have even 1 percent doubt about the hazards of ourmind children, as robotics expert Hans Moravec calls them.

Thats just not true. Im the one who articulated the dangers,Kurzweil said. The promise and peril are deeply intertwined, hecontinued. Fire kept us warm and cooked our food and also burned downour houses . . . . Furthermore, there are strategies to control theperil, as there have been with biotechnology guidelines. He summarizedthe three stages of the human response to new technology as Wow!, Uh-Oh,and What Other Choice Do We Have but to Move Forward? The list ofthings humans can do better than computers is getting smaller andsmaller, he said. But we create these tools to extend our longreach.

Just as, two hundred million years ago, mammalian brains developed aneocortex that eventually enabled humans to invent language andscience and art and technology, by the 2030s, Kurzweil predicts, wewill be cyborgs, with nanobots the size of blood cells connecting us tosynthetic neocortices in the cloud, giving us access to virtual realityand augmented reality from within our own nervous systems. We will befunnier; we will be more musical; we will increase our wisdom, hesaid, ultimately, as I understand it, producing a herd of Beethovens andEinsteins. Nanobots in our veins and arteries will cure diseases andheal our bodies from the inside.

He allows that Musks bte noire could come true. He notes that our A.I.progeny may be friendly and may not be and that if its notfriendly, we may have to fight it. And perhaps the only way to fightit would be to get an A.I. on your side thats even smarter.

Kurzweil told me he was surprised that Stuart Russell had jumped onthe peril bandwagon, so I reached out to Russell and met with him inhis seventh-floor office in Berkeley. The 54-year-old British-Americanexpert on A.I. told me that his thinking had evolved and that he nowviolently disagrees with Kurzweil and others who feel that cedingthe planet to super-intelligent A.I. is just fine.

Russell doesnt give a fig whether A.I. might enable more Einsteins andBeethovens. One more Ludwig doesnt balance the risk of destroyinghumanity. As if somehow intelligence was the thing that mattered andnot the quality of human experience, he said, with exasperation. Ithink if we replaced ourselves with machines that as far as we knowwould have no conscious existence, no matter how many amazing thingsthey invented, I think that would be the biggest possible tragedy.Nick Bostrom has called the idea of a society of technologicalawesomeness with no human beings a Disneyland without children.

There are people who believe that if the machines are more intelligentthan we are, then they should just have the planet and we should goaway, Russell said. Then there are people who say, Well, wellupload ourselves into the machines, so well still have consciousnessbut well be machines. Which I would find, well, completelyimplausible.

Russell took exception to the views of Yann LeCun, who developed theforerunner of the convolutional neural nets used by AlphaGo and isFacebooks director of A.I. research. LeCun told the BBC that therewould be no Ex Machina or Terminator scenarios, because robots would notbe built with human driveshunger, power, reproduction,self-preservation. Yann LeCun keeps saying that theres no reason whymachines would have any self-preservation instinct, Russell said.And its simply and mathematically false. I mean, its so obvious thata machine will have self-preservation even if you dont program it inbecause if you say, Fetch the coffee, it cant fetch the coffee ifits dead. So if you give it any goal whatsoever, it has a reason topreserve its own existence to achieve that goal. And if you threaten iton your way to getting coffee, its going to kill you because any riskto the coffee has to be countered. People have explained this to LeCunin very simple terms.

Russell debunked the two most common arguments for why we shouldntworry: One is: Itll never happen, which is like saying we are drivingtowards the cliff but were bound to run out of gas before we get there.And that doesnt seem like a good way to manage the affairs of the humanrace. And the other is: Not to worrywe will just build robots thatcollaborate with us and well be in human-robot teams. Which begs thequestion: If your robot doesnt agree with your objectives, how do youform a team with it?

Last year, Microsoft shut down its A.I. chatbot, Tay, after Twitteruserswho were supposed to make her smarter through casual andplayful conversation, as Microsoft put itinstead taught her how toreply with racist, misogynistic, and anti-Semitic slurs. bush did9/11, and Hitler would have done a better job than the monkey we havenow, Tay tweeted. donald trump is the only hope weve got. Inresponse, Musk tweeted, Will be interesting to see what the mean timeto Hitler is for these bots. Only took Microsofts Tay a day.

With Trump now president, Musk finds himself walking a fine line. Hiscompanies count on the U.S. government for business and subsidies,regardless of whether Marcus Aurelius or Caligula is in charge. Muskscompanies joined the amicus brief against Trumps executive orderregarding immigration and refugees, and Musk himself tweeted against theorder. At the same time, unlike Ubers Travis Kalanick, Musk has hung inthere as a member of Trumps Strategic and Policy Forum. Its veryElon, says Ashlee Vance. Hes going to do his own thing no matterwhat people grumble about. He added that Musk can be opportunisticwhen necessary.

I asked Musk about the flak he had gotten for associating with Trump. Inthe photograph of tech executives with Trump, he had looked gloomy, andthere was a weary tone in his voice when he talked about the subject. Inthe end, he said, its better to have voices of moderation in the room with the president. There are a lot of people, kind of the hard left,who essentially want to isolateand not have any voice. Very unwise.

Eliezer Yudkowsky is a highly regarded 37-year-old researcher who istrying to figure out whether its possible, in practice and not just intheory, to point A.I. in any direction, let alone a good one. I met himat a Japanese restaurant in Berkeley.

How do you encode the goal functions of an A.I. such that it has anOff switch and it wants there to be an Off switch and it wont try toeliminate the Off switch and it will let you press the Off switch, butit wont jump ahead and press the Off switch itself? he asked over anorder of surf-and-turf rolls. And if it self-modifies, will itself-modify in such a way as to keep the Off switch? Were trying towork on that. Its not easy.

I babbled about the heirs of Klaatu, HAL, and Ultron taking over theInternet and getting control of our banking, transportation, andmilitary. What about the replicants in Blade Runner, who conspire tokill their creator? Yudkowsky held his head in his hands, then patientlyexplained: The A.I. doesnt have to take over the whole Internet. Itdoesnt need drones. Its not dangerous because it has guns. Itsdangerous because its smarter than us. Suppose it can solve the sciencetechnology of predicting protein structure from DNA information. Then itjust needs to send out a few e-mails to the labs that synthesizecustomized proteins. Soon it has its own molecular machinery, buildingeven more sophisticated molecular machines.

If you want a picture of A.I. gone wrong, dont imagine marchinghumanoid robots with glowing red eyes. Imagine tiny invisible syntheticbacteria made of diamond, with tiny onboard computers, hiding insideyour bloodstream and everyone elses. And then, simultaneously, theyrelease one microgram of botulinum toxin. Everyone just falls over dead.

Only it wont actually happen like that. Its impossible for me topredict exactly how wed lose, because the A.I. will be smarter than Iam. When youre building something smarter than you, you have to get itright on the first try.

I thought back to my conversation with Musk and Altman. Dont getsidetracked by the idea of killer robots, Musk said, noting, The thingabout A.I. is that its not the robot; its the computer algorithm inthe Net. So the robot would just be an end effector, just a series ofsensors and actuators. A.I. is in the Net . . . . The important thingis that if we do get some sort of runaway algorithm, then the human A.I.collective can stop the runaway algorithm. But if theres large,centralized A.I. that decides, then theres no stopping it.

Altman expanded upon the scenario: An agent that had full control ofthe Internet could have far more effect on the world than an agent thathad full control of a sophisticated robot. Our lives are already sodependent on the Internet that an agent that had no body whatsoever butcould use the Internet really well would be far more powerful.

Even robots with a seemingly benign task could indifferently harm us.Lets say you create a self-improving A.I. to pick strawberries,Musk said, and it gets better and better at picking strawberries andpicks more and more and it is self-improving, so all it really wants todo is pick strawberries. So then it would have all the world bestrawberry fields. Strawberry fields forever. No room for humanbeings.

But can they ever really develop a kill switch? Im not sure Id wantto be the one holding the kill switch for some superpowered A.I.,because youd be the first thing it kills, Musk replied.

Altman tried to capture the chilling grandeur of whats at stake: Itsa very exciting time to be alive, because in the next few decades we areeither going to head toward self-destruction or toward human descendantseventually colonizing the universe.

Right, Musk said, adding, If you believe the end is the heat deathof the universe, it really is all about the journey.

The man who is so worried about extinction chuckled at his ownextinction joke. As H. P. Lovecraft once wrote, From even the greatestof horrors irony is seldom absent.

CORRECTION: An earlier version of this story gave an incorrect date for the accident that killed the operator of a self-driving Tesla. It happened in May 2016.

Jeff Bezos: The C.E.O. of e-commerce and delivery giant Amazon and the owner of The Washington Post has already sparred with Trump. But Trump could come after Bezos for anti-trust issues, too: Trump is on the record as saying Amazon is controlling so much of what they are doing. The fact that The Washington Post has been reporting on Trump, often critically, probably does not endear Bezos to Trump, either.

Tim Cook: Trump has repeatedly criticized Apple for making its products overseas, and has called on the company to start building their damn computers and things in America. Cook must also contend with tariffs that will inevitably arise if Trump gets the U.S. into a trade war with China. And then theres the fact that Trump denounced Apple in 2016 for refusing a court order to cooperate with an F.B.I. request to unlock an iPhone belonging to one of the shooters in the San Bernardino terrorist attack last year.

Jack Dorsey: Twitter, already a tech company struggling with employee retention and a falling stock price, has been forced to contend with its role in handing Trump a megaphone to spout his opinions, whether those include attacking a union leader or merely suggesting the U.S. stock up on nuclear arms. Dorsey was also excluded by Trump from the tech summit at Trump Tower in December, reportedly as retribution for not allowing the Trump team to use an emoji-fied version of the #CrookedHillary hashtag. Sad!

Visit link:

Elon Musks Billion-Dollar Crusade to Stop the A.I ...