Teaching AI Systems to Behave Themselves – New York Times

In some cases, researchers are working to ensure that systems dont make mistakes on their own, as the Coast Runners boat did. Theyre also working to ensure that hackers and other bad actors cant exploit hidden holes in these systems. Researchers like Googles Ian Goodfellow, for example, are exploring ways that hackers could fool A.I. systems into seeing things that arent there.

Modern computer vision is based on what are called deep neural networks, which are pattern-recognition systems that can learn tasks by analyzing vast amounts of data. By analyzing thousands of dog photos, a neural network can learn to recognize a dog. This is how Facebook identifies faces in snapshots, and its how Google instantly searches for images inside its Photos app.

But Mr. Goodfellow and others have shown that hackers can alter images so that a neural network will believe they include things that arent really there. Just by changing a few pixels in the photo of elephant, for example, they could fool the neural network into thinking it depicts a car.

That becomes problematic when neural networks are used in security cameras. Simply by making a few marks on your face, the researchers said, you could fool a camera into believing youre someone else.

If you train an object-recognition system on a million images labeled by humans, you can still create new images where a human and the machine disagree 100 percent of the time, Mr. Goodfellow said. We need to understand that phenomenon.

Another big worry is that A.I. systems will learn to prevent humans from turning them off. If the machine is designed to chase a reward, the thinking goes, it may find that it can chase that reward only if it stays on. This oft-described threat is much further off, but researchers are already working to address it.

Mr. Hadfield-Menell and others at U.C. Berkeley recently published a paper that takes a mathematical approach to the problem. A machine will seek to preserve its off switch, they showed, if it is specifically designed to be uncertain about its reward function. This gives it an incentive to accept or even seek out human oversight.

Much of this work is still theoretical. But given the rapid progress of A.I. techniques and their growing importance across so many industries, researchers believe that starting early is the best policy.

Theres a lot of uncertainty around exactly how rapid progress in A.I. is going to be, said Shane Legg, who oversees the A.I. safety work at DeepMind. The responsible approach is to try to understand different ways in which these technologies can be misused, different ways they can fail and different ways of dealing with these issues.

An earlier version of a picture caption with this article identified the three people in the picture in the wrong order. They are Dario Amodei, standing, and from left, Paul Christiano and Geoffrey Irving.

A version of this article appears in print on August 14, 2017, on Page B1 of the New York edition with the headline: When Robots Have Minds Of Their Own.

Here is the original post:

Teaching AI Systems to Behave Themselves - New York Times