Microsoft AI’s next leap forward: Helping you play video games – CNET

Could you be playing the next big video game with your voice?

Voice assistants can seem supersmart. Ask my Amazon Alexa why the sky is blue, and you'll get a lesson in light refraction through the atmosphere.

Ask it what CNET is and things start to break down.

"In addition CNET currently has region-specific and language-specific editions."

Well, sure. Then I asked Alexa when the Super Bowl was, right before Sunday night's game. It responded:

"Super Bowl 50's winner is Denver Broncos."

That's one of the biggest contradictions with voice assistants. They can control your lights, play music and even tell you silly jokes. But despite their growing presence in our lives, their capabilities are still very limited.

So far, the way many companies have made them better is to hand-code each response. For example, someone at Amazon could go into Alexa's code and teach it what CNET is and when the next Super Bowl will take place.

Microsoft thinks it's found a different way. It's inviting app developers and companies to use its technology, feeding questions, giving responses and learning what needs to be fixed along the way.

The software giant isn't the only one looking for new uses for artificial intelligence, which, in shorthand, is essentially software that can learn, adapt and act in more subtle, sophisticated ways. Facebook is training its AI with all sorts of software tools, including one in its Oregon data center that's trying to teach a computer to create an original piece of art after looking at a series of pictures. Google, meanwhile, is teaching its AI to play board games. And IBM is refining its AI, called Watson, by feeding it data from all manner of businesses.

Microsoft has had its share of public AI efforts too. It offers a voice assistant in its Windows PC and phone software called Cortana, which will happily jot down reminders and answer trivia questions.

It has also released experiments like Tay, a Twitter chatbot that learned from conversations with people. The experiment, however, was quickly taken offline after people taught it to hate feminists, praise Adolf Hitler and solicit sex.

This time around, Microsoft is taking a more measured approach by offering its AI tools to developers. So far, the results have been encouraging.

A security footage startup called Prism has started using Microsoft's tools to help organize playback video. Prism identifies when there's an object in the video that wasn't there before. Then it sends an image from that clip to Microsoft to identify what's in the picture and gets responses back like "dog" or "package."

This could take hours for a person to do, but combining Prism's technology with Microsoft's AI means a search to see how many packages came to the front desk that day takes mere moments. "It's unfathomable to think about how much data there is," said Adam Planas, a creative director at Prism.

Microsoft's doing the same with voice commands, offering apps not just transcriptions of what I say, but an estimation of what it means, too. That is, if a video game is expecting to hear me say "how old are you" and I say "you look really young," it'll know I basically mean the same thing.

That's a big improvement over the voice command software Alexander Mejia and his team at Human Interact were using before they turned to Microsoft. Their project, Starship Commander, is a new virtual reality game entirely controlled by the player's voice.

"When people put on the headset, they start role-playing, they get into character," he said. "They want to be the starship commander and go forth and have an adventure."

The goal, he said, is to make players feel completely natural talking to the game. Part of that is by creating a slick-looking game that immerses the player to the point that they feel as though they are on a starship. Then, the game has to coax the player into talking enough that after a while, it's just natural. The only downside is that the game will require an internet connection to send your voice commands to Microsoft for processing.

But the upside is that process is "crazy fast," said Sophie Wright, vice president of business development at Human Interact (who also doubles as a character in the game).

Microsoft believes that by inviting developers to use its technology, they can help train its AI. Aside from the 5,000 engineers Microsoft has working on artificial intelligence, more than 424,000 outside developers have signed up to try it out too.

"I think we're on the cusp of a breakthrough," said Andrew Shuman, a corporate vice president at Microsoft who leads the company's AI research group. Once AI is able to understand us better, they can start truly helping in our daily lives. Imagine being able to ask a security camera where you left your car keys.

"You can set up for real user delight," Shuman said.

Does the Mac still matter? Apple execs tell why the MacBook Pro was over four years in the making, and why we should care.

Tech Enabled: CNET chronicles tech's role in providing new kinds of accessibility.