Google is using machine learning to improve the quality of Duo calls – The Verge

Google has rolled out a new technology to improve audio quality in Duo calls when the service cant maintain a steady connection called WaveNetEQ. Its based on technology from Googles DeepMind division that aims to replace audio jitter with artificial noise that sounds just like human speech, generated using machine learning.

If youve ever made a call over the internet, chances are youve experienced audio jitter. It happens when packets of audio data sent as part of the call get lost along the way or otherwise arrive late or in the wrong order. Google says that 99 percent of Duo calls experience packet loss: 20 percent of these lose over 3 percent of their audio, and 10 percent lose over 8 percent. Thats a lot of audio to replace.

Every calling app has to deal with this packet loss somehow, but Google says that these packet loss concealment (PLC) processes can struggle to fill gaps of 60ms or more without sounding robotic or repetitive. WaveNetEQs solution is based on DeepMinds neural network technology, and it has been trained on data from over 100 speakers in 48 different languages.

Here are a few audio samples from Google comparing WaveNetEQ against NetEQ, a commonly used PLC technology. Heres how it sounds when its trying to replace 60ms of packet loss:

Heres a comparison when a call is experiencing packet loss of 120ms:

Theres a limit to how much audio the system can replace, though. Googles tech is designed to replace short sounds, rather than whole words. So after 120ms, it fades out and produces silence. Google says it evaluated the system to make sure it wasnt introducing any significant new sounds. Plus, all of the processing also needs to happen on-device since Google Duo calls are end-to-end encrypted by default. Once the calls real audio resumes, WaveNetEQ will seamlessly fade back to reality.

Its a neat little bit of technology that should make calls that much bit easier to understand when the internet fails them. The technology is already available for Duo calls made on Pixel 4 phones, thanks to the handsets December feature drop, and Google says its in the process of rolling it out to other unnamed handsets.

Here is the original post:
Google is using machine learning to improve the quality of Duo calls - The Verge