Facebooks new AI-powered moderation tool helps it catch billions of fake accounts – The Verge

Facebook is opening up about the behind-the-scenes tools it uses to combat fake account creation on its platforms, and the company says it has a new artificial intelligence-powered method known as Deep Entity Classification (DEC) thats proved especially effective.

DEC is a machine learning model that doesnt just take into account the activity of the suspect account, but it also evaluates all of the surrounding information, including the behaviors of the accounts and pages the suspect account interacts with. Facebook says its reduced the estimated volume of spam and scam accounts by 27 percent.

So far, DEC has helped Facebook thwart more than 6.5 billion fake accounts that scammers and other malicious actors created or tried to create last year. A vast majority of those accounts are actually caught in the account creation process, and even those that do get through tend to get discovered by Facebooks automated systems before they are ever reported by a real user.

Still, Facebook estimates that around 5 percent of all 2.89 billion monthly active users currently on the platform are fake accounts belonging to what Facebook considers violators of its terms of service. That typically means scammers, spammers, and people attempting to phish vulnerable users or use other methods of securing sensitive personal information for some sort of financial or identity theft scheme.

Thats where DEC comes in. It takes a sophisticated and holistic approach to analyzing user behavior that takes in around 20,000 features per profile; for instance, itll take into account the friending activity of an account the suspicious and potentially fake account sent a friend request to, and not just the suspicious account itself. The goal is to combat the ways malicious actors replicate genuine behavior. Over time, Facebook says the savvy spammers will get better and better at pretending to be real users, at least in the way Facebooks automated systems view one. DEC is supposed to take advantage of that by looking deeper into how the accounts that account interacts with behave on the platform, too.

This will be vitally important to Facebook as the 2020 US presidential election approaches. Promoting spam and trying to scam users are just one facet of the fake account problem on Facebook. The company has already acknowledged foreign operations from Iran, Russia, and elsewhere dedicated to using social platforms to influence news narratives, voting behavior, and other integral election matters. And those operations are getting more sophisticated as time goes on.

Last year, Facebook and Twitter shut down a sprawling network of fake accounts pushing pro-Trump messaging that used AI tools to generate real-looking profile photos. These werent scraped photos, but generated ones using neural networks, making them harder to flag as fake. Its these kinds of methods that will keep Facebook hard at work trying to stay one step ahead, and the company is acknowledging that its DEC approach will need to be continually reworked to ensure that it can remain effective against the spammers ever-changing strategies.

Continued here:

Facebooks new AI-powered moderation tool helps it catch billions of fake accounts - The Verge