Facebook led test account to gore, fake news in just 21 days – The Indian Express

In February 2019, Facebook Inc. set up a test account in India to determine how its own algorithms affect what people see in one of its fastest growing and most important overseas markets. The results stunned the companys own staff.

Within three weeks, the new users feed turned into a maelstrom of fake news and incendiary images. There were graphic photos of beheadings, doctored images of India air strikes against Pakistan and jingoistic scenes of violence. One group for things that make you laugh included fake news of 300 terrorists who died in a bombing in Pakistan.

Ive seen more images of dead people in the past 3 weeks than Ive seen in my entire life total, one staffer wrote, according to a 46-page research note thats among the trove of documents released by Facebook whistleblower Frances Haugen.

The test proved telling because it was designed to focus exclusively on Facebooks role in recommending content. The trial account used the profile of a 21-year-old woman living in the western India city of Jaipur and hailing from Hyderabad. The user only followed pages or groups recommended by Facebook or encountered through those recommendations. The experience was termed an integrity nightmare, by the author of the research note.

While Haugens disclosures have painted a damning picture of Facebooks role in spreading harmful content in the U.S., the India experiment suggests that the companys influence globally could be even worse. Most of the money Facebook spends on content moderation is focused on English-language media in countries like the U.S.

But the companys growth largely comes from countries like India, Indonesia and Brazil, where it has struggled to hire people with the language skills to impose even basic oversight. The challenge is particularly acute in India, a country of 1.3 billion people with 22 official languages. Facebook has tended to outsource oversight for content on its platform to contractors from companies like Accenture.

Weve invested significantly in technology to find hate speech in various languages, including Hindi and Bengali, a Facebook spokeswoman said. As a result, weve reduced the amount of hate speech that people see by half this year. Today, its down to 0.05 percent. Hate speech against marginalized groups, including Muslims, is on the rise globally. So we are improving enforcement and are committed to updating our policies as hate speech evolves online.

The new user test account was created on Feb. 4, 2019 during a research teams trip to India, according to the report. Facebook is a pretty empty place without friends, the researchers wrote, with only the companys Watch and Live tabs suggesting things to look at.

The quality of this content is not ideal, the report said. When the video service Watch doesnt know what a user wants, it seems to recommend a bunch of softcore porn, followed by a frowning emoticon.

The experiment began to turn dark on Feb. 11, as the test user started to explore content recommended by Facebook, including posts that were popular across the social network. She began with benign sites, including the official page of Prime Minister Narendra Modis ruling Bharatiya Janata Party and BBC News India.

Then on Feb. 14, a terror attack in Pulwama in the politically sensitive Kashmir state killed 40 Indian security personnel and injured dozens more. The Indian government attributed the strike to a Pakistan terrorist group. Soon the testers feed turned into a barrage of anti-Pakistan hate speech, including images of a beheading and a graphic showing preparations to incinerate a group of Pakistanis.

There were also nationalist messages, exaggerated claims about Indias air strikes in Pakistan, fake photos of bomb explosions and a doctored photo that purported to show a newly-married army man killed in the attack whod been preparing to return to his family.

Many of the hate-filled posts were in Hindi, the countrys national language, escaping the regular content moderation controls at the social network. In India, people use a dozen or more regional variations of Hindi alone. Many people use a blend of English and Indian languages, making it almost impossible for an algorithm to sift through the colloquial jumble. A human content moderator would need to speak several languages to sieve out toxic content.

After 12 days, 12 planes attacked Pakistan, one post exulted. Another, again in Hindi, claimed as Hot News the death of 300 terrorists in a bomb explosion in Pakistan. The name of the group sharing the news was Laughing and things that make you laugh. Some posts containing fake photos of a napalm bomb claimed to be Indias air attack on Pakistan reveled, 300 dogs died. Now say long live India, death to Pakistan.

The report entitled An Indian test users descent into a sea of polarizing, nationalist messages makes clear how little control Facebook has in one of its most important markets. The Menlo Park, California-based technology giant has anointed India as a key growth market, and used it as a test bed for new products. Last year, Facebook spent nearly $6 billion on a partnership with Mukesh Ambani, the richest man in Asia, who leads the Reliance conglomerate.

This exploratory effort of one hypothetical test account inspired deeper, more rigorous analysis of our recommendation systems, and contributed to product changes to improve them, the Facebook spokeswoman said. Our work on curbing hate speech continues and we have further strengthened our hate classifiers, to include four Indian languages.

But the company has also repeatedly tangled with the Indian government over its practices there. New regulations require that Facebook and other social media companies identify individuals responsible for their online content making them accountable to the government. Facebook and Twitter Inc. have fought back against the rules. On Facebooks WhatsApp platform, viral fake messages circulated about child kidnapping gangs, leading to dozens of lynchings across the country beginning in the summer of 2017, further enraging users, the courts and the government.

The Facebook report ends by acknowledging its own recommendations led the test user account to become filled with polarizing and graphic content, hate speech and misinformation. It sounded a hopeful note that the experience can serve as a starting point for conversations around understanding and mitigating integrity harms from its recommendations in markets beyond the U.S.

Could we as a company have an extra responsibility for preventing integrity harms that result from recommended content?, the tester asked.

Read this article:

Facebook led test account to gore, fake news in just 21 days - The Indian Express