Last month, the video platform TikTok banned far-right extremists Britain First and Tommy Robinson, the latest action taken by a tech platform to address hateful and extreme content by sanctioning abusers. Platforms embrace of deplatforming as the default tool for repeated or severe violations of terms of service shows progress in prioritising the issue of online extremism, but as a tool, it is a blunt instrument that may not be equally valuable in all circumstances. Not all platforms can or will address all content equally efficiently, and whether they should requires an assessment of unintended consequences. Whether those factors are correctly balanced by platforms, or deplatforming is simply the most straightforward tool at their disposal, remains to be seen.
Addressing harmful content that could lead to hate, extremism, and terrorism is critical for tech platforms, sometimes for legal compliance and other times simply because it is imperative to protect their users and our communities. For a sense of scale, recent transparency reports show that between January and June of 2019, Twitter took action against almost 600,000 accounts for violating policies related to hate and Facebook took action against 17.8 million pieces of content based on terrorist propaganda concerns and 15.5 million related to hate speech between January and September of 2019. The Global Internet Forum to Counter Terrorism asserts that its joint hashing database the shared mechanism for large tech companies such as Facebook, Microsoft, Twitter, YouTube, and others to post or find terrorism-related content has over 200,000 pieces of unique content. When these actions manifest as banning a user, the result can be severe: an oft-cited example of the success of deplatforming is that of far-right provocateur Milo Yiannopoulos, who may be as much as $2 million in debt following bans that have removed his ability to benefit financially from his notoriety. Alex Jones media outlet InfoWars had about 1.4 million daily views of its site and users before being banned from YouTube and Facebook, and 715,000 afterward, according to the New York Times analysis.
On the other hand, these results raise questions regarding whether platforms are efficient in carrying out bans. Jones, for example, launched Infowars is Back on Facebook an hour after it banned Infowars. Proxy channels emerged on YouTube, sharing Jones videos with over 1.6 million viewers, including 550,000 views in a thirty day period, and 10,000 subscribers. Lesser known antisemitic and white supremacist channels have managed to circumvent attempted bans. If the strategy to address online extremism must be whack-a-mole, there is considerable room to improve efficiency in finding users and content to ban, implementing bans, and finding and removing proxies.
Beyond efficiency is effectiveness: banning an individual or group may feel cathartic, but whether it achieves the desired result of degrading and helping defeat extremists and their movements is a far more central question. The verdict on that is, unfortunately, unclear.
Researchers at Georgia Institute of Technology looked at bans on Reddit, concluding that users that experienced sanctions from Reddit for hate speech left Reddit entirely, reduced hate speech on Reddit by 80-90 percent, and many also migrated to new Reddit threads. Audrey Alexanders study for the George Washington University Program on Extremism shows that mass bans of Islamic State (IS) followers on Twitter deteriorates [IS] followers ability to gain traction on the platform, likely hindering their reach to potential recruits and acknowledges that the decay on Twitter corresponded with IS strategic shift to Telegram as its platform of choice.
Strategic success for mass bans has often been interpreted (1) as digital decay for the individual platform in question, rather than the integrated online ecosystem, and (2) in terms of the volume of users and their hateful content rather than the escalation or de-escalation of extremism.
Telegram, for example, became the platform of choice for jihadists as mainstream platforms began to use bans, removing IS sympathizers ability to recruit followers from a mainstream audience, but driving their online communications underground to a less-visible and less-regulated platform. Now it is also becoming a destination of the global white supremacist movement.
Similar platform migration has led to extremist use of VK, the Russian Facebook-equivalent; Gab, far right-extremists Twitter-equivalent; and lesser-known sites that their users would move to if those platforms began regulating, which, as ADL analysis suggests, could be WrongThink, minds.com, toko.tech, MeWe, or freezoxee. The evolution of the chans is illustrative: bringing attention to 4chan or 8chan may have led to particular actions to limit extremist content on them, but also led 8chan to go dark and return several times, and also gave rise to Endchan, 7chan, and myriad other copycat sites that aim to circumvent attempts to regulate them.
According to an analysis by ADL and the Network Contagion Research Institute, during months when a Twitter mass ban took place corresponded to more than double the percent of new members on Gab than a typical month. The frequency with which the users referenced the ban, and the corresponding spiteful references to censorship (e.g. fascistbook and goolag) suggests that the new users are joining Gab due to mass bans on another platform, and that being banned fueled their anger not self-reflective anger for the behavior that got them banned, but toward the authorities than banned them. Another study reached similar conclusions, looking at Facebook and VK. This analysis suggests that the grievances that fuel far-right extremism may be heightened in users that are banned from mainstream platforms, and that those grievances are then expressed in fora with less oversight and a higher portion of like-minded members. In other words, there is a distinct possibility that deplatforming trades high exposure to a broad population for more extreme exposure to other extremists. And no amount of whack-a-mole will prevent extremists from finding the next forum on which they can post their hate and recruit new followers, with authorities potentially unaware of the platform migration.
Removing users and content also hinders investigation and research into the threat. Imagine an individual that poses a security concern and whose primary means of being discovered by law enforcement is online behavior for example, Conor Climo, whose online conversations and support for the Feuerkrieg Division led law enforcement to search his home, where they found bomb making materials and evidence of violent plots. If such a suspect were removed from all platforms that could be accessed by law enforcement and informants, then plots may continue, but out of sight. Further, researchers looking into such behavior to inform policymakers and the public no longer have visibility into concerning behavior once it is removed, which could distort public opinion and decision-making based on an inaccurate picture of threats.
Deplatforming may limit the breath of hate and extremism on mainstream platforms but increase extremists motivations to plot, doing so in secret. On the other hand, allowing hate unfettered access to the worlds most powerful megaphones to recruit more to their cause is similarly risky. Neither, of course, is an acceptable outcome, which is why comprehensive approaches and comprehensive research into what works is needed. Whether providing law enforcement more opportunities to track extremism, tech platforms better ways to implement terms of service enforcement, or promoting good speech to overwhelm hate and extremism online comprehensive, integrated approaches are necessary.
Read more:
Weighing the Value and Risks of Deplatforming GNET
- Judge blocks Texas bill aimed at preventing social media companies from deplatforming people based on their ... - AOL - February 25th, 2024
- Introducing FIRE's Campus Deplatforming Database | The Foundation for Individual Rights and Expression - Foundation for Individual Rights in Education - February 9th, 2024
- Alex Jones To Tucker Carlson: My Sin Was Decompartmentalizing The American Population - RealClearPolitics - December 11th, 2023
- Does 'deplatforming' work to curb hate speech and calls for ... - October 16th, 2023
- TIMELINE: Trump reinstated from 'permanent' social media bans, but will that last? - WCYB - March 31st, 2023
- Twitter Found Trump Didn't Violate Policy, But Banned Him Anyway - December 28th, 2022
- Social media misogyny: The new way Andrew Tate brought us the same old hate - December 28th, 2022
- Google Pays Sharath Bulusu: Well-meaning regulation will help market mature - The Indian Express - December 28th, 2022
- WordPress discussed deplatforming The New York Post after Hunter Biden ... - December 20th, 2022
- App Store 'Gatekeepers' Urged To Deplatform "Dangerous" Twitter Itself - December 12th, 2022
- FACT CHECK: Did Elon Musk Call Twitter Users Upset By Trumps ... - December 12th, 2022
- 'Deplatforming the President': Twitter Files Part 3 reveals events that led to removal of Donald Trump - Business Today - December 12th, 2022
- Twitter Files Part 2: Top Twitter execs interfered with US election before banning Donald Trump - msnNOW - December 12th, 2022
- Conservatives Are Mad Kanye Is Being Censored for Antisemitism ... - November 25th, 2022
- Chronicles Magazine : A Magazine of American Culture - November 17th, 2022
- Amazon accused of absurd and unacceptable censorship after book ... - November 17th, 2022
- PayPal's ideological deplatforming spurs proposed UK amendment to stop ... - November 17th, 2022
- Andrew Tates popularity decreases following deplatforming from social ... - November 7th, 2022
- This is for people who think deplatforming doesn't work - November 7th, 2022
- 4 Great Email Templates For Leaving Your Job - Jobbio Journal - October 31st, 2022
- What is Social Media? - TechTarget - October 31st, 2022
- The dangers of deplatforming - October 31st, 2022
- A Middlebury College Expert Explains the Ways Social Media Fuels Hate and How to Stop It - Seven Days - October 23rd, 2022
- Alex Jones ordered to pay $965 million after misinformation campaign targeting Sandy Hook families - Yahoo Finance - October 15th, 2022
- Kanye West Bank Ban Is An Indefensible Slippery Slope - OutKick - October 15th, 2022
- My New Paper: Donald Trump Should Remain Deplatformed from Facebook, Twitter, and YouTube Despite the High Bar That Platforms Should Apply to the... - October 15th, 2022
- As Washington Post targets Catholic hospitals, every religious institution needs to build defenses - Washington Examiner - October 15th, 2022
- TMCs Derek OBrien steps in to give legitimacy to The Wires Meta joke that turned out to be a hoax - OpIndia - October 15th, 2022
- LeBron James' The Shop Will Not Air Episode Featuring Kanye West After Controversial Remarks By The Rapper - Yardbarker - October 15th, 2022
- Verizon Blacklists One America News Network Following ... - Breitbart - October 7th, 2022
- Conference aims to make trust and safety hot topics in computer science - The Stanford Daily - October 7th, 2022
- Sneako: Andrew Tate 2.0 Has Been Kicked Off YouTube | 2022 - The Latch - October 7th, 2022
- An Early Conservative Victory in the War on Big Tech | Opinion - Newsweek - October 7th, 2022
- Students respond to upcoming Congress to Campus event - Macalester College The Mac Weekly - October 7th, 2022
- Supreme Court Is Putting the Future of Section 230 Protections on Its Docket - Gizmodo - October 7th, 2022
- How to fundraise when the powerful want to stop you - Open Democracy - October 7th, 2022
- Piers Morgan Has Just Grilled Andrew Tate in a Live TV Interview - We Got This Covered - October 7th, 2022
- Kiwi Farms Is Down After Cloudflare Boots the Site as a Customer - September 29th, 2022
- How two Republicans in Florida are dominating the news - POLITICO - September 29th, 2022
- "White Psychodrama" and the culture wars: A self-reinforcing cycle, going nowhere - Salon - September 29th, 2022
- Regulating Misinformation on Twitter is a Problem, But There Are Bigger Ones on the Horizon - Tech Policy Press - September 29th, 2022
- It's Time to Stop Platforming the Big Lie Playbook - Tech Policy Press - September 29th, 2022
- Alex Jones' defamation trials show deplatforming's limits : NPR - September 21st, 2022
- Judges ruling against Twitter to protect the First Amendment did the opposite - MSNBC - September 21st, 2022
- Libs of TikTok vs. The Washington Post - Idaho State Journal - September 13th, 2022
- Journalist Glenn Greenwald scorches unholy alliance of government Democrats, corporate media and Big Tech - Fox News - September 13th, 2022
- Deplatforming Andrew Tate is the best way to deal with him - Los Angeles Loyolan - September 5th, 2022
- Andrew Tate joins Rumble after Big Tech deplatforming, will post ... - August 28th, 2022
- Pro-Masculine Influencer Andrew Tate Joins Rumble After Big Tech ... - August 28th, 2022
- NBC Journalist Who Attacked Libs Of TikTok Once Took Credit For ... - August 28th, 2022
- Web3 Will Make Or Break Social Media - Forbes - August 28th, 2022
- 2022 Tech Trends | The Future Today Institute - August 20th, 2022
- Commentary: Twitter fast becoming a thought-policed sandbox - The Herald Journal - August 20th, 2022
- What the Trial of Alex Jones Revealed About His Finances - The New York Times - August 12th, 2022
- Big Tech's Ban on Alex Jones Fans the Flames of His Conspiracy Theories - Reason - August 4th, 2022
- A New Cloud Company Looking to Disrupt The Old Order - Digital Journal - August 4th, 2022
- Nazis try to get on the ballot in Massachusetts - but may not have collected enough signatures - Universal Hub - August 4th, 2022
- Meme Creators Want Instagram To Change Its Content Moderation Policies And Stop Deplatforming Them - BuzzFeed News - July 26th, 2022
- Alexs War: A film about the de-platformed conspiracy theorist hits iTunes July 29 - BGR - July 26th, 2022
- Dozens of States Are Jumping on the Social Media Censorship Bandwagon - Gizmodo - July 26th, 2022
- What is Mike Lees TEAM Act and how would it reform Big Tech? | Opinion - Deseret News - July 18th, 2022
- Amid fears of election subversion, DeSantis' handpicked secretary of state will oversee his race - Salon - July 18th, 2022
- Internal Twitter messages show employees targeting Libs of TikTok ... - July 9th, 2022
- Leaked internal messages show Twitter employees debating whether to ban us - July 9th, 2022
- Trump Appeals Dismissal Of Preposterous Suit Against Twitter For Tortious Deplatforming - Above the Law - June 29th, 2022
- Loud music and take-out alcohol. Here are all the new Florida laws going into effect - WKMG News 6 & ClickOrlando - June 29th, 2022
- Study suggests identity advancement plays a key role in the downplaying of Trump's unethical behaviors - PsyPost - June 29th, 2022
- The brewing fight to keep abortion info online - The Verge - June 29th, 2022
- Yes, Elon, There is a Better Way. - International Policy Digest - June 10th, 2022
- Crypto Is An Invaluable Tool In The Fight Against Financial Oppression - - FinanceFeeds - May 25th, 2022
- Law's Jessie Hill discussed the legality of social media "deplatforming" - The Daily | Case Western Reserve University - May 11th, 2022
- Court Dropkicks Trump Twitter LOLsuit For THAT'S NOT HOW ANY OF THIS WORKS - Above the Law - May 11th, 2022
- Commentary: Is Ron DeSantis the Trump of the future? - Palm Beach Post - May 11th, 2022
- Jesus, endless war and the irresistible rise of American fascism - Salon - May 11th, 2022
- Who did Twitter ban during the Trump era, and why? - Los Angeles Times - April 28th, 2022
- The Unholy Alliance Between the US Security Apparatus and Big Tech - Fair Observer - April 28th, 2022
- Jimmy Kimmel Weighs-in on Not De-Platforming Joe Rogan: Its Not as Black and White as Sometimes the Media Makes It Out to Be - EssentiallySports - April 28th, 2022
- 'More tears and tantrums on Twitter are guaranteed' - The Week UK - April 28th, 2022
- Social media has a serious disinformation problem. But it can be fixed - The Indian Express - April 13th, 2022
- Social media company Minds and Daryl Davis tell Joe Rogan ... - April 13th, 2022