Twitter’s Fact-Checking System Has a Major Blind Spot: Anything Divisive
After Elon Musk closed his $44 billion deal for Twitter, he tweeted that the network’s mission was to become “the most accurate source of information about the world.” He pointed toward a new fact-checking system called Community Notes, calling it “a game changer for improving accuracy on Twitter.” But the tool has serious limitations.
Community Notes asks fact-checking volunteers to add more context — a “note” — to misleading or incorrect tweets. Other users can then vote on whether a note is helpful or not, and a machine-learning algorithm determines what notes should be shown more broadly on the site.
But in its current design, Community Notes doesn’t address lies that are divisive, according to current and former employees, and an analysis of its open-source algorithm by Bloomberg News. When volunteers add fact-checking notes to tweets, they only become visible to the public if users from a “diversity of perspectives” are able to agree that a note is “helpful.” Twitter uses an algorithm to place volunteers along an opinion spectrum based on their voting history. Without agreement from both sides of this spectrum, the vast majority of inaccurate tweets — especially on the most divisive topics — go unaddressed.
The data shows that approximately 96% of all fact-checking notes contributed by the Twitter community didn’t pass through to public view. That means to date, more than 30,000 notes have gone undisplayed on Twitter for failing to meet the algorithm’s broad-consensus requirements, according to a Bloomberg analysis of Twitter’s data.
Misinformation on topics including abortion, Covid-19 and election integrity frequently goes unflagged by the system because voters from the two sides of the spectrum, which largely mirrors conservative or liberal views, can’t agree if notes are accurate. Instead, only a handful of notes — roughly 13 a day — are posted and typically address content unrelated to politically divisive topics.
Relying more heavily on the system, as Musk says he plans to, means Twitter would miss a lot of high-severity, highly controversial issues, according to a former machine-learning engineer at Twitter, who spoke on the condition of anonymity for fear of retaliation by the company.
Community Notes, called Birdwatch prior to Musk’s arrival, is meant to address issues with how past fact-checking efforts worked. Musk believes Twitter had too much direct power, equivalent to censorship, over what users say. Since taking over the company in late October, he has revoked Twitter’s policy about fact-checking Covid misinformation, dissolved the company’s Trust and Safety council, and fired thousands of employees, including staff responsible for tracking dangerous or inaccurate posts.
When applied to uncontroversial posts, Community Notes can work as intended. Because Musk has more than 100 million followers on the site, his own posts get a higher level of attention and sometimes get enough votes from volunteers to be successfully fact-checked.
It works like this:
In this case, the note was able to garner support from a “diversity of perspectives” — as determined by Twitter’s algorithm. Musk himself chimed in to congratulate the system for flagging the fake image.
When notes are applied, they can have an impact: People are less likely to agree with the substance of a tweet or choose to like or retweet it after seeing the context. “While Community Notes covers a wide range of topics, we expect it to be particularly impactful on topics where people have less entrenched beliefs, for example novel news topics or events,” said Keith Coleman, a vice president at Twitter running the Community Notes product.
But wide-ranging agreement by volunteers is often unattainable. On Twitter, which had 237 million daily active users in June, fewer than 1,400 notes have been approved and shown to the public since the program began in 2021.
For example, a recent tweet from the Family Research Council, an Evangelical lobbying group opposed to abortion and LGBT rights, said that “Abortion is never medically necessary to save the life of a mother,” an assertion rejected by the American College of Obstetricians and Gynecologists.
The same day the abortion tweet was posted, a Community Notes volunteer added a clarifying note, which then was judged by the other volunteers. Twitter’s system used votes from 169 users, filtering out some ratings that were deemed to be “low quality” using a complex ranking system. Ultimately, 164 of the 169 votes, or 97%, found the note “helpful,” but it wasn’t enough.
This is intentional, according to the program’s documentation. Twitter says that “note statuses aren’t reached by majority rule,” rather the program “takes into account not only how many contributors rated a note as helpful or unhelpful, but also whether people who rated it seem to come from different perspectives.” The note correcting the Family Research Council tweet didn’t get enough support from “both sides” and so all of the “helpful’’ votes were effectively nullified.
A former member of the Community Notes team who was directly involved in the product told Bloomberg News that Musk’s vision for the program has strayed from its original intent. “Twitter knew it could provide helpful information, but it could never replace actual fact-checking,” the former employee said. “It can never address political misinfo in its current form.”
Misinformation can travel further while the thousands of fact checks wait for enough votes from varied perspectives. For example, former UFC fighter Jake Shields recently said the National Institutes of Health had “added Ivermectin to its list of treatments” for Covid-19.
Despite no evidence that the NIH ever took this step, a note correcting the tweet was unable to attract a single positive rating from right-leaning users. To date, the post, which has been shared more than 45,000 times on Twitter, lacks any clarification.
This phenomenon isn’t limited to right-wing posts. Notes that attempt to add context to tweets from liberal accounts frequently end up with lopsided voting patterns in the opposite direction.
Representative Ilhan Omar, for example, tweeted in June that “religious freedom is dead in America” after the US Supreme Court ruled that a public high school football coach could pray on the field after games. A note providing additional details from the ruling garnered “helpful” ratings from users concentrated on the right side of the algorithm’s opinion spectrum. However, the note was never displayed because it lacked sufficient support from left-leaning voters.
This system can be confusing to users who assume that votes from all perspectives are being valued equally. A November tweet from Musk even suggested that was the case. “The community notes feature is awesome. Our goal is to make Twitter the most accurate source of information on Earth, without regard to political affiliation.”
But in practice, ideological affiliation is one of the primary ways votes are given value. Hundreds of users can agree that a note is useful, but if volunteers from the other side of the opinion spectrum choose not to vote, it’s highly unlikely a note will be displayed.
Part of the challenge for Twitter could be an unwillingness from users to engage with notes that go against their political goals. A recent research paper out of MIT’s Sloan School of Management found that the partisanship of Twitter users was directly predictive of voting behavior on the Community Notes system.
The analysis found that users were significantly more likely to engage with notes written by those who shared their political perspective. Voters were also far more likely to rate notes helpful when they shared a political identity with the note’s author. The paper even suggested “it’s possible that partisanship is motivating users to volunteer for, and contribute to, Birdwatch in the first place.”
Without users from opposite perspectives who are willing to vote in good faith on divisive issues, the type of content that’s publicly flagged by the Community Notes system tends to be comparatively mild in nature. This can be seen by analyzing the notes themselves:
Most Notes Are in Limbo If They Fact-Check the Left or Right
Part of the problem may be that Community Notes is being tasked with moderating types of content that it wasn’t designed to address. People familiar with the project, who declined to be named discussing non-public matters, say that it is best suited as a complement to Twitter’s existing fact-checking policies — a chance to add context to nonpolarizing tweets where additional information may actually change somebody’s mind. An employee who worked on misinformation issues at the company explained that the project was originally pitched as an experimental prototype, not something that would need to scale to handle the problem broadly.
Historically, fact-checking was assigned to internal employees who worked on Twitter’s trust and safety team. But those employees only reviewed tweets in a few specific categories, including Covid-19 and voting-related misinformation, and only when it was shared by high-profile accounts, like politicians. The system led to a backlash from people who alleged that tech companies like Twitter were silencing voices of specific ideologies.
Community Notes’ crowd-sourced nature and judgment-free approach might give a much-needed political cushion to a social media company under scrutiny, according to Sahar Massachi, executive director of the Integrity Institute, a think tank focused on social media governance.
“Nothing that it produces should get a cable news personality talking about sinister designs by Twitter,” he said. “And that’s probably one reason it launched when other equally great ideas did not.”
Since Community Notes was first created in early 2021, the project has largely existed in a limited state with just a small group of users. But the project has taken on an elevated role under Musk, who frequently tweets positively about the program. In the days before Musk took charge, the project was pushed to a wider US audience. On Dec. 12, the company began to roll out Community Notes to a global audience.
More Notes Submitted Since Musk Took Over Twitter
As Twitter moves away from manual fact-checking, Community Notes may have to play an outsized role in Twitter’s strategy. And as designed, that would leave a large portion of misinformation on Twitter unchecked.
“If every side has a veto, it’s not a substitute for actual fact-checking,” Massachi said. Community Notes “is a really interesting idea and I’m rooting for it. But we shouldn’t expect it to solve all of our problems.”