Labeling the credibility of information sources does not shift the consumption of news away from low-quality sources or reduce belief in widely circulated inaccurate claims among average internet users, but providing an indicator of sources’ quality may improve the news diet quality of the heaviest consumers of misinformation, shows a new study by New York University’s Center for Social Media and Politics.
The study, which appears in the journal Science Advancescentered on credibility ratings determined by NewsGuard, a browser extension that rates news and other information sites in order to guide users in assessing the trustworthiness of the content they come across online.
“While it’s encouraging that most of us rely on credible sources of information, there are many who turn to sites of questionable trustworthiness, which raises concerns about misperceptions people may hold,” says Kevin Aslett, a postdoctoral researcher at the Center for Social Media and Politics (CSMaP) and the lead author of the paper. “However, while our study shows that, overall, credibility ratings have no discernible effect on misperceptions or online news consumption behavior of the average user, our findings suggest that the heaviest consumers of misinformation — those who rely on low-credibility sites — may move toward higher-quality sources when presented with news reliability ratings.”
In the study, May and June 2020, the researchers conducted a random sample of more than 3,000 online participants to install the NewsGuard browser extension, which embeds source-level indicators of news reliability into users’ search engine results pages, social feeds, and visited URLs. Different “shield” symbols are placed in-feed to provide visual summaries of sources’ quality:
- Green shield = reliable source
- Red shield = unreliable source
- Gray shield = source with user-generated content
- Gold shield = satire
To measure the effect of these source labels, survey data were collected over two time periods (May 28-June 9 and June-19-June 30).
In addition to this panel survey, the researchers also collected anonymized digital trace data to characterize the quality of news consumption of a subset of approximately 1,000 participants. These quality measures made use of the same NewsGuard ratings shown to study participants as they encountered news links in their browsers.
Previous studies have shown that exposure to misinformation and online fake news can increase belief in misperceptions, heighten cynicism toward politics, lower trust in the news media, and increase affective polarization (ie, disdain for supporters of the other political party). Moreover, prior research also suggests that expert source ratings, when shown alongside a mock news article, can influence the perceived truthfulness of the article’s claim.
With these earlier findings in mind, the researchers tested whether in-feed source reliability labels can counteract these effects by shifting news consumption from unreliable to more reliable sources, increasing trust in mainstream media and reliable sources, and/or mitigating political polarization and cynicism.
In order to measure whether source reliability information affects belief in misinformation as well as in accurate claims, the respondents were asked to judge the veracity of five widely circulated statements about the Black Lives Matter movement and five similarly well-circulated statements about COVID-19 — some true and some false.