In recent months, questions have arisen about big tech’s unparalleled influence over what news and information people see online. Potential political bias and censorship in search engine results are a big part of the conversation. Is the concern well-founded?
To evaluate political bias in search results, Stanford researchers focused on evaluating the news sources that appeared on the first page of a Google search result for every candidate running for federal office in the 2018 U.S. election. (Image credit: Getty Images)
According to newly published research by Stanford scholars, there appears to be no political favoritism for or against either major political party in the algorithm of a popular search engine.
Stanford scholars reviewed the first page of Google search results for every candidate running for federal office in the 2018 U.S. election over a six-month period. After a systematic audit of about 4 million URLs scraped from the search engine, they found that sources from either end of the political spectrum are not being excluded from results. For the most part, the researchers found that the news sources most commonly held a relatively centrist point of view.
"Our data suggest that Google’s search algorithm is not biased along political lines, but instead emphasizes authoritative sources," said Jeff Hancock , a professor of communication in the Stanford School of Humanities and Sciences , and author on the study that recently published in the Proceedings of the Association for Computing Machinery on Human-Computer Interaction . "I think audits of large-scale algorithms that play such an important role in so many aspects of our lives are crucial. We need to be able to trust that these AI systems aren’t biased in important ways, and without audits, it’s difficult to assess these opaque algorithms."
Evaluating search media
Internet search plays a key role in today’s political process. "Recent studies have shown that web users are more likely to find and trust news through search than social media," said Danaë Metaxa , a doctoral candidate in computer science and first author on the study. "We’ve also heard concern from many sources including the White House that search may be biased (for example against conservative media), a claim that our audit methodology is well-suited to investigate systematically and thoroughly."In their audit, the researchers focused on news sources that appeared on the first page of a Google search query for every candidate running for federal office in the 2018 election. This included over 3,000 candidates running for 225 seats in the U.S House of Representatives and the Senate. Of these, 878 candidates were on the ballot in the general election.
They focused on Google searches only because of its dominant market share in web search. About 90 percent of all searches are through Google, the researchers noted.
Identifying partisanship, political bias
To evaluate political bias in search results, the researchers focused on evaluating the news sources that appeared on the first page of a Google search result for the candidate. Social media platforms, personal campaign websites and encyclopedia entries were excluded as they could not meaningfully be labeled with a partisanship score, the researchers noted.The researchers used a scoring method created by researchers at Harvard University’s Berkman Klein Center for their analysis of mainstream media coverage of the 2016 United States presidential election. In that work, the researchers assigned a partisan score to a source based on who shared their content over Twitter. They looked at some 44,000 users who they labeled as being likely conservative or liberal. Their political leanings were determined by their retweets of the two general election party candidates - Donald Trump or Hillary Clinton. Rarely did users retweet both candidates. From there, the researchers were able to evaluate partisanship.
Here, Metaxa and Hancock applied those scores to the sources that appeared in their Google search results.
Examining the distribution of partisanship in their data set, the researchers found that overall, candidates’ search results were comprised mostly of centrist news sources that were offset slightly by center-left news outlets. The researchers did identify two large peaks of highly partisan sources on the left and especially the right, but in general, they found no bias for either party. Distribution was roughly normal, with a prevalence of centrist sites, they report in the paper.
"This is good news from a trust point of view, but more audits along more dimensions are needed," Hancock said.
How incumbents and challengers compare
The researchers also wanted to know what search engine results for a candidate might reveal about how elections played out in the real world.They looked at how search engine results for candidates compared with their incumbency status. Political scientists have shown that incumbents are more moderate and centrist in their positions than challengers. Here, the researchers wanted to know whether search media would reflect that trend.
The researchers found that it did. Search inquiries for incumbent candidates resulted in a "strikingly" lower level of partisanship compared to challengers.
The researchers were also curious to know how that then compared to election outcome.
The researchers found that election winners and losers have very different search results - but these results might be attributed to a candidates’ incumbency status. Metaxa pointed out that political science has shown that a main effect driving election outcome is incumbency status; incumbents are overwhelmingly likely to win. The researchers’ data, consistent with political science theory, showed that incumbent candidates’ search results were more moderate and mainstream, compared with challengers.
Metaxa believes that with more work, there is a possibility for using search to gain real-time insight into elections, like a digital polling mechanism.
Other co-authors include James Landay , who is the Anand Rajaraman and Venky Harinarayan Professor in the School of Engineering , and Joon Sung Park from the University of Illinois at Urbana-Champaign.
This research was supported by the John S. and James L. Knight Foundation and the Stanford Project on Democracy and the Internet.