Comment on this article

Fact-checking Images

by Soo-yeon Hwang

Fact-checking images is tough. There are many fact-checking websites out there, and Google even has a separate meta search engine dedicated to fact-checking[1]; yet, for images, you must be able to sufficiently describe the images in words. Or, you can do a Reverse Image Search[2] and see if the results include a fact-checking source, which can be a hit or miss. Moreover, not all images online have been verified by fact checkers.

Two months ago, Google introduced a feature to find fact-checking information in Google Images search results.[3] In addition to the fact check features in Google Search and Google News, Google Images search results are now supposed to include a “Fact check” label under the thumbnail image results. Here is what it actually looks like in a search (the label appears next to the website URL under the thumbnail; it is in red square below):


You can find a few problems here: First off, the label is very hard to notice. Second, it appears next to only one source (PolitiFact) out of the entire search results. The second problem may be because Google relies on third-party fact-checkers and on top of that, a tool called ClaimReview which publishers can use to communicate to search engines that an image has been verified. Therefore, if a fact-checking service is not publishing their contents via ClaimReview, the label does not appear in Google Images search results.

Another test of the Google tools shows their shortcomings. When searched the Google Fact Check Explorer (see the link in reference #1) with the phrase “nuclear plant deformed daisy” (without the quotation marks,) there was no search result. This phrase represents a piece of misinformation that deformed daisies have been found near the Fukushima nuclear power plant in Japan, and they were mutated due to nuclear radiation. However, according to Snopes, a fact-checking website, the mutation due to nuclear radiation part is false.[4] Mutated daisies have been found in other surroundings and it does not have to do with nuclear radiation per se. Google Images search results with the same phrase show no “Fact check” label, either, probably due to the issue with ClaimReview -- in other words, no fact-checkers using ClaimReview had contents about this particular misinformation.

With artificial intelligence (AI) becoming more and more sophisticated, the fact-checking problem gets worse. The advent of AI tools has made it much easier to manipulate graphic media (images and videos) with confusingly realistic results; the technologies are not perfect yet, but are getting harder and harder to detect with naked eye, surprisingly quickly.[5] It is called “deepfake” , and deepfake detection is currently a hot research topic in computer science; which means that there is no definitive answer to the problem yet.[6]

Why do we care about fact-checking images? In 2015 - 2016, Stanford University researchers showed 170 high school students a social media post with the image of the “nuclear plant deformed daisy” and assessed their responses about how they evaluated the information.[7] The results look troublesome: the researchers say that the students “were captivated by the photograph and relied on it to evaluate the trustworthiness of the post.” Less than 20% of the students questioned the source of the social media post or the source of the image. Nearly 40% argued that the post provided strong evidence because it offered pictorial evidence. 25% argued that the evidence was not strong because it showed flowers and not other forms of living beings. Overall, the inclusion of a photograph in the social media post greatly influenced the information validity assessment of the high school students.

Again, fact-checking images is not easy, and the problems are getting worse with advancing AI and deepfake technologies. Evidences such as the Stanford study that shows the power of imagery in information evaluation present a unique challenge to the librarians who teach information literacy. Therefore, in addition to utilizing technologies to check the trustworthiness of images, which is not perfect, librarians would need to teach the students and the public how to best evaluate online information that includes graphic media. In this arms race with fabricated information aided by multimedia, teaching users how to best navigate content is as important as technological advancement.

Copyright 2020 by Soo-yeon Hwang.

About the author: Soo-yeon Hwang is the Web Services Librarian and Assistant Professor at Sam Houston State University. She has a PhD in Communication and Information from Rutgers University, and MS in Information from the University of Michigan, Ann Arbor. She has professional experience in software development, technical writing, testing, and technical support.

[6] Here is a digest of some of the current deepfake detection approaches: