A collaboration between Carnegie Mellon University’s Frank-Ratchye STUDIO for Creative Inquiry and the Carnegie Museum of Art aims to identify, annotate and organize the massive body of work of photographer Charles "Teenie" Harris.
The project has been awarded a National Endowment for the Humanities grant to create a set of image identification tools using machine learning and computer vision techniques. The software developed by the STUDIO will be open-source and compliant with international digital image standards, allowing the tool to be applied to collections across the globe.
From the 1930s through the 1970s, Harris chronicled life in Pittsburgh’s black neighborhoods for the Pittsburgh Courier, one of America’s most influential black newspapers of the 20th century. He left behind an archive of more than 80,000 photographic negatives over more than 40 years. When the museum acquired his body of work in 2001, most came without identifying information.
The Carnegie Museum of Art has been working to add information, such as names of individuals, to the digital images through interviews with contemporaries of Harris and the original community members documented in his photographs. To date, about 2,000 images, or just 2 percent, have been identified.
"Image scans and a digital catalog have allowed us to share Harris’ work faster and with more people than traditional paper-based methods; moreover, digital technology allowed people to contribute their memories and knowledge to the archive more efficiently than ever before," said Louise Lippincott, curator of fine arts at Carnegie Museum of Art. "This new collaboration with CMU will provide us with another source of insight and understanding into the photographer’s methods and subjects-a topic of concern to everyone who works with images and archives."
The idea for the project arose from School of Art Professor Golan Levin’s undergraduate Interactive Art and Computational Design class. Zaria Howard, a junior in the School of Art pursuing a Bachelor of Humanities and Arts degree in art and statistics, developed a keen interest in the archive and the complex problems of identifying individuals, locations, dates and other visual elements across the collection. Golan, who is the director of the STUDIO, is leading the project, along with David Newbury, Enterprise Software and Data Architect at J. Paul Getty Trust.
"It’s exciting to be able to use computer vision and ML to add detail to the narrative Teenie Harris left Pittsburgh," Zaria Howard said. "Using technology to rediscover Black history is rare and yet crucial to understanding American history as a whole."
The new software will create machine-learning-based tools for image analysis and annotation.
For example, the software will be able to identify individual faces that appear in different images throughout the archive. These potential matches could then be shown to interview subjects who could confirm whether or not the matches are correct.
As a person interacts with the software, it will become "smarter" and better-equipped to make more accurate matches. A similar protocol could be used to identify other objects, locations or features in the photographs and add this metadata automatically to the images. This process would make the archive more easily searchable for researchers and the public.