Researcher looks at how software design controls our interactions with technology

CHAMPAIGN, Ill. — University of Illinois professor Ben Grosser sees his fingers as engaged in a choreographed dance as he uses the trackpad on his computer or scrolls through a list of on his phone. The software is directing the way he, and all of us, move as we use our technology all day, every day.

“I started to become aware, not only of how active my fingers are in manipulating these interfaces, but also of how choreographed my fingers feel. It’s a set of movements. It feels like I’m being driven, as much as I’m driving,” said Grosser, a professor of new media in the School of Art and Design and a researcher with the National Center for Supercomputing Applications , where he is developing a new initiative in critical technology studies.

“I’m always interested in something we’re doing that we think of as natural, but that is part of a designed experience,” he said.

Grosser is developing a new initiative in critical technology studies at the National Center for Supercomputing Applications. He made a video last year of his hands manipulating the trackpad on his computer, which was part of a faculty art show at Krannert Art Museum.

Photo by L. Brian Stauffer

Grosser made a video last year of his hands manipulating the trackpad on his computer.

“Look at how delicate these movements really are. There’s a lot of soft touching, almost caressing,” he said.

Grosser recently made a video supercut of scenes from the Netflix series “House of Cards” showing characters in the show using technology. It is the first in a series of three videos for his “ Touching Software ” project.

He is interested in both how actors manipulate the props and how the props manipulate the actors. For example, one scene shows a man caressing the mouse of his computer while scrolling the image of a woman on the screen. Other scenes show a character’s finger or hand movements being directed by software while illuminated by the glow of a bright screen in a dark room. Grosser noted the show has used novel ways to portray technological concepts, such as showing the content of text messages onscreen as the characters are texting one another. Yet little attention has been paid to these other human-tech interactions.

Grosser noted that "House of Cards" initially focused on showing the content of messages onscreen, but it now spends more time featuring personal uses of technology in scenes without any dialogue.

GIF created by Ben Grosser

He’s seen a progression in how technology is portrayed over the span of the series, from early experiments that focus on interface content to later examples that feature quiet and personal uses of technology in scenes without any dialogue. Like Grosser himself, the characters are compelled to touch the items.

“(The designers) want to make devices that almost create an emotional connection. Something about the seamlessness of these objects – the smoothness, the metal and glass – gets you connected to them,” he said.

He is also interested in how data on viewers’ habits drives the plot of “House of Cards.”

“It’s well-known as a show that is partially driven in content based on surveillance analysis of what viewers are doing,” Grosser said. “It’s an algorithmic show. (Netflix) has the ability to know so much about us, and they can make shows specifically to activate emotions.”

Grosser said scenes from “House of Cards” often show a character’s hand movements illuminated by the glow of a bright screen in a dark room.

GIF created by Ben Grosser

One of the main components of the show is the ubiquitous use of technology.

“The more a show portrays technology seamlessly – with everyone in constant communication – the more normalized that behavior becomes in everyday culture, and the more it supports the narrative of technology as neutral,” Grosser said. “As humans, we have the responsibility to be critical of something we give so much power to.”

We also need to be aware that while a piece of technology is designed with an intended purpose, that is not the only thing for which it is being used, he said. For example, Facebook is intended to allow us to stay connected with one another, but it is also collecting information about us and showing us ads to click on, Grosser said. And because Facebook quantifies all social activity, those metrics affect what we like and who we friend. Adobe products are meant to enable creation, but how those products are designed affects what we create with them, he said. Intentional or not, software design reflects the viewpoint of its creators.

“The tendency to think about the technology we engage with all day long as natural is problematic,” Grosser said. “Software represents a particular point of view because of who made it and why they made it.”

He is planning on more video supercuts and is currently choosing another show that uses technology as a plot device. He is hoping to find shows to use on HBO and Amazon, to compare how these three new media networks portray technology.