Unpacking the ’black box’ to build better AI models

Stefanie Jegelka, a newly-tenured associate professor in the Department of Electrical Engineering and Computer Science at MIT, develops algorithms for deep learning applications and studies how deep learning models behave and what they can learn. Credits : Photo: M. Scott Brauer - Stefanie Jegelka seeks to understand how machine-learning models behave, to help researchers build more robust models for applications in biology, computer vision, optimization, and more. When deep learning models are deployed in the real world, perhaps to detect financial fraud from credit card activity or identify cancer in medical images, they are often able to outperform humans. But what exactly are these deep learning models learning? Does a model trained to spot skin cancer in clinical images, for example, actually learn the colors and textures of cancerous tissue, or is it flagging some other features or patterns? These powerful machine-learning models are typically based on artificial neural networks that can have millions of nodes that process data to make predictions. Due to their complexity, researchers often call these models "black boxes" because even the scientists who build them don't understand everything that is going on under the hood. Stefanie Jegelka isn't satisfied with that "black box" explanation. A newly tenured associate professor in the MIT Department of Electrical Engineering and Computer Science, Jegelka is digging deep into deep learning to understand what these models can learn and how they behave, and how to build certain prior information into these models.