Dietrich Graduate Students Advance Neuro Studies with AI

Three graduate students from Carnegie Mellon University’s Dietrich College of Humanities and Social Sciences - Andrew Luo ,  Joel Ye and  Gabriel Sarch - are using artificial intelligence to advance neuroscience research.

Their work, co-authored with faculty and/or post-doctoral researchers from the  Department of Psychology ,  Neuroscience Institute and  Department of Machine Learning , presents new or improved methods for understanding human vision and cognition.

Luo, a Ph.D. candidate in the Neuroscience Institute and Machine Learning Department, has developed Brain Diffusion for Visual Exploration (BrainDiVE), a data-driven approach to synthesize images predicted to activate different regions of the brain. This technique bypasses the current approach that requires category-specific stimuli in order to characterize differences between regions of the brain. With this new technique, researchers can improve their understanding of human visual cortex using diffusion models. Luo worked in collaboration with Margaret M. Henderson, post-doctoral research in the Neuroscience Institute, Leila Wehbe, an assistant professor in the Neuroscience Institute and Machine Learning Department, and Mike Tarr, the Kavcic-Moura Professor of Cognitive and Brain Science, on the article that is available on  Arxiv , a preprint server for articles published prior to peer review.

Ye, a Ph.D. candidate in the Neuroscience Institute, has developed the program Neural Data Transformer 2 (NDT2), a spatiotemporal transformer for neural spiking activity. The model reads the pattern of spikes that occur when neurons in the brain fire in a coordinated fashion. This approach improves upon current models that are limited to individual experiments, robbing deep network models of a vast amount of potential data. NDT2 aims to leverage the various data sources that span multiple sessions, participants and experimental tasks. The ability to aggregate data opens new opportunities to decode behavior from brain activity efficiently. Ye worked in collaboration with Wehbe, Jennifer Collinger and Robert Gaunt, both assistant professors in the neuroscience institute, on the article published on BioRxiv , a preprint server for articles published prior to peer review.

Sarch, a Ph.D. candidate in the Neuroscience Institute and Machine Learning Department, has developed a technique to train neural networks to predict how the brain will respond to images from large-scale datasets of natural scenes. Sarch has enhanced this technique by using a process called "brain dissection" to identify how different brain regions selectivity encode spatial features, such as depth, surface normals, curvature and object relations. Understanding how different areas of the brain selectively encode these spatial characteristics enhances our knowledge of how a coherent 3D percept of the world is constructed. This work will improve how researchers approach functional characteristics of vision, specifically the human visual cortex, when viewing natural scenes. Sarch worked in collaboration with Tarr, Wehbe, and Katerina Fragkiadaki, the JPMorgan Chase Associate Professor of Computer Science in the Machine Learning Department, on an article to be published in the Neural Information Processing Systems proceedings and available publicly on  BioRxiv.

All three graduate students are presenting their work at the 2023  NeurIPS conference , Dec. 10-16 in New Orleans.

"Carnegie Mellon’s graduate students shine with their cutting-edge blend of disciplines in probing the mysteries of the mind and brain," said  Tarr , who also is head of the Department of Psychology. "At NeurIPS, their presentations showcase an impressive command of both Computational Neuroscience and Machine Learning, reflecting deep, foundational expertise."

CMU Neuroscience Major Researches Mindfulness Meditation

Smith Named Co-Director of CMU-Pitt Neuroscience Partnership

Celebrating 10 Years of Dietrich Foundation Support