Researchers at the University of Bonn, TH Köln, and MindPort GmbH breathe life into virtual avatars
Exterior view of the Lightstage - It enables the creation of highly realistic avatars thanks to a multitude of light sources and cameras. The research project "Gen-AIvatar - Universal creation of emotional and diverse avatars through generative AI" is receiving NEXT.IN.NRW funding of around 2 million euros. Scientists at the University of Bonn and TH Köln are working with MindPort GmbH to develop a novel, AI-supported solution for generating emotions for realistic avatars. Gen-AIvatar develops innovative and cost-effective methods for creating lifelike avatars. "Holistic animation is particularly important to us, where voice, body movement, gestures, and facial expressions are in harmony, thus expressing authentic emotions in the avatar," emphasizes Reinhard Klein, who coordinates "With Gen-AIvatar, we want to simplify the previously highly specialized and complex creation of animated 3D characters so that even people without in-depth technical knowledge can create professional, authentic avatars and use them in virtual reality scenarios, for example," adds Roland Klemke from the Cologne Game Lab (CGL) at TH Köln. Together with David Lähner, CEO of MindPort GmbH, and their teams, they are developing the "Gen-AIvatars."
Researchers at the University of Bonn are setting up the Lightstage - where real people can be transformed into virtual avatars using a large number of cameras.
Wide range of applications The emotional avatars are also intended for practical use. This could include training courses for human resources management or customer service, for example, to practice how to deal with angry customers. "Virtual reality training has already been successfully established in many companies for hard skills training. There is also a high demand for soft skills training, but the methods currently available for animating avatars pose a major hurdle. Either they are very expensive or the quality is not sufficient for efficient training. This is exactly where we come in with new AI methods," explains David Lähner from MindPort GmbH. In addition, further applications are being considered, for example in virtual experiments in neuroscience.
’Gen-Alvatars’ are designed to reflect human emotions through realistic facial expressions
Gen-AIvatar aims to offer developers, teachers, and companies the opportunity to implement high-quality virtual reality applications with emotionally responsive virtual characters without high costs or time expenditure. This approach is being practically tested within the project by the Cologne Game Lab at TH Köln, which will develop a virtual reality presentation training program based on this AI.
Excellent infrastructure for AI research
The technological basis for the project is provided by the newly created Visual Computing Incubator (VCI) at the University of Bonn: a state-of-the-art light stage. This dome-shaped arrangement of controllable light sources and synchronized cameras makes it possible to simultaneously capture people and their interactions with objects or other people from hundreds of perspectives. "The data obtained in this way forms the basis for the development of highly realistic avatars that faithfully reproduce even the finest nuances such as skin textures, reflection properties, and facial expressions. A particular focus here is on the authentic representation of a broad emotional spectrum," explains Prof. Klein.
Here, researchers are investigating the AI-based production of realistically animated historical clothing for virtual reality, film, museums, and games.
