In a light-filled room at Galerie Eigenheim in Weimar, guests slip on VR headsets and headphones - just like their counterparts in Aachen, Lübeck, and Ilmenau. Seconds later, they find themselves sharing the same virtual environment, even though they are physically miles apart. Instead of static tiles on a screen, they appear to one another as avatars: smiling, waving, moving freely through the space, and even shaking hands. Voices sound so spatial and lifelike that you almost feel tempted to lean in and whisper to the person next to you.
This is the final demonstration of the MULTIPARTIES research project, carried out by TU Ilmenau in collaboration with Consensive GmbH in Weimar and Brandenburg Labs in Ilmenau. The project’s goal: to make video conferencing feel more natural-making digital work and communication easier, more engaging, and more human.
The problem is familiar to many: after just two hours of video calls, you feel drained, even though you haven’t left your chair. Your eyes keep drifting to your own camera feed, your colleague from Hamburg seems strangely distant, and conversations falter the moment several people speak at once. Online meetings have transformed the workplace-but they rarely feel like real meetings.
More than just faces in tiles
This is where MULTIPARTIES comes in. The project explores how extended reality (XR) can make virtual encounters so vivid that you almost forget you’re not in the same room.To create this sense of co-presence, the partners developed a 3D communication system based on Consensive GmbH’s VR4More platform and advanced spatial audio technology from Brandenburg Labs. The result goes far beyond a grid of faces on a screen. Participants appear as realistic 3D avatars with gestures, facial expressions, and positional sound. Turn your head in the virtual space, and you’ll hear someone’s voice coming from the side in real time. You can walk over to them, form small groups, or even greet each other with a handshake. All of this unfolds immersively through VR headsets and headphones - making digital interaction feel strikingly natural.
To make this possible, three research groups at TU Ilmenau pooled their expertise and made use of the cutting-edge facilities at the VR Center of the Ilmenau Interactive Immersive Technologies Center (I3TC).
A research team from Virtual Worlds and Digital Games (VWDS) examined how avatars, menus, and virtual tools need to be designed so that XR meetings feel as natural and intuitive as possible - even when participants not only want to talk to each other, but also collaborate with objects in the room. Head of Group Prof. Wolfgang Broll explains:
Our research shows that even subtle differences in avatar designfor example, whether the representation is more stylized or highly realistic - can have a major impact on how participants perceive and interact with one another in a virtual meeting.
A second team, from the Electronic Media Technology Group (EMT), developed methods to integrate spatial sound into the environment as seamlessly and realistically as possible. Head of Group Dr. Stephan Werner explains:
We explored how to capture rooms acoustically with the highest possible precision and how to use the resulting data for machine learning. This enables us to predict how people perceive spatial sound - and to adapt audio playback in real time. That’s essential to ensure that virtual conversations sound just as effortless as face-to-face ones.
Finally, researchers at the Audiovisual Technology Group (AVT), investigated under the leadership of Prof. Alexander Raake how users experience this new form of communication - for example, how different visual representations affect perception and how participant behavior changes. Their work focused on aspects of audiovisual quality as well as the so-called quality of experience. The team not only presented their findings at international conferences but also released an open-access dataset. At the same time, their insights fed into the work of international bodies such as the International Telecommunication Union, with the goal of developing global standards for extended reality technologies-standards that will allow the quality of XR applications to be assessed consistently and reliably.
Greater social proximity and opportunities for social participation
For audio pioneer Prof. Karlheinz Brandenburg, CEO of Brandenburg Labs and senior professor at TU Ilmenau, the project marks an important step forward:
This project has brought us closer to the goal of making digital work and communication more natural. MULTIPARTIES is an important milestone in breaking down barriers in video conferences and fostering a greater sense of social closeness. In doing so, we are also opening the door to new forms of social participation.
The results are in fact promising well beyond video conferencing. In the future, they could be applied to smart headphones, industrial remote maintenance, hybrid conferences, or even museum experiences. What is still a research demonstrator today could, within just a few years, become part of everyday life: online meetings that truly feel as if you are there in person.
