The Rise of Avatars in Automotive AI Assistants: A New Era of Interaction

Advancements in generative and agentic AI have captured headlines recently, most notably for their power to transform interaction between humans and technology across industries. But there’s another significant trend that’s also poised to change the way we speak to our virtual assistants, especially in the car.

Traditionally, voice assistants have lacked a visual presence, limiting them to one channel of interaction. With recent developments leading to increasingly human-like assistants, voice-based interaction – at its best – feels a little bit like speaking to someone on the phone. However, this mode of communication has its drawbacks compared to face-to-face interactions. In phone conversations, we often face challenges such as talking over one another, struggling to interpret tones, or inadvertently using gestures like nodding.

When it comes to interacting with an in-car AI assistant, similar issues arise. Users frequently find themselves unsure about where to direct their voice, when to speak, and whether their command is being processed or was missed. In this case, there is no opportunity for face-to-face contact to provide those essential visual cues.

However, recent developments in automotive technology are paving the way for a solution: visual and embodied avatars. With the increasing availability of screen space in vehicles, avatars – some even with a physical presence on the dashboard – can bridge the communication gap and enhance user experiences by providing visual feedback and cues that lead to smoother interactions.

Benefits of Visual Representation
Incorporating avatars into voice interfaces offers numerous advantages. Features like eye contact and facial expressions not only have the potential to increase status awareness and thus reduce errors and misunderstandings, but to also create an immersive conversational experience through more intuitive communication. Avatars can provide a recognizable face for brands, enhance recall and facilitate connection with users. In addition, the combination of avatars and AI can work to make technology feel more familiar and less intimidating.

Visualizing data that is being used by autonomous systems can significantly enhance trust. A visual avatar can increase systems’ transparency by showing users which data points are being utilized. For example, if a pedestrian is detected and the car slows down to ensure safety, visual elements can explain this decision, and thus help users understand and trust the car's actions.

Visual avatars can also illustrate the tasks AI-based agents are performing. When interacting with a navigation agent, the avatar can display a map to show that the system is retrieving directions. If the system is checking weather conditions, the avatar might use magnifying goggles to highlight its "research." This playful and informative approach enhances user understanding of next-generation systems and fosters greater trust in AI.

Visual avatars can be customized according to user preferences or dressed in seasonal outfits. Such personalization and fun can create a deeper connection between users and their cars, transforming the avatar into a digital companion users care about.

By integrating visual avatars, we aim to elevate the in-car voice interface experience and make it more engaging, trustworthy, and personalized. This vision not only enhances the functionality of AI systems but also brings a human touch to technology for a more intuitive and enjoyable user experience.

Advancing State-of-the-Art User Interaction
The interest in adding avatars to in-car assistants is growing rapidly. Various manufacturers are exploring different approaches: Renault has introduced its intelligent co-pilot avatar, "Reno," which made its debut in the Renault 5 E-Tech electric vehicle in September 2024. Following the expansion of a multi-year partnership with Renault, Cerence AI’s Chat Pro now brings new capabilities to Reno and enhances natural communication. Mercedes-Benz showcased its MBUX virtual assistant at CES 2024, including a visual representation able to show “emotions” with the goal of creating more natural and empathic interactions. In China, smart is already offering users customization features for their avatars, while NIO's Nomi stands out globally as the first avatar with a physical presence on the dashboard. Most recently, at CES 2025, HARMAN attracted attention by unveiling Luna, a figure that adds a visual component to their AI voice system, powered by Cerence Assistant.

Developing AI avatars to support voice user interfaces offers designers more than just a visual dimension. By carefully designing eye-gaze, movement patterns, and facial expressions to match the assistant's current tasks or responses, the voice user experience becomes significantly more natural and immersive. Advances in AI technology not only improve the sophistication of users’ conversations with the system, but also the entire appearance, body posture, and behavior of the visual avatar can be dynamically generated on the fly. This results in a highly responsive and lifelike digital assistant that enhances the overall user interaction.

In addition to our work with global automakers to bring in-car companions to life with generative AI, we are also passionate about research that helps us continue to understand users and their interaction preferences.

Cerence AI participated in the EMMI project, a research project that launched in September of 2020. Over a runtime of three years, Cerence AI and our partners have pushed the limits of empathic human-machine interaction, including research on increasing trust in autonomous driving through visual avatars.

Looking ahead, we are exploring user preferences and investigating new ways to merge visual avatars with smart solutions. As context-aware assistants learn to anticipate users’ needs and desires, what Microsoft envisioned with Clippy 27 years ago – a helpful, likeable co-pilot - might now finally become a reality. What was once considered a playful addition appreciated mostly by children is now on the brink of becoming a necessary element in creating the experiences of the future, with visual avatars poised to become a standard feature in any new car.