Multimodal Interaction

Go beyond voice with the most intuitive interaction available. Speak, gesture, write, or glanceyour automotive assistant will understand.

Our industry-leading AI for cars just got smarter.

Cerence sets the bar for voice in cars, building on decades of experience using conversational artificial intelligence and deep learning to perfect the way machines speak and listen. But sometimes it’s more convenient to communicate with a flick of the hand, a scribble, or a pointed look—so we’ve evolved Cerence Drive to understand handwriting, gestures, emotions, and even the human gaze.

Multimodal interaction optimizes safety and user experience by accommodating whatever kind of communication comes naturally in each driving situation, including speech, handwriting, gaze detection, and other nonverbal modalities. Like a friend, your assistant just gets it, shifting seamlessly from one mode of human interaction to the next.

 

Flexible, adaptive, and future-proof

Cerence Drive is ready for future mobility, accommodating drivers with interaction that enhances focus, optimizes productivity, and adapts to their changing needs and responsibilities.

Versatile

On a conference call? Users can write commands or questions in lieu of speaking. If they’re curious about an attraction, shop, or restaurant on the roadside, they can just look at it to launch a targeted dialogue with the assistant.

Safer

By adapting seamlessly to situational human interaction, the multimodal assistant helps minimize distraction and frees the driver to focus on the road.

Ready for Autonomous

Multimodality will be especially vital in cars of the future, when autonomous driving will enable users to transition to a passenger role while staying prepared to take the wheel. Not only will gaze detection allow effortless engagement with the landscape (enhancing the sightseeing that users expect from future mobility, studies show), but multimodal alerting will optimize safety by combining vibration, visual cues, and auditory prompts to tell drivers it’s time to take over.

 

Just one look: Gaze – a new standard for natural human interaction through AI

Cerence Drive is designed to give your assistant the intuition of a human copilot.With gaze detection, the interaction is easier and more natural than ever before. It’s like talking with a friend about passing sights—a friend who happens to have advanced GPS, encyclopedic knowledge, and inexhaustible memory for individual habits and preferences.

Ease for users

A user needs only to look at an object of interest—in or outside the vehicle—and speak casually about it for the assistant to direct its intelligence to the same object. Whatever is said—from “Who built that?” to “Is there a children’s menu there?” or “What does this red light mean?”—the assistant will understand and deliver an accurate, contextual response—without the need for prompts, scripts, or keywords.

More human artificial intelligence

Multimodal interaction combines eye trajectory and head angle with precision car positioning and 3D environmental modeling to track users’ gaze. Thanks to Cerence Drive’s Just Talk feature and contextual reasoning, users can engage conversationally, without a “wake-up” word, and the assistant knows exactly what they’re referring to.

 

Noteworthy reading skills: Handwriting recognition

Though best known for advanced speech recognition and AI, Cerence Drive is equally skilled in its ability to read. Fluent in 50+ languages and infinite writing styles, it applies text-to-speech translation to messages written on a simple pad, mounted on the center console.

Flexibility

No need to write neatly: your assistant reads print or cursive, recognizes single and multi-characters, and can decipher a messy scrawl. Informed by artificial intelligence and Deep Learning based on neural networks, it has the intelligence to save time by predicting what users want to say or ask.

Convenience

When users need to communicate quietly, multitask while speaking, or just jot something down, it’s a relief to have an assistant that understands handwriting. Potential uses include composing messages, selecting contacts, scribbling destinations, and changing the radio station.

Focus

Cerence Drive’s literacy allows drivers to write when it comes naturally—or when speaking and gaze are not possible. To keep drivers’ eyes on the road, feedback and suggestions are provided in direct sight or in audio.