The onboard AI Personal Assistant employs AI and connected car technologies to deeply understand the passengers and to develop with them in delivering the most appropriate kind of information to make the car trip as safe and reassuring as possible, as well as in supporting communication between the driver and family members.

1) Voice-operation

AI Personal Assistant uses the latest AI technology (speech recognition and natural language processing) to correctly understand passengers’ speech and to converse with them using natural-sounding language even amid the noise generated when the car is on the move. Voiceprint identification and a camera allow it to identify individual passengers, to learn about them from their conversation and movements, and to provide services that best match their tastes and needs. AI Personal Assistant also features the hybrid speech recognition system that combines cloud speech recognition (AI-based speech recognition via cloud) and local speech recognition (AI-based speech recognition via onboard system). The local system provides speech recognition accuracy equivalent to the cloud system with a lower latency even in an environment where there is no network connection.

2) Context recognition

AI Personal Assistant uses speech recognition technology and gesture recognition technology to recognize the driver’s intentions and states (actions, emotions^*1, drowsiness, concentration, identification, etc.). Based on this recognition, it provides appropriate information at suitable times.

3) Connectivity with diverse device and content

AI Personal Assistant supports connectivity with various devices and contents. For example, it can connect with Pepper the personal robot by Softbank Robotics Corp. in one’s living room to enable remote operation of the vehicle’s air conditioning; check the vehicle’s gasoline level, door lock status, etc.; and communicate with household members for a glimpse of what robots will offer in the future.

The MMC booth will also feature demonstrations of its Android Auto™ compatible app “EV Support,” which enables a driver to search for EV charging spots via in-car display. MMC is the first automaker to distribute a genuine Android Auto™ compatible app*³.

Other attractions include: Smart Door Bell (beta version) which connects with the home doorbell over the Internet and allows passengers to check on and communicate with visitors to their home while the vehicle is on the move; and Driving Data Recorder (beta version) which uses a compact camera to record steering angle, vehicle speed, and other vehicle information to offer a new way to enjoy car trips.