Augmented reality (AR) is used to place digital information on top of the real world. Mercedes-Benz’s MBUX navigation system head-up display (HUD) on the 2021 Mercedes S-class does this with floating arrows to inform the driver where to turn. This helpful feature reduces the need to look away from the road whilst using navigation. Panasonic Automotive has gone a few steps further and unveiled an AR HUD system at the recent annual CES technology conference.

The Panasonic Automotive AR HUD system uses artificial intelligence (AI) similar to what we’ve seen used by driver-assistance systems such as the Tesla Full Self-Driving (FSD) beta. In addition to showing the road ahead, the system detects and informs the driver about foreign objects, other vehicles, pedestrians, and cyclists. In a demo video, it even shares the height of an underpass—important information if you’re driving a tall vehicle or hauling a tall trailer. Each item is labelled by the system for what Panasonic Automotive calls a ‘seamless, more engaged, and informed driver experience.’

The HUD features eye-tracking to project images correctly regardless of where the driver is looking. If it notices that the driver’s head has turned, it adjusts the HUD elements being projected in less than 300 milliseconds to compensate. The AI system’s navigation feature has to be extremely accurate in order to render the path down the road, arrows of where to turn, and lane markings. Vibration control is also critical. A bumpy road may toss the car and driver around, but according to Panasonic Automotive, the digital icons will continue to stay locked onto their real-world targets.

Advanced optics in the sensors determine what an object is at 33 feet and beyond even in low light and at night. That’s coupled with 3D imaging radar that can capture a 180-degree field of view in front of the vehicle at up to nearly 300 feet away, plus laser and holographic technology for 4K-resolution renderings.

Source: https://www.caranddriver.com/news/a35195014/panasonic-augmented-reality-head-up-display-cars-2024/