Car infotainment systems are getting bigger, but a startup's new prototype goes further than something like Ford's pillar-to-pillar touchscreen: It turns the windshield into a full-color 3D head-up display .
Distance Technologies, founded by the co-founders of enterprise headset maker Varjo, showed off its design last week at the Augmented World Expo in Long Beach, California. The company's goal is to take front-facing displays beyond simple, flat overlays. Its early efforts are impressive, but they also illustrate how difficult (and risky) it could be to mount it on a real car.
Distance Technologies' prototype uses an LCD panel pointed at a windshield with a reflective coating, projecting a transparent image onto it. It is conceptually similar to some existing car HUDs, which can be found in a host of aftermarket accessories and integrated options from companies like Volvo and Mercedes-Benz. But while these tend to involve flat, ghostly projections on a narrow portion of the windshield, this system is much broader and convincingly spatial.
Distance Technologies' prototype works a bit like a projected version of a 3D monitor without glasses. Its large LCD panel is covered with a parallax barrier that can display a slightly different image to each eye, while tracking cameras help determine where the driver is looking and redraw the image accordingly. Co-founder and CMO Jussi Mäkinen says the parallax barrier hardware is just a stopgap while they iterate on the software technology: “our core is in software-defined optics,” Mäkinen says. CEO Urho Konttori was a bit cagey about the precise mechanics in an interview with my colleague Sean Hollister, but overall the effect is a semi-transparent 3D screen that covers half of a normal-sized windshield for the driver.
I tested the prototype in controlled conditions outside the car (a hotel room with a slanted glass simulating a windshield) and got a sense of its capabilities. The projection creates a large, fairly sharp display that allows for anything from common HUD elements like speedometers to detailed 3D displays. Vehicle makers can integrate voice and gesture controls, and the prototype is connected to an Ultraleap hand tracker, so you can do things like press a notification to accept a phone call without taking your eyes off the road. The full-color projection can display video, meaning features currently relegated to car infotainment screens, such as streaming a rear-facing camera, could be located on the side of the windshield.
The most eye-catching hypothetical features involve things like AR night vision, but there's a long way to go
The most striking hypothetical features would take advantage of lidar or other vehicle sensors to add sophisticated augmented reality elements. In cars, Distance Technologies hopes the display can realistically place virtual signs in its surroundings or paint over areas of darkness with night vision scans. In airplanes, pilots could see a detailed 3D topographic map projected in the cockpit without needing to look through a headset or eyepiece.
But the prototype, developed over the past few months, has many of the rough edges one would expect from a new display technology. It apparently costs a lot more than the hundreds of dollars price Distance is pointing at manufacturers. It's also too dark. The demo I saw was about 100 nits bright, enough to see in a hotel room, but virtually invisible in a bright outdoor environment, where you'd need something closer. 10,000 nits. And a large LCD panel is required to project a proportionally large screen.
Eye tracking adds its own set of challenges. The screen image is constantly redrawn to compensate for where the driver is looking, but at the moment there's incredibly high latency, so some demos falter when you move. The sensors seem to lose track of your eyes easily: looking away does it, but apparently so does having long hair or wearing a hat. And if this happens, everything goes crazy. That sentient 3D projection becomes a set of distorted lines crisscrossing the windshield. It's a downright terrifying driving scenario which means good safety measures will be vital.
But my biggest problem, by far, was that using the screen just hurt. I started by looking at some of the more complex demos: a 3D representation of a topographic map and a car navigating through a city. (In real life, the former could be used by airplane pilots, the latter reduced to a corner minimap for cars.) Within minutes, I felt like I was going a little cross-eyed. During simpler experiences, like a flat projected speedometer, my eyes still felt noticeably tired. After leaving the room, I had a slight headache for hours.
I have a slightly unusual vision situation (one of my eyes is slightly nearsighted and I don't wear glasses) and while this rarely causes problems when driving in real life or with normal AR headsets, it might explain parts of my experience. That being said, I spoke to another person who commented that the demo also made their eyes tired. If even a sizable minority of drivers experience the same thing, that seriously undermines Distance Technologies' narrative.
My eyes were working here more than usual, not less.
The startup's dream is for a driver to be able to see useful information mixed with the landscape, so they're not constantly shifting focus between their surroundings and their dashboard. And prototyping can sometimes achieve impressive results. In a demo that allowed me to move a video around the scene, I found a position where I could barely move my eyes between it and my real-life surroundings. (Maintaining that position while driving 60 miles per hour, of course, could pose more challenges.) But overall, my eyes were working harder than usual, not less.
Konttori says Distance Technologies is still defining its target market. The team is reportedly in touch with some consumer automakers, looking to establish the kind of relationships they developed at Varjo, whose partners include Volvo and Kia. The company could also end up focusing on niche markets like military vehicles or airplanes, where HUDs are already likely to be found.
If Distance Technologies solves the prototype's big problems and finds commercial partners, we will have to hope that automotive companies use the technology responsibly. Drivers are already irritated and sometimes distracted by an excess of screen-related functions. AR adds a number of new potential points of failure. In the near future, it's hard to see truly important information moving permanently to your windshield via this prototype, at least without a backup system. Still, it's definitely better than driving a Vision Pro.
