VR eye tracking could shape your relationships with virtual people

Well over a year ago, I tried a Kickstarter-funded virtual reality headset that could track not just your head, but your pupils. It was called Fove, and it’s now open for preorders, with a price of $599 and a shipping date of early 2017. (Kickstarter backers will get their versions toward the end of 2016, and people who preorder in the first week will get a discounted price of $549.) At the VR Developers Conference in San Francisco, I was able to test out the latest version, and it’s come a long way — even if its creators admit that it’s ready for everyday use just yet.

Outside VR, eye tracking in entertainment is usually an aid for aiming and navigating in video games, whether that’s shifting the camera in an action game playing Asteroids with your eyes. Inside VR, it’s most practical for foveated rendering, where a headset saves processing power by rendering the part of a screen you’re directly watching in fine detail and blurring everything else. This could let people make more complex and beautiful VR experiences, or drive down the cost of VR-ready PCs, and multiple companies are working on it.

Fove has demos that showcase both these options, including a turret shooter and a detailed still scene with foveated rendering. But it’s also working on a more experimental project that CEO Yuka Kojima describes as interactive cinema. In this case, that’s a short vignette where you play a soldier being interrogated by a terrorist group. During the scene, looking at various parts of the environment triggers specific events, shaping the course of the story.

Some of these moments are simple: if you look down at an ashtray while your interrogator asks questions, he’ll shout at you and sweep it off the table. Others would have been difficult to detect if Kojima hadn’t been there pointing them out. If you examine the flashbang grenades on a terrorist’s uniform, for example, you fade into a full-fledged flashback, and a fellow soldier later rescues you with a flashbang attack. I only got to that point, apparently, because I’d followed Kojima’s instructions to avoid looking at the soldier in a photo lineup earlier, throwing the terrorists off his trail.

This kind of interactive storytelling is nothing new for video games. But compared to hitting buttons on a controller or even moving your hands in a VR game, eye motion is almost involuntary. With simple headset tracking, you can point your face in one direction and furtively glance in another. When Fove is watching your pupils, there’s no such option. The computer always knows what you’re doing.

You can’t trick the computer

All of this is mostly theoretically interesting, of course. Fove has a single experience that’s a few minutes long; since it was only in Japanese, I couldn’t even understand the dialog without Kojima’s explanations. For now, she says Fove is aiming at selling to developers and public installations like arcades; headsets are apparently being installed in 7,000 Korean and Japanese internet cafes next February in an attempt to kickstart development. The headset doesn’t have a motion controller like the Rift, Vive, or PlayStation VR, and its catalog isn’t as robust.

The hardware is commendably solid for an indie VR headset. Its 2560x1440 screen (the same resolution as the Gear VR) is crisp, and the head tracking, which uses the same camera as the OSVR HDK 2, is nicely precise. But it doesn’t rival the industrial design of the Rift or PSVR, nor does it have the room-scale tracking of the Vive. It’s a product for people who are excited about building for one specific, experimental VR technology, not a credible competitor to big-name consumer headsets. By the former metric, though, it’s made great strides since 2015 — and it’s pushing forward in areas that, so far, we haven’t seen companies like Oculus and Valve talk much about.