In a new research paper, Google has detailed the tech behind its impressive Project Starline demo from this year’s I/O conference. Project Starline is essentially a 3D video chat booth that aims to replace a one-on-one 2D video conference call with an experience that feels like you’re actually sitting in front of a real human being.
It sounds simple, but Google’s research paper highlights just how many challenges there are in tricking your brain into thinking there’s a real human being sitting just a few feet away from you. Obviously the image needs to be high resolution and free of distracting artifacts, but it also needs to look correct from your relative position in the booth. Audio is another challenge, as the system needs to make it sound like a person’s words are coming from their actual mouth. And then there’s just the small matter of eye-contact.
But, eventually, the hope is that Project Starline could offer a similar feeling of presence as virtual or augmented reality, without users needing to wear bulky headsets or trackers.
The display unit and its various tracking hardware.
The paper details exactly how much hardware is needed to start to solve these problems. The system is built around a large 65-inch 8K panel which runs at 60Hz. Around it, Google’s engineers have arranged three “capture pods’’…