Google is working on abooth that uses 3D imagery on a 3D display to create a lifelike image of the people on both sides. While it’s still experimental, “Project Starline” builds on years of research and acquisitions and could be the core of a more personal-feeling video meeting shortly.
The system was onlyof unsuspecting participants, who were asked to enter a room with a heavily obscured screen and camera setup. Then the screen lit up with a of a loved one, but in a way none of them expected:
“I could feel and see her; it was like this 3D experience. It was like she was here.” “I felt like I could touch him!” “It felt like she and I were in the same room.”
CEO Sundar Pichai explained that this “experience” was made possible with high-resolution cameras and custom depth sensors, almost certainly related to theseof people and locations into interactive 3D scenes:
The cameras and sensors — probably a dozen or more hidden around the display — capture the person from multiple angles and figure out their exact shape, creating a live 3D model of them. After a lot of compression and processing, this model and all the color and lighting information are then (after a lot of compression and processing) sent to the other person’s setup, which shows it in convincing 3D. It even and bodies to adjust the image to their perspective. (There’s more on an early version of the technique here.)
But 3D TVs have fallen by the wayside; no one wants to wear special glasses for hours, and the quality of glasses-free 3D was generally pretty bad. So what’s making this particular 3D image?
Pichai said, “We have developed a breakthrough light field display,” probably with the help of the people and IP it scooped up from Lytro, the light field cameraoff the ground and dissolved in 2018. Light field cameras and displays create and show 3D imagery using various techniques that are to explain or show in 2D. The startup Looking several incredibly arresting to view in person, showing 3D models and photographic scenes that genuinely look like tiny holograms.
Whether Google’s approach is similar or different, the effect appears equally impressive, as the participants indicate. They’ve been testing this internally and are getting ready to send out units to partners in various industries (such as medicine) where the feeling of a person’sdifference. At this point, Project Starline is still a prototype and probably a costly one — so don’t expect to get one in your soon. But it’s not wild to think that a consumer version of this light field setup may be available down the . .