The author demonstrates "holding" a virtual object with a SoftKinetic-powered Oculus Rift.
The coolest announcement of CES 2016, to me, was the news that Lenovo and Google are working together to put the search giant's nifty Project Tango sensors into a forthcoming consumer smartphone.
Project Tango, like similar efforts from Microsoft, Apple, and Intel, is trying to give smartphones and other devices the ability to perceive depth and space the way humans do.
That sounds nerdy, but the potential effects of the shift towards depth perception are profound: Get ready for apps that help you measure furniture or guide you straight to your friends in a crowded club.
It's the future of tech, as our devices learn to make more sense of the world around us. And it's just starting to sneak into all the tech we use.
SoftKinetic, a startup that Sony bought in late 2015, has been doing this for a while now. In fact, it's the technology platform behind the new BMW 7 Series' very high-tech dashboard gesture controls, where a wave or a swipe answers the phone or turns up the music.
Check it out in action:
This is just the first wave of what this technology can do, says SoftKinetic CMO Eric Krzselo.
At first, Krzselo says, this technology's most natural on-ramp will be via the smartphone, since depth perception software can help tremendously with camera features like autofocus.
But once the sensor is in more phones, developers can give their apps the ability to "read" where the human hand is in three-dimensional space, as demonstrated in the BMW 7 Series, it means you can also have people interact with digital objects in virtual or augmented reality, with their own hands.
In plainer terms: I got to use an Oculus Rift virtual reality headset, souped up with a SoftKinetic sensor, that let me wave my hands in front of my face and see them reflected in the screen. I didn't need a controller or a keyboard to grab and throw boxes around in virtual reality. All I needed to do was wave.
Matt Weinberger
Google's Project Tango let me run around and shoot robots.
As a Sony subsidiary, SoftKinetic obviously has strong relationships there, and this could well make its way into a future version of the Sony PlayStation 4 virtual reality headset, Krzselo says.
In another virtual reality demo, I used SoftKinetic's gesture technology to control holographically-projected controls on the dashboard of a car, pushing virtual buttons.
Microsoft and Intel have strong investments in this area too, as similar sensors are making their way to the HoloLens holographic computer and Intel-powered systems like the HTC Vive. If you want to see the future, check out Tiltbrush, a VR paint program that lets you step inside a painting, using the depth effect to grand effect.
In short, the future of smartphones is deep, as we get more hardware that understands the world around us like never before.