Could Microsoft's Kinect be reborn as a virtual reality controller?

The Kinect port's removal from the Xbox One S might have lead some to believe that the accessory has reached the end of its life, but a recent Microsoft blog post suggests otherwise.

The post details the research efforts of Microsoft's research lab in Cambridge, who have been hard at work repurposing the Kinect camera accessory as a device for tracking hand and finger movements as part of a project called Handpose.

The blog post details how the researchers expect the technology to be used in an office setting, to use your fingers to skip between slides in a powerpoint presentation or to hang up on a colleague over Skype, but where the technology gets really interesting is in its ability to simulate hand movement in a virtual environment.

But more interestingly, this Handpose technology is also shown being used by people wearing virtual reality headsets, completely removing the need for a controller at all.

Microsoft's VR Plans

At present, Microsoft has not announced any firm VR hardware for the Xbox but with Sony gearing up for the release of its own PlayStation VR there are rumors that Microsoft has more decisive plans to get the Xbox in on the action.

Microsoft briefly mentioned at its E3 2016 press conference that the computing power of Project Scorpio will pave the way for the company to integrate support for the Oculus Rift headset. This is opposed to Sony, who has built its own headset from the ground up.

When it comes to controllers, Sony already has the hardware with its PlayStation Move motion controllers which were originally released for the PS3.

But could Microsoft be preparing a similar move for its own motion-controller?

Motion controllers killed the gamepad

When the first Oculus Rift developer kits were released, users relied upon traditional controllers to interact with its games, but when HTC entered the VR race it made its motion controllers a central part of the Vive experience.

However, while these controllers are a great deal more immersive than a traditional gamepad, you're conscious that you're interacting with a piece of plastic in your hand.

By removing the controller entirely, Handpose allows you to interact directly with the environment instead of using a controller as a proxy.

Two steps forward, one step back

This development might eliminate one set of VR's problems, but in doing so it completely removes any haptic feedback from the environment.

When you use the HTC's controller to pick up a mug in Job Simulator you might not be able to feel the mug itself, but you can feel your finger on the trigger of the controller.

But were you to use the Kinect to simulate picking up a mug there would be nothing in your hand at all, nothing pressing against your skin, and no weight in your hand.

Surprisingly though, the researchers have found that the lack of haptic feedback hasn't been as much of a problem as they'd anticipated. Partly this is due to the way they've designed the experiences with the Kinect in mind, with controls that are thin enough to allow your fingers to touch for example.

But also the researchers have found that they can use other senses such as sight and sound to convince people that they're touching something real.

What this suggests is that being able to physically see your hands and hear them interact with the environment builds such a great sense of presence that your brain doesn't mind when you can't actually feel anything with them.

A new lease of life for Kinect

It's early days yet for the new technology which may never see the light of day, but the videos of the technology look promising.

VR can only get so immersive while it relies upon relatively inarticulate motion controllers. This technology could end up allowing a much more fluid and natural interaction with a virtual world.

People talk about the importance of 'presence' in VR and if it works this technology could take this idea of presence to the next level. With the HTC Vive you might feel present in a space, but the only part of your body that feels truly present is your eyes.

But with Handpose you might one day be able to look down and see your real hands in VR, and maybe even one day your whole body. You won't just be a floating set of eyes and controllers in a virtual space, but a single connected body.

It's too soon to form any definitive statements on the technology yet, but Handpose has given an exciting glimpse into Microsoft's vision for virtual interaction.

June 29, 2016 at 10:30PM
Jon Porter