Microsoft's Experimental Keyboard Recognises Gestures

Microsoft’s research division has found a way by which PC users can enjoy gesture commands without involving intrusive hardware. Microsoft Research’s Type-Hover-Swipe keyboard gets its motion sensing capabilities from a 16 x 4 array of infrared proximity sensors placed within key spaces of a custom made 3D printed keyboard. The infrared sensor nodes are mounted on a printed circuit board and inserted between the keycap and into the silicon membrane. As compared to the Kinect sensor on the Xbox One that captures full HD resolution data, the sensors of Type-Hover-Swipe keyboard can capture only a miniscule 64 pixels. While 64 pixels might seem incredibly low resolution, it is enough to identify the number of fingers and their movement above the keyboard.


The Type-Hover-Swipe keyboard accumulates binary motion history image and intensity motion history image form a motion signature for every gesture. The system can perform operations such as swiping left and right, swiping up and down, zooming in and out in pictures and even controlling a car’s movements in a game. Microsoft says that in the future they will be able to able to refine this system to detect gestures even more accurately as they have found out that they can combine raw sensor data to the nearest neighbour data to generate groundtruth data in an offline process. Once they refine this technique they will be able to create full touch based experiences.

Gesture based systems such as Leap motion controllers worked in their own ecosystem of apps, but Microsoft’s system works on the entire Windows ecosystem. The Type-Hover-Swipe keyboard will surely make using Windows 8 easier but we would like you not be ready with your money just yet as Microsoft Research products may or may not make their way into the market. For the meanwhile, you can have a take a look at the keyboard in action in this video below.


Source: Microsoft Research via Engadget Posted in: #Gadgets

Nice innovation but what would be the cost of a keyboard which would be implemented with sensors?

Apple's recent patent also sounds familiar!

Looks like this is a keyboard with integrated invisible mouse.

Nice one, but I am just wondering what If I will just move my fingers randomly over the keyboard without actually aiming to perform any action?

Keyboard will still sense something and do something on the system, which is not desired by user?