Microsoft’s research and development arm has shown off a novel mechanical keyboard prototype which incorporates elements of natural user interaction such as touch and low-effort motion gestures. The goal is to enhance user interfaces on desktop by allowing for smooth transitions between text entry and gestures.
The system uses an array of proximity sensors embedded in the keyboard itself and is coupled to a software that uses the motion signature technique which utilizes pairs of motion history images and a random forest classifier to robustly recognize a large set of motion gestures.
It works like magic and we have an impressive video right after the break to prove it…
According to a post over at the Microsoft Research blog, the Windows maker’s prototype gesture keyboard can sense dynamic, temporal gestures.
“Gestures can enhance traditional desktop interaction,” Microsoft says. “For example, switching between applications using hover gestures above the keyboard, or navigating documents with core swipes and pinch to zoom.”
The system can even lend itself to games with controls such as a virtual steering wheel.
It can sense rapid motion directly on top of the keyboard and in the narrow band of hover without he need for any external sensing. What dark magic is this, you ask?
Easy, they are using a sixteen-by-four array of proximity sensors interspersed between the key caps of the mechanical keyboard. Check it out in action in the below video by Microsoft Research.
Although this provides for a low-resolution of 64 pixels, sensor data is captured 300 times per second and passed to the underlying software smart enough to visually discern your fingers and other parts of the hand moving above the keyboard.
The project is code-named Type-Hover-Swipe and is in the prototyping phase so currently there’s no telling whether Microsoft’s hardware arm will turn this thing into an actual product.
Yes, they’re using keycaps from an Apple keyboard!
Does an augmented gesture keyboard make sense, do you think?