You have misplaced the remote control of the TV and wish it could just dance to the tune of your fingers? Now it will, thanks to the India-born Professor Ramesh Raskar and his team at the Massachusetts Institute of Technology. The team has developed a bi-directional display interface (BiDi) screen that recognizes hand gestures.

The optical sensors embedded in the screen make it capable of capturing both touch and off-screen gestures. "The BiDi Screen uses a sensor layer, separated a small distance from a normal LCD display. A mask image is then displayed on the LCD. When the bare sensor layer views the world through the mask, information about the distance to objects in front of the screen can be captured and decoded by a computer," says the project team. For now, the LCD can be operated through a finger gesture from a distance of 20 inches. The system can be incorporated into thin devices like a cell phone and make it touch-free.

Many other researchers have been working on such gestural interfaces like Pranav Mistry’s SixthSense. They allow computer users to drag windows around a screen simply by pointing at them and moving their fingers, or to rotate a virtual object through three dimensions with a flick of the wrist. “Existing systems usually involve having a roomful of expensive cameras or wearing tracking tags on your fingers; while BiDi does not require the use of cameras, lenses, projectors, or special gloves,” says Matthew Hirsch, a member of the team.

While MIT’s Raskar is making computers understand hand gestures, Microsoft is on the threshold of making devices recognize human reactions. The new Xbox Natal experience will allow the TV to judge the actions in the room. It will recognize how many people are present, will be able to judge what they want to see on their screen, and will even play physically without a remote controller.