Techniques for using human gestures to control gesture unaware programs

Abstract

A capture device can detect gestures made by a user. The gestures can be used to control a gesture unaware program.

What is claimed is: 
     
         1 . A system, comprising:
 a computing environment including a processor coupled to a computer readable storage medium, the computer readable storage medium including executable instructions for a program, wherein a user interface for the program is configured to only receive non-gesture user interface commands;   a capture device coupled to the computing environment, wherein the capture device is configured to capture images and identify gestures in the images, wherein the capture device is configured to capture audio and identify audio commands in the captured audio; and   wherein the non-gesture user interface commands used to control the user interface of the program are generated from gestures identified in the images and audio commands identified in the captured audio.   
     
     
         2 . The system of  claim 1 , wherein the capture device is further configured to generate scancodes for keys of keyboards based on the gestures and send the scancodes to the computing environment. 
     
     
         3 . The system of  claim 1 , wherein the capture device is further configured to generate packets that include mouse button state and mouse position information based on the gestures and send the packets to the computing environment. 
     
     
         4 . The system of  claim 1 , wherein the capture device is configured to generate state information for a videogame controller and send the state information to the computing environment. 
     
     
         5 . The system of  claim 1 , wherein the computing environment is configured to receive gesture information, generate keyboard messages, and send the keyboard messages to the program. 
     
     
         6 . The system of  claim 1 , wherein the computing environment is configured to receive gesture information, generate mouse messages, and send the mouse messages to the program. 
     
     
         7 . The system of  claim 1 , wherein the computing environment is configured to receive gesture information, generate state information for a videogame controller, and send the state information for the videogame controller to the program. 
     
     
         8 . A computer readable storage medium including executable instructions, the computer readable storage medium comprising:
 instructions for identifying a gesture from an image captured by a camera;   instructions for generating a command for a tactile based user input device based on the identified gesture; and   instructions for sending the command to a gesture unaware program configured to process the command.   
     
     
         9 . The computer readable storage medium of  claim 8 , wherein the instructions for identifying the gesture in the image further comprise:
 instructions for comparing image data generated from the image to a plurality of filters, each filter associated with a different gesture;   instructions for generating a confidence score for each comparison;   instructions for selecting the gesture associated with the filter having the highest confidence score.   
     
     
         10 . The computer readable storage medium of  claim 8 , wherein the instructions for generating the command further comprise:
 instructions for identifying a keyboard command associated with the gesture;   instructions for generating a keyboard message including the keyboard command; and   instructions for sending the keyboard message to the program.   
     
     
         11 . The computer readable storage medium of  claim 8 , wherein the instructions for generating the command further comprise:
 instructions for identifying a mouse status associated with the gesture; and   instructions for sending information indicative of the mouse status to the program.   
     
     
         12 . The computer readable storage medium of  claim 8 , wherein the instructions for generating the command further comprise:
 instructions for identifying a videogame controller status associated with the gesture; and   instructions for sending information indicative of the videogame controller status to the program.   
     
     
         13 . The computer readable storage medium of  claim 8 , further comprising:
 instructions for determining that the program registered for specific tactile-based user input; and   instructions for loading a gesture library that maps gestures to the specific tactile-based user input.   
     
     
         14 . The computer readable storage medium of  claim 8 , further comprising:
 instructions for executing a plurality of programs, each program associated with a window and different gestures;   instructions for determining that the program is the currently focused window; and   instructions for mapping gestures associated with the program to tactile based user input device commands.   
     
     
         15 . A device method, comprising:
 receiving image data from a camera;   identifying a gesture from the image data;   generating a command for a tactile based user input device based on the identified gesture; and   sending the command to a computing environment coupled to the device.   
     
     
         16 . The method of  claim 15 , wherein generating the command further comprises:
 generating a packet of information, the packet of information formatted to emulate a packet generated by a mouse.   
     
     
         17 . The method of  claim 15 , wherein generating the command further comprises:
 generating a scancode indicative of a keyboard action, the scancode formatted to emulate a scancode generated by a keyboard.   
     
     
         18 . The method of  claim 15 , wherein generating the command further comprises:
 generating a packet of information, the packet of information formatted to emulate a packet generated by a videogame controller.   
     
     
         19 . The method of  claim 15 , wherein identifying the gesture further comprises:
 comparing the image to a plurality of filters, each filter associated with a different gesture;   generating a confidence score for each comparison; and   selecting the gesture associated with the filter having the highest confidence score.   
     
     
         20 . The method of  claim 15 , further comprising:
 determining that a program running on the computing environment is gesture unaware; and   loading a gesture library that maps gestures to tactile based user input device commands.