Method and apparatus for identifying input features for later recognition

Abstract

Disclosed are methods and apparatuses to recognize actors during normal system operation. The method includes defining actor input such as hand gestures, executing and detecting input, and identifying salient features of the actor therein. A model is defined from salient features, and a data set of salient features and/or model are retained, and may be used to identify actors for other inputs. A command such as “unlock” may be executed in response to actor input. Parameters may be applied to further define where, when, how, etc. actor input is executed, such as defining a region for a gesture. The apparatus includes a processor and sensor, the processor defining actor input, identifying salient features, defining a model therefrom, and retaining a data set. A display may also be used to show actor input, a defined region, relevant information, and/or an environment. A stylus or other non-human actor may be used.

1 . A method, comprising:
 defining a preliminary geometric feature for an object;   defining a first region in a field of view of a first imaging device to identify the object, wherein the first region in the field of view is less than the full field of view of the first imaging device;   detecting, by the first imaging device, the object at a first point in time based on the preliminary geometrics feature, the object being within the first region performing a first action associated with a first instruction; in response to detecting the first action by the object, executing the first instruction;   identifying, by a processing device, a defining feature of the object as the object performs the first action, wherein the defining feature of the object is different than the preliminary geometric feature;   generating a model of the object, wherein the model includes the defining feature of the object;   detecting, by the first imaging device, the object at a second point in time based on the model of the object, the object being within the first region performing a second action associated with a second instruction, wherein the second action is different than the first action; and   in response to detecting the second action by the object, executing the second instruction.   
     
     
         2 . The method of  claim 1  further comprising:
 defining a limiting parameter for executing the first instruction or the second instruction; and 
 executing the first instruction or the second instruction within the liming parameter. 
 
     
     
         3 . The method of  claim 1 , wherein:
 the object comprises a human hand; and   the first action comprises at least one of a hand gesture or a hand posture.   
     
     
         4 . The method of  claim 1 , further comprising:
 detecting, by the first imaging device, the object at a third point in time based on the model of the object, the object being within the first region performing a third action associated with a third instruction, wherein the third action is different than the first action and the second action; and   in response to detecting the third action, executing the third instruction.   
     
     
         5 . The method of  claim 1 , wherein the first region is substantially fully within the field of view. 
     
     
         6 . The method of  claim 1 , wherein the object comprises an end-effector. 
     
     
         7 . The method of  claim 1 , further comprising identifying a plurality of defining features of the object in the first region. 
     
     
         8 . The method of  claim 1 , further comprising detecting another object using the preliminary geometric feature. 
     
     
         9 . The method of  claim 1 , further comprising:
 defining a second region in the field of view of the first imaging device;   detecting, by the first imaging device, the object at a third point in time based on the model, the object being within the second region performing one of the first action associated with the first instruction, the second action associated with the second instruction, or a third action associated with a third instruction; and   in response to detecting the first action, the second action, or the third action, executing the first instruction, the second action, or the third action, respectively.   
     
     
         10 . The method of  claim 1 , further comprising:
 defining a second region in a field of view of a second imaging device;   detecting, by the second imaging device, the object based on the model, the object being within the second region performing one of the first action associated with the first instruction, the second action associated with the second instruction, or a third action associated with a third instruction; and   in response to detecting the first action, the second action, or the third action, executing the first instruction, the second action, or the third action, respectively.   
     
     
         11 . An apparatus, comprising:
 a sensor configured to detect an object at a first point in time and a second point in time within a first region of a field of view of the sensor; and   a processing device coupled to the sensor, wherein the processing device is configured to:
 determine the object is within the field of view at the first point in time based on a preliminary geometric feature of the object, the object being within the first region performing a first action is associated with a first instruction, 
 in response to detecting the first action by the object, execute the first instruction; 
 identify a defining feature of the object as the object performs the first action, wherein the defining feature of the object is different than the preliminary geometric feature; 
 generate a model of the object, wherein the model includes the defining feature of the object; 
 determine the object is within the field of view at the second point in time based on the model of the object, the object being within the first region performing a second action is associated with a second instruction; and 
 in response to detecting the second action by the object, execute the second instruction. 
   
     
     
         12 . The apparatus of  claim 11 , wherein the processing device is configured to use the preliminary geometric feature to identify a second object. 
     
     
         13 . The apparatus of  claim 11 , wherein the sensor is an image sensor. 
     
     
         14 . The apparatus of  claim 11 , wherein the sensor is an audio sensor. 
     
     
         15 . The apparatus of  claim 11 , further comprising a display coupled to the processing device, wherein:
 the sensor comprises an image sensor, the image sensor adapted to obtain at an image in the field of view with the object therein; and   the display is configured to display at least a portion of the field of view.   
     
     
         16 . The apparatus of  claim 11 , wherein the sensor comprising a first image sensor and a second image sensor. 
     
     
         17 . The apparatus of  claim 16 , wherein the first image sensor and second image sensor are disposed in a stereo configuration. 
     
     
         18 . The apparatus of  claim 11 , wherein the object comprises a stylus. 
     
     
         19 . A method, comprising:
 defining, by a processing device, a preliminary geometric feature for an object;   defining, by the processing device, a region in a field of view of an imaging device to identify the object, wherein the region in the field of view is less than the full field of view of the imaging device;   detecting, by the imaging device, the object at a first point in time based on the preliminary geometrics feature, the object performing a first action being associated with a first instruction;   in response to detecting the first action by the object, executing the first instruction;   identifying, by the processing device, a defining feature of the object as the object performs the first action;   generating a model of the object wherein the model includes the defining feature of the object;   detecting, by the imaging device, the object performing a second action at a second point in time based on the model of the object; and   in response to detecting the second action by the object, executing a second instruction.   
     
     
         20 . The method of  claim 19 , wherein the defining feature comprises at least one of an edge feature, a shape feature, a motion feature, a convex hull model, a concave hull feature a joint feature, a convex feature, a concave feature, a complex geometric feature, an abstract geometries feature, or a ratio of dimension.