Object selection using hit test tracks

Abstract

A method for labeling the pixels within a selected visual area of at least one image frame containing that visual area from a sequence of image frames stored in memory and operative to be displayed on an interactive display so that a user may subsequently select the selected visual area on a pixel accurate, frame accurate basis. To label the selected visual area within an image frame, the scene within that image frame is segmented to identify the selected visual area, each pixel within that selected visual area is then labeled with an area identifier which is unique to that selected visual area, and the pixels containing the area identifiers are mapped into an item buffer. The item buffer is then compressed and stored within a labeled portion of memory linked with the stored frame image from which the item buffer was derived. When a user subsequently selects a pixel within any frame image of the sequence of frame images the pixel is decompressed within the labeled portion of memory corresponding to the pixel in the selected frame image to determine the area identifier for the selected pixel. This area identifier is then used for a number of purposes, such as to identify an area within the frame image corresponding to the selected pixel, or to cause some action related to the selected pixel to be performed.

1 . A method for labeling and subsequently identifying selected areas within images from a sequence of one or more images operative to be displayed by a display of a computer having a memory for storing the images, comprising the steps of: 
 (a) identifying an area to be labeled within an image from said sequence of images;    (b) labeling every pixel within said area with an area identifier which is unique to said area;    (c) storing each labeled pixel in a labeled portion of memory linked to said image;    (d) repeating steps (a) through (c) for each selected area within each image from said sequence of images;    (e) interrogating said memory in response to a users selection of a pixel location within a selected area from a selected image displayed on said display to locate a labeled portion corresponding to said selected image;    (f) evaluating said labeled portion corresponding to said selected image to locate an area identifier corresponding to said pixel location; and    (g) identifying said area identifier to said computer as an indication of said selected area.    
     
     
         2 . The method for labeling and subsequently identifying as recited in  claim 1 , and further comprising the step of (h) comparing said area identifier corresponding to said pixel location to a table containing additional information about said selected area.  
     
     
         3 . The method for labeling and subsequently identifying as recited in  claim 2 , wherein said additional information includes a name for said selected area.  
     
     
         4 . The method for labeling and subsequently identifying as recited in  claim 2 , wherein said additional information includes an action to be performed by said computer which corresponds to said selected area.  
     
     
         5 . The method for labeling and subsequently identifying as recited in  claim 1 , wherein said images are video frames from a sequence of video frames, and wherein step (a) includes the step of segmenting each area to be labeled from within each video frame using a pattern recognizer.  
     
     
         6 . The method for labeling and subsequently identifying as recited in  claim 1 , wherein said images are video frames from a sequence of video frames, and wherein step (a) includes the step of manually segmenting each area to be labeled from within each video frame.  
     
     
         7 . The method for labeling and subsequently identifying as recited in  claim 1 , wherein said images are rendered frames from a sequence of one or more rendered frames, and wherein step (a) includes the step of rendering said rendered frames to identify each area to be labeled from within each rendered frame.  
     
     
         8 . The method for labeling and subsequently identifying as recited in  claim 1 , wherein said selected areas correspond to visual objects and abstract areas within said images.  
     
     
         9 . The method for labeling and subsequently identifying as recited in  claim 1 , wherein step (b) includes the steps of: 
 mapping said area into an item buffer corresponding to said image; and    assigning said area identifier to each pixel within said area to form labeled pixels within said item buffer corresponding to pixel locations within said image.    
     
     
         10 . The method for labeling and subsequently identifying as recited in  claim 9 , wherein step (c) includes the steps of: 
 compressing said item buffer;    storing said compressed item buffer in said labeled portion of memory; and    storing an image identifier with said compressed item buffer in said labeled portion of memory to link said labeled portion of memory to said image.    
     
     
         11 . The method for labeling and subsequently identifying as recited in  claim 10 , wherein said images are compressed for storage in said memory, wherein said item buffer is compressed using a lossless compression format.  
     
     
         12 . The method for labeling and subsequently identifying as recited in  claim 11 , wherein a normal resolution item buffer is utilized when said images are compressed at a ratio equal to or less than a predetermined threshold and a subsampled low resolution item buffer is utilized when said images are compressed at a ratio greater than said predetermined threshold.  
     
     
         13 . The method for labeling and subsequently identifying as recited in claim  109  wherein step (c) further includes the step of storing a labeled portion of memory identifier with said image identifier to distinguish said labeled portion of memory to said computer from other portions of said memory.  
     
     
         14 . The method for labeling and subsequently identifying as recited in  claim 10 , wherein step (c) further includes the step of storing a compression format indicator with said image identifier for indicating a compression format for said compressed item buffer to said computer.  
     
     
         15 . The method for labeling and subsequently identifying as recited in  claim 10 , wherein step (c) further includes the step of storing a pixel bit depth indicator with said image identifier for indicating a bit depth for each labeled pixel to said computer when decompressing said labeled pixel.  
     
     
         16 . The method for labeling and subsequently identifying as recited in  claim 10 , wherein step (c) further includes the step of storing an area to name mapping table with said image identifier containing a name for said selected area which is reported to said computer when said selected area is identified to said computer in step (g).  
     
     
         17 . The method for labeling and subsequently identifying as recited in  claim 10 , wherein step (c) further includes the step of storing an area to event mapping table with said image identifier containing an event corresponding to said selected area which is reported to and performed by said computer when said selected area is identified to said computer in step (g).  
     
     
         18 . A method for aiding a user's selection of areas within images represented by a series of related temporal tracks of image data stored in a memory of a computer, the computer being operative to selectively display said images on a display of the computer by accessing the temporal tracks of image data from the memory, each of the temporal tracks of image data being operative to be offset in the memory by a fixed time from other temporal tracks of image data stored in the memory, comprising the steps of: 
 (a) identifying an area within an image which could be selected by said user;    (b) labeling each pixel within said area with an area identifier which is unique to said area;    (c) storing each labeled pixel in a labeled track in said memory corresponding to said image;    (d) repeating steps (a) through (c) for each area within each image which could be selected by said user;    (e) searching through said memory in response to said user's selection of a pixel location within a selected area of a selected image to locate a labeled track corresponding to said selected image;    (f) searching said labeled track for a labeled pixel corresponding to said pixel location; and    (g) retrieving said area identifier corresponding to said labeled pixel from said memory to indicate said selected area to said computer.    
     
     
         19 . The method for aiding a user's selection of areas within images as recited in  claim 18 , and further comprising the step of: 
 (h) comparing said area identifier corresponding to said pixel location to a table containing additional information about said selected area.    
     
     
         20 . The method for aiding a user's selection of areas within images as recited in  claim 19 , wherein said additional information includes a name for said selected area to be communicated to said user by said computer.