Heuristic motion detection methods and systems for interactive applications

Abstract

A method is provided for motion detection comprising acquiring a series of images of an audience in a viewing area comprising a current image and a previous image, determining a plurality of optical flow vectors, each representing movement of one of a plurality of visual elements from a first location in the previous image to a second location in the current image, storing the optical flow vectors in a current vector map associated with time information, and determining motion by calculating an intensity ratio between the current vector map and at least one prior vector map. The audience is in a theater or other venue having at least one view screen. A video camera captures images of the audience. Audience movements are interpreted and used to control images on the view screen.

What is claimed is: 
     
         1 . A method for detecting motion of an audience, comprising:
 acquiring a series of images of the audience, the series of images comprising a current image and a previous image, wherein said audience comprises one or more human participants;   determining a plurality of optical flow vectors, each optical flow vector representing movement of one of a plurality of visual elements from a first location in the previous image to a second location in the current image, wherein at least some of said visual elements correspond to images of at least part of one or more human participants in the audience;   determining and storing average velocities for said plurality of optical flow vectors for a plurality of different time points; and   using said average velocities, calculating a motion index, wherein the motion index can have positive or negative values depending on motion direction.   
     
     
         2 . The method of  claim 1 , further comprising, prior to determining the plurality of optical flow vectors, applying a difference-based filter between consecutive images in the series of images. 
     
     
         3 . The method of  claim 1 , further comprising, prior to determining the plurality of optical flow vectors, applying a polygon-based filter delimiting areas of the series of images to be excluded when determining the plurality of optical flow vectors. 
     
     
         4 . The method of  claim 1 , wherein the audience comprises a plurality of human participants located in a spectator area,
 the method further comprising, prior to determining the plurality of optical flow vectors, applying a polygon-based filter delimiting areas of the series of images to be excluded when determining the plurality of optical flow vectors, wherein said areas to be excluded comprise areas which correspond to aisles for travel by human participants.   
     
     
         5 . The method of  claim 1 , wherein determining a plurality of optical flow vectors comprises the application of a pyramidal Lucas-Kanade algorithm. 
     
     
         6 . The method of  claim 1 ,
 further comprising calculating an intensity ratio,   wherein calculating the intensity ratio comprises the application of the following formula:   
       
         
           
             
               
                 M 
                  
                 
                   ( 
                   
                     
                       v 
                       t 
                     
                     , 
                     
                       VPos 
                       n 
                     
                     , 
                     
                       VNeg 
                       m 
                     
                   
                   ) 
                 
               
               = 
               
                 { 
                 
                   
                     
                       
                         
                           
                             B 
                             + 
                             
                               0.5 
                                
                               E 
                             
                           
                           n 
                         
                         · 
                         100 
                       
                     
                     
                       
                         
                           if 
                            
                           
                               
                           
                            
                           
                             v 
                             t 
                           
                         
                         > 
                         0 
                       
                     
                     
                       
                           
                       
                     
                   
                   
                     
                       
                         
                           - 
                           
                             
                               B 
                               + 
                               
                                 0.5 
                                  
                                 E 
                               
                             
                             m 
                           
                         
                         · 
                         100 
                       
                     
                     
                       
                         
                           if 
                            
                           
                               
                           
                            
                           
                             v 
                             t 
                           
                         
                         < 
                         0 
                       
                     
                     
                       
                           
                       
                     
                   
                   
                     
                       0 
                     
                     
                       
                         
                           if 
                            
                           
                               
                           
                            
                           
                             v 
                             t 
                           
                         
                         = 
                         0 
                       
                     
                     
                       
                           
                       
                     
                   
                 
               
             
           
         
         wherein:
 B is the number of scores below v t , if v t >0 or number of scores above v t , if v t <0 
 E is the number of scores equal to v t    
 n is the memory size of positive velocities 
 m is the memory size of negative velocities 
 v are velocities 
 
         considering the Optical Flow vectors
     V Pos n   =[u   0    . . . u   n-1 ] 
 
         where μ i  is a positive velocity obtained in an instant ≦t (u i >0),
     V Neg n   =[w   0    . . . w   n-1 ] 
 
         and where w i  is a negative velocity obtained in an instant ≦t (w i <0). 
       
     
     
         7 . The method of  claim 1 , further comprising storing the optical flow vectors, and eliminating optical flow vectors with an absolute value less than a predefined threshold relative to image resolution. 
     
     
         8 . The method of  claim 1 , wherein the audience is located in at least one of a theater, a cinema, an arena, or viewing stands. 
     
     
         9 . The method of  claim 1 , further comprising displaying a moving image to the audience, and controlling one or more aspects of the moving image based on the motion index. 
     
     
         10 . The method of  claim 1 ,
 wherein the audience comprises a plurality of human participants,   the method comprising controlling one or more elements on a view screen based on at least one of arm movements and body movements of the audience.   
     
     
         11 . The method of  claim 1 , further comprising providing instructions to the audience regarding how to control one or more images using physical motions. 
     
     
         12 . The method of  claim 1 , wherein the audience comprises a plurality of human participants, wherein the audience collectively plays a game on a video screen by gesturing, said gesturing being detected by at least one video camera, and wherein the game comprises moving an element on the video screen horizontally back and forth. 
     
     
         13 . An interactive video system, comprising:
 a computer processing unit;   an audience area for accommodating an audience;   a video camera which is pointed at the audience area;   a video screen; and   a memory, and instructions stored in the memory that, when executed on the processing unit, cause the computer system to:   acquire a series of images comprising a current image and a previous image, wherein the images comprise images of the audience area;   determine a plurality of optical flow vectors, each representing movement of one of a plurality of visual elements from a first location in the previous image to a second location in the current image;   determine and store average velocities for said plurality of optical flow vectors for a plurality of different time points; and   using said average velocities, calculating a motion index, wherein the motion index can have positive or negative values depending on motion direction.   
     
     
         14 . The interactive video system of  claim 13 , wherein the memory further comprises instructions that, when executed on the processing unit, prior to determining the plurality of optical flow vectors, apply a difference-based filter between consecutive images in the series of images. 
     
     
         15 . The interactive video system of  claim 13 ,
 wherein the audience area comprises one or more aisles provided for travel by human participants;   wherein the instructions stored in the memory further comprise instructions that, when executed on the processing unit, prior to determining the plurality of optical flow vectors, apply a polygon-based filter delimiting areas of the series of images to be excluded when determining the plurality of optical flow vectors; and   wherein said areas to be excluded by the filter comprise areas which correspond to said aisles.   
     
     
         16 . The interactive video system of  claim 13 ,
 wherein the audience area comprises seats for accommodating a plurality of human participants, and wherein the video camera and the video screen are oriented towards at least some of said seats.   
     
     
         17 . The interactive video system of  claim 13 , wherein the system is located in one of a theater, a cinema, or an arena. 
     
     
         18 . The interactive video system of  claim 13 , wherein the memory further comprises instructions that, when executed on the processing unit, cause the video screen to display a moving image, and further to control one or more aspects of the moving image based on the motion index. 
     
     
         19 . The interactive video system of  claim 13 , further comprising instructions that, when executed on the processing unit, store the optical flow vectors and, prior to storing the optical flow vectors, eliminate optical flow vectors with an absolute value less than a predefined threshold relative to image resolution. 
     
     
         20 . The interactive video system of  claim 13 ,
 wherein the video screen and the video camera are both oriented towards the audience area;   wherein the audience area comprises at least one of: a plurality of seats, and an audience standing area; and   wherein the memory further comprises instructions that, when executed on the processing unit, display a video on the video screen, and control one or more elements of said video based on physical motion in the audience area which is detected by the video camera.