Method and device for identifying and extracting images of multiple users, and for recognizing user gestures

Abstract

A method identifies and extracts images of one or more users. The method obtains an array of pixel values that constitutes a scene image and a corresponding array of depth values that constitutes a depth map. The depth map and the image are registered. The method obtains an array containing the 3D positions of the pixel value points in real-world coordinates by coordinate transformation of the depth map and image. The method then clusters the points into groups according to their relative positions so each group contains points in the same region of space and corresponds to a user location. The method defines individual volumes of interest around each user location. The method selects points from the array of 3D positions located in the volumes of interest to obtain segmentation masks for each user. The segmentation masks are then applied to the image to extract images of the users.

What is claimed is: 
     
         1 . A method for identifying and extracting images of one or more users in an interactive environment comprising the steps of:
 obtaining a depth map ( 7 ) of a scene in the form of an array of depth values, and an image ( 8 ) of said scene in the form of a corresponding array of pixel values, said depth map ( 7 ) and said image ( 8 ) being registered;   applying a coordinate transformation to said depth map ( 7 ) and said image ( 8 ) for obtaining a corresponding array ( 15 ) containing the 3D positions in a real-world coordinates system and pixel values points;   grouping said points according to their relative positions, by using a clustering process ( 18 ) so that each group contains points that are in the same region of space and correspond to a user location ( 19 );   defining individual volumes of interest ( 20 ) each corresponding to one of said user locations ( 19 );   selecting, from said array ( 15 ) containing the 3D positions and pixel values, the points located in said volumes of interest for obtaining segmentation masks ( 35 ) for each user;   applying said segmentation masks ( 35 ) to said image ( 8 ) for extracting images of said users.   
     
     
         2 . The method of  claim 1  comprising the step of determining said individual volumes of interest ( 20 ) by determining individual volumes of interest bounds ( 16 ′) separating said groups. 
     
     
         3 . A method according to  claim 2  for identifying and segmenting one or more users in an interactive environment for providing an interactive virtual reality game wherein a plurality of users are separated from each other and from a background. 
     
     
         4 . A method according to  claim 3  wherein gestures and/or body poses of individual users are recognized and used for controlling said game. 
     
     
         5 . A method according to  claim 1  for identifying and segmenting one or more users in an interactive environment for providing an interactive virtual reality game wherein a plurality of users are separated from each other and from a background. 
     
     
         6 . A method according to  claim 5  wherein gestures and/or body poses of individual users are recognized and used for controlling said game. 
     
     
         7 . A device for identifying and extracting images of multiple users in an interactive environment scene comprising:
 a video camera for capturing an image from the scene;   a depth perception device for providing depth information about said scene;   at least one computer processor for processing said depth information and said image information;   wherein said device comprises means for using individual volumes of interest from said scene for each user.   
     
     
         8 . The device according to  claim 7 , wherein the device comprises means for obtaining adaptive volumes of interest for each user. 
     
     
         9 . A device, according to  claim 7 , for identifying and segmenting one or more users in an interactive environment for providing an interactive virtual reality game wherein a plurality of users are separated from each other and from a background. 
     
     
         10 . A device according to  claim 9  wherein gestures and/or body poses of individual users are recognized and used for controlling said game. 
     
     
         11 . A device, according to  claim 8 , for identifying and segmenting one or more users in an interactive environment for providing an interactive virtual reality game wherein a plurality of users are separated from each other and from a background. 
     
     
         12 . A device according to  claim 11  wherein gestures and/or body poses of individual users are recognized and used for controlling said game.