Method and apparatus for processing a 3d scene

Abstract

A method and an apparatus for processing a 3D scene are disclosed. At least one virtual reference viewpoint in the 3D scene is determined (41). A map of registered real cast shadows of objects in the 3D scene from an input image captured by a camera positioned at a viewpoint distinct from the virtual reference viewpoint is obtained (42), said map of real cast shadows being registered with regards to the virtual reference viewpoint. Parameters for at least one light source in the 3D scene are determined (44) using the map of registered real cast shadows and at least one map of virtual shadows of objects in the 3D scene cast by the at least one light source from the virtual reference viewpoint.

1 . A method for processing a 3D scene, comprising:
 determining at least one virtual reference viewpoint in the 3D scene,   obtaining a map of registered real cast shadows of objects in the 3D scene from an input image captured by a camera positioned at a viewpoint distinct from the virtual reference viewpoint, said map of real cast shadows being registered with regards to the virtual reference viewpoint, and   determining parameters for at least one light source in the 3D scene using the map of registered real cast shadows and at least one map of virtual shadows of objects in the 3D scene cast by the at least one light source from the virtual reference viewpoint.   
     
     
         2 . An apparatus for processing a 3D scene comprising:
 means for determining at least one virtual reference viewpoint in the 3D scene,   means for obtaining a map of registered real cast shadows of objects in the 3D scene from an input image captured by the camera positioned at a viewpoint distinct from the virtual reference viewpoint, said map of real cast shadows being registered with regards to the virtual reference viewpoint, and   means for determining parameters for at least one light source in the 3D scene using the map of registered real cast shadows and at least one map of virtual shadows of objects in the 3D scene cast by the at least one light source from the virtual reference viewpoint.   
     
     
         3 . The method according to  claim 1 , wherein obtaining the map of registered real cast shadows comprises:
 detecting the real cast shadows in the input image to obtain a map of detected real cast shadows, and   registering the map of detected real cast shadows with regards to the virtual reference viewpoint.   
     
     
         4 . The method according to  claim 1 , wherein obtaining the map of registered real cast shadows comprises:
 registering the input image with regards to the virtual reference viewpoint to obtain a registered image, and   detecting the real cast shadows in the registered image.   
     
     
         5 . The method according to  claim 1 , wherein using the map of registered real cast shadows and at least one map of virtual shadows of objects in the 3D scene cast by the at least one light source from the virtual reference viewpoint, comprises matching the map of registered real cast shadows and the at least one map of virtual shadows and selecting the map of virtual shadows that best matches the map of registered real cast shadows. 
     
     
         6 . The method according to  claim 1 , wherein the virtual reference viewpoint corresponds to a viewpoint from which a real scene represented by the 3D scene has been previously captured by the camera. 
     
     
         7 . The method according to  claim 1 , wherein the virtual reference viewpoint in the 3D scene is determined as being a viewpoint for which a number of pixels overlapping in an image captured from the virtual reference viewpoint and an image captured from the viewpoint of the camera is above a determined value. 
     
     
         8 . The method according to  claim 1 , wherein when the virtual reference viewpoint and the camera's viewpoint do not satisfy a determined criterion, a new virtual reference viewpoint is determined. 
     
     
         9 . The method according to  claim 8 , wherein the virtual reference viewpoint and the camera's viewpoint do not satisfy the determined criterion if a number of pixels overlapping in an image captured from the virtual reference viewpoint and an image captured from the camera's viewpoint is below a determined value. 
     
     
         10 . The method according to  claim 8 , further comprising:
 comparing the poses of the virtual reference viewpoint and of the new virtual reference viewpoint with the camera poses, and   selecting among the virtual reference viewpoint and the new virtual reference viewpoint, the virtual reference viewpoint closest to the camera's viewpoint,   light source parameters in the 3D scene being determined with regards to the selected virtual reference viewpoint.   
     
     
         11 . The method according to  claim 1 , wherein determining said at least one virtual reference viewpoint in the 3D scene, comprises:
 detecting movement of the camera capturing the real scene, and   predicting a camera trajectory in the 3D scene as a function of the detected movement,   said at least one virtual reference viewpoint being determined according to the predicted camera trajectory.   
     
     
         12 . The method according to  claim 1 , wherein determining said at least one virtual reference viewpoint in the 3D scene comprises determining and storing at least one shadow map associated to a light source of a 3D scene. 
     
     
         13 . The method according to  claim 1 , further comprising:
 detecting geometry changes in the 3D scene, and   determining areas in the input image captured by the camera that are impacted by the geometry changes,   said determined areas being discarded when determining parameters for at least one light source in the 3D scene using the virtual reference viewpoint until virtual shadow maps associated to the virtual reference viewpoint are updated based on the geometry changes.   
     
     
         14 . (canceled) 
     
     
         15 . The apparatus according to  claim 2 , wherein obtaining the map of registered real cast shadows comprises:
 detecting the real cast shadows in the input image to obtain a map of detected real cast shadows, and   registering the map of detected real cast shadows with regards to the virtual reference viewpoint.   
     
     
         16 . The apparatus according to  claim 2 , wherein obtaining the map of registered real cast shadows comprises:
 registering the input image with regards to the virtual reference viewpoint to obtain a registered image, and   detecting the real cast shadows in the registered image.   
     
     
         17 . The apparatus according to  claim 2 , wherein using the map of registered real cast shadows and at least one map of virtual shadows of objects in the 3D scene cast by the at least one light source from the virtual reference viewpoint, comprises matching the map of registered real cast shadows and the at least one map of virtual shadows and selecting the map of virtual shadows that best matches the map of registered real cast shadows. 
     
     
         18 . The apparatus according to  claim 2 , wherein the virtual reference viewpoint corresponds to a viewpoint from which a real scene represented by the 3D scene has been previously captured by the camera. 
     
     
         19 . The apparatus according to  claim 2 , wherein the virtual reference viewpoint in the 3D scene is determined as being a viewpoint for which a number of pixels overlapping in an image captured from the virtual reference viewpoint and an image captured from the viewpoint of the camera is above a determined value. 
     
     
         20 . A non-transitory computer-readable medium storing instructions operative, when executed by a processor, to perform at least:
 determining at least one virtual reference viewpoint in the 3D scene,   obtaining a map of registered real cast shadows of objects in the 3D scene from an input image captured by a camera positioned at a viewpoint distinct from the virtual reference viewpoint, said map of real cast shadows being registered with regards to the virtual reference viewpoint, and   determining parameters for at least one light source in the 3D scene using the map of registered real cast shadows and at least one map of virtual shadows of objects in the 3D scene cast by the at least one light source from the virtual reference viewpoint.