Image synthesizing apparatus and method, position detecting apparatus and method, and supply medium

Abstract

When a CG image is synthesized with an actually photographic image (dynamic image), it is controlled that the synthesized image becomes unnatural. An image pickup section 2 picks up a plate 1 while moving. When synthesizing a ball formed through CG with a continuous image picked up by the image pickup section 2, how 12 patterns (in the portion enclosed by a frame 21 ) Aj (j=1, . . . , 12) present nearby a ball 5 are transformed in the continuous image picked up by the image pickup section 2 is shown by a rotational transform function and a rectilinear transform function to obtain the synthesis position of the projected image of the ball 5 in each image by using the rotational transform function and rectilinear transform function.

1 . An image synthesizing apparatus for synthesizing the projected image of an object with at least first and second images, the apparatus comprising: 
 first means for obtaining a first characteristic point nearby a synthesis position for synthesizing said object among the characteristic points projected onto said first image;    second means for obtaining first coordinates of said first characteristic point on a three-dimensional coordinate system corresponding to said first image;    third means for obtaining second coordinates of a second characteristic point corresponding to said first characteristic point on a three-dimensional coordinate system corresponding to said second image among the characteristics points projected onto said second image;    fourth means for obtaining coordinate transform functions of said first coordinates and said second coordinates;    fifth means for obtaining third coordinates on the three-dimensional coordinate system of said synthesis position for synthesizing said object with said first image corresponding to said first image;    sixth means for obtaining fourth coordinates on a three-dimensional coordinate system corresponding to said second image by applying said coordinate transform functions to said third coordinates; and    seventh means for synthesizing the projected image of said object at a position corresponding to said fourth coordinates of said second image.    
     
     
         2 . The image synthesizing apparatus according to  claim 1 , 
 wherein said coordinate transform functions are functions related to rotational transform and rectilinear transform.    
     
     
         3 . The image synthesizing apparatus according to  claim 1 , wherein 
 said sixth means obtains said fourth coordinates in accordance with values obtained by adding first three-dimensional vector to said third coordinates,    said seventh means synthesizes the projected image of said object at a position corresponding to values obtained by adding second three-dimensional vector to said fourth coordinates.    
     
     
         4 . The image synthesizing apparatus according to  claim 3 , 
 wherein at least one of said first and second three-dimensional vectors is zero vector.    
     
     
         5 . The image synthesizing apparatus according to  claim 1 , wherein said fourth means provides a weight larger than that of a fourth characteristic point farther from a synthesis position for synthesizing said object for a third characteristic point closer to the synthesis position out of said first characteristic points and thereby, obtains coordinate transform functions of said first and second coordinates.  
     
     
         6 . The image synthesizing apparatus according to  claim 1 , wherein said first characteristic points are obtained from points in which at least one of a brightness signal and a color difference signal greatly changes compared to adjacent points.  
     
     
         7 . The image synthesizing apparatus according to  claim 1 , wherein 
 said first image is an image obtained by actually photographing a predetermined object at the first time,    said second image is an image obtained by actually photographing said object at the second time later than said first time, and    said object is formed through computer graphics.    
     
     
         8 . An image synthesizing method for synthesizing the projected image of an object with first and second images, the method comprising: 
 the first step of obtaining a first characteristic point nearby a synthesis position for synthesizing said object among the characteristic points projected onto said first image;    the second step of obtaining first coordinates on the three-dimensional coordinate system of said first characteristic point corresponding to said first image;    the third step of obtaining second coordinates of a second characteristic point corresponding to said first characteristic point on a three-dimensional coordinate system corresponding to said second image among the characteristic points projected onto said second image;    the fourth step of obtaining coordinate transform functions of said first and second coordinates;    the fifth step of obtaining third coordinates on the three-dimensional coordinate system of said synthesis position for synthesizing said object with said first image corresponding to said first image;    the sixth step of obtaining fourth coordinates on a three-dimensional coordinate system corresponding to said second image by applying said coordinate transform functions to said third coordinates; and    the seventh step of synthesizing the projected image of said object at a position corresponding to the fourth coordinates of said second image.    
     
     
         9 . The image synthesizing method according to  claim 8 , wherein said coordinate transform functions are functions related to rotational transform and rectilinear transform.  
     
     
         10 . The image synthesizing method according to  claim 8 , wherein 
 said fourth coordinates are obtained in accordance with values obtained by adding first three-dimensional vector to said third coordinates in said sixth step, and    the projected image of said object is synthesized at a position corresponding to values obtained by adding second three-dimensional vector to said fourth coordinates in said seventh step.    
     
     
         11 . The image synthesizing method according to  claim 10 , wherein at least one of said first and second three-dimensional vectors is zero vector.  
     
     
         12 . The image synthesizing method according to  claim 8 , wherein coordinate transform functions of said first and second coordinates are obtained by providing a weight larger than that of fourth characteristic points farther from a synthesis position for synthesizing said object for third characteristic points closer to the synthesis position among said first characteristic points in said fourth step.  
     
     
         13 . The image synthesizing method according to  claim 8 , wherein said first characteristic points are obtained from points in which at least one of a brightness signal and a color difference signal greatly changes compared to adjacent points.  
     
     
         14 . The image synthesizing method according to  claim 8 , wherein 
 said first image is an image obtained by actually photographing a predetermined object at the first time,    said second image is an image obtained by actually photographing said object at the second time later than said first time; and    said object is an object formed through computer graphics.    
     
     
         15 . A supply medium for supplying a computer program used for an image synthesizing apparatus for synthesizing the projected image of an object with at least first and second images, the computer program comprising: 
 the first step of obtaining first characteristic points nearby a synthesis position for synthesizing said object among the characteristic points projected onto said first image;    the second step of obtaining first coordinates of said first characteristic point on a three-dimensional coordinate system corresponding to said first image;    the third step of obtaining second coordinates of a second characteristic point corresponding to said first characteristic point on a three-dimensional coordinate system corresponding to said second image among the characteristic points projected onto said second image;    the fourth step of obtaining coordinate transform functions of said first coordinates and said second coordinates;    the fifth step of obtaining third coordinates of said synthesis position for synthesizing said object with said first image on a three-dimensional coordinate system corresponding to said first image;    the sixth step of obtaining fourth coordinates on a three-dimensional coordinate system corresponding to said second image by applying said coordinate transform functions to said third coordinates; and    the seventh step of synthesizing the projected image of said object at a position corresponding to said fourth coordinates of said second image.    
     
     
         16 . A position detecting apparatus for detecting the positional relation between objects projected onto a first image and a second image, the apparatus comprising: 
 first means for obtaining a first characteristic point nearby said object among the characteristic points projected onto said first image;    second means for obtaining first coordinates of said first characteristic point on a three-dimensional coordinate system corresponding to said first image;    third means for obtaining second coordinates of a second characteristic point corresponding to said first characteristic point on a three-dimensional coordinate system corresponding to said second image among the characteristic points projected onto said second image;    fourth means for obtaining third coordinates of said object on a three-dimensional coordinate system corresponding to said first image;    fifth means for obtaining fourth coordinates of said object on a three-dimensional coordinate system corresponding to said second image;    sixth means for obtaining coordinate transform functions of said first coordinates and said second coordinates;    seventh means for obtaining fifth coordinates on a three-dimensional coordinate system corresponding to said second image by applying said coordinate transform functions to said third coordinates; and    eighth means for detecting the difference between said fourth coordinates and said fifth coordinates.    
     
     
         17 . A position detecting method for detecting the positional relation between objects projected onto a first image and a second image, the method comprising; 
 the first step of obtaining a first characteristic point nearby said object among the characteristic points projected onto said first image;    the second step of obtaining first coordinates of said first characteristic point on a three-dimensional coordinate system corresponding to said first image;    the third step of obtaining second coordinates of a second characteristic point corresponding to said first characteristic point on a three-dimensional coordinate system corresponding to said second image among the characteristic points projected onto said second image;    the fourth step of obtaining third coordinates of said object on a three-dimensional coordinate system corresponding to said first image;    the fifth step of obtaining fourth coordinates of said object on a three-dimensional coordinate system corresponding to said second image;    the sixth step of obtaining coordinate transform functions of said first coordinates and said second coordinates;    the seventh step of obtaining fifth coordinates on a three-dimensional coordinate system corresponding to said second image by applying said coordinate transform functions to said third coordinates; and    the eighth step of detecting the difference between said fourth coordinates and said fifth coordinates.    
     
     
         18 . A supply medium for supplying a computer program used for a position detecting apparatus for detecting the positional relation between objects projected onto a first image and a second image, the computer program comprising: 
 the first step of obtaining a first characteristic point nearby said object among the characteristic points projected onto said first image;    the second step of obtaining first coordinates of said first characteristic point on a three-dimensional coordinate system corresponding to said first image;    the third step of obtaining second coordinates of a second characteristic point corresponding to said first characteristic point on a three-dimensional coordinate system corresponding to said second image among the characteristic points projected onto said second image;    the fourth step of obtaining third coordinates of said object on a three-dimensional coordinate system corresponding to said first image;    the fifth step of obtaining fourth coordinates of said object on a three-dimensional coordinate system corresponding to said second image;    the sixth step of obtaining coordinate transform functions of said first coordinates and said second coordinates;    the seventh step of obtaining fifth coordinates on a three-dimensional coordinate system corresponding to said second image by applying said coordinate transform functions to said third coordinates; and    the eighth step of detecting the difference between said fourth coordinates and said fifth coordinates.    
     
     
         19 . An image synthesizing apparatus for synthesizing the projected image of a second object by assuming that said second object is present at a predetermined position of each of a plurality of photographic images on which a first object photographed by an image pickup device is photographed, the apparatus comprising: 
 image-pickup-device-position computing means for computing the most-probable position of said image pickup device when photographing each of said photographic images;    characteristic point position computing means for computing the most-probable position of a characteristic point of said first object;    virtual projected position computing means for computing a virtual projected position on which the most-probable position of said characteristic point is projected when performing photography by using said image pickup device present at said most-probable position;    distortion value computing means for computing a distortion value in accordance with the difference between said virtual projected position and the position of the projected image of said characteristic point actually photographed on each of said photographic images;    correcting means for correcting a position to which the projected image of said second object is set in accordance with the distortion value computed by said distortion value computing means;    projected image computing means for computing the projected image of said second object when photographing said second object from said image pickup device at the most-probable position by assuming that said second object is present at the position corrected by said correcting means; and    synthesizing means for synthesizing the projected image of said second object computed by said projected image computing means with each of said photographic images.    
     
     
         20 . The image synthesizing apparatus according to  claim 19 , wherein said distortion value computing means computes said distortion value through interpolation in accordance with said characteristic point.  
     
     
         21 . An image synthesizing method for synthesizing the projected image of said second object with each of a plurality of photographic images on which a first object is photographed by an image pickup device by assuming that a second object is present at a predetermined position, the method comprising: 
 the image-pickup-device-position computing step for computing the most-probable position of said image pickup device when photographing each of said photographic images;    the characteristic point position computing step of computing the most-probable position of a characteristic point of said first object;    the virtual projected position computing step for computing a virtual projected position on which the most-probable position of said characteristic point is projected when performing photography by using said image pickup device set at said most-probable position;    the distortion value computing step of computing a distortion value in accordance with the difference between said virtual projected position and the position of the projected image of said characteristic point actually photographed on each of said photographic images;    the correcting step of correcting a position to which the projected image of said second object is set in accordance with the distortion value computed by said distortion value computing step;    the projected image computing step for computing the projected image of said second object when photographing said second object from said image pickup device at the most-probable position by assuming that said second object is located at the position corrected in said correcting step; and    the synthesizing step of synthesizing the projected image of said second object computed in said projected image computing step with each of said photographic images.    
     
     
         22 . The image synthesizing method according to  claim 21 , wherein said distortion value is computed through interpolation in accordance with said characteristic point in said distortion value computing step.  
     
     
         23 . A supply medium for supplying a computer program used for an image synthesizing apparatus for synthesizing the projected image of a second object with each of a plurality of photographic images on which a first object photographed by an image pickup device is photographed by assuming that the second object is present at a predetermined position, the computer program comprising: 
 the image-pickup-device-position computing step of computing the most-probable position of said image pickup device when photographing each of said photographic images;    characteristic point position computing step of computing the most-probable position of a characteristic point of said first object;    virtual projected position computing step of computing a virtual projected position to which the most-probable position of said characteristic point is projected when performing photography by using said image pickup device set at the most-probable position;    the distortion value computing step of computing a distortion value in accordance with the difference between said virtual projected position and the position of the projected image of said characteristic point actually photographed on each of said photographic images;    the correcting step of correcting a position to which the projected image of said second object is set in accordance with the distortion value computed in said distortion value computing step;    the projected image computing step of computing the projected image of said second object when photographing said second object from said image pickup device at the most-probable position by assuming that said second object is present at the position corrected in said correcting step; and    synthesizing step of synthesizing the projected image of said second object computed in said projected image computing step with each of said photographic images.