Image processing apparatus and method, and program

Abstract

An image processing apparatus includes a motion compensation parameter calculation unit that calculates a parameter for compensating a relative motion in an image pickup apparatus between picked-up images, a first connection line setting unit that sets as a first connection line a central position in an overlapping area, a moving subject detection unit that generates subject information by detecting a moving subject using the images in the overlapping area, a subject information projection unit that projects the subject information in the overlapping area on a sweep axis, a second connection line setting unit that performs setting of a second connection line on the basis of the projected subject information, an image pickup condition detection unit that detects an image pickup condition in which a two-dimensional rotation is occurred, and a connection line selection unit that selects the first or second connection line as a connection line.

What is claimed is: 
     
         1 . An image processing apparatus comprising:
 a motion compensation parameter calculation unit to calculate a parameter for compensating a relative motion in an image pickup apparatus that performs image pickup between a plurality of picked-up images, some of the picked-up images are being overlapped with each other;   a first connection line setting unit to set as a first connection line a position with respect to a changing direction of the image pickup direction in an overlapping area;   a moving subject detection unit to generate subject information by detecting a moving subject using the picked-up images in the overlapping area;   a subject information projection unit to project the subject information in the overlapping area;   a second connection line setting unit to set a second connection line at least in part on a basis of the projected subject information;   an image pickup condition detection unit to detect an image pickup condition in which a two-dimensional rotation between the picked-up images has occurred; and   a connection line selection unit to select one of the first connection line and the second connection line as a connection line for adjacent picked-up images in accordance with a detection result of the image pickup condition detection unit.   
     
     
         2 . The image processing apparatus according to  claim 1 ,
 wherein the image pickup condition detection unit detects a two-dimensional rotation angle between the picked-up images or an angle in the image pickup direction of the image pickup apparatus which is orthogonal to the sweep axis, and   wherein the connection line selection unit selects the first connection line when the rotation angle or the angle in the image pickup direction is larger than a previously set threshold range and selects the second connection line when the rotation angle or the angle in the image pickup direction is within the threshold range;   wherein the moving subject detection unit uses an evaluation value in accordance with a motion amount of the moving subject as the subject information;   wherein the subject information projection unit projects the subject information on the sweep axis by integrating evaluation values at the respective positions on the picked-up image in a direction orthogonal to the sweep axis.   
     
     
         3 . The image processing apparatus according to  claim 2 ,
 wherein the image pickup condition detection unit detects the angle by using a detection signal from an attitude detection unit provided in the image pickup apparatus.   
     
     
         4 . The image processing apparatus according to  claim 2 , further comprising:
 an image projection unit to utilize the motion compensation parameter and to project the plurality of picked-up images on a panoramic coordinate system; and   an image synthesis unit to connect the picked-up images projected on the panoramic coordinate system at the selected connection line to generate a panoramic image.   
     
     
         5 . An image processing method comprising the steps of:
 calculating, using a motion compensation parameter calculation unit, a parameter for compensating a relative motion in an image pick up apparatus that performs image pick up between a plurality of picked-up images, wherein some of the picked up images overlap with each other;   setting, using a first connection line setting unit, as a first connection line a position with respect to a changing direction of an image pick up direction in an overlapping area;   generating, using a moving subject detection unit, subject information by detecting a moving subject using the picked-up images in the overlapping area;   projecting, using a subject information projecting unit, the subject information in the overlapping area;   setting, using a second connection line setting unit, a second connection line at least in part on a basis of the projected subject information;   detecting, using an image pick up condition detection unit, an image pick up condition in which a two-dimensional rotation unit between the picked-up images has occurred; and   selecting, using a connection line selection unit, one of the first connection line and the second connection line as a connection line for adjacent picked-up images in accordance with a detection result of the image pick up condition detection unit.   
     
     
         6 . A computer readable medium on which is stored a computer program which when implemented by a computer causes the computer to:
 calculating a parameter for compensating a relative motion in an image pick up apparatus that performs image pick up between a plurality of picked-up images, wherein some of the picked up images overlap with each other;   setting, as a first connection line, a position with respect to a changing direction of the image pick up direction in an overlapping area;   generating, subject information by detecting a moving subject using the picked-up images in the overlapping area;   projecting the subject information in the overlapping area;   setting a second connection line at least in part on a basis of the projected subject information;   detecting an image pick up condition in which a two-dimensional rotation unit between the picked-up images has occurred; and   selecting one of the first connection line and the second connection line as a connection line for adjacent picked-up images in accordance with a detection result of the image pick up condition detection unit.