Apparatus to perform alignment to images, image processing method to perform alignment to images, and computer readable non-transitory memory to perform alignment to images

Abstract

An apparatus includes at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions to detect a feature amount from each of a plurality of images different in in-focus position in an optical axis direction, and align the plurality of images by using the feature amount. The at least one processor further executes instructions to detect the feature amount by preferentially using a focusing area of each of the plurality of images.

What is claimed is: 
     
         1 . An apparatus, comprising:
 at least one memory configured to store instructions; and   at least one processor in communication with the at least one memory and configured to execute the instructions to:   arrange a region in each of a plurality of images different in in-focus position in an optical axis direction;   detect a feature amount from the region in each of the plurality of images; and   align the plurality of images by using the feature amount, and   in a case where the detected feature amount does not satisfy a predetermined condition, re-arrange the region.   
     
     
         2 . The apparatus according to  claim 1 , wherein the at least one processor further executes instructions to generate a composite image by extracting respective focusing areas of the plurality of images. 
     
     
         3 . The apparatus according to  claim 2 , wherein the composite image has a depth of field deeper than a depth of field of each of the plurality of images. 
     
     
         4 . The apparatus according to  claim 1 , wherein the feature amount is a motion vector. 
     
     
         5 . The apparatus according to  claim 1 , wherein the at least one processor further executes instructions to arrange the region based on a focusing area of each of the plurality of images. 
     
     
         6 . The apparatus according to  claim 2 , wherein the at least one processor further executes instructions to detect the focusing area of each of the plurality of images by using contrast of the plurality of images. 
     
     
         7 . The apparatus according to  claim 4 , wherein the at least one processor further executes instructions to:
 arrange a plurality of blocks in the region in each of the plurality of images; and   detect a feature amount from each of the plurality of blocks in one image of the plurality of images.   
     
     
         8 . The apparatus according to  claim 7 , wherein the at least one processor further executes instructions to arrange a plurality of blocks similar to the plurality of blocks in each of the plurality of blocks. 
     
     
         9 . The apparatus according to  claim 8 , wherein the at least one processor further executes instructions to re-arrange the plurality of blocks based on a focusing area. 
     
     
         10 . The apparatus according to  claim 9 , wherein the at least one processor further executes instructions to re-arrange the plurality of blocks with at least one of a changed number of blocks, a changed size of each of the blocks, and changes positions of the blocks. 
     
     
         11 . The apparatus according to  claim 7 , wherein the predetermined condition is a condition where a number of the motion vectors detected is greater than a predetermined number. 
     
     
         12 . The apparatus according to  claim 7 , wherein the predetermined condition is a condition where a magnitude of each of the motion vectors is greater than a predetermined threshold value. 
     
     
         13 . The apparatus according to  claim 7 , wherein the predetermined condition is a condition where deviation of the motion vectors is less than a predetermined threshold value. 
     
     
         14 . An apparatus comprising:
 a sensor to pick up a plurality of images at different in-focus positions in an optical axis direction;   at least one memory configured to store instructions; and   at least one processor in communication with the at least one memory and configured to execute the instructions to:   arrange a region in each of the plurality of images;   detect a feature amount from the region in each of the plurality of images; and   align the plurality of images by using the feature amount, and   in a case where the detected feature amount does not satisfy a predetermined condition, re-arrange the region.   
     
     
         15 . A method comprising:
 arranging a region in each of a plurality of images different in in-focus position in an optical axis direction;   detecting a feature amount from the region in each of the plurality of images; and   aligning the plurality of images by using the feature amount, and   in a case where the detected feature amount does not satisfy a predetermined condition, re-arranging the region.   
     
     
         16 . A non-transitory computer-readable storage medium which stores a program for causing a computer of an apparatus to execute a method, the method comprising:
 arranging a region in each of a plurality of images different in in-focus position in an optical axis direction;   detecting a feature amount from the region in each of the plurality of images; and   aligning the plurality of images by using the feature amount, and   in a case where the detected feature amount does not satisfy a predetermined condition, re-arranging the region.