Image processing apparatus and image processing method

Abstract

An image processing apparatus of one embodiment executes a super-resolution processing which generate a high resolution image of a frame from low resolution images of a plurality of frames by performing alignment of pixels on a reference frame on a standard frame which is made to have high resolution and executing reconfiguration of an image based on a result of the aligning. The image processing apparatus includes a plurality of arithmetic processing units which can operate in parallel. When executing alignment between the standard frame and the reference frame, the plurality of arithmetic processing units operate in parallel a first alignment processing and a second alignment processing which are in the reverse relation to each other in terms of the relation of the standard and the reference, and the first alignment processing and the second alignment processing share their processing results mutually.

What is claimed is: 
     
         1 . An image processing apparatus which performs alignment of pixels on a reference frame on a standard frame to be made to have high resolution, and executes reconfiguration of an image based on a result of the aligning, to thereby generate a high resolution image of a frame from low resolution images of a plurality of frames, the image processing apparatus comprising:
 a main memory to store a video taken in from outside and to output a video which has been image processed; and   a plurality of arithmetic processing units which can operate in parallel;   wherein the plurality of arithmetic processing units, at the time of executing alignment between the standard frame and the reference frame, operate in parallel a first alignment processing and a second alignment processing which are in the reverse relation to each other in terms of the relation of the standard and the reference, and the first alignment processing and the second alignment processing share their processing results.   
     
     
         2 . The image processing apparatus as recited in  claim 1 , wherein with respect to a pixel of which a corresponding point pair on the reference frame and the standard frame has already been calculated by one processing of the first alignment processing and the second alignment processing, calculation result for the corresponding point pair is used by the other processing of the first alignment processing and the second alignment processing. 
     
     
         3 . The image processing apparatus as recited in  claim 2 , further comprising:
 a memory unit to provide memory areas, the memory areas which can be accessed by each of the first alignment processing and the second alignment processing, which store their processing results, and which are prepared for each pixel of an input image for each of the first alignment processing and the second alignment processing;   wherein when the one processing has finished calculation of the corresponding point pair, and the other processing has not finished the calculation of the relevant corresponding point pair, the one processing reflects calculation result thereof in the memory area corresponding to the other processing, and thereby the other processing uses the calculation result.   
     
     
         4 . The image processing apparatus as recited in  claim 1 , wherein: each of the first alignment processing and the second alignment processing performs alignment to pixels of each of the reference frames in the reverse sequence mutually, and when one processing of the first alignment processing and the second alignment processing has completed the alignment for the all pixels at the reference frame thereof in first, the one processing takes over the alignment for a part of the non-processed pixels on the reference frame which the other processing of the first alignment processing and the second alignment processing targets. 
     
     
         5 . The image processing apparatus as recited in  claim 1 , wherein: each of the first alignment processing and the second alignment processing performs alignment to pixels on a line at the same location on each of the reference frames in the reverse sequence mutually, and when one processing of the first alignment processing and the second alignment processing has completed the processing for the relevant one line in first, the one processing takes over the alignment for a part of the non-processed pixels on the one line which the other processing of the first alignment processing and the second alignment processing targets. 
     
     
         6 . The image processing apparatus as recited in  claim 1 , wherein: reconfiguration processing which reconfigures the image from each of the standard frames performs a pipeline operation upon receiving the alignment result which has been processed in parallel by each of the alignment processings. 
     
     
         7 . An image processing method used in an image processing apparatus which is provided with a plurality of arithmetic processing units which can be operated in parallel, generate a high resolution image of a frame from low resolution images of a plurality of frames by performing alignment of pixels on a reference frame on a standard frame to be made to have high resolution and performing reconfiguration of an image based on a result of alignment, the image processing method, comprising steps of:
 operating in parallel two alignment processings which are in the reverse relation to each other in terms of the relation of the standard and the reference, at the time of performing alignment between the standard frame and the reference frame; and   sharing processing results of the two alignment processing by the two alignment processing.   
     
     
         8 . An image processing apparatus which generate a high resolution image of a frame by performing alignment of pixels on a reference frame of a low resolution image on a standard frame of a low resolution image and executing reconfiguration of an image of the standard frame based on a result the alignment, the image processing apparatus, comprising:
 a main memory to store a video taken in from outside and to output a video which has been image processed;   a first arithmetic processing unit to execute a first alignment processing to perform alignment between a standard frame and a reference frame which are extracted from the video taken in the main memory;   a second arithmetic processing unit to execute a second alignment processing to perform alignment between a standard frame and a reference frame which are extracted from the video, in parallel to the first alignment processing, while in the relation of the standard frame and the reference frame extracted from the video the standard and the reference is reversed; and   a cache memory which is commonly provided for the first arithmetic processing unit and the second arithmetic processing unit, and stores processing results of the first alignment processing and the second alignment processing so that the first alignment processing and the second alignment processing share their processing results.