Image processing apparatus, image capturing system, image processing method, and recording medium

Abstract

An information processing apparatus obtain a first image in first projection, and a second image in second projection; transforms projection of a first corresponding area of the first image to generate a third image in the second projection; identifies a plurality of feature points in the second image and the third image; determines a second corresponding area in the third image based on the plurality of feature points; corrects the second corresponding area based on a plurality of blocks in the third image, which are determined through matching a plurality of blocks divided from the second image to the third image; transforms projection of a plurality of points in the corrected corresponding area to obtain location information indicating locations of the plurality of points in the first image; and stores the location information in association with the plurality of points in the second image in the second projection.

1 . An image processing apparatus comprising processing circuitry configured to:
 obtain a first image in first projection, and a second image in second projection;   transform projection of a first corresponding area of the first image, which corresponds to the second image, from the first projection to the second projection, to generate a third image in the second projection;   identify a plurality of feature points, respectively, in the second image and the third image;   determine a second corresponding area in the third image, which corresponds to the second image, based on the plurality of feature points respectively identified in the second image and the third image;   correct the second corresponding area based on a plurality of blocks in the third image, which are determined through matching a plurality of blocks divided from the second image to corresponding areas of the third image;   transform projection of a plurality of points in the corrected corresponding area of the third image, from the second projection to the first projection, to obtain location information indicating locations of the plurality of points that have been obtained through transformation in the first image; and   store, in a memory, the location information indicating the locations of the plurality of points in the first image in the first projection, in association with the plurality of points in the second image in the second projection.   
     
     
         2 . The image processing apparatus of  claim 1 , wherein, in a process of correcting, the processing circuitry is configured to:
 divide the second image into the plurality of blocks;   determine an area of the third image that corresponds to each one of the plurality of blocks of the second image, to obtain the plurality of blocks of the third image that match the plurality of blocks of the second image;   calculate, for each representative point of the second corresponding area, a motion vector from the representative point to a point of the corresponding block of the third image that corresponds to the representative point; and   correct the representative point of the second corresponding area based on the motion vector, to generate the corrected second corresponding area.   
     
     
         3 . The image processing apparatus of  claim 2 , wherein the processing circuitry is configured to:
 correct the motion vector that is calculated, based on similarity between the first image and the third image; and   correct the representative point of the second corresponding area, based on the corrected motion vector.   
     
     
         4 . The image processing apparatus of  claim 2 , wherein the processing circuitry is configured to:
 correct the motion vector that is calculated, based on light variance in the third image; and   correct the representative point of the second corresponding area, based on the corrected motion vector.   
     
     
         5 . The image processing apparatus of  claim 1 , wherein the first image is an equirectangular projection image, and the second image is a perspective projection image. 
     
     
         6 . The image processing apparatus of  claim 1 , wherein the image processing apparatus includes at least one of a smart phone, tablet personal computer, notebook computer, desktop computer, and server computer. 
     
     
         7 . An image capturing system comprising:
 the image processing apparatus of  claim 1 ; and   a first image capturing device configured to capture surroundings of a target object to obtain the first image in the first projection and transmit the first image in the first projection to the image processing apparatus; and   a second image capturing device configured to capture the target object to obtain the second image in the second projection and transmit the second image in the second projection to the image processing apparatus.   
     
     
         8 . An image processing method comprising:
 obtaining a first image in first projection, and a second image in second projection;   transforming projection of a first corresponding area of the first image, which corresponds to the second image, from the first projection to the second projection, to generate a third image in the second projection,   identifying a plurality of feature points, respectively, in the second image and the third image;   determining a second corresponding area in the third image, which corresponds to the second image, based on the plurality of feature points respectively identified in the second image and the third image;   correcting the second corresponding area based on a plurality of blocks in the third image, which are determined through matching a plurality of blocks divided from the second image to corresponding areas of the third image;   transforming projection of a plurality of points in the corrected second corresponding area of the third image, from the second projection to the first projection, to obtain location information indicating locations of the plurality of points that have been obtained through transformation in the first image; and   storing, in a memory, the location information indicating the locations of the plurality of points in the first image in the first projection, in association with the plurality of points in the second image in the second projection.   
     
     
         9 . The image processing method of  claim 8 , wherein the correcting includes:
 dividing the second image into the plurality of blocks;   determining an area of the third image that corresponds to each one of the plurality of blocks of the second image, to obtain the plurality of blocks of the third image that match the plurality of blocks of the second image;   calculating, for each representative point of the second corresponding area, a motion vector from the representative point to a point of the corresponding block of the third image that corresponds to the representative point; and   correcting the representative point of the second corresponding area based on the motion vector, to generate the corrected second corresponding area.   
     
     
         10 . A non-transitory recording medium which, when executed by one or more processors, cause the processors to perform an image processing method, comprising:
 obtaining a first image in first projection, and a second image in second projection;   transforming projection of a first corresponding area of the first image, which corresponds to the second image, from the first projection to the second projection, to generate a third image in the second projection;   identifying a plurality of feature points, respectively, in the second image and the third image;   determining a second corresponding area in the third image, which corresponds to the second image, based on the plurality of feature points respectively identified in the second image and the third image;   correcting the second corresponding area based on a plurality of blocks in the third image, which are determined through matching a plurality of blocks divided from the second image to corresponding areas of the third image;   transforming projection of a plurality of points in the corrected second corresponding area of the third image, from the second projection to the first projection, to obtain location information indicating locations of the plurality of points that have been obtained through transformation in the first image; and   storing, in a memory, the location information indicating the locations of the plurality of points in the first image in the first projection, in association with the plurality of points in the second image in the second projection.