Information processing apparatus and control method thereof

Abstract

An information processing apparatus sets a first area and a second area corresponding to image areas of a tracking target, which are different from each other, for a template image included in a video sequence and including an image of the tracking target. The apparatus extracts a first area feature and a second area feature from the first area and the second area using a feature extraction NN, extracts a search area feature from a search area image using the feature extraction NN, and performs correlation calculation and deriving a first feature correlation map and a second feature correlation map. The apparatus integrates the first feature correlation map and the second feature correlation map and deriving an integrated feature, and detects the tracking target from the search area image based on the integrated feature.

What is claimed is: 
     
         1 . An information processing apparatus for tracking a tracking target in a video sequence, comprising:
 one or more hardware processors; and   one or more memories storing one or more programs configured to be executed by the one or more hardware processors, the one or more programs including instructions for:   performing objectness detection for detecting an image area with a relatively high object existence likelihood in an image in the video sequence;   setting a first area and a second area corresponding to image areas of the tracking target, which are different from each other, for a template image included in the video sequence and including an image of the tracking target, wherein the first area and the second area are set based on the image area detected by the performing objectness detection for the template image;   extracting a first area feature and a second area feature, which are template features, from the first area and the second area using a feature extraction neural network (NN);   extracting a search area feature from a search area image included in the video sequence and following the template image using the feature extraction NN;   performing correlation calculation between each of the first area feature and the second area feature and the search area feature and deriving a first feature correlation map and a second feature correlation map, which are correlation maps;   integrating the first feature correlation map and the second feature correlation map and deriving an integrated feature; and   detecting the tracking target from the search area image based on the integrated feature,   wherein the objectness detection is executed for a plurality of template images, and   an image feature is clustered for the image area, of the image area based on the objectness detection for the template image, having at least a predetermined overlap with the tracking target, and the first area and the second area are set based on the image area corresponding to a cluster with a large appearance count.   
     
     
         2 . The apparatus according to  claim 1 , wherein the first feature correlation map and the second feature correlation map are integrated based on a weighting factor corresponding to the first feature correlation map and the second feature correlation map. 
     
     
         3 . The apparatus according to  claim 2 , wherein the one or more programs further include instructions for:
 calculating the weighting factor, and   the calculating includes a neural network that calculates the weighting factor according to the tracking target using the search area feature.   
     
     
         4 . The apparatus according to  claim 1 , wherein the one or more programs further include instructions for:
 dividing the image into a plurality of areas by giving the same values to pixels which are likely to include the same object, and   the first area and the second area are set based on an image area of the tracking target estimated based on area division of the template image.   
     
     
         5 . The apparatus according to  claim 1 , wherein
 the image feature is a feature of the neural network included in the performing objectness detection.   
     
     
         6 . The apparatus according to  claim 1 , wherein
 the tracking target is an animal, and   the first area and the second area are areas corresponding to parts of the animal, which are different from each other.   
     
     
         7 . The apparatus according to  claim 1 , wherein
 the first area is an area including the second area.   
     
     
         8 . The apparatus according to  claim 1 , wherein
 in the extracting the first area feature and the second area feature, an inclusion area feature for an inclusion area including the first area and the second area is extracted using the feature extraction NN, and area features corresponding to the first area and the second area are cut out from the inclusion area feature, thereby extracting the first area feature and the second area feature.   
     
     
         9 . The apparatus according to  claim 2 , wherein
 the weighting factor is derived by the detecting the tracking target by learning such that an image feature effective for detection of the tracking target is obtained from the search area feature based on the integrated feature.   
     
     
         10 . The apparatus according to  claim 1 , wherein
 the feature extraction NN is learned such that importance is placed on a local color feature in, of the first area and the second area, a relatively small area.   
     
     
         11 . The apparatus according to  claim 1 , wherein
 the feature extraction NN is learned such that importance is placed on a global shape feature in, of the first area and the second area, a relatively large area.   
     
     
         12 . A control method of an information processing apparatus for tracking a tracking target in a video sequence, comprising:
 performing objectness detection for detecting an image area with a relatively high object existence likelihood in an image in the video sequence;   setting a first area and a second area corresponding to image areas of the tracking target, which are different from each other, for a template image included in the video sequence and including an image of the tracking target, wherein the first area and the second area are set based on the image area detected by the performing objectness detection for the template image;   extracting a first area feature and a second area feature, which are template features, from the first area and the second area using a feature extraction neural network (NN);   extracting a search area feature from a search area image included in the video sequence and following the template image using the feature extraction NN;   performing correlation calculation between each of the first area feature and the second area feature and the search area feature and deriving a first feature correlation map and a second feature correlation map, which are correlation maps;   integrating the first feature correlation map and the second feature correlation map and deriving an integrated feature;   detecting the tracking target from the search area image based on the integrated feature, wherein the objectness detection is executed for a plurality of template images; and   clustering an image feature for the image area, of the image area based on the objectness detection for the template image, having at least a predetermined overlap with the tracking target, and the first area and the second area are set based on the image area corresponding to a cluster with a large appearance count.   
     
     
         13 . A non-transitory computer-readable recording medium storing a program that, when executed by a computer, causes the computer to perform a control method of an information processing apparatus for tracking a tracking target in a video sequence, comprising:
 performing objectness detection for detecting an image area with a relatively high object existence likelihood in an image in the video sequence;   setting a first area and a second area corresponding to image areas of the tracking target, which are different from each other, for a template image included in the video sequence and including an image of the tracking target, wherein the first area and the second area are set based on the image area detected by the performing objectness detection for the template image;   extracting a first area feature and a second area feature, which are template features, from the first area and the second area using a feature extraction neural network (NN);   extracting a search area feature from a search area image included in the video sequence and following the template image using the feature extraction NN;   performing correlation calculation between each of the first area feature and the second area feature and the search area feature and deriving a first feature correlation map and a second feature correlation map, which are correlation maps;   integrating the first feature correlation map and the second feature correlation map and deriving an integrated feature;   detecting the tracking target from the search area image based on the integrated feature, wherein the objectness detection is executed for a plurality of template images; and   clustering an image feature for the image area, of the image area based on the objectness detection for the template image, having at least a predetermined overlap with the tracking target, and the first area and the second area are set based on the image area corresponding to a cluster with a large appearance count.