Tracking point detecting device and method, program, and recording medium

Abstract

A tracking point detecting device includes: a frame decimation unit for decimation the frame interval of a moving image configured of multiple frame images continuing temporally; a first detecting unit for detecting, of two consecutive frames of the decimated moving image, a temporally-subsequent frame pixel corresponding to a predetermined pixel of a temporally-previous frame; a forward-direction detecting unit for detecting the pixel corresponding to a predetermined pixel of a temporally-previous frame of the decimated moving image, at each of the decimated frames in the same direction as time; an opposite-direction detecting unit for detecting the pixel corresponding to the detected pixel of a temporally-subsequent frame of the decimated moving image, at each of the decimated frames in the opposite direction of time; and a second detecting unit for detecting a predetermined pixel of each of the decimated frames by employing the pixel positions detected in the forward and opposite directions.

1 . A tracking point detecting device comprising:
 frame decimation means configured to perform decimation of the frame interval of a moving image made up of a plurality of frame images which continue temporally;   first detecting means configured to detect, of two consecutive frames of said moving image of which the frames were decimated, a temporally subsequent frame pixel corresponding to a predetermined pixel of a temporally previous frame as a tracking point;   forward-direction detecting means configured to perform forward-direction detection for detecting the pixel corresponding to a predetermined pixel of a temporally previous frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the same direction as time in order;   opposite-direction detecting means configured to perform opposite-direction detection for detecting the pixel corresponding to said detected pixel of a temporally subsequent frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the opposite direction as to time in order; and   second detecting means configured to detect a predetermined pixel of each of said decimated frames as a tracking point by computation employing information representing the position of the pixel detected with said forward-direction detection, and the position of the pixel detected with said opposite-direction detection.   
   
   
       2 . The tracking point detecting device according to  claim 1 , further comprising:
 reduction means configured to reduce a moving image made up of a plurality of frame images which continue temporally;   wherein said frame decimation means perform decimation of the frame interval of said reduced moving image;   and wherein said first detecting means and said second detecting means each detect a tracking point of the frames of said reduced moving image.   
   
   
       3 . The tracking point detecting device according to  claim 2 , further comprising:
 conversion means configured to convert the position of the pixel of the tracking point detected by said second detecting means into the position of the pixel of said tracking point of the frames of said moving image not reduced.   
   
   
       4 . The tracking point detecting device according to  claim 1 , further comprising:
 candidate setting means configured to set a plurality of pixels serving as candidates, of a temporally previous frame of said moving image of which the frames were decimated;   wherein said first detecting means detect each of the pixels of a temporally subsequent frame corresponding to each of the pixels serving as the candidates of a temporally previous frame as a tracking point candidate;   and wherein said forward-direction detecting means detect each of the pixels corresponding to each of the pixels serving as candidates of a temporally previous frame at each of said decimated frames in the forward direction;   and wherein said opposite-direction detecting means detect each of the pixels corresponding to the pixel detected as said tracking point candidate of a temporally subsequent frame at each of said decimated frames in the opposite direction;   and wherein said second detecting means detect each of a plurality of pixels as a tracking point candidate at each of said decimated frames by computation employing information representing the position of each of the pixels detected with said forward-direction detection, and the position of each of the pixels detected with said opposite-direction detection.   
   
   
       5 . The tracking point detecting device according to  claim 4 , with information representing the position of a predetermined pixel of said plurality of pixels serving as candidates at said temporally previous frame, set by said candidate setting means, information representing the position of the pixel detected by said first detecting means as a tracking point candidate at said temporally subsequent frame corresponding to said predetermined pixel, information representing the position of the pixel of each of said decimated frames corresponding to said predetermined pixel detected in the forward direction by said forward-direction detecting means, information representing the position of the pixel of each of said decimated frames corresponding to said predetermined pixel detected in the opposite direction by said opposite-direction detecting means, information representing the positions of said predetermined pixel, and the pixel detected by said second detecting means as the tracking point candidate of each of said decimated frames corresponding to said tracking point candidate being correlated and taken as a set of tracking point candidate group, said tracking point detecting device further comprising:
 storage means configured to store the same number of sets of tracking point candidate groups as the number of said pixels serving as candidates set by said candidate setting means.   
   
   
       6 . The tracking point detecting device according to  claim 5 , wherein said first detecting means calculate the sum of absolute differences of the pixel values of a block made up of pixels with a predetermined pixel of a temporally previous frame as the center, and the pixel value of a plurality of blocks made up of pixels with each of a plurality of pixels at the periphery of the pixel of the position corresponding to said predetermined pixel at said temporally subsequent frame as the center, and detect, of said plurality of blocks, the pixel serving as the center of the block with the value of said sum of absolute differences as the smallest, as a tracking point. 
   
   
       7 . The tracking point detecting device according to  claim 6 , wherein said first detecting means set a plurality of blocks made up of pixels with each of pixels within a motion detection pixel range which is a predetermined area with a predetermined pixel of said temporally previous frame as the center, as the center, detect the pixel of said tracking point corresponding to each of the pixels within said motion detection pixel range, and detect the coordinate value calculated based on the coordinate value of the pixel of said tracking point corresponding to each of the pixels within said motion detection pixel range as the position of the tracking point of a temporally subsequent frame corresponding to a predetermined pixel of a temporally previous frame. 
   
   
       8 . The tracking point detecting device according to  claim 7 , further comprising:
 difference value calculating means configured to calculate the value of the sum of absolute differences of a pixel value within a predetermined area with the pixel of a tracking point detected beforehand of a further temporally previous frame as compared to said temporally previous frame as the center, and a pixel value within a predetermined area with each of said plurality of pixels serving as candidates, of said temporally previous frame, set by said candidate setting means as the center; and   distance calculating means configured to calculate the distance between said pixel detected in the forward direction, and said pixel detected in the opposite direction at the frame positioned in the middle temporally, of said decimated frames, based on information representing the pixel position of each of said decimated frames detected in said forward direction, and information representing the pixel position of each of said decimated frames detected in said opposite direction, stored in said storage means.   
   
   
       9 . The tracking point detecting device according to  claim 8 , wherein said calculated value of the sum of absolute differences, and said calculated distance are compared with predetermined values respectively, thereby detecting a plurality of pixels satisfying a condition set beforehand from said plurality of pixels serving as candidates set by said candidate setting means, and one pixel of said plurality of pixels serving as candidates set by said candidate setting means is determined based on the information of the position of each pixel satisfying said predetermined condition, and of a plurality of tracking point groups stored by said storage means, the tracking point group corresponding to said determined one pixel is taken as the tracking point at each frame. 
   
   
       10 . The tracking point detecting device according to  claim 1 , further comprising:
 frame interval increment/decrement means configured to increment/decrement the frame interval to be decimated by said frame decimation means based on the value of the sum of absolute differences between a pixel value within a predetermined area with a predetermined pixel of a temporally previous frame as the center, and a pixel value within a predetermined area with the pixel of said temporally subsequent frame detected by said first detecting means as the center, of consecutive two frames of said moving image of which the frames were decimated.   
   
   
       11 . The tracking point detecting device according to  claim 1 , further comprising:
 template holding means configured to hold an image shot beforehand as a template;   object extracting means configured to extract an object not displayed on said template from a predetermined frame image of said moving image; and   pixel determining means configured to determine a pixel for detecting said tracking point from the image of said extracted object.   
   
   
       12 . The tracking point detecting device according to  claim 1 , said first detecting means comprising:
 area extracting means configured to extract the area corresponding to a moving object based on a frame of interest, the temporally previous frame of the frame of interest, and the temporally subsequent frame of the frame of interest, of said moving image of which the frames were decimated; and   intra-area detecting means configured to detect the pixel of said frame of interest corresponding to a predetermined pixel of said temporally previous frame, from the area extracted by said area extracting means.   
   
   
       13 . The tracking point detecting device according to  claim 12 , said area extracting means comprising:
 first screen position shifting means configured to shift the screen position of said frame of interest based on a screen motion vector obtained between said frame of interest and the temporally previous frame of said frame of interest;   first frame difference calculating means configured to calculate the difference between the image of said frame of interest of which the screen position is shifted, and the image of the temporally previous frame of said frame of interest;   second screen position shifting means configured to shift the screen position of said frame of interest based on a screen motion vector obtained between said frame of interest and the temporally subsequent frame of said frame of interest;   second frame difference calculating means configured to calculate the difference between the image of said frame of interest of which the screen position is shifted, and the image of the temporally subsequent frame of said frame of interest; and   AND-area extracting means configured to extract an AND area between the pixel corresponding to said difference calculated by said first frame difference calculating means, and the pixel corresponding to said difference calculated by said second frame difference calculating means, as the area corresponding to an object.   
   
   
       14 . A tracking point detecting method comprising the steps of:
 decimation of the frame interval of a moving image made up of a plurality of frame images which continue temporally;   detecting, of two consecutive frames of said moving image of which the frames were decimated, a temporally subsequent frame pixel corresponding to a predetermined pixel of a temporally previous frame as a tracking point;   forward-direction detecting for detecting the pixel corresponding to a predetermined pixel of a temporally previous frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the same direction as time in order;   opposite-direction detecting for detecting the pixel corresponding to said detected pixel of a temporally subsequent frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the opposite direction as to time in order; and   detecting a predetermined pixel of each of said decimated frames as a tracking point by computation employing information representing the position of the pixel detected with said forward-direction detection, and the position of the pixel detected with said opposite-direction detection.   
   
   
       15 . A program causing a computer to function as a tracking point detecting device comprising:
 frame decimation means configured to perform decimation of the frame interval of a moving image made up of a plurality of frame images which continue temporally;   first detecting means configured to detect, of two consecutive frames of said moving image of which the frames were decimated, a temporally subsequent frame pixel corresponding to a predetermined pixel of a temporally previous frame as a tracking point;   forward-direction detecting means configured to perform forward-direction detection for detecting the pixel corresponding to a predetermined pixel of a temporally previous frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the same direction as time in order;   opposite-direction detecting means configured to perform opposite-direction detection for detecting the pixel corresponding to said detected pixel of a temporally subsequent frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the opposite direction as to time in order; and   second detecting means configured to detect a predetermined pixel of each of said decimated frames as a tracking point by computation employing information representing the position of the pixel detected with said forward-direction detection, and the position of the pixel detected with said opposite-direction detection.   
   
   
       16 . A recording medium in which the program according to  claim 14  is recorded. 
   
   
       17 . A tracking point detecting device comprising:
 a frame decimation unit configured to perform decimation of the frame interval of a moving image made up of a plurality of frame images which continue temporally;   a first detecting unit configured to detect, of two consecutive frames of said moving image of which the frames were decimated, a temporally subsequent frame pixel corresponding to a predetermined pixel of a temporally previous frame as a tracking point;   a forward-direction detecting unit configured to perform forward-direction detection for detecting the pixel corresponding to a predetermined pixel of a temporally previous frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the same direction as time in order;   an opposite-direction detecting unit configured to perform opposite-direction detection for detecting the pixel corresponding to said detected pixel of a temporally subsequent frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the opposite direction as to time in order; and   a second detecting unit configured to detect a predetermined pixel of each of said decimated frames as a tracking point by computation employing information representing the position of the pixel detected with said forward-direction detection, and the position of the pixel detected with said opposite-direction detection.   
   
   
       18 . A program causing a computer to function as a tracking point detecting device comprising:
 a frame decimation unit configured to perform decimation of the frame interval of a moving image made up of a plurality of frame images which continue temporally;   a first detecting unit configured to detect, of two consecutive frames of said moving image of which the frames were decimated, a temporally subsequent frame pixel corresponding to a predetermined pixel of a temporally previous frame as a tracking point;   a forward-direction detecting unit configured to perform forward-direction detection for detecting the pixel corresponding to a predetermined pixel of a temporally previous frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the same direction as time in order;   an opposite-direction detecting unit configured to perform opposite-direction detection for detecting the pixel corresponding to said detected pixel of a temporally subsequent frame of said moving image of which the frames were decimated, at each frame of said decimated frames in the opposite direction as to time in order; and   a second detecting unit configured to detect a predetermined pixel of each of said decimated frames as a tracking point by computation employing information representing the position of the pixel detected with said forward-direction detection, and the position of the pixel detected with said opposite-direction detection.