AV signal processing apparatus for detecting a boundary between scenes, method, recording medium and computer program therefor

Abstract

The invention provides an AV signal processing apparatus and method by which a boundary between scenes is detected so that recorded video data can be played back beginning with an arbitrary scene. First, video data inputted is divided into video segments or audio segments or, if possible, into both of video and audio segments. Then, feature amounts representative of features of the segment are calculated, and then, similarity measurement between segments is performed using the feature amounts. Thereafter, it is discriminated whether or not the segment corresponds to a break of a scene. Thus, the video-audio processing apparatus uses the dissimilarity measurement criterion and the feature amounts calculated as above to determine, regarding each segment as the reference segment at present, in which one of the past and the future with respect to the reference segment the ratio of presence of neighboring similar segments is higher, and investigates the pattern of the change of the ratio to discriminate whether or not the reference segment corresponds to a boundary of a scene.

1 - 12 . (canceled)  
     
     
         13 . An AV signal processing apparatus for detecting a boundary between scenes, comprising: 
 feature amount extraction means for extracting feature amounts of segments each formed from a series of frames which form an AV signal;    similarity measurement means for measuring a similarity between a segment and other segments in a predetermined time domain between a past time and a future time using said feature amounts;    similar segments detection means for detecting similar segments according to said similarity for each segment in said predetermined time domain;    similar segments counting means for counting said similar segments in said past and said future in said predetermined time domain for each segment;    boundary likelihood measurement calculation means for calculating boundary likelihood measurement value according to a counted amount of said similar segments in said predetermined time domain for each segment;    pattern detection means for detecting a pattern of existence of said boundary likelihood measurement values in said predetermined time domain; and    boundary discrimination means for discriminating a boundary of a scene according to said pattern.    
     
     
         14 . The AV signal processing apparatus according to  claim 13 , wherein said AV signal includes at least one of a video signal and an audio signal.  
     
     
         15 . The AV signal processing apparatus according to  claim 14 , further comprising audio segment production means or detecting, when the AV signal includes a video signal, a shot which is a basic unit of a video segment to produce an audio segment.  
     
     
         16 . The AV signal processing apparatus according to  claim 14 , further comprising audio segment production means for using, when the AV signal includes an audio signal, at least one of the feature amounts of the audio signal and a no sound period to produce an audio segment.  
     
     
         17 . The AV signal processing apparatus according to  claim 14 , wherein the feature amounts of the video signal at least include a color histogram.  
     
     
         18 . The AV signal processing apparatus according to  claim 14 , wherein the feature amounts of the video signal at least include at least one of a sound volume and a spectrum.  
     
     
         19 . The AV signal processing apparatus according to  claim 13 , wherein said boundary discrimination means compares the measurement value with a preset threshold value to discriminate whether or not a reference segment is a boundary of the scene.  
     
     
         20 . An AV signal processing apparatus for detecting a boundary between scenes, comprising: 
 an extractor operable to extract a feature amount of segments each formed from a series of frames which form an AV signal;    a similarity measurer operable to measure a similarity between a segment and other segments in a predetermined time domain between a past time and a future time using said feature amounts;    a detector operable to detect similar segments according to said similarity for each segment in said predetermined time domain;    a counter operable to count similar segments in said past and said future in said predetermined time domain for each segment;    a boundary likelihood measurer operable to calculate a boundary likelihood measurement value according to a counted amount of said similar segments in said predetermined time domain for each segment;    a detector operable to detect a pattern of existence of said boundary likelihood measurement values in said predetermined time domain; and    a discriminator operable to discriminate a boundary of a scene according to said pattern.    
     
     
         21 . The AV signal processing apparatus according to  claim 20 , wherein said AV signal includes at least one of a video signal and an audio signal.  
     
     
         22 . The AV signal processing apparatus according to  claim 21 , further comprising a detector operable to detect, when the AV signal includes a video signal, a shot which is a basic unit of a video segment to produce an audio segment.  
     
     
         23 . The AV signal processing apparatus according to  claim 21 , further comprising a detector operable to determine, when the AV signal includes an audio signal, at least one of the feature amounts of the audio signal and a no sound period to produce an audio segment.  
     
     
         24 . The AV signal processing apparatus according to  claim 21 , wherein the feature amounts of the video signal at least include a color histogram.  
     
     
         25 . The AV signal processing apparatus according to  claim 21 , wherein the feature amounts of the video signal at least include at least one of a sound volume and a spectrum.  
     
     
         26 . The AV signal processing apparatus according to  claim 20 , wherein said discriminator compares the measurement value with a preset threshold value to discriminate whether or not a reference segment is a boundary of the scene.  
     
     
         27 . A method of detecting a boundary between scenes in an AV signal comprising the steps of: 
 extracting feature amounts of segments each formed from a series of frames which form an AV signal;    measuring a similarity between a segment and other segments in a predetermined time domain between a past time and a future time using said feature amounts;    detecting similar segments according to said similarity for each segment in said predetermined time domain;    counting said similar segments in said past and said future in said predetermined time domain for each segment;    calculating boundary likelihood measurement value according to a counted amount of said similar segments in said predetermined time domain for each segment;    detecting a pattern of existence of said boundary likelihood measurement values in said predetermined time domain; and    discriminating a boundary of a scene according to said pattern.    
     
     
         28 . The method of  claim 27 , wherein said AV signal includes at least one of a video signal and an audio signal.  
     
     
         29 . The method of  claim 28 , further comprising a step of: 
 detecting, when the AV signal includes a video signal, a shot which is a basic unit of a video segment, and    producing an audio segment.    
     
     
         30 . The method of  claim 28 , further comprising a step of: 
 producing an audio segment, when the AV signal includes an audio signal, by using at least one of the feature amounts of the audio signal and a no sound period.    
     
     
         31 . The method of  claim 28 , wherein the feature amounts of the video signal include at least one color histogram.  
     
     
         32 . The method of  claim 28 , wherein the feature amounts of the video signal include at least one of a sound volume and a spectrum.  
     
     
         33 . The method of  claim 27 , wherein said step of discriminating comprises: 
 comparing the measurement value with a preset threshold value; and    discriminating whether or not a reference segment is a boundary of the scene.    
     
     
         34 . A recording medium having recorded thereon a program for detecting a boundary between scenes in an AV signal said program describing steps of: 
 extracting feature amounts of segments each formed from a series of frames which form an AV signal;    measuring a similarity between a segment and other segments in a predetermined time domain between a past time and a future time using said feature amounts;    detecting similar segments according to said similarity for each segment in said predetermined time domain;    counting said similar segments in said past and said future in said predetermined time domain for each segment;    calculating boundary likelihood measurement value according to a counted amount of said similar segments in said predetermined time domain for each segment;    detecting a pattern of existence of said boundary likelihood measurement values in said predetermined time domain; and    discriminating a boundary of a scene according to said pattern.    
     
     
         35 . The recording medium of  claim 34 , wherein said AV signal includes at least one of a video signal and an audio signal.  
     
     
         36 . The recording medium of  claim 35 , wherein said program further describes the steps of: 
 detecting, when the AV signal includes a video signal, a shot which is a basic unit of a video segment, and    producing an audio segment.    
     
     
         37 . The recording medium of  claim 35 , wherein said program further describes the steps of: 
 producing an audio segment, when the AV signal includes an audio signal, by using at least one of the feature amounts of the audio signal and a no sound period.    
     
     
         38 . The recording medium of clam  35 , wherein the feature amounts of the video signal include at least one color histogram.  
     
     
         39 . The recording medium of  claim 35 , wherein the feature amounts of the video signal include at least one of a sound volume and a spectrum.  
     
     
         40 . The recording medium of  claim 34 , wherein said step of discriminating comprises: 
 comparing the measurement value with a preset threshold value; and    discriminating whether or not a reference segment is a boundary of the scene.    
     
     
         41 . A computer program embodied on a computer readable medium, for detecting a boundary between scenes in an AV signal said program describing steps of: 
 extracting feature amounts of segments each formed from a series of frames which form an AV signal;    measuring a similarity between a segment and other segments in a predetermined time domain between a past time and a future time using said feature amounts;    detecting similar segments according to said similarity for each segment in said predetermined time domain;    counting said similar segments in said past and said future in said predetermined time domain for each segment;    calculating boundary likelihood measurement value according to a counted amount of said similar segments in said predetermined time domain for each segment;    detecting a pattern of existence of said boundary likelihood measurement values in said predetermined time domain; and    discriminating a boundary of a scene according to said pattern.    
     
     
         42 . The computer program of  claim 41 , wherein said AV signal includes at least one of a video signal and an audio signal.  
     
     
         43 . The computer program of  claim 42 , further describing the steps of: 
 detecting, when the AV signal includes a video signal, a shot which is a basic unit of a video segment, and    producing an audio segment.    
     
     
         44 . The computer program of  claim 42 , further describing the steps of: 
 producing an audio segment, when the AV signal includes an audio signal, by using at least one of the feature amounts of the audio signal and a no sound period.    
     
     
         45 . The computer program of  claim 42 , wherein the feature amounts of the video signal include at least one color histogram.  
     
     
         46 . The computer program of  claim 42 , wherein the feature amounts of the video signal include at least one of a sound volume and a spectrum.  
     
     
         47 . The computer program of  claim 41 , wherein said step of discriminating comprises: 
 comparing the measurement value with a preset threshold value; and    discriminating whether or not a reference segment is a boundary of the scene.