Apparatus and method for summarizing moving-picture using events, and computer-readable recording medium storing computer program for controlling the apparatus

Abstract

A moving-picture summarizing apparatus and method using events, and a computer-readable recording medium having embodied thereon a computer program for controlling the apparatus. The moving-picture summarizing apparatus includes: a video summarizing unit combining or segmenting shots considering a video event component detected from a video component of a moving-picture, and outputting the combined or segmented result as a segment; and an audio summarizing unit combining or segmenting the segment on the basis of an audio event component detected from an audio component of the moving-picture, and outputting a summarized result of the moving-picture, wherein the video event is an effect inserted where the content of the moving-picture changes and the audio event is the type of sound by which the audio component is identified to the apparatus can correctly combine or segment shots based on their contents, summarize a moving-picture differentially according to genre, and summarize a moving-picture at a high speed.

1 . A moving-picture summarizing apparatus using events, comprising: 
 a video summarizing unit combining or segmenting shots considering a video event component detected from a video component of a moving-picture, and outputting the combined or segmented result as a segment; and    an audio summarizing unit combining or segmenting the segment on a basis of an audio event component detected from an audio component of the moving-picture, and outputting a summarized result of the moving-picture,    wherein the video event is an effect inserted where the content of the moving-picture changes and the audio event is a type of sound by which the audio component is identifiable.    
   
   
       2 . The moving-picture summarizing apparatus of  claim 1 , wherein the video summarizing unit comprises: 
 a video event detector detecting the video event component from the video component;    a scene transition detector detecting a scene transition portion from the video component and creating time and color information of shots corresponding to a same scene using the detected result;    a video shot combining/segmenting unit measuring a similarity of the shots using the color information of the shots received from the scene transition detector, and combining or segmenting the shots on a basis of the measured similarity and the video event component.    
   
   
       3 . The moving-picture summarizing apparatus of  claim 2 , wherein the video event detector detects a single color frame located in a center of a fade effect from the video component and outputs the detected single color frame as the video event component, and the video event corresponds to the fade effect.  
   
   
       4 . The moving-picture summarizing apparatus of  claim 2 , wherein the video event is a fade effect, a dissolve effect, or a wipe effect.  
   
   
       5 . The moving-picture summarizing apparatus of  claim 2 , wherein the video shot combining/segmenting unit comprises: 
 a buffer storing the color information of the shots received from the scene transition detector;    a similarity calculator reading a first predetermined number of a color information belonging to a search window from the stored color information and calculating color similarity of the shots using the read color information; and    a combining unit comparing the color similarity with a threshold value and combining the compared two shots in response to the compared result.    
   
   
       6 . The moving-picture summarizing apparatus of  claim 5 , wherein the video shot combining/segmenting unit further comprises a segmenting unit which segments the combined result based on the video event component when the combined result has the video event component.  
   
   
       7 . The moving-picture summarizing apparatus of  claim 5 , wherein the similarity calculator calculates the color similarity using the following equation:  
     
       
         
           
             
               
                 Sim 
                 ⁡ 
                 
                   ( 
                   
                     
                       H 
                       1 
                     
                     , 
                     
                       H 
                       2 
                     
                   
                   ) 
                 
               
               = 
               
                 
                   ∑ 
                   
                     n 
                     = 
                     1 
                   
                   N 
                 
                 ⁢ 
                 
                   min 
                   ⁡ 
                   
                     [ 
                     
                       
                         
                           H 
                           1 
                         
                         ⁡ 
                         
                           ( 
                           n 
                           ) 
                         
                       
                       , 
                       
                         
                           H 
                           2 
                         
                         ⁡ 
                         
                           ( 
                           n 
                           ) 
                         
                       
                     
                     ] 
                   
                 
               
             
             , 
             and 
           
         
       
       wherein Sim(H 1, H   2 ) represents the color similarity of the two shots, H 1 (n) and H 2  (n) represent color histograms of the two shots, N is the level of the histogram, and min(x, y) represents a minimum value of x and y.  
     
   
   
       8 . The moving-picture summarizing apparatus of  claim 5 , wherein the first predetermined number represents a size of the search window and is differently set according to Electronic Program Guide (EPG) information.  
   
   
       9 . The moving-picture summarizing apparatus of  claim 2 , wherein the audio summarizing unit comprises: 
 an audio characteristic value generator extracting audio features from an audio component for each frame of the moving picture, and outputting, as audio characteristic values, an average deviation and a standard deviation of the audio features for a second predetermined number of frames;    an audio event detector detecting the audio event component using the audio characteristic values; and    a recombining/resegmenting unit combining or segmenting the segment based on the audio event component and outputting the combined or segmented result as a summarized result of the moving-picture.    
   
   
       10 . The moving-picture summarizing unit of  claim 9 , wherein the audio characteristic value generator comprises: 
 a frame divider dividing the audio component of the moving-picture into frame units each with a predetermined time;    a feature extractor extracting audio features of the divided frame units; and    an average/standard deviation calculating unit calculating an average deviation and a standard deviation of a second predetermined number of audio features for the second predetermined number of frames, extracted by the feature extractor, and outputting the calculated average and standard deviation as the audio characteristic values.    
   
   
       11 . The moving-picture summarizing unit of  claim 9 , wherein the audio feature is a Mel-Frequency Cepstral Coefficient (MFCC), a Spectral Flux, a Centroid, a Rolloff, a ZCR, an Energy, or Pitch information.  
   
   
       12 . The moving-picture summarizing apparatus of  claim 9 , wherein the audio event component is one of music, speech, an environment sound, hand clapping, a shout of joy, a clamor, and silence.  
   
   
       13 . The moving-picture summarizing apparatus of  claim 12 , wherein the audio event component is music, and 
 wherein the recombining/resegmenting unit combines a plurality of neighboring segments in which the music exists.    
   
   
       14 . The moving-picture summarizing apparatus of  claim 12 , wherein the audio component is hand-clapping, the clamor, or silence, 
 wherein the recombining/resegmenting unit divides a single segment in which the hand clapping, the clamor, or silence exists into two by a division event frame, and    wherein the division event frame is a frame existing at the scene transition portion detected by the scene transition detector.    
   
   
       15 . The moving-picture summarizing apparatus of  claim 1 , further comprising: 
 a metadata generator generating metadata of the summarized result of the moving-picture; and    a storage unit storing the generated metadata and the summarized result.    
   
   
       16 . The moving-picture summarizing apparatus of  claim 1 , further comprising: 
 a summarizing buffer buffering the segment received from the video summarizing unit; and    a display unit displaying the buffered result received from the summarizing buffer and the audio component of the moving-picture, wherein the video summarizing unit outputs a previous segment to the summarizing buffer whenever a new segment is generated.    
   
   
       17 . The moving-picture summarizing apparatus of  claim 1 , wherein the video component of the moving-picture includes Electronic Program Guide (EPG) information and a video component contained in a television broadcast signal.  
   
   
       18 . The moving-picture summarizing apparatus of  claim 1 , wherein the audio component of the moving-picture includes Electronic Program Guide (EPG) information and an audio component contained in a television broadcast signal.  
   
   
       19 . A moving-picture summarizing method comprising: 
 combining or segmenting shots considering a video event component detected from a video component of a moving-picture, and outputting the combined or segmented result being a segment; and    combining or segmenting the segment on a basis of an audio event component detected from an audio component of the moving-picture, and obtaining a summarized result of the moving-picture, wherein the video event is an effect inserted where the content of the moving-picture changes and the audio event is a type of sound by which the audio component is identifiable.    
   
   
       20 . The moving-picture summarizing method of  claim 19 , wherein the combining or segmenting shots comprises: 
 detecting the video event component from the video component;    detecting a scene transition portion from the video component and creating time and color information of shots corresponding to a same scene using the detected result; and    measuring a similarity of the shots from the color information of the shots and combining or segmenting the shots using the measured similarity and the video event component.    
   
   
       21 . The moving-picture summarizing method of  claim 20 , wherein the combining or segmenting the segment comprises: 
 extracting audio features for each frame from the audio component, and outputting as audio characteristic values an average deviation and a standard deviation of audio features for a second predetermined number of frames;    detecting the audio event component using the audio characteristic values; and    combining or segmenting the segment based on the audio event component, and outputting the combined or segmented result as a summarized result of the moving-picture.    
   
   
       22 . A computer-readable recording medium having embodied thereon a computer program for controlling a moving-picture summarizing apparatus performing a moving-picture summarizing method using events, the method comprising: 
 combining or segmenting shots considering a video event component detected from a video component of a moving-picture, and outputting the combined or segmented result being a segment; and    combining or segmenting the segment on a basis of an audio event component detected from an audio component of the moving-picture, and obtaining a summarized result of the moving-picture,    wherein the video event is an effect inserted where the content of the moving-picture changes and the audio event is a type of sound by which the audio component is identifiable.    
   
   
       23 . The computer-readable reading medium of  claim 22 , wherein the combining or segmenting shots comprises: 
 detecting the video event component from the video component;    detecting a scene transition portion from the video component and creating time information and color information of shots corresponding to a same scene using the detected result; and    measuring a similarity of the shots from the color information of the shots and combining or segmenting the shots using the measured similarity and the video event component.    
   
   
       24 . The computer-readable recording medium of  claim 23 , wherein the combining or segmenting the segment comprises: 
 extracting audio features for each frame from the audio component, and outputting, as audio characteristic values, an average deviation and a standard deviation of audio features for a second predetermined number of frames;    detecting the audio event component using the audio characteristic values; and    combining or segmenting the segment based on the audio event component, and outputting the combined or segmented result as a summarized result of the moving-picture.