Apparatus, medium, and method segmenting video sequences based on topic

Abstract

Provided are an apparatus, medium, and method segmenting video sequences based on a topic. The apparatus may include a start-shot determination unit detecting a plurality of key-frames by using character information from video sequences including a plurality of frames to determine the detected key-frames as start-shots for each topic, and a topic list creation unit creating a topic list by using the start-shots for each topic.

1 . An apparatus topic based segmenting a video program, the apparatus comprising:
 a start-shot determination unit to detect a plurality of key-frames based on character information from video sequences including a plurality of frames to determine the detected key-frames as start-shots for each topic; and   a topic list creation unit to create a topic list based on the start-shots for each topic.   
     
     
         2 . The apparatus of  claim 1 , wherein the start-shot determination unit detects key-frames based on clothing information of at least one main character. 
     
     
         3 . The apparatus of  claim 1 , wherein the topic list creation unit organizes frames existing between a current topic start-shot and a next topic start-shot into a current topic episode, and adds the current topic episode to the start-shot of each topic in the topic list. 
     
     
         4 . The apparatus of  claim 1 , wherein the start-shot determination unit comprises:
 a pre-processing unit to determine frames belonging to a respective scene by detecting scene change among frames included in the video sequences and to obtain a number of main characters appearing in the video sequences;   a face detection unit to detect faces from the determined frames belonging to the respective scene to determine face detection frames; and   a key-frame determination unit to cluster the determined face detection frames according to the main characters corresponding to the number of main characters to determine the key-frames.   
     
     
         5 . The apparatus of  claim 4 , wherein the pre-processing unit detects the scene change by calculating similarity between a current frame and a previous frame. 
     
     
         6 . The apparatus of  claim 4 , wherein the pre-processing unit obtains the number of main characters from an electronic program guide (EPG) signal. 
     
     
         7 . The apparatus of  claim 4 , wherein the pre-processing unit comprises:
 a thumbnail image creation unit to create thumbnail images for input frames;   a scene change detection unit to detect the scene change using similarity of color histograms between thumbnail images of neighboring frames; and   a number-of-main-characters determination unit to determine the number of main characters by analyzing an EPG signal.   
     
     
         8 . The apparatus of  claim 4 , wherein the face detection unit comprises:
 a thumbnail image re-organization unit to create an integral image for thumbnail images of input frames and to re-organize the thumbnail images using the integral image;   a sub-window generation unit to generate a sub-window for the re-organized thumbnail images; and   a classifying unit to determining whether the sub-window includes a face.   
     
     
         9 . The apparatus of  claim 8 , wherein the face detection unit divides the thumbnail images of the input frames into a plurality of sections comprising a section having a highest probability of detecting the face, and sequentially provides the plurality of sections to the thumbnail image re-organization unit in descending order from the section having the highest probability of detecting the face to a section having a lowest probability of detecting the face. 
     
     
         10 . The apparatus of  claim 4 , wherein the key-frame determination unit comprises:
 a clothing information extraction unit to extract clothing information from a face detection frame;   a character clustering unit to perform a character clustering method based on the extracted clothing information; and   a main character determination unit to select a cluster corresponding to the main character from a plurality of clusters, clustered in the character clustering unit, corresponding to the number of main characters and to provide frames included in the selected cluster as key-frames of each topic.   
     
     
         11 . The apparatus of  claim 10 , wherein the clothing information comprises a clothing color histogram. 
     
     
         12 . A method of topic based segmenting of video sequences, the method comprising:
 detecting a plurality of key-frames based on character information from video sequences including a plurality of frames to determine the detected key-frames as start-shots for each topic; and   creating a topic list based on the start-shots for each topic.   
     
     
         13 . The method of  claim 12 , wherein the determination of the start-shots includes detecting key-frames based on clothing information of at least one main character. 
     
     
         14 . The method of  claim 12 , wherein the creation of the topic list organizes frames existing between a current topic start-shot and a next topic start-shot into a current topic episode, and adds the current topic episode to the start-shot of each topic in the topic list. 
     
     
         15 . The method of  claim 12 , wherein the determination of the start-shots comprises:
 detecting a scene change from the frames included in the video sequences to determine frames belonging to a respective scene and obtaining a number of main characters appearing in the video sequences;   detecting faces from the determined frames belonging to the respective scene to determine face detection frames; and   clustering the determined face detection frames according to the main characters corresponding to the number of main characters to determine the face detection frames as key-frames.   
     
     
         16 . The method of  claim 15 , wherein the scene change is detected by creating thumbnail images of input frames and using similarity of color histograms between thumbnail images of neighboring frames. 
     
     
         17 . The method of  claim 15 , wherein the number of main characters is obtained by analyzing an electronic program guide (EPG) signal. 
     
     
         18 . The method of  claim 15 , wherein the detection of the faces comprises:
 creating an integral image for thumbnail images of input frames and re-organizing the thumbnail images using the integral image;   generating a sub-window for the re-organized thumbnail images; and   determining whether the sub-window comprises a face.   
     
     
         19 . The method of  claim 18 , wherein the detection of the faces further comprises dividing the thumbnail images of the input frames into a plurality of sections comprising a section having a highest probability of detecting a face, and sequentially providing the thumbnail images for the thumbnail image re-organizing in descending order from the section having the highest probability of detecting the face to a section having a lowest probability of detecting the face. 
     
     
         20 . The method of  claim 15 , wherein the determination of the key-frames comprises:
 extracting clothing information from the face detection frames;   performing a character clustering method based on the extracted clothing information; and   selecting a cluster corresponding to the main character from a plurality of clusters corresponding to the number of main characters and providing frames included in the selected cluster as the key-frames of each topic.   
     
     
         21 . A medium comprising computer readable code to implement a method of topic based segmenting of video sequences, the method comprising:
 detecting a plurality of key-frames based on character information from video sequences including a plurality of frames to determine the detected key-frames as start-shots for each topic; and   creating a topic list based on the start-shots for each topic.