Real time video summarization

Abstract

System, apparatus, method, and computer readable media for on-the-fly captured video summarization. A video stream is incrementally summarized in concurrence with generation of the stream by a camera module. Saliency of the video stream summary is maintained as the stream evolves by updating the summary to include only the most significant frames. In one exemplary embodiment, saliency is determined by optimizing an objective function including terms that are indicative of both the diversity of a selection, and how representative the selection is to the processed portion of the video data corpus. A device platform including a CM and comporting with the exemplary architecture may provide video camera functionality at ultra-low power, and/or with ultra-low storage resources, and/or with ultra-low communication channel bandwidth.

What is claimed is: 
     
         1 . A captured video data stream summarization method, comprising:
 receiving a stream of consecutively exposed video data frames from a camera hardware module (CM);   iteratively evaluating a stream summary comprising one or more of the video data frames based on each new set of frames received from the CM, wherein the evaluating further comprises:
 scoring each non-incumbent frame in a new set of frames received from the CM with respect to the other frames in the new set, and with respect to each incumbent frame included in the stream summary from a prior iteration; and 
 selecting frames, from a pool including the non-incumbent and the incumbent frames, based on a solution to an objective function that optimizes the frame scoring for the selection; and 
   updating the stream summary stored in a memory in response to selecting a non-incumbent frame in preference over an incumbent frame.