System and method for tracking content timeline in the presence of playback rate changes

Abstract

A system and method for controlling a media player for replacement content, such as dynamic ad insertion. The system tracks video watermarks from a content stream, where the input content timeline is being modified by a user exercising the transport controls of a digital video recorder (DVR). A Detector Engine receives decoded video frames and extracts a time-offset field, a VP1 payload, and a Cyclic Redundance Check (CRC) field in each video frame. A Content Timeline Tracker monitors and analyzes the output of the Detector Engine to produce a piecewise linear approximation of the content timeline, wherein playback rate changes by a user in an upstream device can be tracked. This enables the playback of auxiliary content which is synchronized to a watermark timeline recovered from the received content in cases where the recovered timeline has a non-linear mapping to real time. When the estimated speed is changing due to user-controlled trick play of recorded content, estimated speed deviates from the user intended speed profile because of imperfect playback of the media player. The system includes additional filtering of estimated speed to produce a Boolean updated speed which is asserted sparsely at estimated control segment endpoints in an attempt to delineate constant slope (constant speed) control segments.

1 . A method comprising:
 receiving video content having a video watermark embedded therein;   decoding video frames from the received video content;   using a detector engine to receive the decoded video frames and extract at least a time-offset, a payload, and an error detection field in a plurality of the decoded video frames, wherein the detector engine generates an output for the plurality of decoded video frames that includes a time-offset and a payload derived from the video watermark; and   using a content timeline tracker to monitor and analyze the output of the detector engine, to produce a piecewise linear approximation of the content timeline, wherein the playback rate changes by a user in an upstream device can be tracked, thereby enabling the playback of auxiliary content which is synchronized to a watermark timeline recovered from the received content when the recovered timeline has a non-linear mapping to real time.   
     
     
         2 . The method of  claim 1  further comprising using matching logic to compare the watermark data extracted from the current frame with at least some watermark data extracted from the previous frame and to set a data repetition flag to TRUE if they match and otherwise set it to FALSE. 
     
     
         3 . The method of  claim 2  wherein the matching logic determines if consecutive watermark data are repeated, wherein this information can be later used to discriminate between actual payload repetition, such as time offset repetition in high frame-rate video or fragment repetition, or frame repetition in pause-and-seek playback rate change, skip and pause. 
     
     
         4 . The method of  claim 1  wherein the content timeline tracker analyzes at least the time_offset and data repetition flag values to produce estSpeed, a piecewise linear approximation of the content timeline which can track playback rate changes initiated by a user on an upstream device. 
     
     
         5 . The method of  claim 1  wherein the Content Timeline Tracker implements a state machine to recognize patterns in the recovered timeline and estimate the control segment boundaries. 
     
     
         6 . The method of  claim 1  wherein the piecewise linear approximation of the content timeline includes an estimate of the playback speed and further comprising using the estimate of the playback speed in controlling a replacement media player, whereby the number of seek commands required to track main content in minimized. 
     
     
         7 . The method of  claim 1  wherein the content timeline tracker recognizes one or more patterns selected from the group consisting of: initial tracker state; pause; pause seek; oneXOnset; oneXplay; JumpOnset; and Jumping. 
     
     
         8 . A device comprising:
 a detector engine receiving decoded video having a video watermark embedded therein, the detector engine extracting at least a time-offset, a payload, and an error detection field in a plurality of the decoded video frames, wherein the detector engine generates an output for the plurality of decoded video frames that includes a time-offset and a payload derived from the video watermark; and   a content timeline tracker that monitors and analyzes the output of the detector engine and produces a piecewise linear approximation of the content timeline, wherein the playback rate changes by a user in an upstream device can be tracked, thereby enabling the playback of auxiliary content which is synchronized to a watermark timeline recovered from the received content when the recovered timeline has a non-linear mapping to real time.   
     
     
         9 . The device of  claim 8  wherein the detector engine uses matching logic to compare the watermark data extracted from the current frame with at least some watermark data extracted from the previous frame and to set a data repetition flag to TRUE if they match and otherwise set it to FALSE. 
     
     
         10 . The device of  claim 9  wherein the matching logic determines if consecutive watermark data are repeated, wherein this information can be later used to discriminate between actual payload repetition, such as time_offset repetition in high frame-rate video or fragment repetition, or frame repetition in pause-and-seek playback rate change, skip and pause. 
     
     
         11 . The method of  claim 8  wherein the content timeline tracker analyzes at least the time_offset and data repetition flag values to produce estSpeed, a piecewise linear approximation of the content timeline which can track playback rate changes initiated by a user on an upstream device. 
     
     
         12 . The device of  claim 8  wherein the content timeline tracker implements a state machine to recognize patterns in the recovered timeline and estimate the control segment boundaries. 
     
     
         13 . The device of  claim 8  wherein the piecewise linear approximation of the content timeline includes an estimate of the playback speed and wherein the content timeline tracker uses the estimate of the playback speed to control a replacement media player, whereby the number of seek commands required to track main content in minimized. 
     
     
         14 . The device of  claim 8  wherein the content timeline tracker recognizes one or more patterns selected from the group consisting of: initial tracker state; pause; pause seek; oneXOnset; oneXplay; JumpOnset; and Jumping. 
     
     
         15 . A computer program product embodied on one or more non-transitory computer readable media, comprising:
 program code for receiving video content having a video watermark embedded therein;   program code for decoding video frames from the received video content;   program code for using a Detector Engine to receive the decoded video frames and extract at least a time-offset, a payload, and an error detection field in a plurality of the decoded video frames, wherein the Detector Engine generates an output for the plurality of decoded video frames that includes a time-offset and a payload derived from the video watermark; and   program code for using a Content Timeline Tracker to monitor and analyze the output of the Detector Engine, to produce a piecewise linear approximation of the content timeline, wherein the playback rate changes by a user in an upstream device can be tracked, thereby enabling the playback of auxiliary content which is synchronized to a watermark timeline recovered from the received content when the recovered timeline has a non-linear mapping to real time.   
     
     
         16 . The computer program product of  claim 15  further comprising using matching logic to compare the watermark data extracted from the current frame with at least some watermark data extracted from the previous frame and to set a data repetition flag to TRUE if they match and otherwise set it to FALSE. 
     
     
         17 . The computer program product of  claim 16  wherein the matching logic determines if consecutive watermark data are repeated, wherein this information can be later used to discriminate between actual payload repetition, such as time_offset repetition in high frame-rate video or fragment repetition, or frame repetition in pause-and-seek playback rate change, skip and pause. 
     
     
         18 . The computer program product of  claim 15  wherein the Content Timeline Tracker analyzes at least the time_offset and data repetition flag values to produce estSpeed, a piecewise linear approximation of the content timeline which can track playback rate changes initiated by a user on an upstream device. 
     
     
         19 . The computer program product of  claim 15  wherein the Content Timeline Tracker implements a state machine to recognize patterns in the recovered timeline and estimate the control segment boundaries. 
     
     
         20 . The computer program product of  claim 15  wherein the piecewise linear approximation of the content timeline includes an estimate of the playback speed and further comprising using the estimate of the playback speed in controlling a replacement media player, whereby the number of seek commands required to track main content in minimized.