Method for Synchronizing Additional Signal to Primary Signal

Abstract

The present invention relates to a method for synchronizing an additional signal to a primary signal. Synchronization information for a primary signal is generated by extracting at least one signal feature sequence of the primary signal and comparing it to DB feature sequences stored in a database. If the signal feature sequence matches one of the DB feature sequences to a predetermined degree, then synchronization information of the matching DB feature sequences is allocated to the primary signal at a position specified by the signal feature sequence. The synchronization information is transmitted to a playback device, which outputs an additional signal to the primary signal based on the synchronization information.

1 . A method for synchronizing an additional signal to a primary signal including the steps:
 Generation of DB feature sequences for a database from a primary signal while the primary signal is being transmitted from a broadcast station to a playback device,   Generation of synchronization information for the primary signal in a synchronization server by extracting at least one signal feature sequence of the primary signal and comparing it to DB feature sequences stored in the database; if the signal feature sequence matches one of the DB feature sequences to a predetermined degree, then synchronization information of the matching DB feature sequences is allocated to the primary signal at a position specified by the signal feature sequence,   
       wherein an audio track of the main signal is transformed into the frequency domain section by section by an Fast Fourier Transform, wherein the sections transformed into the frequency domain are not longer than 50 milliseconds and the features are frequency peaks which are above a certain threshold value,
 Transmission of the synchronization information to a playback device, which outputs an additional signal synchronously to the primary signal based on the synchronization information. 
 
     
     
         2 . The method according to  claim 1 ,
 wherein   the time of the signal feature sequence in the primary signal, which matches the DB feature sequence, is determined and this time is used as an extraction time in order to generate the synchronization information.   
     
     
         3 . The method according to  claim 1 ,
 wherein   the DB feature sequence is allocated time information, which defines a particular DB time relative to the DB feature sequence that is used to generate the synchronization information.   
     
     
         4 . The method according to  claim 1 ,
 wherein   synchronization information is allocated to the additional signal by extracting a signal feature sequence of the additional signal and comparing it to DB feature sequences stored in a database; if the signal feature sequence matches one of the DB feature sequences to a predetermined degree, then synchronization information is allocated to the additional signal at a position specified by the signal feature sequence or   synchronization information is allocated to the additional signal manually or   an additional signal is used to which synchronization information has already been allocated in advance.   
     
     
         5 . The method according to  claim 1 ,
 wherein   
       the synchronization information comprises one or more of the following pieces of data:
 Time information, which describes a time of the primary and/or additional signal relative to a detected signal feature sequence, 
 One or more identification markers, which describe the content of the primary and/or additional signal, 
 A time stamp, which describes a particular time in the primary and/or additional signal, 
 A time offset, which describes the time interval required to relay a primary and/or additional signal from a particular point of a transmission path to the actual output of the primary and/or additional signal on the playback device. 
 
     
     
         6 . The method according to  claim 1 ,
 wherein   the synchronization information is generated in a synchronization server, which is embodied independently of a playback device.   
     
     
         7 . The method according to  claim 1 ,
 wherein   in the playback device, the additional signal is synchronized to the primary signal
 in that the primary signal and the additional signal are output on the playback device and each contain one or more time markers as synchronization information based on which the playback device synchronously allocates the additional signal to the primary signal or 
 based on a playback time measured in the playback device by means of a clock, to which playback time the additional signal is allocated by means of the synchronization information in such a way that the additional signal is output synchronously to the primary signal. 
   
     
     
         8 . The method according to  claim 1 ,
 wherein   if several signal feature sequences match corresponding DB feature sequences to a predetermined degree, then these signal feature sequences are evaluated according to predetermined rules and the signal feature sequence with the best evaluation is selected for allocating the synchronization information to the primary signal at a position specified by the signal feature sequence.   
     
     
         9 . The method according to  claim 8 ,
 wherein   the predetermined rules include one or more of the following rules:
 The larger the number of matching features of a feature sequence is, the better the evaluation is. 
 The larger the number of matching features of a feature sequence is in relation to the number of features contained in the feature sequence, the better the evaluation is. 
 A classification of the matching of the individual features is performed; the less distance there is between matching features, the better the classification that is assigned to the corresponding matching and this classification is then taken into consideration in the overall evaluation of the matching of a feature sequence. 
   
     
     
         10 . The method according to  claim 1 ,
 wherein   to calibrate a transmission path from a server to a playback device and/or to calibrate the latency in a playback device for outputting a media signal on the playback device, a reference signal is output,   which is simultaneously received by a corresponding sensor; the output reference signal and the received reference signal are compared to each other in order to determine a time interval required for relaying the reference signal and/or for actually outputting it to the playback device, and   this time interval is used as a time offset in order to determine an output time at which a signal is output on the media playback device.   
     
     
         11 . The method according to  claim 10 ,
 wherein   the reference signal includes an audio signal and the sensor is a microphone and/or the reference signal includes a video signal and the sensor is a camera.   
     
     
         12 . The method according to  claim 10 ,
 wherein   the time interval is determined by determining the transmission time and the reception time of the reference signal; the time interval is derived from the time difference between these two times and the measurement of at least one of the two times is carried out by comparing an extracted reference signal feature sequence to one or more previously stored reference signal feature sequences.   
     
     
         13 . The method according to  claim 1 , wherein
 an additional signal is transmitted from a synchronization server, which is embodied to be independent of a playback device, to the playback device and in the synchronization server, synchronization information is generated, which relates to a server time of the synchronization server that is measured in the synchronization server with a clock provided there; in the playback device, a playback device clock is provided for measuring the playback time, which is synchronized with the server time at least once, and   a time drift of the playback time relative to the server time is measured and this time drift is taken into account in the synchronization of the additional signal to the primary signal.   
     
     
         14 . The method according to  claim 13 ,
 wherein   the time drift is determined by multiple transmission of a time signal of the server time to the playback device or multiple transmission of the playback time to the synchronization server and comparison of the transmitted time signal to the locally present time signal in order to calculate a respective time difference; the time drift is determined based on the deviations of the time differences.   
     
     
         15 . The method according to  claim 1 , wherein
 synchronization information is generated, which describes how an additional signal is to be synchronized to a primary signal, wherein   based on the synchronization information, a determination is made as to how much time is available for transmitting the additional signal from a predetermined server to a playback device and based on this time, either one or more encoding methods are selected or adjusted automatically and/or one or more transmission paths are selected.   
     
     
         16 . The method according to  claim 1 ,
 wherein the sections transformed into the frequency domain are not longer than 32 milliseconds and, in particular, no longer than 16 ms.   
     
     
         17 . A method for synchronizing an additional signal to a primary signal, comprising:
 generating DB feature sequences for a database from a primary signal while the primary signal is being transmitted from a broadcast station to a playback device;   generating synchronization information for the primary signal in a synchronization server by extracting at least one signal feature sequence of the primary signal and comparing it to DB feature sequences stored in the database;   allocating synchronization information of the matching DB feature sequences to the primary signal at a position specified by the signal feature sequence, if the signal feature sequence matches one of the DB feature sequences to a predetermined degree;   transforming an audio track of the main signal into the frequency domain section by section by a Fast Fourier Transform, wherein the sections transformed into the frequency domain are not longer than 50 milliseconds and the features are frequency peaks which are above a certain threshold value; and   transmitting the synchronization information to a playback device, which outputs an additional signal synchronously to the primary signal based on the synchronization information.   
     
     
         18 . A system for synchronizing an additional signal to a primary signal, comprising:
 a synchronization server storing DB feature sequences from a primary signal to a database-while transmitting the primary signal and generating synchronization information for the primary signal by extracting at least one signal feature sequence of the primary signal and comparing it to DB feature sequences stored in the database, allocating synchronization information of the matching DB feature sequences to the primary signal at a position specified by the signal feature sequence, if the signal feature sequence matches one of the DB feature sequences to a predetermined degree, transforming an audio track of the main signal into the frequency domain section by section by a Fast Fourier Transform, wherein the sections transformed into the frequency domain are not longer than 50 milliseconds and the features are frequency peaks which are above a certain threshold value, and transmitting the synchronization information; and   a playback device receiving the primary signal and outputs an additional signal synchronously to the primary signal based on the synchronization information.