Video synchronization based on audio

Abstract

A content reproduction screen displays a plurality of content items. A video reproduction processing section reproduces substantially simultaneously videos of a plurality of content items on a screen. An audio reproduction processing section reproduces substantially simultaneously audio signals of a plurality of content items. During reproduction of a plurality of content items, a comparison is made between a current reproduction position of the audio signal in an audio reproduction processing section and a current reproduction position of each video in the video reproduction processing section, and, if a discrepancy between the compared current reproduction positions is equal to or greater than a threshold value, the current reproduction position of the video is corrected on the basis of the current reproduction position of the audio signal. In this way, the current reproduction position of each video is synchronized with the reproduction position of the audio signal.

What is claimed is: 
     
         1 . A computer-implemented method for reproducing a content item comprising a combination of audio data and video data, said method comprising:
 an audio reproduction step of reproducing an audio signal based on the audio data for reproduction of the content item;   a video reproduction step of reproducing a video based on the video data for reproduction of the content item;   a detection step of detecting a discrepancy between a reproduction position of the audio signal and a reproduction position of the video during the reproduction of the content item; and   a correction step of, when the discrepancy detected by said detection step is greater than a threshold value, correcting the reproduction position of the video on a basis of the reproduction position of the audio signal.   
     
     
         2 . The method as claimed in  claim 1 , wherein said audio reproduction step mixes a plurality of audio signals based on audio data of a plurality of content items into one audio signal and reproduces the mixed audio signal,
 said video reproduction step reproduces a plurality of videos in a parallel manner based on video data of the plurality of content item, and   said detection step detects a discrepancy between a current reproduction position of the mixed audio signal and a reproduction position of each of the plurality of videos.   
     
     
         3 . The method as claimed in  claim 1 , wherein said correction step calculates a correction amount for causing the reproduction position of the video to match the reproduction position of the audio signal and correcting the reproduction position of the video on a basis of the calculated correction amount and the reproduction position of the audio signal. 
     
     
         4 . The method as claimed in  claim 1 , which further comprises a step of updating a number-of-correction parameter each time the discrepancy is detected, and
 wherein said step of correcting the reproduction position of the video calculates the correction amount on a basis of the number-of-correction parameter.   
     
     
         5 . A content reproduction apparatus for reproducing a content item comprising a combination of audio data and video data, said content reproduction apparatus comprising:
 a processing unit configured to:
 reproduce an audio signal based on the audio data for reproduction of the content item; 
 reproduce a video based on the video data for reproduction of the content item; 
 detect a discrepancy between a reproduction position of the audio signal and a reproduction position of the video during the reproduction of the content item; and 
 when the detected discrepancy is greater than a threshold value, correct the reproduction position of the video on a basis of the reproduction position of the audio signal. 
   
     
     
         6 . A non-transitory computer-readable storage medium containing instructions executable by a computer to perform a method for reproducing a content item comprising a combination of audio data and video data, said method comprising: an audio reproduction step of reproducing an audio signal based on the audio data for reproduction of the content item;
 a video reproduction step of reproducing a video based on the video data for reproduction of the content item;   a detection step of detecting a discrepancy between a reproduction position of the audio signal and a reproduction position of the video during the reproduction of the content item; and   a correction step of, when the discrepancy detected by said detection step is greater than a threshold value, correcting the reproduction position of the video on a basis of the reproduction position of the audio signal.   
     
     
         7 . A computer-implemented method for reproducing substantially simultaneously a plurality of content items each comprising audio data and video data, said method comprising:
 an audio reproduction step of reproducing substantially simultaneously a plurality of audio signals on a basis of respective audio data of the plurality of content items;   a video reproduction step of reproducing substantially simultaneously a plurality of videos on a basis of respective video data of the plurality of content items;   an acquisition step of, when a content item has been newly designated as an object of reproduction during reproduction of the plurality of content items, acquiring a current reproduction position of the audio signals;   a first determination step of determining a reproduction position of a video of the newly-designated content item on a basis of the current reproduction position of the audio signals acquired by said acquisition step; and   a step of starting reproduction of the video of the newly-designated content item at the reproduction position of the video of the newly-designated content item determined by said first determination step.   
     
     
         8 . The method as claimed in  claim 7 , which further comprises:
 a second determination step of, when a content item has been newly designated as an object of reproduction during reproduction of the plurality of content items, determining a reproduction position of an audio signal of the newly-designated content item on a basis of the current reproduction position of the audio signal; and   a step of starting reproduction of the audio signal of the newly-designated content item at the reproduction position of the audio signal of the newly-designated content item determined by said second determination step.   
     
     
         9 . The method as claimed in  claim 7 , wherein, in said audio reproduction step, reproduction of the plurality of audio signals is controlled by one audio player, and
 wherein said acquisition step acquires the current reproduction position of the audio signals from the audio player.   
     
     
         10 . The method as claimed in  claim 7 , wherein said acquisition step acquires the current reproduction position of the audio signals from a clock that provides time information functioning as a basis for controlling reproduction of the content items. 
     
     
         11 . The method as claimed in  claim 7 , wherein the content item has start position data indicative of a start position of the content item, and
 wherein said video reproduction position determination step determines the reproduction position of the video on a basis of the acquired current reproduction position of the audio signal and the start position data of the content item.   
     
     
         12 . A content reproduction apparatus for reproducing substantially simultaneously a plurality of content items each comprising audio data and video data, said content reproduction apparatus comprising:
 a processing unit configured to:
 reproduce substantially simultaneously a plurality of audio signals on a basis of respective audio data of the plurality of content items; 
 reproduce substantially simultaneously a plurality of videos on a basis of respective video data of the plurality of content items; 
 when a content item has been newly designated as an object of reproduction during reproduction of the plurality of content items, acquire a current reproduction position of the audio signals; 
 determine a reproduction position of a video of the newly-designated content item on a basis of the acquired current reproduction position of the audio signals; and 
 control to start reproduction of the video of the newly-designated content item at the determined reproduction position of the video of the newly-designated content item. 
   
     
     
         13 . A non-transitory computer-readable storage medium containing instructions executable by a computer to perform a method for reproducing substantially simultaneously a plurality of content items each comprising audio data and video data, said method comprising:
 an audio reproduction step of reproducing substantially simultaneously a plurality of audio signals on a basis of respective audio data of the plurality of content items;   a video reproduction step of reproducing substantially simultaneously a plurality of videos on a basis of respective video data of the plurality of content items;   an acquisition step of, when a content item has been newly designated as an object of reproduction during reproduction of the plurality of content items, acquiring a current reproduction position of the audio signals;   a determination step of determining a reproduction position of a video of the newly-designated content item on a basis of the current reproduction position of the audio signals acquired by said acquisition step; and   a step of starting reproduction of the video of the newly-designated content item at the reproduction position of the video of the newly-designated content item determined by said determination step.   
     
     
         14 . A computer-implemented method for creating a musical work based on combination of a plurality of performance content items, the performance content items each comprising video data and audio data, said method comprising:
 a step of displaying a music creation screen having a plurality of blocks arranged in a matrix configuration for displaying the plurality of performance content items constituting the musical work, each of the blocks being capable of reproducing a video based on the video data of any one of the plurality of performance content items;   a change step of, in response to a change instruction given by a user, changing the performance content item, placed in a user-selected one of the blocks, over to another performance content item selected by the user; and   a step of, in response to a reproduction instruction given by the user, reproducing videos based on the video data of the performance content items placed in one or more of the blocks selected by the user and reproducing performance sound based on the audio data of the performance content items placed in the selected one or more blocks.   
     
     
         15 . The method as claimed in  claim 14 , wherein the plurality of blocks arranged in a matrix configuration are constructed in such a manner that a time axis constituting the musical work is represented on one axis while types of performance sound constituting the musical work are represented on other axis. 
     
     
         16 . The method as claimed in  claim 14 , wherein said change step further comprises a step of displaying candidate-for-section information, indicative of one or more performance content items placeable in the block selected by the user, on a different region than the music creation screen, and
 one performance content item selected by the user from the candidate-for-section information displayed on the different region is placed in the selected block.   
     
     
         17 . The method as claimed in  claim 14 , which further comprises a step of storing a plurality of performance content items currently placed in the plurality of blocks as a musical work having a construction defined by positions of the blocks in which the plurality of performance content items are placed. 
     
     
         18 . The method as claimed in  claim 17 , wherein the video data and the audio data of each of the performance content items are stored separately from each other. 
     
     
         19 . The method as claimed in  claim 14 , wherein the plurality of blocks arranged in a matrix configuration includes a block that displays in realtime a performance content item representative of a performance input by the user. 
     
     
         20 . A music creation apparatus for creating a musical work based on combination of a plurality of performance content items, the performance content items each comprising video data and audio data, said music creation apparatus comprising:
 a processing unit configured to:
 display a music creation screen having a plurality of blocks arranged in a matrix configuration for displaying the plurality of performance content items constituting the musical work, each of the blocks being capable of reproducing a video based on the video data of any one of the plurality of performance content items; 
   in response to a change instruction given by a user, change the performance content item, placed in a user-selected one of the blocks, over to another performance content item selected by the user; and   in response to a reproduction instruction given by the user, reproduce videos based on the video data of the performance content items placed in one or more of the blocks selected by the user and reproduce performance sound based on the audio data of the performance content items placed in the selected one or more blocks.   
     
     
         21 . A non-transitory computer-readable storage medium containing instructions executable by a computer to perform a method for creating a musical work based on combination of a plurality of performance content items, the performance content items each comprising video data and audio data, said method comprising:
 a step of displaying a music creation screen having a plurality of blocks arranged in a matrix configuration for displaying the plurality of performance content items constituting the musical work, each of the blocks being capable of reproducing a video based on the video data of any one of the plurality of performance content items;   a change step of, in response to a change instruction given by a user, changing the performance content item, placed in a user-selected one of the blocks, over to another performance content item selected by the user; and   a step of, in response to a reproduction instruction given by the user, reproducing videos based on the video data of the performance content items placed in one or more of the blocks selected by the user and reproducing performance sound based on the audio data of the performance content items placed in the selected one or more blocks.   
     
     
         22 . A music creation system for creating a musical work based on combination of a plurality of performance content items, the performance content items each comprising video data and audio data, said music creation system comprising a server and a client terminal interconnected via a network,
 said server including a database storing the plurality of performance content items,   said client terminal comprising:
 a processing unit configured to:
 acquire, from the database of said server, a plurality of performance content items constituting the musical work; 
 display a music creation screen having a plurality of blocks arranged in a matrix configuration for displaying the plurality of performance content acquired from the database of the server, each of the blocks being capable of reproducing a video based on the video data of any one of the plurality of performance content items; 
 in response to a change instruction given by a user, acquire a user-selected one of the performance content items from the database of the server and change the performance content item placed in a selected one of the blocks over to the acquired performance content item; and 
 in response to a reproduction instruction given by the user, reproduce videos based on the video data of the performance content items placed in one or more of the blocks selected by the user and reproduce performance sound based on the audio data of the performance content items placed in the selected one or more blocks.