3D stereoscopic/multiview video processing system and its method

Abstract

Disclosed is a stereoscopic/multiview three-dimensional video processing system and its method. In the present invention, stereoscopic/multiview three-dimensional video data having a plurality of images at the same time are coded into a plurality of elementary streams. The plural elementary streams output at the same time are multiplexed according to the user's selected display mode to generate a single elementary stream. After packetization of the single elementary stream continuously generated, information about the stereoscopic/multiview three-dimensional video multiplexing method and the selected display mode information are added to the packet header of the stream. Then the packetized elementary stream is sent to the image reproducer or stored in storage media. The present invention multiplexes the multi-channel elementary streams having the same temporal and spatial information, thereby minimizing the overlapping header information, and performs streaming of data suitable for the user's demand and the user system environments.

What is claimed is:  
     
         1 . A stereoscopic/multiview three-dimensional video processing system, which is based on MPEG-4, the system comprising: 
 a compressor for processing input stereoscopic/multiview three-dimensional video data to generate field-based elementary streams of multiple channels, and outputting the multi-channel elementary streams into a single integrated elementary stream;    a packetizer for receiving the elementary streams from the compressor per access unit and packetizing the received elementary streams; and    a transmitter for processing the packetized stereoscopic/multiview three-dimensional video data and transferring or storing the processed video data.    
     
     
         2 . The system as claimed in  claim 1 , wherein the compressor comprises: 
 a three-dimensional object encoder for encoding the input stereoscopic/multiview three-dimensional video data to output multi-channel field-based elementary streams; and    a three-dimensional elementary stream mixer for integrating the multi-channel field-based elementary streams into a single elementary stream, and outputting the same.    
     
     
         3 . The system as claimed in  claim 2 , wherein the three-dimensional object encoder outputs elementary streams in the unit of 4-channel fields including odd and even fields of a left three-dimensional stereoscopic image and odd and even fields of a right three-dimensional stereoscopic image, when the input data are three-dimensional stereoscopic video data.  
     
     
         4 . The system as claimed in  claim 2 , wherein the three-dimensional object encoder outputs N×2 field-based elementary streams to the three-dimensional elementary stream mixer, when the input data are N-view's multiview video data.  
     
     
         5 . The system as claimed in  claim 2 , wherein the compressor comprises: 
 an object descriptor stream generator for generating an object descriptor stream for representing the attributes of multiple multimedia objects;    a scene description stream generator for generating a scene description stream for representing the temporal and spatial correlations among objects; and    a two-dimensional encoder for encoding 2-dimensional multimedia data.    
     
     
         6 . The system as claimed in  claim 2 , wherein the three-dimensional elementary stream mixer generates a single elementary stream by selectively using a plurality of elementary streams input through multiple channels according to a display mode for stereoscopic/multiview three-dimensional video selected by a user.  
     
     
         7 . The system as claimed in  claim 6 , wherein the display mode is any one mode selected from a two-dimensional video display mode, a three-dimensional video field shuttering display mode for displaying three-dimensional video images by field-based shuttering, a three-dimensional stereoscopic video frame shuttering display mode for displaying three-dimensional video images by frame-based shuttering, and a multiview three-dimensional video display mode for sequentially displaying images at a required frame rate.  
     
     
         8 . The system as claimed in  claim 6 , wherein the three-dimensional elementary stream mixer multiplexes 4-channel field-based elementary streams of stereoscopic three-dimensional video output from the three-dimensional object encoder into a single-channel access unit stream using 2-channel elementary streams in the order of the odd field elementary stream of a left image and the even field elementary stream of a right image, when the display mode is the three-dimensional video field shuttering display mode.  
     
     
         9 . The system as claimed in  claim 6 , wherein the three-dimensional elementary stream mixer multiplexes 4-channel field-based elementary streams of stereoscopic three-dimensional video output from the three-dimensional object encoder into a single-channel access unit stream using 4-channel elementary streams in the order of the odd field elementary stream of a left image, the even field elementary stream of the left image, the odd field elementary stream of a right image, and the even field elementary stream of the right image, when the display mode is the three-dimensional video frame shuttering display mode.  
     
     
         10 . The system as claimed in  claim 6 , wherein the three-dimensional elementary stream mixer multiplexes 4-channel field-based elementary streams of stereoscopic three-dimensional video output from the three-dimensional object encoder into a single-channel access unit stream using 2-channel elementary streams in the order of the odd field elementary stream of a left image and the even field elementary stream of the left image, when the display mode is the two-dimensional video display mode.  
     
     
         11 . The system as claimed in  claim 6 , wherein the three-dimensional elementary stream mixer multiplexes N×2 field-based elementary streams of N-view video output from the three-dimensional object encoder into a single-channel access unit stream sequentially using the individual viewpoints in the order of odd field elementary streams and even field elementary streams by viewpoints, when the display mode is the three-dimensional multiview video display mode.  
     
     
         12 . The system as claimed in  claim 1 , wherein when processing the elementary streams into a single-channel access unit stream and sending them to the packetizer, the compressor sends the individual elementary stream to the packetizer by adding at least one of image discrimination information representing whether the elementary stream is display discrimination information representing the display mode of the stereoscopic/multiview three-dimensional video selected by a user, and viewpoint information representing the number of viewpoints of a corresponding video image that is a multiview video image.  
     
     
         13 . The system as claimed in  claim 12 , wherein the packetizer receives a single-channel stream from the compressor per access unit, packetizes the received single-channel stream, and then constructs a packet header based on the additional information, wherein the packet header includes an access unit start flag representing which byte of a packet payload is the start of the stream, an access unit end flag representing which byte of the packet payload is the end of the stream, an image discrimination flag representing whether the elementary stream output from the compressor is two- or three-dimensional video data, a decoding time stamp flag, a composition time stamp flag, a viewpoint information flag representing the number of viewpoints of the video image, and a display discrimination flag representing the display mode.  
     
     
         14 . A stereoscopic/multiview three-dimensional video processing method, which is based on MPEG-4, the method comprising: 
 (a) receiving three-dimensional video data, determining whether a corresponding video image is a stereoscopic or multiview video image, and processing the corresponding video data according to the determination result to generate multi-channel field-based elementary streams;    (b) multiplexing the multi-channel field-based elementary streams in a display mode selected by a user to output a single-channel elementary stream;    (c) packetizing the single-channel elementary stream received; and    (d) processing the packetized stereoscopic/multiview three-dimensional video image and sending or storing the processed video image.    
     
     
         15 . The method as claimed in  claim 14 , wherein the step (a) of generating the elementary streams comprises: 
 outputting elementary streams in the unit of 4-channel fields including odd and even fields of a left three-dimensional stereoscopic image and odd and even fields of a right three-dimensional stereoscopic image, when the input data are three-dimensional stereoscopic video data; and    outputting N×2 field-based elementary streams, when the input data are Nview's multiview video data.    
     
     
         16 . The method as claimed in  claim 15 , wherein the multiplexing step (b) further comprises: multiplexing 4-channel field-based elementary streams of stereoscopic three-dimensional video into a single-channel access unit stream using 2-channel elementary streams in the order of the odd field elementary streams of a left image and the even field elementary streams of a right image, when the display mode is a three-dimensional video field shuttering display mode.  
     
     
         17 . The method as claimed in  claim 15 , wherein the multiplexing step (b) further comprises: multiplexing 4-channel field-based elementary streams of stereoscopic three-dimensional video into a single-channel access unit stream using 4-channel elementary streams in the order of the odd field elementary stream of a left image, the even field elementary stream of the left image, the odd field elementary stream of a right image and the even field elementary stream of the right image, when the display mode is a three-dimensional video frame shuttering display mode.  
     
     
         18 . The method as claimed in  claim 15 , wherein the multiplexing step (b) further comprises: multiplexing 4-channel field-based elementary streams of stereoscopic three-dimensional video into a single-channel access unit stream using 2-channel elementary streams in the order of the odd field elementary stream of a left image and the even field elementary stream of the left image, when the display mode is a two-dimensional video display mode.  
     
     
         19 . The method as claimed in  claim 15 , wherein the multiplexing step (b) further comprises: multiplexing N×2 field-based elementary streams of N-view video into a single-channel access unit stream sequentially using the individual viewpoints in the order of odd field elementary streams and even field elementary streams by viewpoints, when the display mode is a three-dimensional multiview video display mode.  
     
     
         20 . The method as claimed in  claim 14 , wherein the multiplexing step (b) comprises: processing multiview three-dimensional video images to generate multi-channel elementary streams and using time information acquired from an elementary stream of one channel among the multi-channel elementary streams to acquire synchronization with elementary streams of the other viewpoints, thereby acquiring synchronization among the three-dimensional video images.  
     
     
         21 . The system as claimed in  claim 1 , wherein the DecoderConfigDescriptor includes a 3D video image stream type so as to process a stereoscopic/multiview 3D video image.