System, method, and computer program product for real time transparency-based compositing

Abstract

A system, method, and computer program product for compositing rendered image data in near real time. The input pixel streams that constitute the rendered image data can be video streams, for example. Each input pixel stream can originate from its own graphics processing unit. Each pixel includes a set of color coordinates, such as red, green, and blue (RGB) coordinates, plus an alpha value. Compositing is performed by an image combiner implemented in either hardware or software. The image combiner accepts two or more input pixel streams, and performs compositing on corresponding pixels from each input pixel stream. The compositing process takes into account the color coordinates of each corresponding pixel as well as the alpha values. The compositing process also uses depth information that defines whether a given pixel is in the foreground or in the background relative to another corresponding pixel. The result of the compositing process is a resultant pixel stream based on corresponding pixels of each input pixel stream.

What is claimed is:  
     
         1 . A method of combining a plurality of input pixel streams to form a resultant pixel stream, comprising the steps of: 
 a) receiving the plurality of input pixel streams;    b) receiving depth information relating to the relative depth of the input pixel streams;    c) compositing, in depth order, corresponding pixels from each input pixel stream to form the resultant pixel stream in approximately real time.    
     
     
         2 . The method of  claim 1 , wherein each pixel comprises color coordinates and an alpha value.  
     
     
         3 . The method of  claim 1 , wherein step c) comprises the steps of: 
 i) blending the corresponding pixels according to the alpha values of the corresponding pixels and the depth information;    ii) outputting the resultant pixel; and    iii) if the input pixel streams contain additional corresponding pixels, repeating steps (i) and (ii) for the additional corresponding pixels.    
     
     
         4 . The method of  claim 1 , wherein the input pixel streams and resultant pixel stream are video streams.  
     
     
         5 . The method of  claim 1 , wherein the input pixel streams each represent renderings of adjacent sub-volumes, such that the resultant pixel stream represents a rendering of the adjacent sub-volumes viewed collectively.  
     
     
         6 . The method of  claim 1 , wherein the depth information can vary for each frame.  
     
     
         7 . An image combiner for combining a plurality of input pixel streams to form a single resultant pixel stream, comprising: 
 a depth determination module for converting depth information to an indication as to depth order of the input pixel streams; and    one or more blending modules that perform a blending operation on color coordinates of corresponding input pixels, on the basis of said depth order and alpha values associated with said corresponding input pixels.    
     
     
         8 . The system of  claim 7 , wherein the input pixel streams comprise image data output from a graphics processor.  
     
     
         9 . The system of  claim 7 , wherein the input pixel streams comprise image data output from another image combiner.  
     
     
         10 . A computer program product comprising a computer usable medium having computer readable program code means embodied in said medium for causing a program to execute on a computer that combines a plurality of input pixel streams to form a single resultant pixel stream, said computer readable program code means comprising: 
 a first computer program code means for causing the computer to receive the plurality of input pixel streams;    a second computer program code means for causing the computer to receive depth information relating to the relative depth of the input pixel streams; and    a third computer program code means for causing the computer to composite, in depth order, corresponding pixels from each input pixel stream to form the resultant pixel stream in approximately real time.    
     
     
         11 . The computer program product of  claim 10 , wherein each pixel comprises color coordinates and an alpha value.  
     
     
         12 . The computer program product of  claim 10 , wherein said third computer program code means comprises: 
 i) computer program code means for combining the corresponding pixels according to the alpha values of the corresponding pixels and the depth information;    ii) computer program code means for outputting the resultant pixel; and    iii) computer program code means for repeating execution of code means (i) and (ii) for additional corresponding pixels, if the input pixel streams contain additional corresponding pixels.    
     
     
         13 . The computer program product of  claim 10 , wherein the input pixel streams and resultant pixel stream are video streams.  
     
     
         14 . The computer program product of  claim 10 , wherein the input pixel streams each represent renderings of adjacent sub-volumes, such that the resultant pixel stream represents a rendering of the adjacent sub-volumes viewed collectively.  
     
     
         15 . The computer program product of  claim 10 , wherein the depth information can vary for each frame.