Game and screen media content streaming architecture

Abstract

A streaming architecture includes a two-layer architecture with a base layer and an enhanced layer. The base layer encodes computer generated content and generates an encoded bitstream. The enhanced layer encodes and transmit a chroma residual for a region of interest, wherein the encoded chroma residual stored in a UV33 surface that is inserted into a supplemental enhancement information (SEI) of the encoded bitstream from the base layer. A transmitter transmits the encoded bitstream to a receiver.

What is claimed is: 
     
         1 . A streaming architecture, comprising:
 a base layer, wherein the base layer performs encodes computer generated content and generates an encoded bitstream;   an enhanced layer to encode and transmit a chroma residual for a region of interest, wherein the encoded chroma residual stored in a UV33 surface that is inserted into a supplemental enhancement information (SEI) of the encoded bitstream from the base layer; and   a transmitter to transmit the encoded bitstream to a receiver.   
     
     
         2 . The streaming architecture of  claim 1 , wherein the UV33 surface is formatted to store and transmit the chroma residual with the least amount of data to reconstruct a YUV 4:4:4 surface composited with a decoded YUV 4:2:0 surface. 
     
     
         3 . The streaming architecture of  claim 1 , wherein the UV33 surface has a different layout based on different chroma siting location information used during chroma down sampling. 
     
     
         4 . The streaming architecture of  claim 1 , wherein the size of the UV33 surface is same as a YUV 4:2:0 surface with a same width and height of pixels. 
     
     
         5 . The streaming architecture of  claim 1 , wherein the amount of data stored at the UV33 surface is smaller than the data stored in a YUV 4:2:0 surface of the base layer. 
     
     
         6 . The streaming architecture of  claim 1 , wherein in response to the receiver not supporting the enhanced layer, the base layer functions independently to reconstruct the encoded bitstream. 
     
     
         7 . The streaming architecture of  claim 1 , wherein regions of interest are determined by edge detection, Sobel edge detectors, Canny edge detection, edge thinning, thresholding, or any combinations thereof. 
     
     
         8 . The streaming architecture of  claim 1 , wherein the enhanced layer output is transmitted using an SEI message. 
     
     
         9 . The streaming architecture of  claim 1 , wherein the receiver receives the encoded bitstream and parses an SEI syntax to obtain composite YUV 4:4:4 data for each region of interest. 
     
     
         10 . The streaming architecture of  claim 1 , wherein the encoded bitstream is decoded at the receiver into a YUV 4:2:0 format, wherein for each region of interest base layer information is replaced by enhanced layer information. 
     
     
         11 . A method for a media streaming architecture, comprising:
 determining regions of interest in image data;   encoding the image data into a bitstream at a base layer;   encoding the regions of interest using a chroma residual of each region of interest at an enhanced layer;   combining the encoded chroma residual from the enhanced layer in a supplemental enhancement information of the bitstream of the base layer; and   transmitting the bitstream to a receiver.   
     
     
         12 . The method of  claim 11 , wherein the regions of interest are encoded using a UV33 surface. 
     
     
         13 . The method of  claim 11 , wherein the regions of interest are encoded based on a chroma sitting location. 
     
     
         14 . The method of  claim 11 , wherein the base layer contains all information to restore the bit stream at the receiver in response to the receiver not supporting the enhanced layer. 
     
     
         15 . The method of  claim 11 , wherein the regions of interest are those regions that include colorful text and sharp edges. 
     
     
         16 . The method of  claim 11 , wherein the regions of interest are determined by edge detection, Sobel edge detectors, Canny edge detection, edge thinning, thresholding, or any combination thereof. 
     
     
         17 . The method of  claim 11 , wherein the enhanced layer output is transmitted using an SEI message. 
     
     
         18 . The method of  claim 11 , wherein the receiver receives the encoded bitstream and parses an SEI syntax to obtain composite YUV 4:4:4 data for each region of interest. 
     
     
         19 . The method of  claim 11 , wherein the encoded bitstream is decoded at the receiver into a YUV 4:2:0 format, wherein for each region of interest base layer information is replaced by enhanced layer information. 
     
     
         20 . The method of  claim 11 , wherein the receiver is a playback device. 
     
     
         21 . At least one computer readable medium for encoding video frames having instructions stored therein that, in response to being executed on a computing device, cause the computing device to:
 determine regions of interest in image data;   encode the image data into a bitstream at a base layer;   encode the regions of interest using a chroma residual of each region of interest at an enhanced layer;   combine the encoded chroma residual from the enhanced layer in a supplemental enhancement information of the bitstream of the base layer; and   transmit the bitstream to a receiver.   
     
     
         22 . The at least one computer readable medium of  claim 21 , wherein the regions of interest are encoded using a UV33 surface. 
     
     
         23 . The at least one computer readable medium of  claim 21 , wherein the regions of interest are encoded based on a chroma sitting location. 
     
     
         24 . The at least one computer readable medium of  claim 21 , wherein the base layer contains all information to restore the bit stream at the receiver in response to the receiver not supporting the enhanced layer. 
     
     
         25 . The at least one computer readable medium of  claim 21 , wherein the regions of interest are those regions that include colorful text and sharp edges.