Buffer level signaling for rate adaptation in multimedia streaming

Abstract

In a multimedia streaming network where a client has a receiver buffer to store a plurality of packets received from a server so as to compensate for the difference between data transmission amount by the server and data usage amount by the client, the server should be able to adapt the data transmission rate based on the status of the receiver buffer. For rate adaptation purposes, server reconstructs a list of packets in the receiver buffer based on information provided by the client. The information signaled to the server is indicative of the next unit to be decoded and the sequence number of the packet to which the next unit to be decoded belongs.

1 . A method for controlling level of a receiver buffer in a client in a multimedia streaming network, the streaming network comprising a server for providing streaming data in a plurality of packets to the client, wherein at least some of the data packets are stored in the receiver buffer so as to compensate for difference between data transmission amount by the server and the data usage amount by the client, and wherein the packets are decoded in a decoding order based on a plurality of decoding order values associated with a playout order to the client, said method comprising: 
 determining in the client the next packet to be decoded among the packets in the receiver buffer based on the decoding order values; and    signaling to the server information indicative of said next packet to be decoded, so as to allow the client to adjust the streaming data amount provided to the client based on the information.    
     
     
         2 . The method of  claim 1 , wherein the packets are associated with a plurality of units including the next unit to be decoded, and wherein the next packet to be decoded is the packet to which the next unit to be decoded belongs.  
     
     
         3 . The method of  claim 2 , wherein each of the data packets has a sequence number known to both the client and the server, and wherein the information signaled to the server is indicative of the sequence number of next packet to be decoded.  
     
     
         4 . The method of  claim 2 , wherein each of the units has a unit number and each of the data packets has a sequence number known to both the client and the server, and wherein the information signaled to the server is indicative of the unit number of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         5 . The method of  claim 4 , wherein the server maintains a list of units that have been sent and a record of unit number and the sequence number of the packet to which the sent units belong and a mapping between said sequence number and unit number to the decoding order for determining the data units in the receiver buffer based on said mapping so as to adjust the streaming data amount provided to the client based on said determination in the server.  
     
     
         6 . The method of  claim 4 , wherein the information signaled to the server is further indicative of a difference between a scheduled playout time of said next unit to be decoded and the decoding time of said next unit.  
     
     
         7 . The method of  claim 6 , wherein the information signaled to the server is further indicative of the highest sequence number received by the client so as to allow the server to determine the data packets in the receiver buffer.  
     
     
         8 . The method of  claim 2 , wherein each of the units has a timestamp and each of the data packets has a sequence number known to both the client and the server, and wherein the information signaled to the server is indicative of the timestamp of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         9 . A multimedia streaming network comprising: 
 at least a client; and    a server for providing streaming data in a plurality of packets to the client, wherein the client comprises: 
 a receiver buffer for storing at least some of the data packets to be decoded so as to compensate for difference between data transmission amount by the server and data usage amount by the client, and wherein the packets are decoded in a decoding order based on a plurality of decoding values associated with a playout order in the client, and  
 a mechanism for signaling to the server information indicative of the next packet to be decoded among the packets in the buffer based on the decoding order values so as to allow the server to adjust the rate of streaming data provided to the client.  
   
     
     
         10 . The streaming network of  claim 9 , wherein the packets are associated with a plurality of units including the next unit to be decoded, and wherein the next packet to be decoded is the packet to which the next unit to be decoded belongs.  
     
     
         11 . The streaming network of  claim 10 , wherein each of the data packets has a sequence number known to both the client and the server, and wherein the information signaled to the server is indicative of the sequence number of next packet to be decoded.  
     
     
         12 . The streaming network of  claim 10 , wherein each of the units has a unit number and each of the data packets has a sequence number known to both the client and the server, and wherein the information signaled to the server is indicative of the unit number of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         13 . The streaming network of  claim 12 , wherein the server maintains a list of units that have been sent and a record of unit number and the sequence number of the packet to which the sent units belong and a mapping between said sequence number and unit number to the decoding order for determining the data units in the receiver buffer based on said mapping so as to adjust the streaming data amount provided to the client based on said determination in the server.  
     
     
         14 . The streaming network of  claim 12 , wherein the information signaled to the server is further indicative of a difference between a scheduled playout time of said next unit to be decoded and the decoding time of said next unit.  
     
     
         15 . The streaming network of  claim 12 , wherein the information signaled to the server is further indicative of the highest sequence number received by the client so as to allow the server to determine the data packets in the receiver buffer.  
     
     
         16 . The streaming network of  claim 10 , wherein each of the units has a timestamp and each of the data packets has a sequence number known to both the client and the server, and wherein the information signaled to the server is indicative of the timestamp of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         17 . A client device in a multimedia streaming network, the streaming network comprising a server device for providing streaming data in a plurality of packets to the client device, wherein the packets are decoded in a decoding order based on a plurality of decoding values associated with a playout order in the client device, said client device comprising: 
 a receiver buffer for storing at least some of the data packets to be decoded so as to compensate for difference between data transmission amount by the server device and the data usage amount in the client device; and    a mechanism for signaling to the server device information indicative of the next packet to be decoded among the packets in the receiver buffer based on the decoding order values so as to allow the server device to adjust the streaming data amount provided to the client device.    
     
     
         18 . The client device of  claim 17 , wherein the packets are associated with a plurality of units including the next unit to be decoded, and wherein the next packet to be decoded is the packet to which the next unit to be decoded belongs.  
     
     
         19 . The client device of  claim 18 , wherein each of the units has a unit number and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signaled to the server device is indicative of the unit number of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         20 . The client device of  claim 18 , wherein each of the units has a timestamp and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signaled to the server device is indicative of the timestamp of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         21 . The client device of  claim 17 , further comprising 
 a software program having executable codes for determining: 
 the decoding order of the data packets in the receiver buffer based on the decoder order values, and  
 the next packet to be decoded among the data packets in the receiver buffer based on the decoding order values.  
   
     
     
         22 . A server device for providing streaming data in a multimedia streaming network, the multimedia streaming network comprising at least a client device for receiving the streaming data in a plurality of data packets and decoding the data packets in a decoding order based on a plurality of decoding order values associated with a playout order, wherein the client device has a receiver buffer for storing at least some of the data packets so as to compensate for difference between data transmission amount by the server device and the data usage amount by the client device, said server device comprising: 
 a mechanism for receiving information from the client device indicative of the next packet to be decoded among the packets in the receiver buffer based on the decoding order values in the client device; and    a software program for determining the packets in the receiver buffer based on the information so as to adjust the streaming data amount provided to the client device for controlling level of the receiver buffer.    
     
     
         23 . The server device of  claim 22 , wherein the packets are associated with a plurality of units including the next unit to be decoded, and wherein the next packet to be decoded is the packet to which the next unit to be decoded belongs.  
     
     
         24 . The server device of  claim 23 , wherein each of the units has a unit number and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signaled to the server device is indicative of the unit number of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         25 . The server device of  claim 23 , wherein each of the units has a timestamp and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signaled to the server device is indicative of the timestamp of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         26 . A software product embedded in a computer readable media for use in a client device in a multimedia streaming network, the streaming network comprising a server device for providing streaming data in a plurality of packets to the client device, wherein the packets are decoded in a decoding order based on a plurality of decoding values associated with a playout order in the client device, and wherein the client device comprises a receiver buffer for storing at least some of the data packets to be decoded so as to compensate for difference between data transmission, the software product comprising: 
 a code for determining the decoding order of the data packets in the receiver buffer based on the decoding order values; and    a code for determining the next packet to be decoded among the data packets in the receiver buffer based on the decoding order values, so as to provide to the server device information indicative of said next packet to be decoded, allowing the server device to adjust the streaming data amount provided to the client device based on the information for controlling level of receiver buffer.    
     
     
         27 . The software product of  claim 26 , wherein the packets are associated with a plurality of units including the next unit to be decoded, and wherein the next packet to be decoded is the packet to which the next unit to be decoded belongs.  
     
     
         28 . The software product of  claim 27 , wherein each of the units has a unit number and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signaled to the server device is indicative of the unit number of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         29 . The software product of  claim 27 , wherein each of the units has a timestamp and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signal to the server device is indicative of the timestamp of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         30 . A software product embedded in a computer readable media for use in a server device providing streaming data in the multimedia streaming network, the multimedia network comprising at least a client device for receiving the streaming data in a plurality of data packets and decoding the data packets in a decoding order based on a plurality of decoding order values associated with a playout order, wherein the client device has a receiver buffer for storing at least some of the data packets so as to compensate for difference between the transmission amount by the server device and the data usage amount by the client device, said software product comprising: 
 a code for relating the decoding order to the sequence numbers of the data packets having been sent to the client device; and    a code for determining the data packets in the receiver buffer based on said relating and information provided by the client device indicative of the next packet to be decoded in the client device, so as to allow the server device to adjust the streaming data amount provided to the client device for controlling level of the receiver buffer.    
     
     
         31 . The software product of  claim 30 , wherein the packets are associated with a plurality of units including the next unit to be decoded, and wherein the next packet to be decoded is the packet to which the next unit to be decoded belongs.  
     
     
         32 . The software product of  claim 30 , wherein each of the units has a unit number and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signaled to the server device is indicative of the unit number of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.  
     
     
         32 . The software product of  claim 30 , wherein each of the units has a timestamp and each of the data packets has a sequence number known to both the client device and the server device, and wherein the information signaled to the server device is indicative of the timestamp of said next unit to be decoded and the sequence number of the packet to which said next unit belongs.