Latency reduction in remote rendering with adaptive phase shifting

Abstract

Sending streamed data packets from a producer to a consumer. A method includes, at a first entity, sending consumable data packets from the first entity to a second entity at a first consumable packet rate. The method further includes receiving a first phase delta from the second entity, wherein the first phase delta is computed from transmission jitter, computed from timing information in the consumable data packets. The method further includes sending from the first entity consumable data packets at a second consumable packet rate, the second consumable packet rate being dependent on the first phase delta.

What is claimed is: 
     
       1. A computer system enabled to send consumable data packets at variable packet rates based upon measured latency, the system comprising:
 one or more processors; and 
 one or more computer-readable media having stored thereon instructions that are executable by the one or more processors to configure the computer system to send streamed data packets from a producer to a consumer, including instructions that are executable to configure the computer system to perform at least the following: 
 at a first entity, send consumable data packets from the first entity to a second entity at a first consumable packet rate, each consumable data packet comprising a timestamp enabling the consumer to identify when the each consumable data packet had been sent by the first entity; 
 receive a first phase delta from the second entity, wherein the first phase delta is computed at least from transmission jitter, computed from timing information in the consumable data packets, including the included timestamps and information identifying when the each consumable data packet was received at the second entity, the second entity having computed a frequency at which packets are sent from the first entity and the first phase delta being computed to minimize latency, latency being a difference in time between an arrival time of a packet and a beginning time of processing of the packet; 
 compute a second consumable packet rate based upon the received first phase delta; and 
 send from the first entity consumable data packets at second consumable packet rate, the second consumable packet rate being dependent on the first phase delta; 
 wherein the data in the consumable data packets is dependent on state data of the second entity provided to the first entity; 
 wherein the consumable data packets are video packets, the second entity is a consumer of video packets, the first entity is a producer of video packets, and the state data comprises telemetry data. 
 
     
     
       2. The computer system of  claim 1 , wherein the state data is received as state data packets at a first state data packet rate, the one or more computer-readable media further having stored thereon instructions that are executable by the one or more processors to configure the computer system to perform at least the following:
 sending feedback including a second phase delta to the second entity, the second phase delta comprising timing differences between when state data packets are sent from the second entity to the first entity; and 
 receiving the state data packets at a second state data packet rate, the second state data packet rate being dependent on the second phase delta. 
 
     
     
       3. The computer system of  claim 2 , wherein:
 the first phase delta is received in a state data packet; and 
 the second phase delta is sent in a consumable data packet. 
 
     
     
       4. The computer system of  claim 1 , wherein the consumable data packets are telemetry packets, the second entity is a consumer of telemetry packets, and the second entity is a producer of video packets. 
     
     
       5. In a computing environment, a method of receiving streamed data packets from a producer to a consumer, the data packets being dependent on state from the consumer, the method comprising:
 at a second entity, receiving consumable data packets from a first entity at a first consumable packet rate, each consumable data packet comprising a timestamp enabling the second entity to identify when the each consumable data packet had been sent by the first entity; 
 computing transmission jitter based at least on timing information in the consumable data packets; 
 computing a first phase delta based at least on the jitter, the first phase delta computed from the timing information, including the included timestamps and information identifying when the each consumable data packet was received at the second entity, and the first phase delta defining timing differences between when consumable data packets are sent from the first entity, the second entity having computed a frequency at which packets are sent from the first entity and the first phase delta being computed to minimize latency, latency being a difference in time between an arrival time of a packet and a beginning time of processing of the packet; 
 providing the first phase delta to the first entity; and 
 receiving from the first entity consumable data packets at a second consumable packet rate, the second consumable packet rate being dependent on the first phase delta; 
 wherein the data in the consumable data packets is dependent on state data of the second entity provided to the first entity; 
 wherein the consumable data packets are video packets, the second entity is a consumer of video packets, the first entity is a producer of video packets, and the state data comprises telemetry data. 
 
     
     
       6. The method of  claim 5 , wherein the state data is sent as state data packets at a first state data packet rate, the method further comprising:
 receiving feedback including a second phase delta from the first entity, the second phase delta comprising timing differences between when state data packets are sent from the second entity to the first entity; 
 sending the state data packets to the first entity at a second state data packet rate, the second state data packet rate being dependent on the second phase delta. 
 
     
     
       7. The method of  claim 6 , wherein
 the first phase delta is sent in a state data packet; and 
 the second phase delta is received in a consumable data packet. 
 
     
     
       8. The method of  claim 5 , wherein the consumable data packets are telemetry packets, the second entity is a consumer of telemetry packets, and the second entity is a producer of video packets. 
     
     
       9. The method of  claim 5 , wherein the acts are performed iteratively, such that consumable packet rates vary over time as phase deltas vary. 
     
     
       10. In a computing environment, a method of sending consumable data packets from a producer to a consumer at variable packet rates based upon measured latency, the method comprising:
 at a first entity, sending consumable data packets from the first entity to a second entity at a first consumable packet rate, each consumable data packet comprising a timestamp enabling the consumer to identify when the each consumable data packet had been sent by the first entity; 
 receiving a first phase delta from the second entity, wherein the first phase delta is computed at least from transmission jitter, computed from timing information in the consumable data packets, including the included timestamps and information identifying when the each consumable data packet was received at the second entity, the second entity having computed a frequency at which packets are sent from the first entity and the first phase delta being computed to minimize latency, latency being a difference in time between an arrival time of a packet and a beginning time of processing of the packet; 
 computing a second consumable packet rate based upon the received first phase delta; and 
 sending from the first entity consumable data packets at a second consumable packet rate, the second consumable packet rate being dependent on the first phase delta; 
 wherein the data in the consumable data packets is dependent on state data of the second entity provided to the first entity; 
 wherein the consumable data packets are video packets, the second entity is a consumer of video packets, the first entity is a producer of video packets, and the state data comprises telemetry data. 
 
     
     
       11. The method of  claim 10 , wherein the state data is received as state data packets at a first state data packet rate, the method further comprising:
 sending feedback including a second phase delta to the second entity, the second phase delta comprising timing differences between when state data packets are sent from the second entity to the first entity; and receiving the state data packets at a second state data packet rate, the second state data packet rate being dependent on the second phase delta. 
 
     
     
       12. The method of  claim 11 , wherein:
 the first phase delta is received in a state data packet; and 
 the second phase delta is sent in a consumable data packet. 
 
     
     
       13. The method of  claim 10 , wherein the consumable data packets are telemetry packets, the second entity is a consumer of telemetry packets, and the second entity is a producer of video packets. 
     
     
       14. The method of  claim 10 , wherein the acts are performed iteratively, such that consumable packet rates vary over time as phase deltas vary.