Application latency determination method and system for wearable display devices

Abstract

The present invention relates to latency determination for wearable display devices. In particular, the invention is particularly useful with regards to virtual reality (VR) headsets, augmented reality (AR) headsets or head mounted displays (HMD). It is an object to provide a system and method for determining application latency for a wearable display device. It is a further object of to provide a hardware independent system and method for determining application latency between a computing device and a wearable display device. With hardware independent systems and methods, different hardware, e.g. different wearable display devices, can be benchmarked and compared to each other accurately.

The invention claimed is: 
     
       1. An application latency determination method for a wearable display device comprising the steps of:
 sending, using sending hardware, a time-stamped display signal to the wearable display device from a computing device, 
 receiving, from at least one photo sensor arranged to monitor a display of the wearable display, a signal through a microphone input of the computing device, said signal indicating a response of the display to the display signal, 
 time-stamping the received signal via an audio application programming interface (API) of the computing device, and 
 determining, using a processor, an application latency for the wearable display based at least on the sent time-stamped display signal and the time-stamped received signal. 
 
     
     
       2. The application latency determination method according to  claim 1 , wherein the wearable display device includes two separate displays and wherein a separate photo sensor is provided to monitor each display, said method including:
 receiving, from each of two photo sensors, a signal through the microphone input of the computing device, said signal indicating a response of each respective display to the time-stamped display signal, 
 time-stamping the received signals via the API of the computing device, and 
 determining a latency between the two separate displays based on the difference in received times of the signals of the respective photo sensors. 
 
     
     
       3. The application latency determination method according to  claim 1 , further comprising a calibration step, prior to sending the time-stamped display signal, said calibration step including:
 connecting an audio output of the computing device to the microphone input, 
 sending a calibration signal through the audio output to the microphone input, 
 time-stamping the calibration signal and a respective response signal from the microphone input, and 
 determining a time difference between the two signals. 
 
     
     
       4. The application latency determination method according to  claim 3 , wherein if the difference between the calibration signal and respective response signal is less than 2 milliseconds, then sending the time-stamped display signal. 
     
     
       5. The application latency determination method according to  claim 1 , further comprising:
 alternate sending the time-stamped display signal and a second time-stamped display signal, said second time-stamped display signal for inducing a full screen color change of the display from the induced display of said first display signal, and 
 based on time-stamped received signals and the time-stamped display signals, determine application latency for the wearable display device and/or the presence of dropped frames by the wearable display device. 
 
     
     
       6. The application latency determination method according to  claim 5 , wherein the display signals induce black and white displays respectively. 
     
     
       7. The application latency determination method according to  claim 5 , wherein the display signals induce a different color to be overlaid over the previous signal color. 
     
     
       8. The application latency determination method according to  claim 5 , wherein the first and second display signals are sent one frame apart at a desired frame rate. 
     
     
       9. A non-transitory computer readable medium having stored thereon a set of computer executable instructions for carrying out an application latency method for a wearable display device, comprising:
 sending, using sending hardware, a time-stamped display signal to the wearable display device from a computing device, 
 receiving, from at least one photo sensor arranged to monitor a display of the wearable display, a signal through a microphone input of the computing device, said signal indicating a response of the display to the display signal, 
 time-stamping the received signal via an audio application programming interface (API) of the computing device, and 
 determining, using a processor, an application latency for the wearable display based at least on the sent time-stamped display signal and the time-stamped received signal. 
 
     
     
       10. A system for application latency determination for a wearable display device comprising:
 at least one photo sensor capable of being arranged to monitor a display of the wearable display device, wherein an output of the at least one photo sensor is capable of being received by a microphone input of a computing device, 
 a computer-readable non-transitory medium storing computer readable instructions configured to send a time-stamped display signal to the wearable display device from the computing device, and 
 a processor configured to determine an application latency for the wearable display device based at least on the sent time-stamped display signal and a signal received from the photo sensor, through the microphone input and time-stamped by an audio application programming interface (API) of the computing device. 
 
     
     
       11. The system for application latency determination according to  claim 10 , further comprising a computing device having a microphone input and an audio application programming interface (API) for time-stamping signals received from the microphone input. 
     
     
       12. The system for application latency determination according to  claim 10 , further comprising at least two photo sensors, wherein the output of each photo sensor are combined and capable of being received by a single stereo microphone input of the computing device. 
     
     
       13. The system for application latency determination according to  claim 10 , wherein the photo sensor(s) are phototransistor(s). 
     
     
       14. The system for application latency determination according to  claim 10 , wherein an audio output of the computing device is connected to the microphone input and a signal is sent between the two and time-stamped by the audio API, the difference between a time-stamp of the signal from the audio output and the signal from the microphone input is less than 2 milliseconds. 
     
     
       15. The system for application latency determination according to  claim 10 , further comprising an arrangement means connected to the photo sensor(s) for arranging the photo sensor(s) in relation to a display of the wearable display device. 
     
     
       16. The system for application latency determination according to  claim 10 , wherein the wearable display device is a virtual reality device, an augmented reality device, a head mounted display or a combination thereof. 
     
     
       17. The system for application latency determination according to  claim 10 , wherein the computing device is a personal computer, a laptop, a mobile phone, a tablet computer or a wearable computer.