Selective execution of warping for graphics processing

Abstract

Example techniques are described for image processing to selectively perform a warping operation if there is change in orientation of a display device between frames. For example, if there is change in orientation of a display device between processing a first frame and after rendering of a second frame, processing circuitry may perform a warp operation on the second frame to warp image content of the second frame based on a current orientation of the display device. If there is no change in orientation of the display device between processing a third frame and after rendering of a fourth frame, the processing circuitry may bypass the warp operation on the fourth frame to avoid warping entire image content of the fourth frame.

What is claimed is: 
     
         1 . A method for image processing, the method comprising:
 determining, with one or more processors, that there is change in orientation of a display device between processing a first frame and after rendering a second frame;   responsive to the determination that there was change in the orientation between processing the first frame and after rendering the second frame, performing, with the one or more processors, a warp operation on the second frame to warp image content of the second frame based on a current orientation of the display device after rendering the second frame;   determining, with the one or more processors, that there is no change in orientation of the display device between processing a third frame and after rendering of a fourth frame; and   responsive to the determination that there is no change in the orientation of the display device between processing the third frame and after rendering the fourth frame, bypassing a warp operation on the fourth frame to avoid warping entire image content of the fourth frame.   
     
     
         2 . The method of  claim 1 , further comprising:
 determining that there is change in image content between the third frame and the fourth frame; and   responsive to the determination that there is change in the image content between the third frame and the fourth frame, updating a portion, and not the entirety, of a frame buffer that stores image content for the third frame based on the change between the third frame and the fourth frame.   
     
     
         3 . The method of  claim 2 , wherein updating the portion comprises texture mapping tiles of the fourth frame that are different than corresponding tiles of the third frame to the portion of the frame buffer. 
     
     
         4 . The method of  claim 3 , wherein the texture mapping comprises:
 replacing the portion of the frame buffer that stores image content of a tile of the third frame with a tile from the fourth frame based on a current orientation of the display device after rendering the fourth frame.   
     
     
         5 . The method of  claim 2 , further comprising:
 determining to which buffer or buffers a graphics processing unit (GPU) is writing image content of tiles of the fourth frame during rendering of the fourth frame,   wherein determining that there is change in image content between the third frame and the fourth frame comprises determining that there is change in image content of tiles of the fourth frame that correspond to the buffers to which the GPU is writing or wrote image content.   
     
     
         6 . The method of  claim 2 , further comprising:
 determining a first hash value for the third frame; and   determining a second hash value for the fourth frame,   wherein determining that there is change in image content between the third frame and the fourth frame comprises determining that there is change in image content based on a difference between the first hash value and the second hash value.   
     
     
         7 . The method of  claim 1 , further comprising:
 causing a display panel to display the third frame;   determining that there is no change in image content between the third frame and the fourth frame; and   responsive to the determination that that there is no change in the orientation of the display device and that there is no change in image content between the third frame and the fourth frame, causing a display panel to redisplay the third frame.   
     
     
         8 . The method of  claim 1 , wherein performing the warping operation comprises performing one or more of synchronous time warp (STW), STW with depth, asynchronous time warp (ATW), ATW with depth, asynchronous space warp (ASW), or ASW with depth. 
     
     
         9 . The method of  claim 1 , wherein the display device comprises a wearable display device. 
     
     
         10 . The method of  claim 1 , further comprising:
 executing, on the display device, a virtual reality (VR) or augmented reality (AR) application corresponding to the first frame, the second frame, the third frame, and the fourth frame.   
     
     
         11 . The method of  claim 10 , further comprising:
 outputting information indicative of a first orientation of the display device;   receiving image content for the first frame based on the first orientation;   processing the first frame based on the received image content for the first frame;   subsequent to processing the first frame, outputting information indicative of a second orientation of the display device;   receiving image content for the second frame based on the second orientation;   rendering the second frame based on the received image content for the second frame;   outputting information indicative of a third orientation of the display device;   receiving image content for the third frame based on the third orientation;   processing the third frame based on the received image content for the third frame;   subsequent to processing the third frame, outputting information indicative of a fourth orientation of the display device;   receiving image content for the fourth frame based on the fourth orientation; and   rendering the fourth frame based on the received image content for the fourth frame.   
     
     
         12 . A display device for image processing, the device comprising:
 memory configured to store information indicative of orientation of the display device; and   processing circuitry configured to:
 determine, based on the stored information, that there is change in orientation of a display device between processing a first frame and after rendering a second frame; 
 responsive to the determination that there was change in the orientation between processing the first frame and after rendering the second frame, perform a warp operation on the second frame to warp image content of the second frame based on a current orientation of the display device after rendering the second frame; 
 determine, based on the stored information, that there is no change in orientation of a display device between processing a third frame and after rendering of a fourth frame; and 
 responsive to the determination that there is no change in the orientation of the display device between processing the third frame and after rendering the fourth frame, bypass a warp operation on the fourth frame to avoid warping entire image content of the fourth frame. 
   
     
     
         13 . The device of  claim 12 ,
 wherein the memory comprises a frame buffer that stores image content for the third frame, and   wherein the processing circuitry is configured to:
 determine that there is change in image content between the third frame and the fourth frame; and 
 responsive to the determination that there is change in the image content between the third frame and the fourth frame, update the portion, and not the entirety, of the frame buffer that stores the image content for the third frame based on the change between the third frame and the fourth frame. 
   
     
     
         14 . The device of  claim 13 , wherein to update the portion, the processing circuitry is configured to texture map tiles of the fourth frame that are different than corresponding tiles of the third frame to the portion of the frame buffer. 
     
     
         15 . The device of  claim 14 , wherein to texture map, the processing circuitry is configured to:
 replace the portion of the frame buffer that stores image content of a tile of the third frame with a tile from the fourth frame based on a current orientation of the display device after rendering the fourth frame.   
     
     
         16 . The device of  claim 13 , wherein the processing circuitry is configured to:
 determine to which buffer or buffers a graphics processing unit (GPU) is writing image content of tiles of the fourth frame during rendering of the fourth frame,   wherein to determine that there is change in image content between the third frame and the fourth frame, the processing circuitry is configured to determine that there is change in image content of tiles of the fourth frame that correspond to the buffers to which the GPU is writing or wrote image content.   
     
     
         17 . The device of  claim 13 , wherein the processing circuitry is configured to:
 determine a first hash value for the third frame; and   determine a second hash value for the fourth frame,   wherein to determine that there is change in image content between the third frame and the fourth frame, the processing circuitry is configured to determine that there is change in image content based on a difference between the first hash value and the second hash value.   
     
     
         18 . The device of  claim 12 , wherein the processing circuity is configured to:
 cause a display panel to display the third frame;   determine that there is no change in image content between the third frame and the fourth frame; and   responsive to the determination that that there is no change in the orientation of the display device and that there is no change in image content between the third frame and the fourth frame, cause a display panel to redisplay the third frame.   
     
     
         19 . The device of  claim 12 , wherein to perform the warping operation, the processing circuitry is configured to perform one or more of synchronous time warp (STW), STW with depth, asynchronous time warp (ATW), ATW with depth, asynchronous space warp (ASW), or ASW with depth. 
     
     
         20 . The device of  claim 12 , wherein the display device comprises a wearable display device. 
     
     
         21 . The device of  claim 12 , wherein the processing circuitry is configured to:
 execute a virtual reality (VR) or augmented reality (AR) application corresponding to the first frame, the second frame, the third frame, and the fourth frame.   
     
     
         22 . The device of  claim 21 , wherein the processing circuity is configured to:
 output information indicative of a first orientation of the display device;   receive image content for the first frame based on the first orientation;   process the first frame based on the received image content for the first frame;   subsequent to processing the first frame, output information indicative of a second orientation of the display device;   receive image content for the second frame based on the second orientation;   render the second frame based on the received image content for the second frame;   output information indicative of a third orientation of the display device;   receive image content for the third frame based on the third orientation;   process the third frame based on the received image content for the third frame;   subsequent to processing the third frame, output information indicative of a fourth orientation of the display device;   receive image content for the fourth frame based on the fourth orientation; and   render the fourth frame based on the received image content for the fourth frame.   
     
     
         23 . A computer-readable storage medium storing instructions that when executed cause one or more processors of a display device for image processing to:
 determine that there is change in orientation of the display device between processing a first frame and after rendering a second frame;   responsive to the determination that there was change in the orientation between processing the first frame and after rendering the second frame, perform a warp operation on the second frame to warp image content of the second frame based on a current orientation of the display device after rendering the second frame;   determine that there is no change in orientation of the display device between processing a third frame and after rendering of a fourth frame; and   responsive to the determination that there is no change in the orientation of the display device between processing the third frame and after rendering the fourth frame, bypass a warp operation on the fourth frame to avoid warping entire image content of the fourth frame.   
     
     
         24 . The computer-readable storage medium of  claim 23 , wherein the display device comprises a wearable display device. 
     
     
         25 . The computer-readable storage medium of  claim 23 , storing further instructions that cause the one or more processors to:
 execute a virtual reality (VR) or augmented reality (AR) application corresponding to the first frame, the second frame, the third frame, and the fourth frame.   
     
     
         26 . The computer-readable storage medium of  claim 25 , storing further instructions that cause the one or more processors to:
 output information indicative of a first orientation of the display device;   receive image content for the first frame based on the first orientation;   process the first frame based on the received image content for the first frame;   subsequent to processing the first frame, output information indicative of a second orientation of the display device;   receive image content for the second frame based on the second orientation;   render the second frame based on the received image content for the second frame;   output information indicative of a third orientation of the display device;   receive image content for the third frame based on the third orientation;   process the third frame based on the received image content for the third frame;   subsequent to processing the third frame, output information indicative of a fourth orientation of the display device;   receive image content for the fourth frame based on the fourth orientation; and   render the fourth frame based on the received image content for the fourth frame.