Multi-level optical flow estimation framework for stereo pairs of images based on spatial partitioning

Abstract

Techniques related to multi-level optical flow estimation are discussed. Such techniques include partitioning each pair of input images into one or more partitions, separately performing optical flow estimation on the partitions, and merging the separately generated optical flow results into a final optical flow map for the pair of input images.

1 - 20 . (canceled) 
     
     
         21 . At least one volatile memory device or non-volatile storage device comprising instructions to cause at least one programmable circuit to at least:
 cause a neural network to generate first optical flow data based on a first frame and a second frame, the first optical flow data having a first resolution;   generate second optical flow data based on the first frame and the second frame, the second optical flow data having a second resolution, the second resolution lower than the first resolution; and   output an optical flow map associated with the first frame and the second frame, the optical flow map based on the first optical flow data and the second optical flow data.   
     
     
         22 . The at least one volatile memory device or non-volatile storage device of  claim 21 , wherein the neural network is associated with a deep learning model. 
     
     
         23 . The at least one volatile memory device or non-volatile storage device of  claim 21 , wherein the instructions are to cause one or more of the at least one programmable circuit to generate the optical flow map based on a filter. 
     
     
         24 . The at least one volatile memory device or non-volatile storage device of  claim 23 , wherein the filter is to smooth an edge in the optical flow map. 
     
     
         25 . The at least one volatile memory device or non-volatile storage device of  claim 21 , wherein the first resolution corresponds to a resolution of the first frame and the second frame. 
     
     
         26 . The at least one volatile memory device or non-volatile storage device of  claim 21 , wherein the second optical flow data includes a motion vector map. 
     
     
         27 . The at least one volatile memory device or non-volatile storage device of  claim 21 , wherein the first frame and a second frame are consecutive frames of a video. 
     
     
         28 . An apparatus comprising:
 interface circuitry;   instructions; and   at least one programmable circuit to be programmed based on the instructions to:
 cause a neural network to generate first optical flow data based on a first frame and a second frame, the first optical flow data having a first resolution; 
 generate second optical flow data based on the first frame and the second frame, the second optical flow data having a second resolution, the second resolution lower than the first resolution; and 
 output an optical flow map associated with the first frame and the second frame, the optical flow map based on the first optical flow data and the second optical flow data. 
   
     
     
         29 . The apparatus of  claim 28 , wherein the neural network is associated with a deep learning model. 
     
     
         30 . The apparatus of  claim 28 , wherein one or more of the at least one programmable circuit is to generate the optical flow map based on a filter. 
     
     
         31 . The apparatus of  claim 30 , wherein the filter is to smooth an edge in the optical flow map. 
     
     
         32 . The apparatus of  claim 28 , wherein the first resolution corresponds to a resolution of the first frame and the second frame. 
     
     
         33 . The apparatus of  claim 28 , wherein the second optical flow data includes a motion vector map. 
     
     
         34 . The apparatus of  claim 28 , wherein the first frame and a second frame are consecutive frames of a video. 
     
     
         35 . A method comprising:
 generating, with a neural network, first optical flow data based on a first frame and a second frame, the first optical flow data having a first resolution;   generating second optical flow data based on the first frame and the second frame, the second optical flow data having a second resolution, the second resolution lower than the first resolution; and   outputting an optical flow map associated with the first frame and the second frame, the optical flow map based on the first optical flow data and the second optical flow data.   
     
     
         36 . The method of  claim 35 , wherein the neural network is associated with a deep learning model. 
     
     
         37 . The method of  claim 35 , including generating the optical flow map based on a filter. 
     
     
         38 . The method of  claim 37 , wherein the filter is to smooth an edge in the optical flow map. 
     
     
         39 . The method of  claim 35 , wherein the first resolution corresponds to a resolution of the first frame and the second frame. 
     
     
         40 . The method of  claim 35 , wherein the second optical flow data includes a motion vector map.