Merge mode by using subblock-based motion vector predictor lookahead and/or lookbehind

Abstract

A video bitstream is received. The video bitstream includes coded information of a current block in a current picture and of a plurality of reference pictures of the current picture in a reference list. The current block includes a plurality of subblocks. A reference block of the current block is determined. The reference block includes a plurality of reference subblocks corresponding to the plurality of subblocks of the current block. A subblock-level motion vector (MV) is determined for a subblock of the plurality of subblocks of the current block. The subblock-level MV is associated with a sum of a plurality of intermediate vectors. Each of the plurality of intermediate vectors is defined between two different respective pictures of the plurality of reference pictures. The subblock of the plurality of subblocks of the current block is reconstructed based on the subblock-level MV.

What is claimed is: 
     
         1 . A method of video decoding, the method comprising:
 receiving a video bitstream including coded information of a current block in a current picture and of a plurality of reference pictures of the current picture in a reference list, the current block including a plurality of subblocks;   determining a reference block of the current block, the reference block including a plurality of reference subblocks corresponding to the plurality of subblocks of the current block;   determining a subblock-level motion vector (MV) for a subblock of the plurality of subblocks of the current block, the subblock-level MV being associated with a sum of a plurality of intermediate vectors, each of the plurality of intermediate vectors being defined between two different respective pictures of the plurality of reference pictures; and   reconstructing the subblock of the plurality of subblocks of the current block based on the subblock-level MV.   
     
     
         2 . The method of  claim 1 , wherein the plurality of intermediate vectors includes:
 a first intermediate MV from a first one of the plurality of reference subblocks positioned in a first one of the plurality of reference pictures to a first intermediate prediction subblock positioned in a second one of the plurality of reference pictures, and   a second intermediate MV from the first intermediate prediction subblock positioned in the second one of the plurality of reference pictures to a second intermediate prediction subblock positioned in a third one of the plurality of reference pictures.   
     
     
         3 . The method of  claim 2 , wherein:
 the first intermediate prediction subblock is an intermediate prediction subblock of the first one of the plurality of reference subblocks, and   the second intermediate prediction subblock is a prediction subblock of the first intermediate prediction subblock.   
     
     
         4 . The method of  claim 1 , wherein:
 the determining the reference block includes determining the reference block in a first one of the plurality of reference pictures according to a first intermediate MV, and   the plurality of intermediate vectors includes (i) the first intermediate MV from the current block to the reference block in the first one of the plurality of reference pictures, and (ii) a second intermediate MV from a first one of the plurality of reference subblocks to a first intermediate prediction subblock positioned in a second one of the plurality of reference pictures.   
     
     
         5 . The method of  claim 1 , wherein:
 the determining the reference block includes determining the reference block as a reconstructed block in the current picture indicated by a block vector (BV), and   the plurality of intermediate vectors includes (i) the BV from the current block to the reference block in the current picture, and (ii) a first intermediate MV from a first one of the plurality of reference subblocks to a first intermediate prediction subblock positioned in a first one of the plurality of reference pictures.   
     
     
         6 . The method of  claim 1 , wherein the reference list is one of a forward reference list and a backward reference list with respect to the current picture. 
     
     
         7 . The method of  claim 1 , wherein the determining the reference block further comprises:
 deriving a displacement vector (DV) based on a MV of a spatial neighboring coded block of the current block; and   determining the reference block indicated by the DV in a first one of the plurality of reference pictures.   
     
     
         8 . The method of  claim 1 , further comprising:
 determining a spatial neighboring coded block of the current block;   determining a motion vector from the spatial neighboring coded block to a reference block of the spatial neighboring coded block in one of the plurality of reference pictures;   determining a collocated block of the current block in the one of the plurality of reference pictures based on the motion vector, the collocated block including a plurality of collocated subblocks corresponding to the plurality of subblocks of the current block; and   determining a subblock-based MV for the subblock of the plurality of subblocks of the current block based on a MV of one of the plurality of collocated subblocks that corresponds to the subblock of the current block.   
     
     
         9 . The method of  claim 8 , further comprising:
 constructing a merge candidate list including merge candidates indicated by the subblock-level MV and the subblock-based MV; and   reordering the merge candidates based on template costs of the merge candidates.   
     
     
         10 . The method of  claim 1 , wherein a total number of the plurality of intermediate vectors is defined according to a maximum trace depth. 
     
     
         11 . A method of video encoding, the method comprising:
 determining a reference block of a current block in a current picture, the reference block including a plurality of reference subblocks corresponding to a plurality of subblocks of the current block;   determining a subblock-level motion vector (MV) for a subblock of the plurality of subblocks of the current block, the subblock-level MV being associated with a sum of a plurality of intermediate vectors, each of the plurality of intermediate vectors being defined between two different respective pictures of a plurality of reference pictures in a reference list; and   encoding the subblock of the plurality of subblocks of the current block based on the subblock-level MV.   
     
     
         12 . The method of  claim 11 , wherein the plurality of intermediate vectors includes:
 a first intermediate MV from a first one of the plurality of reference subblocks positioned in a first one of the plurality of reference pictures to a first intermediate prediction subblock positioned in a second one of the plurality of reference pictures, and   a second intermediate MV from the first intermediate prediction subblock positioned in the second one of the plurality of reference pictures to a second intermediate prediction subblock positioned in a third one of the plurality of reference pictures.   
     
     
         13 . The method of  claim 12 , wherein:
 the first intermediate prediction subblock is an intermediate prediction subblock of the first one of the plurality of reference subblocks, and   the second intermediate prediction subblock is a prediction subblock of the first intermediate prediction subblock.   
     
     
         14 . The method of  claim 11 , wherein:
 the determining the reference block includes determining the reference block in a first one of the plurality of reference pictures according to a first intermediate MV, and   the plurality of intermediate vectors includes (i) the first intermediate MV from the current block to the reference block in the first one of the plurality of reference pictures, and (ii) a second intermediate MV from a first one of the plurality of reference subblocks to a first intermediate prediction subblock positioned in a second one of the plurality of reference pictures.   
     
     
         15 . The method of  claim 11 , wherein:
 the determining the reference block includes determining the reference block as a reconstructed block in the current picture indicated by a block vector (BV), and   the plurality of intermediate vectors includes (i) the BV from the current block to the reference block in the current picture, and (ii) a first intermediate MV from a first one of the plurality of reference subblocks to a first intermediate prediction subblock positioned in a first one of the plurality of reference pictures.   
     
     
         16 . The method of  claim 11 , wherein the reference list is one of a forward reference list and a backward reference list with respect to the current picture. 
     
     
         17 . The method of  claim 11 , wherein the determining the reference block further comprises:
 deriving a displacement vector (DV) based on a MV of a spatial neighboring coded block of the current block; and   determining the reference block indicated by the DV in a first one of the plurality of reference pictures.   
     
     
         18 . The method of  claim 11 , further comprising:
 determining a spatial neighboring coded block of the current block;   determining a motion vector from the spatial neighboring coded block to a reference block of the spatial neighboring coded block in one of the plurality of reference pictures;   determining a collocated block of the current block in the one of the plurality of reference pictures based on the motion vector, the collocated block including a plurality of collocated subblocks corresponding to the plurality of subblocks of the current block; and   determining a subblock-based MV for the subblock of the plurality of subblocks of the current block based on a MV of one of the plurality of collocated subblocks that corresponds to the subblock of the current block.   
     
     
         19 . The method of  claim 18 , further comprising:
 constructing a merge candidate list including merge candidates indicated by the subblock-level MV and the subblock-based MV; and   reordering the merge candidates based on template costs of the merge candidates.   
     
     
         20 . A method of processing visual media data, the method comprising:
 processing a bitstream of the visual media data according to a format rule, wherein:   the bitstream includes coded information of a current block in a current picture and of a plurality of reference pictures of the current picture in a reference list, the current block including a plurality of subblocks; and   the format rule specifies that:
 a reference block of the current block is determined, the reference block including a plurality of reference subblocks corresponding to the plurality of subblocks of the current block; 
 a subblock-level motion vector (MV) is determined for a subblock of the plurality of subblocks of the current block, the subblock-level MV being associated with a sum of a plurality of intermediate vectors, each of the plurality of intermediate vectors being defined between two different respective pictures of the plurality of reference pictures; and 
 the subblock of the plurality of subblocks of the current block is processed based on the subblock-level MV.