Multiple hypothesis prediction with template matching in video coding

Abstract

A method of encoding or decoding video data includes determining, for a multiple hypothesis prediction (MHP) process, a plurality of prediction templates based on a plurality of weights; comparing the plurality of prediction templates to a current template of a current block; determining a weight from the plurality of weights based on the comparing of the plurality of prediction templates to the current template of the current block; determining one or more prediction hypotheses; determining a prediction signal for the current block based on the one or more prediction hypotheses and the weight; and encoding or decoding the current block based on the prediction signal.

What is claimed is: 
     
         1 . A method of encoding or decoding video data, the method comprising:
 determining, for a multiple hypothesis prediction (MHP) process, a plurality of prediction templates based on a plurality of weights;   comparing the plurality of prediction templates to a current template of a current block;   determining a weight from the plurality of weights based on the comparing of the plurality of prediction templates to the current template of the current block;   determining one or more prediction hypotheses;   determining a prediction signal for the current block based on the one or more prediction hypotheses and the weight; and   encoding or decoding the current block based on the prediction signal.   
     
     
         2 . The method of  claim 1 , wherein determining the plurality of prediction templates based on the plurality of weights comprises:
 determining a first prediction template based on a reference template, a hypothesis template, and a first weight of the plurality of weights; and   determining a second prediction template based on the reference template, the hypothesis template, and a second weight of the plurality of weights.   
     
     
         3 . The method of  claim 2 , wherein determining the first prediction template comprises:
 T pred1 =(1−α 1 )T ref +α 1 T add , wherein T pred1  is the first prediction template, α 1  is the first weight, T ref  is the reference template, and T add  is the hypothesis template.   
     
     
         4 . The method of  claim 3 , wherein determining the second prediction template comprises:
 T pred2 =(1−α 2 )T ref +α 2 T add , wherein T pred2  is the second prediction template, α 2  is the second weight, T ref  is the reference template, and T add  is the hypothesis template.   
     
     
         5 . The method of  claim 2 , wherein the hypothesis template is a first hypothesis template, the method further comprising:
 determining a third prediction template based on the reference template, a second hypothesis template, and the first weight of the plurality of weights; and   determining a fourth prediction template based on the reference template, the second hypothesis template, and the second weight of the plurality of weights.   
     
     
         6 . The method of  claim 1 , further comprising:
 constructing a list based on the comparing of the plurality of prediction templates to the current template of the current block, and   wherein determining the weight comprises determining the weight based on an entry in the list.   
     
     
         7 . The method of  claim 1 , wherein determining the one or more prediction hypotheses comprises determining the one or more prediction hypotheses based on the comparing of the plurality of prediction templates to the current template of the current block. 
     
     
         8 . The method of  claim 1 , wherein determining the prediction signal for the current block based on the one or more prediction hypotheses and the weight comprises:
 applying the weight to at least one prediction hypothesis of the one or more prediction hypotheses; and   combining a base prediction signal and the at least one prediction hypothesis having the weight applied to determine the prediction signal for the current block.   
     
     
         9 . The method of  claim 1 , wherein encoding or decoding the current block based on the prediction signal comprises decoding the current block, wherein decoding the current block comprises:
 determining residual values indicative of a difference between the prediction signal and the current block; and   adding the residual values to the prediction signal to reconstruct the current block.   
     
     
         10 . The method of  claim 1 , wherein encoding or decoding the current block based on the prediction signal comprises encoding the current block, wherein encoding the current block comprises:
 determining residual values indicative of a difference between the prediction signal and the current block; and   signaling information indicative of the residual values.   
     
     
         11 . A device for encoding or decoding video data, the device comprising:
 one or more memories configured to store the video data; and   processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to:
 determine, for a multiple hypothesis prediction (MHP) process, a plurality of prediction templates based on a plurality of weights; 
 compare the plurality of prediction templates to a current template of a current block; 
 determine a weight from the plurality of weights based on the comparing of the plurality of prediction templates to the current template of the current block; 
 determine one or more prediction hypotheses; 
 determine a prediction signal for the current block based on the one or more prediction hypotheses and the weight; and 
 encode or decode the current block based on the prediction signal. 
   
     
     
         12 . The device of  claim 11 , wherein to determine the plurality of prediction templates based on the plurality of weights, the processing circuitry is configured to:
 determine a first prediction template based on a reference template, a hypothesis template, and a first weight of the plurality of weights; and   determine a second prediction template based on the reference template, the hypothesis template, and a second weight of the plurality of weights.   
     
     
         13 . The device of  claim 12 , wherein to determine the first prediction template, the processing circuitry is configured to determine that:
 T pred1 =(1−α 1 )T ref +α 1 T add , wherein T pred1  is the first prediction template, α 1  is the first weight, T ref  is the reference template, and T add  is the hypothesis template.   
     
     
         14 . The device of  claim 13 , wherein to determine the second prediction template, the processing circuitry is configured to determine that:
 T pred2 =(1−α 2 )T ref +α 2 T add , wherein T pred2  is the second prediction template, α 2  is the second weight, T ref  is the reference template, and T add  is the hypothesis template.   
     
     
         15 . The device of  claim 12 , wherein the hypothesis template is a first hypothesis template, and wherein the processing circuitry is configured to:
 determine a third prediction template based on the reference template, a second hypothesis template, and the first weight of the plurality of weights; and   determine a fourth prediction template based on the reference template, the second hypothesis template, and the second weight of the plurality of weights.   
     
     
         16 . The device of  claim 11 , wherein the processing circuitry is configured to:
 construct a list based on the comparing of the plurality of prediction templates to the current template of the current block; and   wherein to determine the weight, the processing circuitry is configured to determine the weight based on an entry in the list.   
     
     
         17 . The device of  claim 11 , wherein to determine the one or more prediction hypotheses, the processing circuitry is configured to determine the one or more prediction hypotheses based on the comparing of the plurality of prediction templates to the current template of the current block. 
     
     
         18 . The device of  claim 11 , wherein to determine the prediction signal for the current block based on the one or more prediction hypotheses and the weight, the processing circuitry is configured to:
 apply the weight to at least one prediction hypothesis of the one or more prediction hypotheses; and   combine a base prediction signal and the at least one prediction hypothesis having the weight applied to determine the prediction signal for the current block.   
     
     
         19 . The device of  claim 11 , wherein to encode or decode the current block based on the prediction signal, the processing circuitry is configured to decode the current block, wherein to decode the current block, the processing circuitry is configured to:
 determine residual values indicative of a difference between the prediction signal and the current block; and   add the residual values to the prediction signal to reconstruct the current block.   
     
     
         20 . A non-transitory computer-readable storage medium storing instructions thereon that when executed cause one or more processors to:
 determine, for a multiple hypothesis prediction (MHP) process, a plurality of prediction templates based on a plurality of weights;   compare the plurality of prediction templates to a current template of a current block;   determine a weight from the plurality of weights based on the comparing of the plurality of prediction templates to the current template of the current block;   determine one or more prediction hypotheses;   determine a prediction signal for the current block based on the one or more prediction hypotheses and the weight; and   encode or decode the current block based on the prediction signal.