Method and Apparatus of Matrix Based Intra Prediction in Image and Video Processing

Abstract

An image or video processing method comprises receiving input data of a current block, mapping an MIP mode index to a default mode, and encoding or decoding the current block by referencing the default mode. Another image or video processing method comprises receiving input data of a current block coded or to be coded by an MIP mode, and parsing or signaling a transpose flag for the current block indicating whether transposing is applied in coding the current block. Reference samples of the current block are prepared from neighboring boundary samples according to the transpose flag. A matrix is selected to be multiplied to the reference samples to generate predicted samples. A transposing process is adaptively applied to the current block according to the transpose flag, and the predicted samples are used to encode or decode the current block.

What is claimed is: 
     
         1 . A method of processing video data in an image or video coding system, comprising:
 receiving input data associated with a current block in a current picture;   mapping a Matrix-based Intra Prediction (MIP) mode index of an MIP block to a default mode, wherein the MIP block is encoded or decoded by an MIP mode of multiple MIP modes according to predicted samples generated by a matrix and neighboring boundary samples in the current picture, wherein each of the multiple MIP modes is mapped to the default mode; and   encoding or decoding the current block by referencing the default mode.   
     
     
         2 . The method of  claim 1 , wherein the MIP block is encoded or decoded by deriving reference samples from the neighboring boundary samples of the MIP block in the current picture, generating intermediate predicted samples by multiplying the reference samples and the matrix, clipping the intermediate predicted samples to a rational range, and up-sampling the clipped intermediate predicted samples to generate the predicted samples. 
     
     
         3 . The method of  claim 2 , wherein the step of deriving reference samples comprises down-sampling the neighboring boundary samples of the MIP block to generate the reference samples of the MIP block. 
     
     
         4 . The method of  claim 2 , wherein the step of deriving reference samples comprises extracting the reference samples by averaging the neighboring boundary samples of the MIP block. 
     
     
         5 . The method of  claim 2 , wherein the step of generating intermediate predicted samples further comprises adding an offset after multiplying the reference samples and the matrix. 
     
     
         6 . The method of  claim 1 , wherein the default mode is Planar mode or DC mode. 
     
     
         7 . The method of  claim 1 , wherein the current block is a chrominance (chroma) block coded or to be coded in Direct Mode (DM) and the MIP block is a corresponding luminance (luma) block of the current block, wherein DM derivation of the current block maps the MIP mode index of the corresponding luma block to the default mode and the current block is encoded or decoded according to the default mode. 
     
     
         8 . The method of  claim 1 , wherein the current block is coded or to be coded by intra prediction and the MIP block is a neighboring block of the current block coded by the MIP mode, wherein intra Most Probable Mode (MPM) derivation of the current block comprises mapping the MIP mode index of the neighboring block to the default mode. 
     
     
         9 . The method of  claim 1 , wherein the current block is the MIP block coded by the MIP mode and a secondary transform is applied to a transform block in the current block, and the step of encoding or decoding the current block by referencing the default mode further comprises selecting a secondary transform set according to the default mode, determining a transform matrix of the selected secondary transform set, and transforming primary transform coefficients of the transform block into secondary transform coefficients based on the transform matrix or transforming secondary transform coefficients of the transform block into primary transform coefficients based on the transform matrix. 
     
     
         10 . An apparatus for performing image or video processing, comprising: a computer processor configured for receiving input data and programming executable on the computer processor for image or video coding by performing steps comprising:
 receiving input data associated with a current block in a current picture;   mapping a Matrix-based Intra Prediction (MIP) mode index of an MIP block to a default mode, wherein the MIP block is encoded or decoded by an MIP mode of multiple MIP modes according to predicted samples generated by a matrix and neighboring boundary samples in the current picture, wherein each of the multiple MIP modes is mapped to the default mode; and   encoding or decoding the current block by referencing the default mode.   
     
     
         11 . The apparatus of  claim 10 , wherein the MIP block is encoded or decoded by deriving reference samples from the neighboring boundary samples of the MIP block in the current picture, generating intermediate predicted samples by multiplying the reference samples and the matrix, clipping the intermediate predicted samples to a rational range, and up-sampling the clipped intermediate predicted samples to generate the predicted samples. 
     
     
         12 . The apparatus of  claim 11 , wherein the step of deriving reference samples comprises down-sampling the neighboring boundary samples of the MIP block to generate the reference samples of the MIP block. 
     
     
         13 . The apparatus of  claim 11 , wherein the step of deriving reference samples comprises extracting the reference samples by averaging the neighboring boundary samples of the MIP block. 
     
     
         14 . The apparatus of  claim 11 , wherein the step of generating intermediate predicted samples further comprises adding an offset after multiplying the reference samples and the matrix. 
     
     
         15 . The apparatus of  claim 10 , wherein the default mode is Planar mode or DC mode. 
     
     
         16 . The apparatus of  claim 10 , wherein the current block is a chrominance (chroma) block coded or to be coded in Direct Mode (DM) and the MIP block is a corresponding luminance (luma) block of the current block, wherein DM derivation of the current block maps the MIP mode index of the corresponding luma block to the default mode and the current block is encoded or decoded according to the default mode. 
     
     
         17 . The apparatus of  claim 10 , wherein the current block is coded or to be coded by intra prediction and the MIP block is a neighboring block of the current block coded by the MIP mode, wherein intra Most Probable Mode (MPM) derivation of the current block comprises mapping the MIP mode index of the neighboring block to the default mode. 
     
     
         18 . The apparatus of  claim 10 , wherein the current block is the MIP block coded by the MIP mode and a secondary transform is applied to a transform block in the current block, and the step of encoding or decoding the current block by referencing the default mode further comprises selecting a secondary transform set according to the default mode, determining a transform matrix of the selected secondary transform set, and transforming primary transform coefficients of the transform block into secondary transform coefficients based on the transform matrix or transforming secondary transform coefficients of the transform block into primary transform coefficients based on the transform matrix. 
     
     
         19 . A non-transitory computer-readable medium storing instructions which when executed by a computer cause the computer to perform steps comprising:
 receiving input data associated with a current block in a current picture;   mapping a Matrix-based Intra Prediction (MIP) mode index of an MIP block to a default mode, wherein the MIP block is encoded or decoded by an MIP mode of multiple MIP modes according to predicted samples generated by a matrix and neighboring boundary samples in the current picture, wherein each of the multiple MIP modes is mapped to the default mode; and   encoding or decoding the current block by referencing the default mode.   
     
     
         20 . The non-transitory computer-readable medium of  claim 19 , wherein the MIP block is encoded or decoded by deriving reference samples from the neighboring boundary samples of the MIP block in the current picture, generating intermediate predicted samples by multiplying the reference samples and the matrix, clipping the intermediate predicted samples to a rational range, and up-sampling the clipped intermediate predicted samples to generate the predicted samples.