Method and device for image encoding/decoding using arbitrary pixels in a sub-block

Abstract

A video encoding/decoding method and apparatus using arbitrary pixels in each sub-block are disclosed. The present disclosure comprises an extractor, a first predictor, a first subtractor, a first transformer, a first quantizer, a first inverse quantizer, a first inverse transformer, a first adder, a second predictor, a second subtractor, a combiner for combining arbitrary residual pixels, a second transformer, a second quantizer, and an encoder whereby arbitrary pixels may be extracted from the respective sub-blocks of a current block and then encoded and decoded into reconstructed arbitrary pixels, which are used to predict and encode the remaining pixels of the respective sub-blocks. According to the disclosed encoding and decoding of a video, an intra prediction of pixels in a block to currently encode or decode is improved in accuracy to increase the video coding efficiency.

1 . An apparatus for encoding a video comprising:
 an extractor for extracting arbitrary pixels from each of sub-blocks of a current block of the video and generating an arbitrary pixel block;   a first predictor for predicting the arbitrary pixel block and generating a predicted arbitrary pixel block;   a first subtractor for subtracting the predicted arbitrary pixel block from the arbitrary pixel block and generating an arbitrary pixel residual block;   a first transformer for transforming the arbitrary pixel residual block;   a first quantizer for performing quantization with respect to the transformed arbitrary pixel residual block;   a first inverse quantizer for performing inverse quantization with respect to the quantized arbitrary pixel residual block;   a first inverse transformer for performing an inverse transform with respect to the inverse quantized arbitrary pixel residual block;   a first adder for adding the inverse transformed arbitrary pixel residual block to the predicted arbitrary pixel block and generating a reconstructed arbitrary pixel block;   a second predictor for predicting the remaining pixels in the respective sub-blocks by using adjacent pixels to the current block and each of reconstructed arbitrary pixels in the reconstructed arbitrary pixel block and generating predicted blocks;   a second subtractor for subtracting the predicted blocks from the current block and generating residual blocks;   a combiner for combining each of inverse transformed arbitrary residual pixels in the inverse transformed arbitrary pixel residual block to the corresponding positions of the residual block;   a second transformer for transforming the residual blocks having the respective inverse transformed arbitrary residual pixels combined;   a second quantizer for performing quantization with respect to the transformed residual blocks; and   an encoder for encoding the quantized residual blocks and generating a bitstream.   
     
     
         2 . The apparatus of  claim 1 , wherein the second transformer uses basic vectors determined according to the sequence of a first one-dimensional transform direction decided according to a prediction direction of the first predictor. 
     
     
         3 . The apparatus of  claim 2 , wherein the second transformer, when applying a horizontal one-dimensional transform first and then a vertical one-dimensional transform, performs the horizontal one-dimensional transform with respect to respective 4×4 blocks using an existing basic vector in their first to third rows and a modified basic vector in their fourth rows, and performs the vertical one-dimensional transform with respect to the respective 4×4 blocks using the modified basic vector in their first columns and the modified basic vector in their second to fourth columns. 
     
     
         4 . The apparatus of  claim 2 , wherein the second transformer, when applying a vertical one-dimensional transform first and then a horizontal one-dimensional transform, performs the vertical one-dimensional transform with respect to respective 4×4 blocks using an existing basic vector in their first to third columns and a modified basic vector in their fourth columns, and performs the horizontal one-dimensional transform with respect to the respective 4×4 blocks using the modified basic vector in their first rows and the modified basic vector in their second to fourth rows. 
     
     
         5 . The apparatus of  claim 1 , further comprising:
 a second inverse quantizer for performing inverse quantization with respect to the quantized residual block;   a second inverse transformer for performing inverse transform with respect to the inverse quantized residual block;   a second adder for adding the inverse transformed residual block to the predicted block and generating a reconstructed current block;   and a combiner for combining the respective reconstructed arbitrary pixels in the reconstructed arbitrary pixel block to the corresponding positions of the reconstructed current block.   
     
     
         6 . The apparatus of  claim 5 , wherein the second inverse transformer uses basic vectors determined according to the sequence of a first one-dimensional inverse transform direction. 
     
     
         7 . The apparatus of  claim 6 , wherein the second inverse transformer decides the sequence of the first one-dimensional inverse transform direction in the reverse order of the sequence of the first one-dimensional transform direction decided at the second transformer. 
     
     
         8 . The apparatus of  claim 6 , wherein the second inverse transformer, when applying a horizontal one-dimensional inverse transform first and then a vertical one-dimensional inverse transform, performs the horizontal one-dimensional inverse transform with respect to respective 4×4 blocks using a modified basic vector in their first rows and an existing basic vector in their second to fourth rows, and performs the vertical one-dimensional inverse transform with respect to the respective 4×4 blocks using the existing basic vector in their first to third columns and the modified basic vector in their fourth columns. 
     
     
         9 . The apparatus of  claim 6 , wherein the second inverse transformer, when applying a vertical one-dimensional inverse transform first and then a horizontal one-dimensional inverse transform, performs the vertical one-dimensional inverse transform with respect to respective 4×4 blocks using a modified basic vector in their first columns and an existing basic vector in their second to fourth columns, and performs the horizontal one-dimensional inverse transform with respect to the respective 4×4 blocks using the existing basic vector in their first to third rows and the modified basic vector in their fourth rows. 
     
     
         10 . An apparatus for encoding a video comprising:
 an extractor for extracting arbitrary pixels from each of sub-blocks of a current block of the video and generating an arbitrary pixel block;   a first predictor for predicting the arbitrary pixel block and generating a predicted arbitrary pixel block;   a first subtractor for subtracting the predicted arbitrary pixel block from the arbitrary pixel block and generating an arbitrary pixel residual block;   a first transformer for transforming the arbitrary pixel residual block;   a first quantizer for performing quantization with respect to the transformed arbitrary pixel residual block;   a first inverse quantizer for performing inverse quantization with respect to the quantized arbitrary pixel residual block;   a first inverse transformer for performing inverse transform with respect to the inverse quantized arbitrary pixel residual block;   a first adder for adding the inverse transformed arbitrary pixel residual block to the predicted arbitrary pixel block and generating a reconstructed arbitrary pixel block;   a second predictor for predicting the remaining pixels in the respective sub-blocks by using adjacent pixels to the respective sub-blocks and each of reconstructed arbitrary pixels in the reconstructed arbitrary pixel block and generating respective predicted sub-blocks;   a second subtractor for subtracting the respective predicted sub-blocks from the respective sub-blocks and generating respective residual sub-blocks;   a combiner for combining each of inverse transformed arbitrary residual pixels in the inverse transformed arbitrary pixel residual block to the corresponding positions of the respective residual sub-blocks;   a second transformer for transforming the respective residual sub-blocks having the respective inverse transformed arbitrary residual pixels combined;   a second quantizer for performing quantization with respect to the respective transformed residual sub-blocks; and   an encoder for encoding the respective quantized residual sub-blocks and generating a bitstream,   wherein encoding and decoding with respect to a sub-block are carried out after encoding and decoding with respect to another sub-block are done.   
     
     
         11 . The apparatus of  claim 10 , wherein the sequence of encoding and decoding of the respective sub-blocks is the sequence of encoding and decoding of intra — 4×4 blocks in H.264/AVC or a raster scanning direction. 
     
     
         12 . The apparatus of  claim 10 , wherein the second predictor decides a prediction direction of each of the sub-blocks and predicts the remaining pixels in the respective sub-blocks in the decided prediction directions. 
     
     
         13 . The apparatus of  claim 12 , wherein the second predictor decides as the prediction direction the direction of one of the adjacent pixels to the respective sub-blocks toward the reconstructed arbitrary pixel with the least difference in value between the adjacent pixel. 
     
     
         14 . The apparatus of  claim 12 , wherein the second predictor generates information on a prediction mode according to the prediction direction of each of the sub-blocks and transmits the information to the encoder. 
     
     
         15 . The apparatus of  claim 1 , wherein the first transformer performs a transform and then an inverse transform with respect to each of arbitrary pixel residual signals of the arbitrary pixel residual block exclusively and independently. 
     
     
         16 . The apparatus of  claim 15 , wherein the first transformer performs the transform with respect to the respective arbitrary pixel residual signals of the arbitrary pixel residual block exclusively and independently by transforming the respective arbitrary pixel residual signals with a modified basic vector. 
     
     
         17 . The apparatus of  claim 16 , wherein the first transformer further performs Hadamard Transform after performing the transform. 
     
     
         18 . The apparatus of  claim 1 , wherein the first inverse transformer performs the inverse transform with respect to the respective arbitrary pixel residual signals exclusively and independently by inverse transforming frequency coefficients from the transform of the respective arbitrary pixel residual signals by using a modified basic vector. 
     
     
         19 . The apparatus of  claim 18 , wherein the first inverse transformer performs inverse Hadamard Transform with respect to the frequency coefficients generated from the transform of the respective arbitrary pixel residual signals and then performs the inverse transform. 
     
     
         20 . The apparatus of  claim 10 , wherein the second transformer uses basic vectors determined according to the sequence of a first one-dimensional transform direction decided according to a prediction direction of the first predictor. 
     
     
         21 . The apparatus of  claim 20 , wherein the second transformer, when applying a horizontal one-dimensional transform first and then a vertical one-dimensional transform, performs the horizontal one-dimensional transform with respect to respective 4×4 blocks using an existing basic vector in their first to third rows and a modified basic vector in their fourth rows, and performs the vertical one-dimensional transform with respect to the respective 4×4 blocks using the modified basic vector in their first columns and the modified basic vector in their second to fourth columns. 
     
     
         22 . The apparatus of  claim 20 , wherein the second transformer, when applying a vertical one-dimensional transform first and then a horizontal one-dimensional transform, performs the vertical one-dimensional transform with respect to respective 4×4 blocks using an existing basic vector in their first to third columns and a modified basic vector in their fourth columns, and performs the horizontal one-dimensional transform with respect to the respective 4×4 blocks using the modified basic vector in their first rows and the modified basic vector in their second to fourth rows. 
     
     
         23 . The apparatus of  claim 10 , further comprising:
 a second inverse quantizer for performing inverse quantization with respect to the respective quantized residual sub-blocks;   a second inverse transformer for performing inverse transform with respect to the respective inverse quantized residual sub-blocks;   a second adder for adding the respective inverse transformed residual sub-blocks to the respective predicted sub-blocks and generating corresponding reconstructed sub-blocks; and   a combiner for combining the respective reconstructed arbitrary pixels in the reconstructed arbitrary pixel block to the corresponding positions of the respective reconstructed sub-blocks,   wherein the first predictor and the second predictor store the respective sub-blocks having the respective reconstructed arbitrary pixels combined and predict the subsequent sub-blocks using the respective stored sub-blocks.   
     
     
         24 . The apparatus of  claim 23 , wherein the second inverse transformer uses basic vectors determined according to the sequence of a first one-dimensional inverse transform direction. 
     
     
         25 . The apparatus of  claim 24 , wherein the second inverse transformer decides the sequence of the first one-dimensional inverse transform direction in the reverse order of the sequence of the first one-dimensional transform direction decided at the second transformer. 
     
     
         26 . The apparatus of  claim 24 , wherein the second inverse transformer, when applying a horizontal one-dimensional inverse transform first and then a vertical one-dimensional inverse transform, performs the horizontal one-dimensional inverse transform with respect to respective 4×4 blocks using a modified basic vector in their first rows and an existing basic vector in their second to fourth rows, and performs the vertical one-dimensional inverse transform with respect to the respective 4×4 blocks using the existing basic vector in their first to third columns and the modified basic vector in their fourth columns. 
     
     
         27 . The apparatus of  claim 24 , wherein the second inverse transformer, when applying a vertical one-dimensional inverse transform first and then a horizontal one-dimensional inverse transform, performs the vertical one-dimensional inverse transform with respect to respective 4×4 blocks using a modified basic vector in their first columns and an existing basic vector in their second to fourth columns, and performs the horizontal one-dimensional inverse transform with respect to the respective 4×4 blocks using the existing basic vector in their first to third rows and the modified basic vector in their fourth rows. 
     
     
         28 . The apparatus of  claim 23 , wherein the second inverse transformer performs inverse transform with respect to frequency coefficients using the existing basic vector. 
     
     
         29 . The apparatus of  claim 23 , wherein the second inverse transformer uses some of the frequency coefficients and some of the respective reconstructed arbitrary pixels to predict the remaining frequency coefficients. 
     
     
         30 . The apparatus of  claim 1 , wherein the first predictor performs a low-pass filtering on the adjacent pixels to the current block and predicts the arbitrary pixel block using down-sampled pixels. 
     
     
         31 . The apparatus of  claim 1 , wherein the first and second predictors predict in the same direction as a prediction direction according to the prediction mode of the current block. 
     
     
         32 . A method for encoding a video comprising:
 extracting arbitrary pixels from each sub-block of a current block of the video and generating an arbitrary pixel block;   firstly predicting the arbitrary pixel block and generating a predicted arbitrary pixel block;   firstly subtracting the predicted arbitrary pixel block from the arbitrary pixel block and generating an arbitrary pixel residual block;   firstly transforming the arbitrary pixel residual block;   firstly performing quantization with respect to the transformed arbitrary pixel residual block;   firstly performing inverse quantization with respect to the quantized arbitrary pixel residual block;   firstly performing inverse transform with respect to the inverse quantized arbitrary pixel residual block;   firstly adding the inverse transformed arbitrary pixel residual block to the predicted arbitrary pixel block and generating a reconstructed arbitrary pixel block;   secondly predicting the remaining pixels in the respective sub-blocks by using adjacent pixels to the current block and each of reconstructed arbitrary pixels in the reconstructed arbitrary pixel block and generating predicted blocks;   secondly subtracting the predicted blocks from the current block and generating residual blocks;   combining each of inverse transformed arbitrary residual pixels in the inverse transformed arbitrary pixel residual block to the corresponding positions of the residual block;   secondly transforming the residual blocks having the respective inverse transformed arbitrary residual pixels combined;   secondly performing quantization with respect to the transformed residual blocks; and   encoding the quantized residual blocks and generating a bitstream.   
     
     
         33 . The method of  claim 32 , further comprising:
 secondly performing inverse quantization with respect to the quantized residual block;   secondly performing inverse transform with respect to the inverse quantized residual block;   secondly adding the inverse transformed residual block to the predicted block and generating a reconstructed current block; and   combining the respective reconstructed arbitrary pixels in the reconstructed arbitrary pixel block to the corresponding positions of the reconstructed current block.   
     
     
         34 . A method for encoding a video comprising:
 extracting arbitrary pixels from each sub-block of a current block of the video and generating an arbitrary pixel block;   firstly predicting the arbitrary pixel block and generating a predicted arbitrary pixel block;   firstly subtracting the predicted arbitrary pixel block from the arbitrary pixel block and generating an arbitrary pixel residual block;   firstly transforming the arbitrary pixel residual block;   firstly performing quantization with respect to the transformed arbitrary pixel residual block;   firstly performing inverse quantization with respect to the quantized arbitrary pixel residual block;   firstly performing inverse transform with respect to the inverse quantized arbitrary pixel residual block;   firstly adding the inverse transformed arbitrary pixel residual block to the predicted arbitrary pixel block and generating a reconstructed arbitrary pixel block;   secondly predicting the remaining pixels in the respective sub-blocks by using adjacent pixels to the respective sub-blocks and each of reconstructed arbitrary pixels in the reconstructed arbitrary pixel block and generating respective predicted sub-blocks;   secondly subtracting the respective predicted sub-blocks from the respective sub-blocks and generating respective residual sub-blocks;   combining each of inverse transformed arbitrary residual pixels in the inverse transformed arbitrary pixel residual block to the corresponding positions of the respective residual sub-blocks;   secondly transforming the respective residual sub-blocks having the respective inverse transformed arbitrary residual pixels combined;   secondly performing quantization with respect to the respective transformed residual sub-blocks; and   encoding the respective quantized residual sub-blocks and generating a bitstream,   wherein encoding and decoding with respect to a sub-block are carried out after encoding and decoding with respect to another sub-block are done.   
     
     
         35 . The method of  claim 34 , further comprising:
 secondly performing inverse quantization with respect to the respective quantized residual sub-blocks;   secondly performing inverse transform with respect to the respective inverse quantized residual sub-blocks;   secondly adding the respective inverse transformed residual sub-blocks to the respective predicted sub-blocks and generating corresponding reconstructed sub-blocks; and   combining the respective reconstructed arbitrary pixels in the reconstructed arbitrary pixel block to the corresponding positions of the respective reconstructed sub-blocks,   wherein the firstly predicting and the secondly predicting store the respective sub-blocks having the respective reconstructed arbitrary pixels combined and predict the subsequent sub-blocks using the respective stored sub-blocks.   
     
     
         36 . An apparatus for decoding a video comprising:
 a decoder for decoding a bitstream and extracting a residual block and a prediction mode;   an inverse quantizer for performing inverse quantization with respect to the residual block;   an extractor for extracting frequency coefficients corresponding to positions of arbitrary pixels in each of sub-blocks from the inverse quantized residual block;   a first inverse transformer for performing inverse transform with respect to the extracted frequency coefficients and generating an inverse transformed arbitrary pixel residual block;   a first predictor for predicting an arbitrary pixel block in a prediction direction according to the prediction mode and generating a predicted arbitrary pixel block;   a first adder for adding the inverse transformed arbitrary pixel residual block to the predicted arbitrary pixel block and generating a reconstructed arbitrary pixel block;   a second inverse transformer for performing inverse transform with respect to the inverse quantized residual block and generating an inverse transformed residual block;   a second predictor for predicting a current block of the video in a prediction direction according to the prediction mode by using adjacent pixels to the current block and each of reconstructed arbitrary pixels in the reconstructed arbitrary pixel block and generating a predicted current block;   a second adder for generating a reconstructed current block by using the inverse transformed residual block and the predicted current block; and   a combiner for combining the respective reconstructed arbitrary pixels to the corresponding positions of the reconstructed current block.   
     
     
         37 . The apparatus of  claim 36 , wherein the second inverse transformer uses basic vectors determined according to the sequence of a first one-dimensional inverse transform direction. 
     
     
         38 . The apparatus of  claim 37 , wherein the second inverse transformer decides the sequence of the first one-dimensional inverse transform direction in the reverse order of the sequence of the first one-dimensional transform direction decided at the second transformer. 
     
     
         39 . The apparatus of  claim 37 , wherein the second inverse transformer, when applying a horizontal one-dimensional inverse transform first and then a vertical one-dimensional inverse transform, performs the horizontal one-dimensional inverse transform with respect to respective 4×4 blocks using a modified basic vector in their first rows and an existing basic vector in their second to fourth rows, and performs the vertical one-dimensional inverse transform with respect to the respective 4×4 blocks using the existing basic vector in their first to third columns and the modified basic vector in their fourth columns. 
     
     
         40 . The apparatus of  claim 37 , wherein the second inverse transformer, when applying a vertical one-dimensional inverse transform first and then a horizontal one-dimensional inverse transform, performs the vertical one-dimensional inverse transform with respect to respective 4×4 blocks using a modified basic vector in their first columns and an existing basic vector in their second to fourth columns, and performs the horizontal one-dimensional inverse transform with respect to the respective 4×4 blocks using the existing basic vector in their first to third rows and the modified basic vector in their fourth rows. 
     
     
         41 . The apparatus of  claim 36 , wherein the first predictor performs a low-pass filtering on the adjacent pixels to the current block and predicts the arbitrary pixel block using down-sampled pixels. 
     
     
         42 . The apparatus of  claim 36 , wherein the first and second predictors predict in the same direction as a prediction direction according to the prediction mode of the current block. 
     
     
         43 . A method for decoding a video comprising:
 decoding a bitstream to extract a residual block and a prediction mode;   performing inverse quantization with respect to the residual block;   extracting frequency coefficients corresponding to positions of arbitrary pixels in each sub-block from the inverse quantized residual block;   firstly performing inverse transform with respect to the extracted frequency coefficients and generating a reconstructed arbitrary pixel residual block;   firstly predicting an arbitrary pixel block in a prediction direction according to the prediction mode and generating a predicted arbitrary pixel block;   firstly adding the reconstructed arbitrary pixel residual block to the predicted arbitrary pixel block and generating a reconstructed arbitrary pixel block;   secondly performing inverse transform with respect to the inverse quantized residual block and generating an inverse transformed residual block;   secondly predicting a current block of the video in a prediction direction according to the prediction mode by using adjacent pixels to the current block and each of reconstructed arbitrary pixels in the reconstructed arbitrary pixel block and generating a predicted current block;   secondly adding for generating a reconstructed current block by using the inverse transformed residual block and the predicted current block; and   combining the respective reconstructed arbitrary pixels to the corresponding positions of the reconstructed current block.   
     
     
         44 . The apparatus of  claim 10 , wherein the first transformer performs a transform and then an inverse transform with respect to each of arbitrary pixel residual signals of the arbitrary pixel residual block exclusively and independently. 
     
     
         45 . The apparatus of  claim 10 , wherein the first inverse transformer performs the inverse transform with respect to the respective arbitrary pixel residual signals exclusively and independently by inverse transforming frequency coefficients from the transform of the respective arbitrary pixel residual signals by using a modified basic vector. 
     
     
         46 . The apparatus of  claim 10 , wherein the first predictor performs a low-pass filtering on the adjacent pixels to the current block and predicts the arbitrary pixel block using down-sampled pixels.