Neural network-based picture filtering method and apparatus, device, and storage medium

Abstract

This application provides a neural network-based picture filtering method performed by an electronic device. The method includes: determining a first data processing mode for to-be-filtered information; performing, when the first data processing mode is a data rearrangement mode, data rearrangement on a reconstructed picture block in the to-be-filtered information according to the first data processing mode, and filtering rearranged to-be-filtered information by using a neural network filter, to obtain a filtered picture block of the rearranged to-be-filtered information; and performing data inverse rearrangement on the filtered picture block of the rearranged to-be-filtered information according to the first data processing mode, to obtain a filtered reconstructed picture block. In this application, data rearrangement is performed on the to-be-filtered information, so that a data distribution characteristic of the rearranged to-be-filtered information is close to a data distribution characteristic of training data, improving a filtering effect of a neural network filter.

What is claimed is: 
     
         1 . A neural network-based picture filtering method, the method comprising:
 determining a first data processing mode for to-be-filtered information, the to-be-filtered information comprising a reconstructed picture block;   when the first data processing mode is one of a plurality of data rearrangement modes, performing data rearrangement on the reconstructed picture block according to the first data processing mode, to obtain rearranged to-be-filtered information;   filtering the rearranged to-be-filtered information by using a neural network filter, to obtain a filtered picture block of the rearranged to-be-filtered information; and   performing data inverse rearrangement on the filtered picture block of the rearranged to-be-filtered information according to the first data processing mode, to obtain a filtered reconstructed picture block.   
     
     
         2 . The method according to  claim 1 , further comprising:
 when the first data processing mode is a data non-rearrangement mode, directly filtering the to-be-filtered information by using the neural network filter, to obtain the filtered reconstructed picture block.   
     
     
         3 . The method according to  claim 1 , wherein the first data processing mode is a data processing mode with a minimum filtering cost in the plurality of data rearrangement modes. 
     
     
         4 . The method according to  claim 1 , wherein the determining the first data processing mode for to-be-filtered information comprises:
 decoding a bitstream to obtain a first flag indicating whether to perform data rearrangement on the to-be-filtered information during filtering; and   determining the data non-rearrangement mode as the first data processing mode when the first flag indicates not to perform data rearrangement on the to-be-filtered information during filtering; and   when the first flag indicates to perform data rearrangement on the to-be-filtered information during filtering, decoding the bitstream to obtain a second flag indicating an index of a data rearrangement mode for the to-be-filtered information, and determining the first data processing mode from the plurality of data rearrangement modes based on the second flag.   
     
     
         5 . The method according to  claim 1 , wherein the determining the first data processing mode for to-be-filtered information comprises:
 decoding a bitstream to obtain a third flag indicating the first data processing mode; and   determining the first data processing mode from the plurality of data processing modes based on the third flag.   
     
     
         6 . The method according to  claim 1 , wherein the plurality of data rearrangement modes comprise at least one of the following: a rotation mode, a mode of horizontal flipping and then rotation, a mode of vertical flipping and then rotation, a mode of diagonal flipping and then rotation, a mode of downsampling and then rotation, and a mode of upsampling and then rotation. 
     
     
         7 . The method according to  claim 6 , wherein the rotation mode comprises any one of rotating left by (N*90)° or rotating right by (N*90)°, N being a positive integer. 
     
     
         8 . The method according to  claim 1 , wherein the to-be-filtered information further comprises at least one of a predicted picture block corresponding to the reconstructed picture block, a boundary strength picture block, a frame type of the reconstructed picture block, and quantization parameter information. 
     
     
         9 . The method according to  claim 8 , wherein the performing data rearrangement on the reconstructed picture block according to the first data processing mode, to obtain rearranged to-be-filtered information comprises:
 performing data rearrangement on the reconstructed picture block according to the first data processing mode, and performing data rearrangement on at least one of the predicted picture block, the boundary strength picture block, the frame type, and the quantization parameter information according to the first data processing mode, to obtain the rearranged to-be-filtered information.   
     
     
         10 . An electronic device, comprising a processor and a memory,
 the memory being configured to store a computer program; and   the processor being configured to execute the computer program, and cause the electronic device to implement a neural network-based picture filtering method including:   determining a first data processing mode for to-be-filtered information, the to-be-filtered information comprising a reconstructed picture block;   when the first data processing mode is one of a plurality of data rearrangement modes, performing data rearrangement on the reconstructed picture block according to the first data processing mode, to obtain rearranged to-be-filtered information;   filtering the rearranged to-be-filtered information by using a neural network filter, to obtain a filtered picture block of the rearranged to-be-filtered information; and   performing data inverse rearrangement on the filtered picture block of the rearranged to-be-filtered information according to the first data processing mode, to obtain a filtered reconstructed picture block.   
     
     
         11 . The electronic device according to  claim 10 , wherein the method further comprises:
 when the first data processing mode is a data non-rearrangement mode, directly filtering the to-be-filtered information by using the neural network filter, to obtain the filtered reconstructed picture block.   
     
     
         12 . The electronic device according to  claim 10 , wherein the first data processing mode is a data processing mode with a minimum filtering cost in the plurality of data processing modes. 
     
     
         13 . The electronic device according to  claim 10 , wherein the determining the first data processing mode for to-be-filtered information comprises:
 decoding a bitstream to obtain a first flag indicating whether to perform data rearrangement on the to-be-filtered information during filtering; and   determining the data non-rearrangement mode as the first data processing mode when the first flag indicates not to perform data rearrangement on the to-be-filtered information during filtering; and   when the first flag indicates to perform data rearrangement on the to-be-filtered information during filtering, decoding the bitstream to obtain a second flag indicating an index of a data rearrangement mode for the to-be-filtered information, and determining the first data processing mode from the plurality of data rearrangement modes based on the second flag.   
     
     
         14 . The electronic device according to  claim 10 , wherein the determining the first data processing mode for to-be-filtered information comprises:
 decoding a bitstream to obtain a third flag indicating the first data processing mode; and   determining the first data processing mode from the plurality of data processing modes based on the third flag.   
     
     
         15 . The electronic device according to  claim 10 , wherein the plurality of data rearrangement modes comprise at least one of the following: a rotation mode, a mode of horizontal flipping and then rotation, a mode of vertical flipping and then rotation, a mode of diagonal flipping and then rotation, a mode of downsampling and then rotation, and a mode of upsampling and then rotation. 
     
     
         16 . The electronic device according to  claim 15 , wherein the rotation mode comprises any one of rotating left by (N*90)° or rotating right by (N*90)°, N being a positive integer. 
     
     
         17 . The electronic device according to  claim 10 , wherein the to-be-filtered information further comprises at least one of a predicted picture block corresponding to the reconstructed picture block, a boundary strength picture block, a frame type of the reconstructed picture block, and quantization parameter information. 
     
     
         18 . The electronic device according to  claim 17 , wherein the performing data rearrangement on the reconstructed picture block according to the first data processing mode, to obtain rearranged to-be-filtered information comprises:
 performing data rearrangement on the reconstructed picture block according to the first data processing mode, and performing data rearrangement on at least one of the predicted picture block, the boundary strength picture block, the frame type, and the quantization parameter information according to the first data processing mode, to obtain the rearranged to-be-filtered information.   
     
     
         19 . A non-transitory computer-readable storage medium, configured to store a computer program and a bitstream, the computer program, when executed by a processor of a computer device, enabling the computer device to perform a neural network-based picture filtering method including:
 determining a first data processing mode for to-be-filtered information, the to-be-filtered information comprising a reconstructed picture block;   when the first data processing mode is one of a plurality of data rearrangement modes, performing data rearrangement on the reconstructed picture block according to the first data processing mode, to obtain rearranged to-be-filtered information;   filtering the rearranged to-be-filtered information by using a neural network filter, to obtain a filtered picture block of the rearranged to-be-filtered information; and   performing data inverse rearrangement on the filtered picture block of the rearranged to-be-filtered information according to the first data processing mode, to obtain a filtered reconstructed picture block.   
     
     
         20 . The non-transitory computer-readable storage medium according to  claim 19 , wherein the method further comprises:
 when the first data processing mode is a data non-rearrangement mode, directly filtering the to-be-filtered information by using the neural network filter, to obtain the filtered reconstructed picture block.