Moving image encoding apparatus and moving image decoding apparatus

Abstract

When a background interpolation-judging section 107 judges that parts areas of a lower layer and an upper layer move significantly, and a visual effect can be obtained by the background interpolation, a controller 103 controls a switch 104 based on the area information of the parts area of the lower layer to perform the background interpolation. A weighted average section 106 takes an average by weighting to the upper layer and the lower layer by using the interpolated background to generate an interpolated image. By using this interpolated image, the prediction-encoding/decoding of the upper layer is performed. On the contrary, when the background interpolation-judging section 107 judges that parts areas do not move so significantly, and a visual effect cannot be obtained by the background interpolation, a background is synthesized by using the output of the interpolated image-forming section 105 to generate an interpolated image. Hence, the shape information for the background interpolation is reduced and high quality of an image can be obtained.

1 . A moving image encoding apparatus which separates one moving image sequence to a lower layer having a low frame rate and an upper layer having a high frame rate, encodes a shape of a parts area for synthesizing said lower layer, on the condition that there is no frame of the lower layer corresponding to the upper layer, and encodes the upper layer by prediction, wherein 
 when the parts area of the lower layer appearing as a background is larger than a predetermined threshold, the moving image encoding apparatus encodes the shape of the parts area, and synthesizes a frame obtained by taking the average by weighting the lower layer and a frame of the lower layer to generate image information, and when said parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the moving image encoding apparatus does not encode the shape of the parts area and generates image information by the weighted average of the lower layer.    
     
     
         2 . A moving image decoding apparatus which synthesizes a lower layer having a low frame rate, on the condition that there is no frame corresponding to an upper layer having a high frame rate, decodes said upper layer by prediction, and superimposes said prediction-decoded upper layer on said lower layer to decode them into one moving image sequence, wherein 
 when the shape of a parts area has been encoded, the moving image decoding apparatus decodes the shape of said parts area, and synthesizes a frame obtained by taking the average by weighting the lower layer and a frame of the lower layer to generate image information, and when the shape of parts area has not been encoded, the moving image decoding apparatus generates said image information by the weighted average of the lower layer.    
     
     
         3 . A moving image encoding apparatus which separates one moving image sequence to a lower layer having a low frame rate and an upper layer having a high frame rate, encodes a shape of a parts area for synthesizing said lower layer, on the condition that there is no frame of the lower layer corresponding to the upper layer, and encodes the upper layer by prediction, wherein 
 the moving image encoding apparatus interpolates a pixel value within the overlapping area of the lower layer, using a pixel value in the periphery of the area appearing as a background, to generate image information.    
     
     
         4 . A moving image decoding apparatus which synthesizes a lower layer having a low frame rate, on the condition that there is no lower layer frame corresponding to an upper layer having a high frame rate, decodes said upper layer by prediction, and superimposes said prediction-decoded upper layer on said lower layer to decode them into one moving image sequence, wherein 
 the moving image decoding apparatus interpolates a pixel value within the overlapping area, using a pixel value in the periphery of the overlapping area of the parts area of the lower layer appearing as a background, to generate image information.    
     
     
         5 . A moving image encoding apparatus according to  claim 1 , wherein the pixel value within said parts area is interpolated by using a pixel value in the periphery of the parts area of the lower layer, and image information is generated by using the interpolated lower layer frame.  
     
     
         6 . A moving image decoding apparatus according to  claim 2 , wherein the pixel value within said parts area is interpolated by using a pixel value in the periphery of the parts area of the lower layer, and image information is generated by using the interpolated lower layer frame.  
     
     
         7 . A moving image encoding apparatus according to  claim 1 , wherein 
 in the case where the number of frames of the lower layer required for the synthesis of the lower layer is not satisfied, the image information is generated by using a frame obtained by interpolating the parts area of said lower layer.    
     
     
         8 . A moving image decoding apparatus according to  claim 2 , wherein 
 in the case where the number of frames of the lower layer required for the synthesis of the lower layer is not satisfied, the image information is generated by using a frame obtained by interpolating the parts area of the lower layer.    
     
     
         9 . A moving image encoding apparatus according to  claim 1 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the parts area of the lower layer appearing as a background of any one of the plurality of frames of the upper layer is larger than the predetermined threshold, the shape of the parts area for synthesizing the lower layer frame is encoded with respect to said plurality of the upper layer frames.    
     
     
         10 . A moving image encoding apparatus according to  claim 3 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the parts area of the lower layer appearing as a background of any one of the plurality of frames of the upper layer is larger than the predetermined threshold, the shape of the parts area for synthesizing the lower layer frame is encoded with respect to said plurality of the upper layer frames.    
     
     
         11 . A moving image encoding apparatus according to  claim 5 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the parts area of the lower layer appearing as a background of any one of the plurality of frames of the upper layer is larger than the predetermined threshold, the shape of the parts area for synthesizing the lower layer frame is encoded with respect to said plurality of the upper layer frames.    
     
     
         12 . A moving image encoding apparatus according to  claim 7 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the parts area of the lower layer appearing as a background of any one of the plurality of frames of the upper layer is larger than the predetermined threshold, the shape of the parts area for synthesizing the lower layer frame is encoded with respect to said plurality of the upper layer frames.    
     
     
         13 . A moving image decoding apparatus according to  claim 2 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the shape of the parts area for synthesizing the said lower layer frame is encoded with respect to any one of said plurality of frames of the upper layer, a frame obtained by taking the average by weighting the lower layer and a frame of the lower layer are synthesized with respect to all of the plurality of frames of the upper layer to generate image information.    
     
     
         14 . A moving image decoding apparatus according to  claim 4 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the shape of the parts area for synthesizing the said lower layer frame is encoded with respect to any one of said plurality of frames of the upper layer, a frame obtained by taking the average by weighting the lower layer and a frame of the lower layer are synthesized with respect to all of the plurality of frames of the upper layer to generate image information.    
     
     
         15 . A moving image decoding apparatus according to  claim 6 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the shape of the parts area for synthesizing the said lower layer frame is encoded with respect to any one of said plurality of frames of the upper layer, a frame obtained by taking the average by weighting the lower layer and a frame of the lower layer are synthesized with respect to all of the plurality of frames of the upper layer to generate image information.    
     
     
         16 . A moving image decoding apparatus according to  claim 8 , wherein 
 in the case where there are a plurality of frames of the upper layer between two adjacent frames of the lower layer, when the shape of the parts area for synthesizing the said lower layer frame is encoded with respect to any one of said plurality of frames of the upper layer, a frame obtained by taking the average by weighting the lower layer and a frame of the lower layer are synthesized with respect to all of the plurality of frames of the upper layer to generate image information.    
     
     
         17 . A moving image encoding apparatus according to  claim 1 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         18 . A moving image encoding apparatus according to  claim 3 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         19 . A moving image encoding apparatus according to  claim 5 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         20 . A moving image encoding apparatus according to  claim 7 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         21 . A moving image encoding apparatus according to  claim 9 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         22 . A moving image encoding apparatus according to  claim 10 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         23 . A moving image encoding apparatus according to  claim 11 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         24 . A moving image encoding apparatus according to  claim 12 , wherein 
 when the parts area of the lower layer appearing as a background is smaller than the predetermined threshold, the parts area of the upper layer is expanded by using the parts area of the lower layer to generate a parts area of image information.    
     
     
         25 . A moving image decoding apparatus according to  claim 2 , wherein 
 when the lower layer frame is synthesized, a pixel value of one of the lower layer frames which exist front and behind of a frame of the lower layer is used, with respect to an area where the first parts area and the second parts area overlap, or an area which is neither the first parts area nor the second parts area.    
     
     
         26 . A moving image decoding apparatus according to  claim 6 , wherein 
 when the lower layer frame is synthesized, a pixel value of one of the lower layer frames which exist front and behind of a frame of the lower layer is used, with respect to an area where the first parts area and the second parts area overlap, or an area which is neither the first parts area nor the second parts area.    
     
     
         27 . A moving image decoding apparatus according to  claim 2 , wherein 
 when the lower layer frame is synthesized, a pixel value of one of the lower layer frames which exist front and behind of a frame of the lower layer is used, with respect to an area where the first parts area and the second parts area overlap, or an area which is neither the first parts area nor the second parts area, and at the time of the synthesis, interpolation is performed by using a pixel value of a frame of said lower layer with respect to a pixel value outside of one parts area of the lower layer, and using a pixel value in the periphery of said parts area with respect to a pixel value inside of said one parts image of the lower layer.    
     
     
         28 . A moving image decoding apparatus according to  claim 4 , wherein 
 when the lower layer frame is synthesized, a pixel value of one of the lower layer frames which exist front and behind of a frame of the lower layer is used, with respect to an area where the first parts area and the second parts area overlap, or an area which is neither the first parts area nor the second parts area, and at the time of the synthesis, interpolation is performed by using a pixel value of a frame of said lower layer with respect to a pixel value outside of one parts area of the lower layer, and using a pixel value in the periphery of said parts area with respect to a pixel value inside of said one parts image of the lower layer.    
     
     
         29 . A moving image decoding apparatus according to  claim 6 , wherein 
 when the lower layer frame is synthesized, a pixel value of one of the lower layer frames which exist front and behind of a frame of the lower layer is used, with respect to an area where the first parts area and the second parts area overlap, or an area which is neither the first parts area nor the second parts area, and at the time of the synthesis, interpolation is performed by using a pixel value of a frame of said lower layer with respect to a pixel value outside of one parts area of the lower layer, and using a pixel value in the periphery of said parts area with respect to a pixel value inside of said one parts image of the lower layer.    
     
     
         30 . A moving image decoding apparatus according to  claim 25 , wherein 
 when the lower layer frame is synthesized, a pixel value of one of the lower layer frames which exist front and behind of a frame of the lower layer is used, with respect to an area where the first parts area and the second parts area overlap, or an area which is neither the first parts area nor the second parts area, and at the time of the synthesis, interpolation is performed by using a pixel value of a frame of said lower layer with respect to a pixel value outside of one parts area of the lower layer, and using a pixel value in the periphery of said parts area with respect to a pixel value inside of said one parts image of the lower layer.    
     
     
         31 . A moving image decoding apparatus according to  claim 26 , wherein 
 when the lower layer frame is synthesized, a pixel value of one of the lower layer frames which exist front and behind of a frame of the lower layer is used, with respect to an area where the first parts area and the second parts area overlap, or an area which is neither the first parts area nor the second parts area, and at the time of the synthesis, interpolation is performed by using a pixel value of a frame of said lower layer with respect to a pixel value outside of one parts area of the lower layer, and using a pixel value in the periphery of said parts area with respect to a pixel value inside of said one parts image of the lower layer.