Self-adaptive prediction method for multi-layer codec

Abstract

Relatively low dynamic range images or image partitions are converted into relatively high dynamic range images or image partitions that comprise reconstructed pixel values having a higher dynamic range than pixel values of the relatively low dynamic range images. Information relating to reconstructed pixel values of the relatively high dynamic range images and pixel values of the relatively low dynamic range images is collected. Prediction parameters are derived from the collected information. A predicted image or image partition is predicted from a relatively low dynamic range image or image partition based on the prediction parameters and comprises predicted pixel values having the higher dynamic range than pixel values of the relatively low dynamic range image or image partition.

The invention claimed is: 
     
       1. A method, comprising:
 generating, by a video decoder based on a number of previously decoded relatively low dynamic range images and the same number of previously decoded relatively high dynamic range residual images, the same number of previously reconstructed relatively high dynamic range images, wherein the number is one or more, wherein each image in the previously reconstructed relatively high dynamic range images is generated based on a respective corresponding image in the previously decoded relatively low dynamic range images and a respective corresponding image in the previously decoded relatively high dynamic range residual images, and wherein each image in the previously reconstructed relatively high dynamic range images comprises reconstructed pixel values that have a higher dynamic range than pixel values of the respective corresponding image in the previously decoded relatively low dynamic range images; 
 collecting, by the video decoder, previous image statistic information relating to reconstructed pixel values of each image in the previously reconstructed relatively high dynamic range images and pixel values of the respective corresponding image in the previously decoded relatively low dynamic range images, wherein the collected previous image statistic information includes high dynamic range statistics of the reconstructed pixel values of each image in the one or more relatively high dynamic range images; 
 deriving, by the video decoder from the collected previous image statistic information including the high dynamic range statistics of the reconstructed pixel values of each image in the previously reconstructed relatively high dynamic range images, one or more current prediction parameters for a currently decoded relatively low dynamic range image that is decoded subsequent to the previously decoded relatively low dynamic range images; 
 generating, by the video decoder, a currently predicted image from the currently decoded relatively low dynamic range image based on the one or more current prediction parameters derived from the collected previous image statistic information, the currently predicted image comprising currently predicted pixel values that have a higher dynamic range than currently decoded pixel values of the currently decoded relatively low dynamic range image and that are predicted from the currently decoded pixel values of the currently decoded relatively low dynamic range image; 
 decompressing, by the video decoder, a portion of enhancement layer (EL) image data in a multi-layer video signal into currently decoded residual pixel values; and 
 combining, by the video decoder, the currently decoded residual pixel values with the currently predicted pixel values of the currently predicted image to generate a currently reconstructed relatively high dynamic range image, the currently reconstructed image comprising currently reconstructed pixel values that have the higher dynamic range than the currently decoded pixel values of the currently decoded relatively low dynamic range image. 
 
     
     
       2. The method of  claim 1 , further comprising:
 generating, based on one or more relatively low dynamic range image partitions of a specific image in the previously decoded relatively low dynamic range images and relatively high dynamic range residual image partition of a specific residual image in the previously decoded residual image, one or more relatively high dynamic range image partitions of a specific reconstructed image in the previously reconstructed relatively high dynamic range images; 
 collecting partition-based information relating to reconstructed pixel values of each image partition in the one or more relatively high dynamic range image partitions of the specific reconstructed image and pixel values of a corresponding image partition in the one or more relatively low dynamic range image partitions of the specific image; 
 deriving, from the collected partition-based information, one or more partition-based prediction parameters for a second image, wherein the second image is among the previously decoded relatively low dynamic range images and is subsequent in time to the specific image in the previously decoded relatively low dynamic range images; and 
 generating a specific predicted image from the second image based on the one or more partition-based prediction parameters derived from the collected partition-based information. 
 
     
     
       3. The method as recited in  claim 1 , wherein the currently decoded relatively low dynamic range image and the previously decoded one or more relatively low dynamic range images are in a group of relatively low dynamic range images representing a scene of video. 
     
     
       4. The method as recited in  claim 1 , wherein the previously decoded relatively low dynamic range images comprise an initial relatively low dynamic range image in a group of relatively low dynamic range images representing a scene of video; further comprising generating an initial predicted image from the initial relatively low dynamic range image based on one or more default prediction parameters, the initial predicted image comprising initial predicted pixel values that have the higher dynamic range than initial pixel values of the initial relatively low dynamic range image, wherein the default prediction parameters are predefined prediction parameters. 
     
     
       5. The method as recited in  claim 1 , wherein the previously decoded relatively low dynamic range images comprise an initial relatively low dynamic range image in a group of relatively low dynamic range images representing a scene of video; further comprising:
 selecting a set of default prediction parameters from one or more sets of default prediction parameters; and 
 generating an initial predicted image from the initial relatively low dynamic range image based on the selected set of default prediction parameters, the initial predicted image comprising initial predicted pixel values that have the higher dynamic range than initial pixel values of the initial relatively low dynamic range image, 
 wherein the default prediction parameters are predefined prediction parameters. 
 
     
     
       6. The method as recited in  claim 1 , wherein the previously decoded relatively low dynamic range images comprise an initial relatively low dynamic range image in a group of relatively low dynamic range images representing a scene of video; further comprising:
 determining non-default prediction parameters for the initial relatively low dynamic range image; 
 generating an initial predicted image from the initial relatively low dynamic range image based on the non-default prediction parameters, the initial predicted image comprising initial predicted pixel values that have the higher dynamic range than initial pixel values of the initial relatively low dynamic range image. 
 
     
     
       7. The method as recited in  claim 1 , further comprising:
 collecting information relating to the currently reconstructed pixel values of the currently reconstructed relatively high dynamic range image and the pixel values of the currently decoded relatively low dynamic range image. 
 
     
     
       8. The method as recited in  claim 1 , wherein the collected previous image statistic information is stored in a look back window that stores information relating to (a) all reconstructed relatively high dynamic range images in a scene of video preceding the currently reconstructed relatively high dynamic range image converted from the currently decoded relatively low dynamic range image and (b) all relatively low dynamic range images in the scene of video preceding the currently decoded relatively low dynamic range image. 
     
     
       9. The method as recited in  claim 1 , wherein the collected previous image statistic information is stored in a sliding window that stores information relating to (a) up to a fixed number of reconstructed relatively high dynamic range images in a scene of video most recently preceding the currently reconstructed relatively high dynamic range image converted from the currently decoded relatively low dynamic range image and (b) up to the fixed number of relatively low dynamic range images in the scene of video most recently preceding the currently decoded relatively low dynamic range image. 
     
     
       10. The method as recited in  claim 1 , wherein the currently decoded relatively low dynamic range image and the previously decoded relatively low dynamic range images are a part of base-layer (BL) image data in a multi-layer video signal. 
     
     
       11. The method as recited in  claim 1 , wherein neither the collected previous image statistic information nor the one or more prediction parameters for the currently decoded relatively low dynamic range image are provided as metadata in a multi-layer video signal that comprises the currently decoded relatively low dynamic range image and the previously decoded relatively low dynamic range images are a part of base-layer (BL) image data in the multi-layer video signal. 
     
     
       12. The method as recited in  claim 1 , wherein the currently decoded relatively low dynamic range image comprises pixel values in one or more of a luminance channel, a chroma channel, a red color channel, a blue color channel, a green color channel, or other primary channels. 
     
     
       13. A method, comprising:
 generating, by a video decoder based on a number of previously decoded relatively low dynamic range image partitions and the same number of previously decoded relatively high dynamic range residual image partitions, the same number of previously reconstructed relatively high dynamic range image partitions, wherein the number is one or more, wherein each image partition in the previously reconstructed relatively high dynamic range image partitions is generated based on a respective corresponding image partition in the previously decoded relatively low dynamic range image partitions and a respective corresponding image partition in the previously decoded relatively high dynamic range residual image partitions, and wherein each image partition in the previously reconstructed relatively high dynamic range image partitions comprises reconstructed pixel values that have a higher dynamic range than pixel values of the respective corresponding image partition in the previously decoded relatively low dynamic range image partitions; 
 collecting, by the video decoder, previous image partition statistic information relating to reconstructed pixel values of each image partition in the previously reconstructed relatively high dynamic range image partitions and pixel values of the respective corresponding image partition in the previously decoded relatively low dynamic range image partitions, wherein the collected previous image partition statistic information includes high dynamic range statistics of the reconstructed pixel values of each image in the previously reconstructed relatively high dynamic range images; 
 deriving, by the video decoder from the collected previous image partition statistic information including the high dynamic range statistics of the reconstructed pixel values of each image in the previously reconstructed relatively high dynamic range images, one or more prediction parameters for a currently decoded relatively low dynamic range image partition that neighbors with the previously decoded relatively high dynamic range image partitions; 
 generating, by the video decoder, a currently predicted image partition from the currently decoded relatively low dynamic range image partition based on the one or more prediction parameters derived from the collected previous image partition statistic information, the currently predicted image partition comprising predicted pixel values that have the higher dynamic range than pixel values of the currently decoded relatively low dynamic range image partition and that are predicted from the currently decoded pixel values of the currently decoded relatively low dynamic range image partition; 
 decompressing, by the video decoder, a portion of enhancement layer (EL) image data in a multi-layer video signal into currently decoded residual pixel values; and 
 combining, by the video decoder, the currently decoded residual pixel values with the currently predicted pixel values of the currently predicted image partition to generate a currently reconstructed relatively high dynamic range image partition, the currently reconstructed relatively high dynamic range image partition comprising currently reconstructed pixel values that have the higher dynamic range than the currently decoded pixel values of the currently decoded relatively low dynamic range image partition. 
 
     
     
       14. The method as recited in  claim 13 , wherein the currently decoded relatively low dynamic range image partition and the previously decoded relatively low dynamic range image partitions are within a relatively low dynamic range image. 
     
     
       15. The method as recited in  claim 13 , wherein the previously decoded relatively low dynamic range image partitions comprise an initial relatively low dynamic range image partition in a relatively low dynamic range image; further comprising generating an initial predicted image partition from the initial relatively low dynamic range image partition based on one or more default prediction parameters, the initial predicted image partition comprising initial predicted pixel values that have the higher dynamic range than initial pixel values of the initial relatively low dynamic range image partition, wherein the default prediction parameters are predefined prediction parameters. 
     
     
       16. The method as recited in  claim 13 , wherein the previously decoded relatively low dynamic range image partitions comprise an initial relatively low dynamic range image partition in a relatively low dynamic range image; further comprising:
 selecting a set of default prediction parameters from one or more sets of default prediction parameters; and 
 generating an initial predicted image partition from the initial relatively low dynamic range image partition based on the selected set of default prediction parameters, the initial predicted image partition comprising initial predicted pixel values that have the higher dynamic range than initial pixel values of the initial relatively low dynamic range image partition, 
 wherein the default prediction parameters are predefined prediction parameters. 
 
     
     
       17. The method as recited in  claim 13 , wherein the previously decoded relatively low dynamic range image partitions comprise an initial relatively low dynamic range image in a relatively low dynamic range image; further comprising:
 determining non-default prediction parameters for the initial relatively low dynamic range image partition; 
 generating an initial predicted image partition from the initial relatively low dynamic range image partition based on the non-default prediction parameters, the initial predicted image partition comprising initial predicted pixel values that have the higher dynamic range than initial pixel values of the initial relatively low dynamic range image partition. 
 
     
     
       18. The method as recited in  claim 13 , further comprising:
 collecting information relating to the reconstructed pixel values of the currently reconstructed relatively high dynamic range image partition and the pixel values of the currently decoded relatively low dynamic range image partition. 
 
     
     
       19. The method as recited in  claim 13 , wherein the collected previous image partition statistic information is stored in a look back window that stores information relating to (a) all reconstructed relatively high dynamic range image partitions in a reconstructed relatively high dynamic range image preceding to the currently reconstructed relatively high dynamic range image partition converted from the currently decoded relatively low dynamic range image partition and (b) all relatively low dynamic range image partitions in a relatively low dynamic range image preceding to the currently decoded relatively low dynamic range image partition. 
     
     
       20. The method as recited in  claim 13 , wherein the collected previous image partition statistic information is stored in a sliding window that stores information relating to (a) up to a fixed number of reconstructed relatively high dynamic range image partitions in a reconstructed relatively high dynamic range image most recently preceding to the currently reconstructed relatively high dynamic range image partition converted from the currently decoded relatively low dynamic range image partition and (b) up to the fixed number of relatively low dynamic range image partitions in a relatively low dynamic range image most recently preceding to the currently decoded relatively low dynamic range image partition. 
     
     
       21. The method as recited in  claim 13 , wherein the currently decoded relatively low dynamic range image partition and the previously decoded relatively low dynamic range image partitions are a part of base-layer (BL) image data in a multi-layer video signal. 
     
     
       22. The method as recited in  claim 13 , wherein neither the collected previous image partition statistic information nor the one or more prediction parameters for the currently decoded relatively low dynamic range image partition are provided as metadata in a multi-layer video signal that comprises the currently decoded relatively low dynamic range image partition and the previously decoded relatively low dynamic range image partitions are a part of base-layer (BL) image data in the multi-layer video signal. 
     
     
       23. The method as recited in  claim 13 , wherein the currently decoded relatively low dynamic range image partition comprises pixel values in one or more of a luminance channel, a chroma channel, a red color channel, a blue color channel, a green color channel, or other primary channels. 
     
     
       24. The method as recited in  claim 13 , wherein the previously decoded relatively low dynamic range image partitions represent one or more of a top neighboring partition, a left neighboring partition, a top-left neighboring partition, or a top-right neighboring partition, in a relatively low dynamic range image with respect to the relatively low dynamic range image partition. 
     
     
       25. A computing device comprising one or more processors and one or more storage media, storing a set of instructions, which when executed by one or more processors cause performance of the method recited in  claim 1 . 
     
     
       26. One or more non-transitory computer-readable storage media storing a set of instructions, which when executed by one or more processors, cause the one or more processors to perform the method recited in  claim 1 . 
     
     
       27. A computing device comprising one or more processors and one or more storage media, storing a set of instructions, which when executed by one or more processors cause performance of the method recited in  claim 13 . 
     
     
       28. One or more non-transitory computer-readable storage media storing a set of instructions, which when executed by one or more processors, cause the one or more processors to perform the method recited in  claim 13 .