Method and apparatus for inverse tone mapping

Abstract

Inverse tone mapping (ITM) aims at generating a single high dynamic range (HDR) image from a low dynamic range (LDR) image. While ITM was frequently used for graphics rendering in the HDR space, the advent of HDR consumer displays (e.g., HDR TV) and the consequent need for HDR multimedia contents open up new horizons for the consumption of ultra-high quality video contents. However, due to the lack of HDR-filmed contents, the legacy LDR videos must be up-converted for viewing on these HDR displays. Unfortunately, the previous ITM methods are not appropriate for HDR consumer displays, and their inverse-tone-mapped results are not visually pleasing with noise amplification or lack of details. In this paper, we propose a convolutional neural network (CNN) based architecture designed for the ITM to HDR consumer displays, called ITM-CNN, and its training strategy for enhancing the performance based on image decomposition using the guided filter. We demonstrate the benefits of decomposing the image by experimenting with various architectures and also compare the performance for different training strategies. To the best of our knowledge, this paper first presents the ITM problem using CNNs for HDR consumer displays, where the network is trained to restore lost details and local contrast. Our ITM-CNN can readily up-convert LDR images for direct viewing on an HDR consumer medium, and is a very powerful means to solve the lack of HDR video contents with legacy LDR videos.

1 . AN apparatus for inverse tone mapping comprising:
 end-to-end CNN that is able to jointly optimize the LDR decomposition and HDR reconstruction phases,   wherein the end-to-end CNN allows all legacy LDR images/video to be up-converted for direct viewing on HDR TVs.