Method and system for end-to-end image processing

Abstract

A method of processing an input image comprises receiving the input image, storing the image in a memory, and accessing, by an image processor, a computer readable medium storing a trained deep learning network. A first part of the deep learning network has convolutional layers providing low-level features extracted from the input image, and convolutional layers providing a residual image. A second part of the deep learning network has convolutional layers for receiving the low-level features and extracting high-level features based on the low-level features. The method feeds the input image to the trained deep learning network, and applies a transformation to the residual image based on the extracted high-level features.

What is claimed is: 
     
         1 . A method of processing an input image, comprising:
 receiving the input image, and storing the image in a memory;   by an image processor:   extracting low-level features from the input image;   generating a residual image devoid of low-level features;   extracting high-level features based on said low-level features;   applying a transformation to said residual image based on said extracted high-level features; and   generating on a display device an output showing said transformed residual image.   
     
     
         2 . The method of  claim 1 , wherein said low-level features comprise denoising features and demosaicing features. 
     
     
         3 . The method of  claim 1 , wherein the input image is a raw image. 
     
     
         4 . The method of  claim 1 , wherein said input image is a demosaiced image. 
     
     
         5 . The method of  claim 1 , further comprising preprocessing said image by applying a bilinear interpolation, prior to said feeding. 
     
     
         6 . The method of  claim 1 , wherein said transformation comprises a non-linear function of color components of each pixel of said residual image. 
     
     
         7 . The method of  claim 1 , wherein said applying said transformation is executed globally to all pixels of said residual image. 
     
     
         8 . The method of  claim 1 , further comprising capturing the input image. 
     
     
         9 . An image capturing and processing system, comprising:
 an imaging device for capturing an image; and   a hardware image processor configured to receive said captured image, to extract low-level features from said captured image, to generate a residual image devoid of low-level features, to extract high-level features based on said low-level features, to apply a transformation to said residual image based on said extracted high-level features, and to generate on a display device an output showing said transformed residual image.   
     
     
         10 . The system of  claim 9 , further comprising said display device. 
     
     
         11 . The system of  claim 9 , wherein said low-level features comprise denoising features and demosaicing features. 
     
     
         12 . The system of  claim 9 , wherein said image processor is configured to apply to said captured image at least one low-level image processing procedure. 
     
     
         13 . The system of  claim 12 , wherein said at least one low-level image processing procedure comprises demosaicing. 
     
     
         14 . The system of  claim 12 , wherein said at least one low-level image processing procedure comprises denoising. 
     
     
         15 . The system of  claim 9 , wherein said image processor is configured to preprocess said captured image by applying a bilinear interpolation, prior to said extraction of said low-level features. 
     
     
         16 . The system of  claim 9 , wherein said transformation comprises a non-linear function of color components of each pixel of said residual image. 
     
     
         17 . The system of  claim 9 , wherein said image processor is configured to apply said transformation globally to all pixels of said residual image. 
     
     
         18 . The system of  claim 9 , wherein said image processor is configured to apply at least one image processing task selected from the group consisting of super-resolution, deblurring, and image enhancement. 
     
     
         19 . A smartphone, comprising the system of  claim 9 . 
     
     
         20 . A tablet, comprising the system of  claim 9 . 
     
     
         21 . A smartwatch, comprising the system of  claim 9 .