Method and system for pixel-wise imaging

Abstract

There is provided a method and system for pixel-wise imaging of a scene. The method including: receiving a pixel-wise pattern, the pixel-wise pattern including a masking value for each pixel in an array of pixels of an image sensor; producing an electronic signal at each pixel when such pixel is exposed to light received from the scene; and directing the electronic signal at each pixel to one or more collection nodes associated with such pixel based on the respective masking value, the one or more collection nodes each capable of integrating the received electronic signal.

1 . A camera to capture pixel-wise imaging of a scene, the camera comprising:
 a per-pixel programmable exposure image sensor comprising an array of pixels, each pixel comprising a photosensitive receptor that produces an electronic signal when exposed to light received from the scene;   a signal storage comprising one or more collection nodes for each pixel in the image sensor, the collection nodes receive an electronic signal from the respective photosensitive receptor in the image sensor and integrate the electronic signal; and   one or more integrated circuits comprising control memory and one or more logic components, the control memory storing a pixel-wise pattern, the pixel-wise pattern comprising a masking value for each pixel of the image sensor, and for each of the pixels, the one or more logical components direct the electronic signal to one or more of the respective collection nodes based on the respective masking value.   
     
     
         2 . The camera of  claim 1 , wherein the one or more collection nodes comprises exactly two collection nodes per pixel, wherein the masking value is a one-digit binary value, and wherein a high binary value indicates directing the electronic signal to one of the collection nodes and a low binary value indicates directing the electronic signal to the other collection node. 
     
     
         3 . The camera of  claim 1 , wherein the control memory receives a new pixel-wise pattern for every subframe, and the one or more logical components direct the electronic signal based on the masking value of the new pixel-wise pattern, each frame comprising a plurality of subframes. 
     
     
         4 . The camera of  claim 1 , wherein the control memory comprises a first memory unit and a second memory unit, the second memory unit storing the pixel-wise pattern for a current subframe and the first memory unit storing the new pixel-wise pattern for an upcoming subframe, wherein the new pixel-wise pattern for the upcoming subframe is loaded into the first memory sequentially, and the pixel-wise pattern for the current subframe is loaded into the second memory simultaneously. 
     
     
         5 . The camera of  claim 1 , wherein the light received from the scene comprises temporally modulated light from a light source and the one or more collection nodes comprises exactly two collection nodes for each pixel, the camera further comprising a processor to determine time-of-flight of the modulated light by measuring phase differences between the modulated light at the light source and the light received at the photosensitive receptor, the one or more logical components directing the electronic signal to one of the collection nodes during a first portion of the modulation of the modulated light and directing the electronic signal to the other collection node during a second portion of the modulation of the modulated light, wherein the predetermined path type is selected from a group consisting of a direct light path, an indirect light path, or a specular light path. 
     
     
         6 . The camera of  claim 1 , wherein the one or more collection nodes comprises exactly two collection nodes for each pixel, the camera in communication with a projector to project a light-pattern onto the scene, the pixel-wise pattern comprising a complementary pattern such that the one or more logical components direct the electronic signal to one of the collection nodes if the associated light received at the respective pixel approximately comprises a predetermined path type, otherwise the one or more logical components direct the electronic signal to the other collection node. 
     
     
         7 . The camera of  claim 1 , wherein the light received from the scene comprises photometric stereo light illumination conditions from a light source and the one or more collection nodes comprises exactly two collection nodes for each pixel, the one or more logical components directing the electronic signal to one of the collection nodes during a first illumination condition and directing the electronic signal to the other collection node during a second illumination condition, the camera further comprising a processor to determine normals of one or more objects in the scene via determining intensity of light received at each pixel. 
     
     
         8 . The camera of  claim 8 , wherein the normals are determined using intensity demosaicing of two or more neighboring pixels. 
     
     
         9 . The camera of  claim 8 , wherein the normals are determined using demosaicing with a ratio of each pixel's collection nodes for two or more neighboring pixels. 
     
     
         10 . The camera of  claim 1 , wherein the light received from the scene comprises structured light illumination conditions from a light source and the one or more collection nodes comprises exactly two collection nodes for each pixel, the one or more logical components directing the electronic signal to one of the collection nodes during a first illumination condition and directing the electronic signal to the other collection node during a second illumination condition, the camera further comprising a processor to determine depth to one or more objects in the scene from intensity of light received at each pixel via triangulation with pixels at the light source and pixels at the image sensor. 
     
     
         11 . The camera of  claim 11 , wherein the depths are determined using intensity demosaicing of two or more neighboring pixels. 
     
     
         12 . The camera of  claim 11 , wherein the depths are determined using demosaicing with a ratio of each pixel's collection nodes for two or more neighboring pixels.