Generalized assorted pixel camera systems and methods

Abstract

Generalized assorted pixel camera systems and methods are provided. In accordance with some embodiments, the generalized assorted pixel camera systems include a color filter array, where the color filter array includes a plurality of primary filters and a plurality of secondary filters. Each filter has a particular spectral response and each filter is formed on a corresponding pixel of a plurality of pixels. Each of the plurality of primary filters and the plurality of secondary filters enhances an attribute of image quality and the information obtained using the plurality of primary filters and the plurality of secondary filters is used to balance spectral resolution, dynamic range, and spatial resolution for generating an image of a plurality of image types.

What is claimed is: 
     
         1 . A color filter array, comprising:
 a plurality of first color filters each having a first spectral response;   a plurality of second color filters each having a second spectral response that is different from the first spectral response;   a plurality of third color filters each having a third spectral response that is different from the first spectral response and different from the second spectral response; and   a plurality of fourth color filters each having a spectral response that is highly correlated with at least a portion of the first spectral response, the second spectral response or the third spectral response, wherein the plurality of fourth filters includes at least four filters that each have a different spectral response;   wherein each of the plurality of first color filters are arranged such that the first color filter is adjacent to a second color filter and a third color filter,   each of the plurality of second color filters are arranged such that the second color filter is adjacent to two first color filters and a fourth color filter,   each of the plurality of third color filters are arranged such that the third color filter is adjacent to two first color filters and a fourth color filter, and   each of the plurality of fourth color filters is arranged such that the fourth color filter is adjacent to two second color filters and two third color filters, and spaced apart from another fourth color filter having the same spectral response by at least three color filters along two orthogonal directions of the color filter array.   
     
     
         2 . The color filter array of  claim 1 , wherein the transmittance of each fourth filter is lower than the transmittance of the plurality of first color filters, the plurality of second filters or the plurality of third color filters that has a spectral response that is highly correlated with that fourth filter. 
     
     
         3 . The color filter array of  claim 1 , wherein the transmittance of each fourth filter is lower than the transmittance of the plurality of first color filters, the transmittance of the plurality of second color filters, and the transmittance of the plurality of third color filters. 
     
     
         4 . The color filter array of  claim 1 , wherein the plurality of first color filters are blue filters, the plurality of second color filters are green filters, and the plurality of third color filters are red color filters. 
     
     
         5 . The color filter array of  claim 4 , wherein the plurality of fourth color filters comprise:
 a plurality of fifth color filters having a fourth spectral response that is highly correlated with at least a portion of the first spectral response and having a transmittance that is lower than the transmittance of the plurality of first color filters;   a plurality of sixth color filters having a fifth spectral response that is highly correlated with a portion of the second spectral response and having a transmittance that is lower than the transmittance of the plurality of second color filters;   a plurality of seventh filters having a sixth spectral response that is highly correlated with a portion of the second spectral response and having a transmittance that is lower than the transmittance of the plurality of second color filters, wherein a wavelength at which the sixth spectral response has peak transmittance is substantially different a wavelength at which the fifth spectral response has peak transmittance; and   a plurality of eighth color filters having an eighth spectral response that is highly correlated with at least a portion of the third spectral response and having a transmittance that is lower than the transmittance of the third color filters.   
     
     
         6 . A camera system, comprising:
 a color filter array, comprising:
 a plurality of first color filters each having a first spectral response; 
 a plurality of second color filters each having a second spectral response that is different from the first spectral response; 
 a plurality of third color filters each having a third spectral response that is different from the first spectral response and different from the second spectral response; and 
 a plurality of fourth color filters each having a spectral response that is highly correlated with at least a portion of the first spectral response, the second spectral response or the third spectral response, wherein the plurality of fourth filters includes at least four filters that each have a different spectral response; 
 wherein each of the plurality of first color filters are arranged such that the first color filter is adjacent to a second color filter and a third color filter, 
 each of the plurality of second color filters are arranged such that the second color filter is adjacent to two first color filters and a fourth color filter, 
 each of the plurality of third color filters are arranged such that the third color filter is adjacent to two first color filters and a fourth color filter, and 
 each of the plurality of fourth color filters is arranged such that the fourth color filter is adjacent to two second color filters and two third color filters, and spaced apart from another fourth color filter having the same spectral response by at least three color filters along two orthogonal directions of the color filter array; and 
   an image sensor comprising a plurality of pixels, wherein the color filter array is disposed in the camera system such that, during operation of the camera system, each color filter of the color filter array corresponds to a pixel of the plurality of pixels.   
     
     
         7 . The camera system of  claim 6 , wherein the transmittance of each fourth filter is lower than the transmittance of the plurality of first color filters, the plurality of second filters or the plurality of third color filters that has a spectral response that is highly correlated with that fourth filter. 
     
     
         8 . The camera system of  claim 6 , wherein the transmittance of each fourth filter is lower than the transmittance of the plurality of first color filters, the transmittance of the plurality of second color filters, and the transmittance of the plurality of third color filters. 
     
     
         9 . The camera system of  claim 8 , wherein the plurality of first color filters are blue filters, the plurality of second color filters are green filters, and the plurality of third color filters are red color filters. 
     
     
         10 . The camera system of  claim 9 , wherein the plurality of fourth color filters comprise:
 a plurality of fifth color filters having a fourth spectral response that is highly correlated with at least a portion of the first spectral response and having a transmittance that is lower than the transmittance of the plurality of first color filters;   a plurality of sixth color filters having a fifth spectral response that is highly correlated with a portion of the second spectral response and having a transmittance that is lower than the transmittance of the plurality of second color filters;   a plurality of seventh filters having a sixth spectral response that is highly correlated with a portion of the second spectral response and having a transmittance that is lower than the transmittance of the plurality of second color filters, wherein a wavelength at which the sixth spectral response has peak transmittance is substantially different a wavelength at which the fifth spectral response has peak transmittance; and   a plurality of eighth color filters having an eighth spectral response that is highly correlated with at least a portion of the third spectral response and having a transmittance that is lower than the transmittance of the third color filters.   
     
     
         11 . The camera system of  claim 6 , further comprising a processor that is configured to generate a plurality of images based on image data captured by the image sensor during a single exposure, wherein each of the plurality of images is of a different image type of a plurality of image types that includes at least two of: a monochrome image, a high dynamic resolution monochrome image, a tri-chromatic image, a high dynamic resolution tri-chromatic image, and a multispectral image. 
     
     
         12 . The camera system of  claim 11 , wherein the processor is further configured to generate a monochrome image using primarily information obtained using pixels corresponding to the plurality of first color filters, the plurality of second color filters, and the plurality of third color filters. 
     
     
         13 . The camera system of  claim 12 , wherein the processor is further configured to generate a low exposure monochrome image using primarily information obtained using pixels corresponding to the plurality of fourth color filters. 
     
     
         14 . The camera system of  claim 13 , wherein the processor is further configured to generate a high dynamic range image by combining information from the low exposure monochrome image and the monochrome image. 
     
     
         15 . The camera system of  claim 11 , wherein the processor is further configured to generate a tri-chromatic image using primarily information obtained using pixels corresponding to the plurality of first color filters, the plurality of second color filters, and the plurality of third color filters. 
     
     
         16 . The camera system of  claim 15 , wherein the processor is further configured to generate the tri-chromatic image using a color reproduction matrix and a linear transformation to combine the information obtained using pixels corresponding to the plurality of first color filters, the plurality of second color filters, and the plurality of third color filters. 
     
     
         17 . The camera system of  claim 16 , wherein the processor is further configured to:
 generate a low exposure tri-chromatic image using primarily information obtained using pixels corresponding to the plurality of fourth color filters; and   generate a high dynamic range tri-chromatic image by combining information from the low exposure tri-chromatic image and information from the tri-chromatic image.   
     
     
         18 . The camera system of  claim 17 , wherein the processor is further configured to generate the low exposure tri-chromatic image by estimating aliasing of the information obtained using pixels corresponding to the plurality of fourth color filters using the information obtained from pixels corresponding to the plurality of first color filters, the plurality of second color filters, and the plurality of third color filters. 
     
     
         19 . The camera system of  claim 11 , wherein the processor is further configured to:
 generate anti-aliased information by performing anti-aliasing on the information obtained using pixels corresponding to the plurality of fourth colored filters; and   generate a multispectral image using the anti-aliased information and information obtained using pixels corresponding to the plurality of first color filters, the plurality of second color filters and the plurality of third color filters to reconstruct spectral reflectance for the multispectral image.