Dynamic range using a monochrome image sensor for hyperspectral and fluorescence imaging and topology laser mapping

Abstract

Hyperspectral, fluorescence, and laser mapping imaging with increased dynamic range is disclosed. A system includes an emitter for emitting pulses of electromagnetic radiation and an image sensor for sensing reflected electromagnetic radiation, wherein the pixel array comprises a plurality of pixels each configurable as a short exposure pixel or a long exposure pixel. The system is such that at least a portion of the pulses of electromagnetic radiation emitted by the emitter comprises one or more of: electromagnetic radiation having a wavelength from about 513 nm to about 545 nm, from about 565 nm to about 585 nm, from about 900 nm to about 1000 nm, an excitation wavelength of electromagnetic radiation that causes a reagent to fluoresce, or a laser mapping pattern.

1 - 30 . (canceled) 
     
     
         31 . A system comprising:
 an emitter that emits a plurality of pulses of electromagnetic radiation in a light deficient environment;   an image sensor comprising a pixel array that senses electromagnetic radiation to generate a plurality of frames in response to the plurality of pulses of electromagnetic radiation; and   an image signal processor that executes instructions;   wherein the instructions executed by the image signal processor comprise combining data from two or more of the plurality of frames to generate an overlay frame comprising color image data and specialty image data; and   wherein the specialty image data comprises one or more of:
 multispectral image data sensed by the image sensor in response to the pixel array sensing a spectral response; 
 fluorescence image data sensed by the image sensor in response to the pixel array sensing a fluorescence relaxation wavelength of electromagnetic radiation; or 
 mapping data for calculating one or more of a dimension, distance, or three-dimensional topography. 
   
     
     
         32 . The system of  claim 31 , wherein the emitter comprises a multispectral source that emits a multispectral wavelength of electromagnetic radiation, and wherein the multispectral wavelength of electromagnetic radiation elicits the spectral response sensed by the pixel array. 
     
     
         33 . The system of  claim 32 , wherein the multispectral source comprises one or more of:
 a first multispectral source that emits electromagnetic radiation within a range from about 513 nm to about 545 nm and a second multispectral source that emits electromagnetic radiation within a range from about 900 nm to about 1000 nm; or   a first multispectral source that emits electromagnetic radiation within a range from about 565 nm to about 585 nm and a second multispectral source that emits electromagnetic radiation within a range from about 900 nm to about 1000 nm.   
     
     
         34 . The system of  claim 31 , wherein the light deficient environment comprises a body cavity, and wherein the emitter is the only source of electromagnetic radiation within the body cavity, and wherein the spectral response is emitted by a tissue within the body cavity. 
     
     
         35 . The system of  claim 31 , wherein the emitter comprises a fluorescence source that emits a fluorescence excitation wavelength of electromagnetic radiation, and wherein the fluorescence excitation wavelength of electromagnetic radiation causes a reagent or tissue to emit the fluorescence relaxation wavelength of electromagnetic radiation. 
     
     
         36 . The system of  claim 35 , wherein the fluorescence source comprises one or more of:
 a first fluorescence source that emits electromagnetic radiation within a range from about 770 nm to about 795 nm; or   a second fluorescence source that emits electromagnetic radiation within a range from about 790 nm to about 815 nm.   
     
     
         37 . The system of  claim 31 , further comprising:
 an endoscope, wherein the image sensor is disposed on a distal end of the endoscope; and   a fluorescent reagent administered to a patient;   wherein the endoscope is disposed within a body cavity of the patient to visualize one or more tissues within the body cavity; and   wherein the fluorescence relaxation wavelength is emitted by the fluorescent reagent in response to the emitter pulsing a fluorescence excitation wavelength of electromagnetic radiation.   
     
     
         38 . The system of  claim 37 , wherein the fluorescent reagent adheres to cancerous cells within the patient, and wherein the fluorescence image data comprises an indication of where the cancerous cells are located within the patient. 
     
     
         39 . The system of  claim 38 , wherein the image signal processor is further configured to provide the fluorescence image data to a corresponding system configured to determine one or more of:
 an identity of the cancerous cells based on the fluorescence image data; or   a location of the cancerous cells based on the fluorescence image data.   
     
     
         40 . The system of  claim 31 , wherein the emitter comprises a mapping source that pulses electromagnetic radiation in a mapping pattern, and wherein the mapping pattern comprises one or more of vertical hashing, horizontal hashing, a dot array, an occupancy grid map, a raster grid of discrete points, or a line. 
     
     
         41 . The system of  claim 31 , wherein the specialty image data within the overlay frame comprises at least a portion of the data from each of the multispectral image data, the fluorescence image data, and the mapping data. 
     
     
         42 . The system of  claim 31 , wherein the specialty image data within the overlay frame comprises at least a portion of the data from each of the multispectral image data and the mapping data. 
     
     
         43 . The system of  claim 31 , wherein the specialty image data within the overlay frame comprises at least a portion of the data from each of the fluorescence image data and the mapping data. 
     
     
         44 . The system of  claim 31 , wherein the specialty image data within the overlay frame comprises an alphanumerical indication of one or more of:
 a dimension of a structure within a scene;   a distance between two or more structures within the scene;   a distance between a tool and another structure within the scene;   a distance between a distal end of an endoscope and another structure within the scene, wherein the image sensor is disposed at the distal end of the endoscope.   
     
     
         45 . The system of  claim 31 , wherein:
 the color image data depicts a scene within a patient body cavity;   the spectral response is emitted by a tissue within the patient body cavity;   the specialty image data within the overlay frame comprises a color overlay indicating where the pixel array sensed the spectral response; and   the color overlay is overlaid on the color image data depicting the patient body cavity.   
     
     
         46 . The system of  claim 45 , wherein the overlay frame further comprises an indication of one or more of a dimension or three-dimensional topography of the tissue within the patient body cavity that emitted the spectral response. 
     
     
         47 . The system of  claim 31 , wherein:
 the color image data depicts a scene within a patient body cavity;   the fluorescence relaxation wavelength is emitted by one or more of a tissue or a reagent within the patient body cavity;   the specialty image data within the overlay frame comprises a color overlay indicating where the pixel array sensed the fluorescence relaxation wavelength; and   the color overlay is overlaid on the color image data depicting the patient body cavity.   
     
     
         48 . The system of  claim 47 , wherein the reagent adheres to a tissue structure within the body cavity, and wherein the overlay frame further comprises an indication of one or more of a dimension or a three-dimensional topography of the tissue or the tissue structure comprising the reagent within the patient body cavity that emitted the fluorescence relaxation wavelength. 
     
     
         49 . The system of  claim 31 , wherein the image sensor comprises:
 a first image sensor comprising a color filter array; and   a second image sensor comprising a monochromatic pixel array.   
     
     
         50 . The system of  claim 49 , wherein at least one of the first image sensor or the second image sensor comprises a pixel array having a plurality of pixels that are independently actuatable as a short exposure pixel or a long exposure pixel.