Fluorescence imaging in a light deficient environment

Abstract

An endoscopic imaging system for use in a light deficient environment includes an imaging device having a tube, one or more image sensors, and a lens assembly including at least one optical elements that corresponds to the one or more image sensors. The endoscopic system includes a display for a user to visualize a scene and an image signal processing controller. The endoscopic system includes a light engine having an illumination source generating one or more pulses of electromagnetic radiation and a lumen transmitting one or more pulses of electromagnetic radiation to a distal tip of an endoscope.

The invention claimed is: 
     
       1. An endoscopic system for use in a light deficient environment comprising:
 an image sensor comprising a pixel array that accumulates electromagnetic radiation and converts the accumulated electromagnetic radiation into an exposure frame comprising pixel integration data, wherein the image sensor operates according to a sensor cycle comprising a plurality of blanking periods and a plurality of readout periods; 
 a light engine that cycles a plurality of sources of electromagnetic radiation according to a variable pulse cycle, wherein the plurality of sources comprises:
 a visible source that pulses a visible wavelength of electromagnetic radiation; 
 a first excitation source that pulses only infrared electromagnetic radiation covering a first excitation waveband that is from 770 nm to 790 nm, and wherein the endoscopic system further comprises a filter that prevents electromagnetic radiation within the first excitation waveband from 770 nm to 790 nm from irradiating the pixel array; and 
 a second excitation source that pulses only infrared electromagnetic radiation covering a second excitation waveband that is from 795 nm to 815 nm, and wherein the endoscopic system further comprises a filter that prevents electromagnetic radiation within the second excitation waveband from 795 nm to 815 nm from irradiating the pixel array; 
 
 a waveguide transmitting one or more pulses of electromagnetic radiation from the light engine to a distal tip of an endoscope; and 
 a controller in communication with the image sensor and the light engine that synchronizes implementation of the sensor cycle and the variable pulse cycle to output a plurality of datasets, wherein each of the plurality of datasets comprises an exposure frame and a waveband assignment, and wherein the plurality of datasets comprises:
 a visible dataset comprising a visible exposure frame, wherein the pixel array accumulates the pixel integration data for the visible exposure frame during a blanking period when the light engine cycles on only the visible source, and wherein the waveband assignment for the visible dataset is the visible wavelength of electromagnetic radiation; and 
 a fluorescence dataset comprising a fluorescence exposure frame, wherein the pixel array accumulates the pixel integration data for the fluorescence exposure frame during a blanking period when the light engine cycles on only one or more of the first excitation source or the second excitation source, and wherein the waveband assignment for the fluorescence dataset is one or more of the first excitation waveband or the second excitation waveband; and 
 wherein the controller determines the waveband assignment for each exposure frame read out by the image sensor based on the synchronization of the sensor cycle and the variable pulse cycle such that the waveband assignment for a first exposure frame indicates which wavelength of electromagnetic radiation was emitted by the light engine during a blanking period immediately preceding readout of the first exposure frame. 
 
 
     
     
       2. The endoscopic system of  claim 1 , further comprising:
 a display; and 
 an image signal processor; 
 wherein the image signal processor renders an overlay frame comprising data extracted from each of the visible dataset and the fluorescence dataset; and 
 wherein the controller provides the overlay frame to be rendered on the display; and wherein the overlay frame is assigned a visible color for use on the display, and wherein the visible color is 8-bit or 16-bit or n-bit. 
 
     
     
       3. The endoscopic system of  claim 1 , further comprising one or more filters that allow electromagnetic radiation between 790 nm and 800 nm and above 815 nm to pass through the one or more filters to the image sensor. 
     
     
       4. The endoscopic system of  claim 1 , wherein the pixel array accumulates electromagnetic radiation emitted by a fluorescent reagent at a fluorescence relaxation wavelength in response to the light engine pulsing only one or more of the first excitation source or the second excitation source; and
 wherein the pixel integration data for the fluorescence dataset indicates a location of the fluorescent reagent within a scene. 
 
     
     
       5. The endoscopic system of  claim 4 , wherein the fluorescent reagent is selected to adhere to one or more tissue structures in the human body, and wherein the one or more tissue structures comprises a nerve, a ureter, a blood vessel, an artery, a blood flow, or a tumor. 
     
     
       6. The endoscopic system of  claim 4 , wherein the fluorescent reagent is selected to adhere to a cancerous cell, and wherein the pixel integration data for the fluorescence dataset indicates a location of the cancerous cell within the scene. 
     
     
       7. The endoscopic system of  claim 1 , wherein the image sensor comprises a first image sensor and a second image sensor;
 wherein the first image sensor and the second image sensor simultaneously output pixel integration data; and 
 wherein the simultaneous outputs from the first image sensor and the second image sensor are processed with an algorithm to produce three-dimensional visualization data. 
 
     
     
       8. The endoscopic system of  claim 7 , further comprising a filter that prevents certain wavelengths of electromagnetic radiation from irradiating the first image sensor such that the first image sensor accumulates electromagnetic radiation only above 815 nm. 
     
     
       9. The endoscopic system of  claim 7 , further comprising a filter that prevents certain wavelengths of electromagnetic radiation from irradiating the second image sensor such that the second image sensor accumulates detects electromagnetic radiation only between 785 nm and 800 nm. 
     
     
       10. The endoscopic system of  claim 7 , wherein:
 the first image sensor comprises one or more filters that filter electromagnetic radiation between 770 nm and 815 nm; and 
 the second image sensor comprises one or more filters that filter electromagnetic radiation between 760 nm and 785 nm and electromagnetic radiation between 800 nm and 850 nm. 
 
     
     
       11. The endoscopic system of  claim 1 , further comprising a filter that blocks electromagnetic radiation between 770 nm and 815 nm. 
     
     
       12. The endoscopic system of  claim 1 , further comprising a filter that blocks electromagnetic radiation between 760 nm and 785 nm and electromagnetic radiation between 800 nm and 850 nm. 
     
     
       13. The endoscopic system of  claim 1 , wherein the light engine further comprises a polarization filter. 
     
     
       14. The endoscopic system of  claim 13 , wherein the polarization filter is located in a pathway of electromagnetic radiation emitted by the plurality of sources. 
     
     
       15. The endoscopic system of  claim 13 , further comprising a lumen, and wherein the image sensor is disposed at a distal end of the lumen, and wherein the polarization filter is located at a proximal end of the lumen. 
     
     
       16. The endoscopic system of  claim 13 , further comprising a lumen, and wherein the image sensor is disposed at a distal end of the lumen, and wherein the polarization filter is located at a distal end of the lumen. 
     
     
       17. The endoscopic system of  claim 1 , wherein each of the first excitation waveband the second excitation waveband is within a near infrared or infrared waveband of the electromagnetic spectrum. 
     
     
       18. The endoscopic system of  claim 1 , further comprising polarization filter. 
     
     
       19. The endoscopic system of  claim 1 , further comprising an image sensor processor that identifies a location of a fluorescent reagent within a scene based on the pixel integration data for the fluorescence dataset. 
     
     
       20. The endoscopic system of  claim 1 , wherein the controller synchronizes the implementation of the sensor cycle and the variable pulse cycle such that:
 the light engine pulses only the visible source during a first blanking period of the image sensor when the pixel array accumulates the electromagnetic radiation and no active pixels in the pixel array are read out; and 
 the light engine cycles off each of the plurality of sources during a first readout period of the image sensor when the image sensor reads out the pixel integration data for the visible dataset. 
 
     
     
       21. The endoscopic system of  claim 20 , wherein the controller synchronizes the implementation of the sensor cycle and the variable pulse cycle such that:
 the light engine pulses only the one or more of the first excitation source or the second excitation source during a second blanking period of the image sensor when the pixel array accumulates the electromagnetic radiation and no active pixels in the pixel array are read out; and 
 the light engine cycles off each of the plurality of sources during a second readout period of the image sensor when the image sensor reads out the pixel integration data for the fluorescence dataset. 
 
     
     
       22. The endoscopic system of  claim 21 , wherein the first blanking period is separate from the second blanking period; and
 wherein the first readout period is separate from the second readout period. 
 
     
     
       23. The endoscopic system of  claim 1 , wherein the visible dataset comprises only color visualization data. 
     
     
       24. The endoscopic system of  claim 1 , wherein the fluorescence dataset comprises only fluorescence visualization data corresponding with an emission by a fluorescent reagent. 
     
     
       25. The endoscopic system of  claim 1 , wherein the visible source is a white light source;
 wherein the pixel array comprises a color filter array; and wherein the pixel integration data for the visible dataset is utilized to generate a Red Green Blue (RGB) color image frame. 
 
     
     
       26. The endoscopic system of  claim 1 , wherein the visible source comprises each of:
 a red source that emits only red light; 
 a green source that emits only green light; and 
 a blue source that emits only blue light. 
 
     
     
       27. The endoscopic system of  claim 26 , wherein the controller instructs the light engine to simultaneously pulse each of the red source, the green source, and the blue source; and wherein the pixel integration data for the visible dataset corresponds with the controller simultaneously pulsing each of the red source, the green source, and the blue source. 
     
     
       28. The endoscopic system of  claim 26 , wherein the controller instructs the light engine to separately pulse the red source, the green source, and the blue source during separate blanking periods for the image sensor such that the visible dataset comprises three separate color datasets comprising:
 a red color dataset comprising a red exposure frame, wherein the pixel array accumulates the pixel integration data for the red exposure frame when the light engine pulses only the red source, and wherein the waveband assignment for the red color dataset is a red waveband of the electromagnetic spectrum; 
 a green color dataset comprising a green exposure frame, wherein the pixel array accumulates the pixel integration data for the green exposure frame when the light engine pulses only the green source, and wherein the waveband assignment for the green color dataset is a green waveband of the electromagnetic spectrum; and 
 a blue color dataset comprising a blue exposure frame, wherein the pixel array accumulates the pixel integration data for the blue exposure frame when the light engine pulses only the blue source, and wherein the waveband assignment for the blue color dataset is a blue waveband of the electromagnetic spectrum. 
 
     
     
       29. The endoscopic system of  claim 1 , wherein the controller instructs the light engine to separately pulse the visible source, the first excitation source, and the second excitation source during separate blanking periods for the image sensor such that:
 the pixel array accumulates only reflected visible light during a first blanking period when the light engine pulses only the visible source; 
 the pixel array reads out the pixel integration data for the visible exposure frame during a first readout period occurring immediately subsequent to the first blanking period; 
 the pixel array accumulates only a fluorescence relaxation emission by a fluorescent reagent during a second blanking period when the light engine pulses only the one or more of the first excitation source or the second excitation source; and 
 the pixel array reads out the pixel integration data for the fluorescence exposure frame during a second readout period occurring immediately subsequent to the second blanking period.