Systems and methods for removing occluding objects in surgical images and/or video

Abstract

The present disclosure is directed to systems and methods for removing an occluding object from a surgical image. An image capture device is inserted into a patient and captures an initial image of a surgical site inside the patient during a surgical procedure. A controller receives the image and determines that the occluding object is present in the initial image. The controller executes a removal algorithm that includes controlling the image capture device to perform a plurality of movements, controlling the image capture device to capture a plurality of images, wherein each image among the plurality of images corresponds to a movement among the plurality of movements, and applying an image filter to combine the initial image and the plurality of images and generate a processed image where the occluding object is removed from the processed image.

What is claimed is: 
     
         1 . A system for removing an occluding object from a surgical image, the system comprising:
 an image capture device configured to be inserted into a patient and capture an initial image of a surgical site and capture a plurality of images based on the presences of the occluding object; and   an image filter configured to combine the captured initial image and the captured plurality of images to generate a processed image where the occluding object is removed from the processed image.   
     
     
         2 . The system of  claim 1 , wherein when generating the processed image where the occluding object is removed from the processed image, the image capture device performs a plurality of movements, wherein each image among the plurality of movements corresponds to a movement among the plurality of movements. 
     
     
         3 . The system of  claim 2 , wherein each movement among the plurality of movements includes a magnitude and a direction. 
     
     
         4 . The system of  claim 2 , wherein each movement among the plurality of movements is predetermined. 
     
     
         5 . The system of  claim 1 , wherein the image filter includes:
 separating the initial image into an initial background image and an initial occluding layer;   separating the plurality of images into a plurality of background images and a plurality of occluding layers; and   combining the initial background images and the plurality of background images to generate the processed image.   
     
     
         6 . The system of  claim 5 , wherein combining the initial background images and the plurality of background images includes:
 registering the initial background image and the plurality of background images; and   overlaying the registered initial background image and the plurality of background images.   
     
     
         7 . The system of  claim 1 , wherein the image filter includes:
 removing the occluding object from the initial image to create an empty space;   registering the plurality of images with the initial image; and   filling the empty space in the initial image with corresponding pixels from the registered plurality of images, wherein the corresponding pixels do not include the occluding object.   
     
     
         8 . The system of  claim 1 , wherein the presence of the occluding object is based on an input from a user. 
     
     
         9 . The system of  claim 1 , further comprising a display that displays the processed image. 
     
     
         10 . A method for removing an occluding object from a surgical image, the method comprising:
 capturing an initial image of a surgical site inside the patient during a surgical procedure with an image capture device;   capturing a plurality of images based on the presence of the occluding object; and   applying an image filter to combine the initial image and the plurality of images and generate a processed image where the occluding object is removed from the processed image.   
     
     
         11 . The method of  claim 10 , wherein capturing the plurality of images based on the presence of the occluding object includes controlling the image capture device to perform a plurality of movements, wherein each image among the plurality of movements corresponds to a movement among the plurality of movements. 
     
     
         12 . The method of  claim 11 , wherein each movement among the plurality of movements includes a magnitude and a direction. 
     
     
         13 . The method of  claim 11 , wherein each movement among the plurality of movements is predetermined. 
     
     
         14 . The method of  claim 10 , wherein applying the image filter includes:
 separating the initial image into an initial background image and an initial occluding layer;   separating the plurality of images into a plurality of background images and a plurality of occluding layers; and   combining the initial background images and the plurality of background images to generate the processed image.   
     
     
         15 . The method of  claim 14 , wherein combining the initial background images and the plurality of background images includes:
 registering the initial background image and the plurality of background images; and   overlaying the registered initial background image and the plurality of background images.   
     
     
         16 . The method of  claim 10 , wherein applying the image filter includes:
 removing the occluding object from the initial image to create an empty space;   registering the plurality of images with the initial image; and   filling the empty space in the initial image with corresponding pixels from the registered plurality of images, wherein the corresponding pixels do not include the occluding object.   
     
     
         17 . The method of  claim 10 , further comprising displaying the processed image. 
     
     
         18 . A system for removing an occluding object from a surgical image, the system comprising:
 an image capture device configured to be inserted into a patient and generate a collection of substitution pixels based on a plurality of images, a location of the occluding object, and a point of view of the image capture device; and   an image filter configured reassemble pixels of the initial image and the collection of substitution pixels to remove the occluding object and create a processed image.   
     
     
         19 . The system of  claim 18 , wherein reassembling pixels of the initial image and the collection of substitution pixel includes, the image filter identifies the pixels belong to a specific object at a specific spatial locations relative to the image capture device;
 removes a relatively nearer pixel of the occluding object; and   substitutes the relatively nearer pixel with relatively further pixel from the collection of substitution pixels.   
     
     
         20 . The system of  claim 18 , wherein the relatively nearer pixel is a pixel that is closer to the image capture device and the relatively further pixel is a pixel that is further from the image capture device.