Computer-implemented systems and methods for intelligent image analysis using spatio-temporal information

Abstract

A computer-implemented method for detecting at least one feature of interest in images captured with an imaging device includes: receiving an ordered set of images and analyzing one or more subsets of the ordered set using a local spatio-temporal processing module. The local spatio-temporal processing module determines presence of characteristics related to the feature of interest in each image of each subset of images and annotates the subset of images. The method also includes processing a set of feature vectors of the ordered set of images using a global spatio-temporal processing module to refine the determined characteristics associated with each subset of images, and calculate one or more values for each image using a timeseries analysis module, the values being representative of the feature of interest and calculated using the refined characteristics associated with each subset of images and spatio-temporal information.

What is claimed is: 
     
         1 . A system for processing images, comprising:
 one or more memory devices storing processor-executable instructions; and   one or more processors configured to execute instructions to cause the system to perform operations to perform plurality of tasks on a set of images, the operations comprising:   receiving the plurality of tasks, wherein at least one task of the plurality of tasks is associated with a request to identify at least one feature of interest in the set of images;   analyzing, using a local spatio-temporal processing module, a subset of images of the set of images to identify presence of characteristics associated with the at least one feature of interest; and   iterating execution of a timeseries analysis module for each task of the plurality of tasks to associate a numerical score for each task with each image of the subset of images.   
     
     
         2 . The system of  claim 1 , wherein the local spatio-temporal processing module outputs subsets of analyzed images of the set of images, wherein each subset is associated with a task of the plurality of tasks. 
     
     
         3 . The system of  claim 1 , wherein the local spatio-temporal processing module determines the presence of characteristics by determining a vector of quality scores, wherein each quality score in the vector of quality scores corresponds to each image of the subset of the images. 
     
     
         4 . The system of  claim 3 , wherein each quality score is an ordinal number between 0 and R, wherein a score 0 represents minimum quality and a score R represents a maximum quality. 
     
     
         5 . The system of  claim 1 , wherein the local spatio-temporal processing module generates a set of feature vectors for features of interest related to the plurality of tasks. 
     
     
         6 . The system of  claim 1 , wherein the operations further comprise:
 analyze, using a global spatio-temporal processing module, sets of feature vectors for the subset of images analyzed by the local spatio-temporal processing module.   
     
     
         7 . The system of  claim 1 , wherein the operations further comprise:
 aggregate output of the local spatio-temporal processing module for each task of the plurality of tasks using the timeseries analysis module.   
     
     
         8 . The system of  claim 1 , wherein the ordered set of images are received directly from the imaging device during a medical procedure. 
     
     
         9 . The system of  claim 1 , wherein the presence of at least one feature of interest is determined from a portion of the captured images. 
     
     
         10 . A non-transitory computer readable medium including instructions that when executed by at least one processor, cause the at least one processor to perform operations to perform a plurality of tasks on a set of images, the operations comprising:
 receiving a plurality of tasks, wherein at least one task of the plurality of tasks is associated with a request to identify at least one feature of interest in the set of images;   analyzing, using a local spatio-temporal processing module, a subset of images of the set of images to identify presence of characteristics associated with the at least one feature of interest; and   iterating execution of a timeseries analysis module for each task of the plurality of tasks to associate a numerical score for each task with each image of the subset of images.   
     
     
         11 . The computer readable medium of  claim 10 , wherein the local spatio-temporal processing module outputs subsets of analyzed images of the set of images, wherein each subset of the subsets of analyzed images is associated with a task of the plurality of tasks. 
     
     
         12 . The computer readable medium of  claim 11 , wherein the local spatio-temporal processing module determines the presence of characteristics by determining a vector of quality scores, wherein each quality score in the vector of quality scores corresponds to each image of the subset of the images. 
     
     
         13 . The computer readable medium of  claim 10 , wherein each quality score is an ordinal number between 0 and R, wherein a score 0 represents minimum quality and a score R represents the maximum quality. 
     
     
         14 . The computer readable medium of  claim 11 , wherein the local spatio-temporal processing module generates a set of feature vectors for features of interest related to each task of the plurality of tasks. 
     
     
         15 . The computer readable medium of  claim 11 , wherein the operations further comprise:
 analyzing, using a global spatio-temporal processing module, sets of feature vectors for the subset of images analyzed by the local spatio-temporal processing module.   
     
     
         16 . The computer readable medium of  claim 11 , wherein the operations further comprise:
 aggregating output of the local spatio-temporal processing module for each task of the plurality of tasks using the timeseries analysis module.   
     
     
         17 . A method for performing a plurality of tasks on a set of input images, the method comprising operations performed by at least one processor, including:
 receiving a plurality of tasks, wherein at least one task of the plurality of tasks is associated with a request to identify at least one feature of interest in the set of images;   analyzing, using a local spatio-temporal processing module, a subset of images of the set of images to identify presence of characteristics associated with at least one feature of interest; and   iterating execution of a timeseries analysis module for each task of the plurality of tasks to associate a numerical score for each task with each image of the subset of images.   
     
     
         18 . The method of  claim 17 , wherein the local spatio-temporal processing module outputs subsets of analyzed images of the set of images, wherein each subset of the subsets of analyzed images is associated with a task of the plurality of tasks. 
     
     
         19 . The method of  claim 17 , wherein the local spatio-temporal processing module determines the presence of characteristics by determining a vector of quality scores, wherein each quality score in the vector of quality scores corresponds to each image of the subset of the images. 
     
     
         20 . The method of  claim 19 , wherein each quality score is an ordinal number between 0 and R, wherein a score 0 represents minimum quality and a score R represents the maximum quality. 
     
     
         21 . The method of  claim 17 , wherein the local spatio-temporal processing module generates a set of feature vectors for features of interest related to each task of the plurality of tasks. 
     
     
         22 . The method of  claim 17 , further comprising:
 analyzing, using a global spatio-temporal processing module sets of feature vectors for the subset of images analyzed by the local spatio-temporal processing module.   
     
     
         23 . The method of  claim 17 , further comprising:
 aggregating output of the local spatio-temporal processing module for each task of the plurality of tasks using the timeseries analysis module.   
     
     
         24 . The method of  claim 17 , wherein the ordered set of images are received directly from the imaging device during a medical procedure. 
     
     
         25 . The method of  claim 17 , wherein the presence of at least one feature of interest is determined from a portion of the captured images.