Systems and methods for automatically interpreting images of microbiological samples

Abstract

In accordance with some embodiments of the disclosed subject matter, provided herein are mechanisms (which can, for example, include systems, methods, and media) for automatically interpreting and classifying images of Gram-stained biological samples.

1 . A method for automatically classifying a biological sample, comprising:
 receiving an image of the biological sample;   extracting a plurality of image patches from the image;   providing each of the plurality of image patches to a trained convolution neural network that was trained using transfer learning and a plurality of images of pre-classified and labeled biological samples;   receiving, for each of the plurality of image patches, a classification from the trained convolution neural network indicating a prediction that the image patch includes an example of a particular class of a plurality of classes;   counting, for each of the plurality of classes, the number of image patches predicted by the trained classifier to include examples of the class; and   classifying, based on the count for each of the plurality of classes, the biological sample.   
     
     
         2 . The method of  claim 1 , wherein the biological sample is a Gram-stained blood culture slide. 
     
     
         3 . The method of  claim 2 , wherein the plurality of classes includes at least a first class corresponding to Gram-negative rods, a second class corresponding to Gram-positive cocci in clusters, and a third class corresponding to Gram-positive cocci in pairs and chains. 
     
     
         4 . The method of  claim 2 , wherein the plurality of classes includes one or more classes selected from the group consisting of Gram-negative cocci in pairs, Gram-negative coccobacilli, Gram-positive coccobacilli, Gram positive cocci in pairs, Gram-positive diptheroidal morphologies, Gram-positive bacilli/clostridial morphologies, and Gram-positive rods in chains. 
     
     
         5 . The method of  claim 1 , wherein the trained convolution neural network is a version of an Inception v3 convolution neural network. 
     
     
         6 . The method of  claim 5 , wherein the Inception v3 convolution neural network was trained to classify a multitude of different object classes from input image data using labeled images including examples of the different object classes, wherein each of the plurality of classes is different than all of the different object classes. 
     
     
         7 . The method of  claim 6 , wherein a final fully connected layer of the Inception v3 convolution neural network was retrained using the plurality of images of pre-classified and labeled biological samples. 
     
     
         8 . The method of  claim 1 , wherein the biological sample comprises at least one of the following liquids: blood, plasma, serum, sputum, saliva, bronchoalveolar lavage fluid, lymph, urine, pleural fluid, ascites fluid, synovial fluid, and fluid obtained by irrigating or washing a body cavity. 
     
     
         9 . (canceled) 
     
     
         10 . The method of  claim 8 , wherein the plurality of classes includes at least a first class corresponding to the presence of a first organism, and a second class corresponding to absence of the first organism. 
     
     
         11 . The method of  claim 8 , wherein the biological sample comprises sputum, wherein the plurality of classes includes at least a first class corresponding to the presence of a squamous epithelial cell, and wherein the method further comprises determining the prevalence of squamous epithelial cells in the sputum. 
     
     
         12 . The method of  claim 1 , wherein the biological sample comprises a fluid prepared using an acid-fast stain. 
     
     
         13 . The method of  claim 12 , wherein the fluid comprises sputum or bronchoalveolar lavage fluid. 
     
     
         14 . The method of  claim 13 , wherein the plurality of classes includes at least a first class corresponding to the presence of mycobacteria, and a second class corresponding to the absence of mycobacteria. 
     
     
         15 . The method of  claim 1 , wherein the biological sample comprises Löwenstein-Jensen medium or Middlebrook broth with presumptive mycobacteria prepared using acid-fast stain. 
     
     
         16 . The method of  claim 15 , wherein the plurality of classes includes at least a first class corresponding to the presence of one or more acid-fast organisms. 
     
     
         17 . The method of  claim 1 , wherein the biological sample comprises a fluid sample prepared using fluorescent probes. 
     
     
         18 . The method of  claim 17 , wherein the fluid comprises sputum or bronchoalveolar lavage fluid. 
     
     
         19 . The method of  claim 18 , wherein the plurality of classes includes at least a first class corresponding to the presence of a first type of organism, and a second class corresponding to the presence of a second organism. 
     
     
         20 . The method of  claim 1 , wherein the biological sample comprises a stool sample prepared using fluorescent probes. 
     
     
         21 . The method of  claim 20 , wherein the plurality of classes includes at least a first class corresponding to the presence of  Cryptosporidium  or  Giardia , and a second class corresponding to the absence of  Cryptosporidium  or  Giardia.    
     
     
         22 . The method of  claim 1 , wherein classifying the biological sample comprises classifying the biological sample based on the class having the highest count or classifying the biological sample based on one or more classes having a count over a threshold. 
     
     
         23 . (canceled) 
     
     
         24 . The method of  claim 22 , wherein each of the one or more classes is associated with a threshold, and the threshold for a first class is different than a threshold for a second class.