Artificial immunohistochemical image systems and methods

Abstract

The disclosure provides a method of generating an artificial immunohistochemistry (IHC) image of cells. The method includes receiving a hematoxylin and eosin (H&E) stained whole slide image (WSI) generated by a brightfield microscopy imaging modality of at least a portion of cells included in a specimen, applying, to the H&E brightfield image, at least one trained model, the trained model being trained to generate the artificial IHC image based on the H&E brightfield image, receiving the artificial IHC image from the trained model.

1 - 30 . (canceled) 
     
     
         31 . A method of generating an artificial immunohistochemistry (IHC) stained image of cells, without IHC staining, comprising:
 receiving, by a computer system having at least one processor, a hematoxylin and eosin (H&E) stained image generated by a brightfield microscopy imaging modality of at least a portion of cells included in a specimen (brightfield H&E image) derived from a patient;   applying by the processor, to the brightfield H&E image, at least one trained model, wherein the trained model is trained to modify pixel intensities of the brightfield H&E image so as to generate the artificial IHC image of the cells based on the brightfield image; and   generating the artificial IHC image by the trained model.   
     
     
         32 . The method of  claim 31 , wherein the specimen comprises a tissue slice or a blood smear from the patient. 
     
     
         33 . The method of  claim 32 , wherein the artificial IHC image is indicative of whether the portion of the cells included in the specimen are positive or negative for an IHC target molecule. 
     
     
         34 . The method of  claim 32 , wherein the specimen is associated with a colorectal cancer, a gastric cancer, a breast cancer, a lung cancer, an endometrial cancer, a colon cancer, a head and neck cancer, an ovarian cancer, a pancreatic cancer, a gastric cancer, a hepatobiliary cancer, or a genitourinary cancer. 
     
     
         35 . The method of  claim 32 , further comprising:
 outputting the artificial IHC image to a prediction model to predict response of the patient to a treatment method.   
     
     
         36 . The method of  claim 32 , wherein the specimen is included in a first group of specimens, and the method further comprises:
 providing a second brightfield H&E image to the trained model; and   receiving a second artificial IHC image from the trained model, wherein the second brightfield H&E image comprises a second group of tissue slices or blood smears, and the second artificial IHC image is indicative of whether cells included in the second group are positive or negative for the IHC target molecule.   
     
     
         37 . The method of  claim 31 , further comprising:
 providing to the trained model a plurality of brightfield H&E images generated by a brightfield microscopy imaging modality of the at least a portion of cells, the plurality of brightfield H&E images generated after the brightfield H&E image is generated; and   receiving a corresponding plurality of artificial IHC images from the trained model.   
     
     
         38 . The method of  claim 31  further comprising:
 generating a report based on the artificial IHC image. 
 
     
     
         39 . The method of  claim 31 , wherein the trained model is trained based on a loss function comprising a discriminator loss, or a generator loss and a discriminator loss. 
     
     
         40 . The method of  claim 31  further comprising preprocessing the brightfield H&E image to increase contrast levels. 
     
     
         41 . The method of  claim 31  further comprising preprocessing brightfield H&E images and IHC images included in training data used to train the trained model. 
     
     
         42 . The method of  claim 31 , wherein the trained model comprises a generator, the generator being trained in part by a discriminator. 
     
     
         43 . The method of  claim 31 , further comprising analyzing nucleic acid data associated with the specimen, and including the results of the analysis in a report. 
     
     
         44 . The method of  claim 31 , further comprising:
 associating the artificial IHC image with information about the specimen; and   providing the artificial IHC image and the associated information about the specimen to a database comprising at least seven hundred and fifty artificial IHC images.   
     
     
         45 . The method of  claim 44 , wherein the information about the specimen comprises a cancer diagnosis associated with the specimen. 
     
     
         46 . The method of  claim 44  further comprising:
 applying one or more drugs to the patient prior to the generation of the brightfield H&E image; 
 wherein the information about the specimen comprises a diagnosis associated with the specimen and an identification of the one or more drugs. 
 
     
     
         47 . A brightfield microscopy imaging modality configured to execute the method of  claim 31 . 
     
     
         48 . The brightfield microscopy imaging modality of  claim 47 , wherein the brightfield microscope imaging modality comprises a brightfield microscope. 
     
     
         49 . A pathology slide analysis system comprising at least one processor and at least one memory, the system configured to
 receive an H&E stained image generated by a brightfield microscopy imaging modality from at least a portion of cells included in a specimen (brightfield H&E image);   apply, via the processor, to the brightfield H&E image, at least one model trained to modify pixel intensities of the brightfield H&E image so as to generate an artificial IHC image based on the brightfield H&E image without IHC staining, the artificial IHC image being indicative of whether the cells included in the specimen are positive or negative for an IHC target molecule;   
       generating the artificial IHC image by the trained model; and
 output the artificial IHC image to at least one of a memory or a display. 
 
     
     
         50 . A method of generating an artificial IHC image of cells without IHC stain, comprising;
 receiving, from a computer system having at least one processor, an H&E stained image generated by a brightfield microscopy imaging modality from at least a portion of cells included in a specimen (brightfield H&E image);   applying, by the processor, to the H&E brightfield image, at least one model trained to modify pixel intensities of the H&E brightfield image so as to generate an artificial IHC image of the cells based on the H&E brightfield image, the artificial IHC image being indicative of whether the cells included in the specimen are positive or negative for an IHC target molecule;   
       generating the artificial IHC image by the trained model; and
 generating a report based on the artificial IHC image of the cells.