Methods and systems for generating colorized organoid images

Abstract

In some embodiments, a method for generating colorized organoid images comprises synthesizing, by a generator, an input feature map from a grayscale image derived from a lightness channel as a conditional input; extracting, by a convolution block attention layer (CBAL); and generating, by the CBAL, a refined output feature map derived from a 1D channel attention map and a 2D spatial attention map. The method further comprises synthesizing, by the generator, color information derived from the lightness channel and the refined output feature map, calculating, by a discriminator based on at least the lightness channel and the color information, a value indicating a probability that the color information is real; and performing the aforementioned steps iteratively until the generator produces the color information which the discriminators can no longer identify as fake.

What is claimed is: 
     
         1 . A method for generating colorized organoid images, the method comprising:
 synthesizing, by a generator, an input feature map from a grayscale image derived from a lightness channel as a conditional input;   extracting, by a convolution block attention layer (CBAL), a 1D channel attention map and a 2D spatial attention map derived from the input feature map;   generating, by the CBAL, a refined output feature map derived from the 1D channel attention map and the 2D spatial attention map;   synthesizing, by the generator, color information derived from the lightness channel and the refined output feature map, wherein the color information comprises an a* channel and a b* channel;   calculating, by a discriminator based on at least the lightness channel and the color information, a value indicating a probability that the color information is real; and   performing the aforementioned steps iteratively until the generator produces the color information which the discriminators can no longer identify as fake.   
     
     
         2 . The method of  claim 1 , wherein the generator comprises a U-net generator. 
     
     
         3 . The method of  claim 1 , wherein the CBAL is integrated within the generator. 
     
     
         4 . The method of  claim 1 , wherein the generator comprises an encoder and a decoder, wherein the method further comprises:
 reducing with the encoder a spatial dimension of the grayscale image while extracting features; and   upsampling with the decoder the extracted features to synthesize the color information.   
     
     
         5 . The method of  claim 4 , wherein the encoder and the decoder are connected by a bottleneck layer. 
     
     
         6 . The method of  claim 1 , wherein the color information is coherent with the lightness channel. 
     
     
         7 . The method of  claim 1 , wherein the discriminator is a convolutional neural network. 
     
     
         8 . The method of  claim 1 , wherein the grayscale image is a cardiac organoid image. 
     
     
         9 . The method of  claim 1 , further comprising evaluating the color information with a weighted patch histogram. 
     
     
         10 . The method of  claim 1 , further comprising calculating patches, by a patch discriminator to distinguish between real colorized organoid patches and fake patches by the generator. 
     
     
         11 . The method of  claim 2 , further comprising calculating patches, by a patch discriminator to distinguish between real colorized organoid patches and fake patches by the U-net generator. 
     
     
         12 . The method of  claim 11 , further comprising calculating a discriminator loss. 
     
     
         13 . The method of  claim 12 , wherein the discriminator maximizes the discriminator loss to correctly classify real and fake patches within images. 
     
     
         14 . The method of  claim 1 , further comprising evaluating accuracy and quality of a generated image by using at least one evaluation metric of PSNR, SSIM, WPH, or a combination thereof. 
     
     
         15 . A method for generating colorized organoid images, the method comprising:
 synthesizing, by a patch generator, an input feature map from a grayscale image derived from a lightness channel as a conditional input;   extracting, by a convolution block attention layer (CBAL), one or more patches of a 1D channel attention map and a 2D spatial attention map derived from the input feature map;   generating, by the CBAL, the one or more patches of a refined output feature map derived from the 1D channel attention map and the 2D spatial attention map;   synthesizing, by the generator, the one or more patches of color information derived from the lightness channel and the refined output feature map, wherein the color information comprises an a* channel and a b* channel;   calculating, by a discriminator based on at least the lightness channel and the color information, a value indicating a probability that the color information is real; and   the generator and the discriminator performing the aforementioned steps iteratively until the generator produces the color information which the discriminators can no longer identify as fake for the one or more patches.   
     
     
         16 . The method of  claim 15 , further comprising evaluating accuracy and quality of a generated image by using at least one evaluation metric of PSNR, SSIM, WPH, or a combination thereof. 
     
     
         17 . The method of  claim 15 , wherein the discriminator is a convolutional neural network. 
     
     
         18 . The method of  claim 15 , further comprising calculating a generator loss to produce colorizations to convincingly fool the discriminator into classifying them as real. 
     
     
         19 . The method of  claim 18 , wherein the generator loss comprises a binary cross-entropy loss and an L1 loss. 
     
     
         20 . A system for capturing fluorescence intricacies of cardiovascular cells (CMs, ECs, and SMCs) in hPSC-derived cardiac organoids, comprising:
 a generator for synthesizing an input feature map from a grayscale image derived from lightness channel as a conditional input;   a convolution block attention layer (CBAL) for extracting a 1D channel attention map and 2D spatial attention map derived from the input feature map, and generating a refined output feature map derived from the 1D channel attention map and the 2D spatial attention map;   the generator for synthesizing by color information derived from the lightness channel and;   the refined output feature map, wherein the color information comprises an a* channel and a b* channel;   a discriminator calculating based on at least the lightness channel and the color information, a value indicating a probability that the color information is real; and   the generator and the discriminator performing the aforementioned steps iteratively until the generator produces the color information which the discriminators can no longer identify as fake.