Image segmentation method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides an image segmentation method and apparatus, an electronic device, and a storage medium. The image segmentation method including: obtaining an image to be segmented; determining a preliminarily segmented image and a target normal vector image that are corresponding to the image to be segmented; and performing image fusion on the preliminarily segmented image and the target normal vector image to obtain a target segmented image.

1 . An image segmentation method, comprising:
 obtaining an image to be segmented;   determining a preliminarily segmented image and a target normal vector image that are corresponding to the image to be segmented; and   performing image fusion on the preliminarily segmented image and the target normal vector image to obtain a target segmented image.   
     
     
         2 . The method according to  claim 1 , wherein the determining the preliminarily segmented image and the target normal vector image that are corresponding to the image to be segmented comprises:
 inputting the image to be segmented to an image segmentation model that has been pre-trained to obtain the preliminarily segmented image and the target normal vector image that are corresponding to the image to be segmented, wherein the image segmentation model is trained based on a sample segmented image, a segmentation marked image corresponding to the sample segmented image, and a sample normal vector image corresponding to the sample segmented image.   
     
     
         3 . The method according to  claim 2 , before the inputting the image to be segmented to the image segmentation model that has been pre-trained, further comprising:
 with the sample segmented image as an input image to a big model that has been pre-established, and with the segmentation marked image and the sample normal vector image that are corresponding to the sample segmented image as expected output images of the big model, training the big model to obtain a teacher model; and   with the sample segmented image as an input image to a small model that has been pre-established, and with a big model segmented image and a big model normal vector image that are corresponding to the sample segmented image and are output by the teacher model as expected outputs of the small model, training the small model to obtain the image segmentation model.   
     
     
         4 . The method according to  claim 3 , wherein the training the big model to obtain the teacher model comprises:
 inputting the sample segmented image to the big model that has been pre-established to obtain the big model segmented image and the big model normal vector image;   calculating a big model segmentation loss between the big model segmented image and the segmentation marked image corresponding to the sample segmented image, and calculating a big model normal vector loss between the big model normal vector image and the sample normal vector image corresponding to the sample segmented image; and   adjusting a model parameter of the big model based on the big model segmentation loss and the big model normal vector loss to obtain the teacher model.   
     
     
         5 . The method according to  claim 4 , wherein the calculating the big model segmentation loss between the big model segmented image and the segmentation marked image corresponding to the sample segmented image comprises:
 calculating the big model segmentation loss between the big model segmented image and the segmentation marked image corresponding to the sample segmented image according to a binary cross entropy loss function; or   calculating the big model segmentation loss between the big model segmented image and the segmentation marked image corresponding to the sample segmented image according to the binary cross entropy loss function and a regional mutual information loss function.   
     
     
         6 . The method according to  claim 4 , wherein the calculating the big model normal vector loss between the big model normal vector image and the sample normal vector image corresponding to the sample segmented image comprises:
 calculating the big model normal vector loss between the big model normal vector image and the sample normal vector image corresponding to the sample segmented image according to a mean square error loss function.   
     
     
         7 . The method according to  claim 3 , wherein the training the small model comprises:
 inputting the sample segmented image to the small model that has been pre-established as an input image to obtain a small model segmented image and a small model normal vector image;   calculating a small model segmentation output loss based on the small model segmented image of the sample segmented image, the segmentation marked image, and the big model segmented image output by the teacher model;   calculating a small model normal vector output loss based on the small model normal vector image of the sample segmented image, the sample normal vector image, and the big model normal vector image output by the teacher model; and   adjusting a model parameter of the small model based on the small model segmentation output loss and the small model normal vector output loss to obtain the image segmentation model.   
     
     
         8 . The method according to  claim 7 , further comprising:
 inputting the small model segmented image output by the small model to a segmented image discriminator that has been pre-trained to obtain a segmentation discrimination result, and determining a segmentation discrimination loss based on the segmentation discrimination result and an expected discrimination result, wherein the segmented image discriminator is trained with the big model segmented image corresponding to the sample segmented image output by the teacher model as a real sample and the small model segmented image output by the small model as a fake sample; and   the adjusting the model parameter of the small model based on the small model segmentation output loss and the small model normal vector output loss comprises:   adjusting a model segmentation parameter of the small model based on the small model segmentation output loss and the segmentation discrimination loss; and   adjusting a model normal vector parameter of the small model based on the small model normal vector output loss.   
     
     
         9 . The method according to  claim 7 , wherein the calculating the small model segmentation output loss based on the small model segmented image of the sample segmented image, the segmentation marked image, and the big model segmented image output by the teacher model comprises:
 calculating a first small model segmentation loss between the small model segmented image of the sample segmented image and the segmentation marked image according to a binary cross entropy loss function, or the binary cross entropy loss function and a regional mutual information loss function;   calculating a second small model segmentation loss between the small model segmented image and the big model segmented image output by the teacher model according to a kullback-leibler divergence loss function; and   determining the small model segmentation output loss based on the first small model segmentation loss and the second small model segmentation loss.   
     
     
         10 . The method according to  claim 7 , wherein the calculating the small model normal vector output loss based on the small model normal vector image of the sample segmented image, the sample normal vector image, and the big model normal vector image output by the teacher model comprises:
 calculating a first small model normal vector loss between the small model normal vector image of the sample segmented image and the sample normal vector image according to a mean square error loss function;   calculating a second small model normal vector loss between the small model normal vector image and the big model normal vector image output by the teacher model according to a kullback-leibler divergence loss function; and   determining the small model normal vector output loss based on the first small model normal vector loss and the second small model normal vector loss.   
     
     
         11 . The method according to  claim 7 , further comprising:
 inputting the small model normal vector image output by the small model to a normal vector image discriminator that has been pre-trained to obtain a normal vector discrimination result, and determining a normal vector discrimination loss based on the normal vector discrimination result and an expected discrimination result, wherein the normal vector image discriminator is trained with the big model normal vector image corresponding to the sample segmented image output by the teacher model as a real sample and the small model normal vector image output by the small model as a fake sample; and   the adjusting the model parameter of the small model based on the small model segmentation output loss and the small model normal vector output loss comprises:   adjusting a model segmentation parameter of the small model based on the small model segmentation output loss; and   adjusting a model normal vector parameter of the small model based on the small model normal vector output loss and the normal vector discrimination loss.   
     
     
         12 . The method according to  claim 1 , wherein the performing image fusion on the preliminarily segmented image and the target normal vector image to obtain the target segmented image comprises:
 for each pixel in the preliminarily segmented image, determining a predicted weight of the pixel based on a predicted pixel value of the pixel in the target normal vector image, and a preset segmentation threshold;   weighting a pixel value of the pixel in the preliminarily segmented image based on the predicted weight to obtain a target pixel value of the pixel; and   determining the target segmented image based on the target pixel value of each pixel in the preliminarily segmented image.   
     
     
         13 . The method according to  claim 1 , after the performing image fusion on the preliminarily segmented image and the target normal vector image, further comprising:
 obtaining shooting angle information of an image shooting apparatus for shooting the image to be segmented, and adjusting the target segmented image based on the shooting angle information.   
     
     
         14 . (canceled) 
     
     
         15 . An electronic device, comprising:
 a processor; and   a storage apparatus, configured to store a program,   wherein the program, when executed by the processor, causes the processor to implement-the an image segmentation method, wherein the method comprising:   obtaining an image to be segmented;   determining a preliminarily segmented image and a target normal vector image that are corresponding to the image to be segmented; and   performing image fusion on the preliminarily segmented image and the target normal vector image to obtain a target segmented image.   
     
     
         16 . A storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, cause implementing the image segmentation method according to  claim 1 . 
     
     
         17 . The electronic device according to  claim 15 , wherein the determining the preliminarily segmented image and the target normal vector image that are corresponding to the image to be segmented comprises:
 inputting the image to be segmented to an image segmentation model that has been pre-trained to obtain the preliminarily segmented image and the target normal vector image that are corresponding to the image to be segmented, wherein the image segmentation model is trained based on a sample segmented image, a segmentation marked image corresponding to the sample segmented image, and a sample normal vector image corresponding to the sample segmented image.   
     
     
         18 . The electronic device according to  claim 17 , before the inputting the image to be segmented to the image segmentation model that has been pre-trained, further comprising:
 with the sample segmented image as an input image to a big model that has been pre-established, and with the segmentation marked image and the sample normal vector image that are corresponding to the sample segmented image as expected output images of the big model, training the big model to obtain a teacher model; and   with the sample segmented image as an input image to a small model that has been pre-established, and with a big model segmented image and a big model normal vector image that are corresponding to the sample segmented image and are output by the teacher model as expected outputs of the small model, training the small model to obtain the image segmentation model.   
     
     
         19 . The electronic device according to  claim 18 , wherein the training the big model to obtain the teacher model comprises:
 inputting the sample segmented image to the big model that has been pre-established to obtain the big model segmented image and the big model normal vector image;   calculating a big model segmentation loss between the big model segmented image and the segmentation marked image corresponding to the sample segmented image, and calculating a big model normal vector loss between the big model normal vector image and the sample normal vector image corresponding to the sample segmented image; and   adjusting a model parameter of the big model based on the big model segmentation loss and the big model normal vector loss to obtain the teacher model.   
     
     
         20 . The electronic device according to  claim 18 , wherein the training the small model comprises:
 inputting the sample segmented image to the small model that has been pre-established as an input image to obtain a small model segmented image and a small model normal vector image;   calculating a small model segmentation output loss based on the small model segmented image of the sample segmented image, the segmentation marked image, and the big model segmented image output by the teacher model;   calculating a small model normal vector output loss based on the small model normal vector image of the sample segmented image, the sample normal vector image, and the big model normal vector image output by the teacher model; and   adjusting a model parameter of the small model based on the small model segmentation output loss and the small model normal vector output loss to obtain the image segmentation model.   
     
     
         21 . The electronic device according to  claim 15 , wherein the performing image fusion on the preliminarily segmented image and the target normal vector image to obtain the target segmented image comprises:
 for each pixel in the preliminarily segmented image, determining a predicted weight of the pixel based on a predicted pixel value of the pixel in the target normal vector image, and a preset segmentation threshold;   weighting a pixel value of the pixel in the preliminarily segmented image based on the predicted weight to obtain a target pixel value of the pixel; and
 determining the target segmented image based on the target pixel value of each pixel in the preliminarily segmented image.