Explainable artificial intelligence (ai) based image analytic, automatic damage detection and estimation system

Abstract

An Artificial Intelligence (AI) based automatic damage detection and estimation system receives images of a damaged object. The images are converted into monochrome versions if needed and analyzed by an ensemble machine learning (ML) cause prediction model that includes a plurality of sub-models that are each trained to identify a cause of damage to a corresponding portion for the damaged object from a plurality of causes. In addition, an explanation for the selection of the cause from the plurality of causes is also provided. The explanation includes image portions and pixels of images that enabled the cause prediction model to select the cause of damage. An ML parts identification model is also employed to identify and labels parts of the damaged object which are repairable and parts that are damaged and need replacement. The cost estimation for the repair and restoration of the damaged object can also be generated.

What is claimed is: 
     
         1 . An Artificial Intelligence (Al) based automatic damage detection and cost estimation system, the system comprising:
 at least one processor;   a non-transitory processor readable medium storing machine-readable instructions that cause the processor to: 
 provide images of a damaged object to at least one Machine Learning (ML) parts identification model, wherein the at least one ML parts identification model includes at least one object detection model and the at least one ML parts identification model identifies various parts and states of the various parts of the damaged object from the images; 
 obtain labels for the various parts and the states of the various parts, wherein the labels obtained include labels for intact parts, replaceable parts and repairable parts of the damaged object; 
 estimate an extent of damage to the damaged object, the estimation of the extent of damage is based on the labels for the damaged parts, the repairable parts and the non-damaged parts; 
 estimate a cost for restoring the damaged object; and 
 output the estimated extent of damage, portions of the images showing the damaged parts and the repairable parts, and the estimated cost of restoring the damaged object. 
   
     
     
         2 . The automatic damage detection and cost estimation system of  claim 1 , wherein the non-transitory processor further comprises instructions that cause the processor is to:
 receive the images of the damaged object, wherein the damaged object is associated with an object category.   
     
     
         3 . The automatic damage detection and cost estimation system of  claim 1 , wherein the at least one object detection model includes a faster region convolution neural network (R-CNN). 
     
     
         4 . The automatic damage detection and cost estimation system of  claim 3 , wherein to output the portions of the images showing the damaged parts, the processor is to:
 generate region proposal networks (RPNs) identifying the various parts and the states of the various parts of the damaged object;   reshape outputs from the RPNs by employing regions of interest pooling into fixed sizes and shapes.   
     
     
         5 . The automatic damage detection and cost estimation system of  claim 4 , wherein to output the portions of the images showing the damaged parts, the processor is to:
 provide the reshaped regions of interest pooling to a fully connected layer of the faster R-CNN.   
     
     
         6 . The automatic damage detection and cost estimation system of  claim 1 , wherein to obtain the labels for the various parts and the states of the various parts the at least one processor is to:
 train the at least one object detection model on training data of a particular object category corresponding to the damaged object, wherein the at least one object detection model is trained for identifying various parts of the damaged object, wherein the training data includes images of different damaged objects of the particular object category with labeled parts.   
     
     
         7 . The automatic damage detection and cost estimation system of  claim 6 , wherein each of the labeled parts are identified in the training data with qualifiers indicative of whether the part is intact and not damaged, whether the part is damaged but repairable, or whether the part is damaged and needs to be replaced. 
     
     
         8 . The automatic damage detection and cost estimation system of  claim 6 , wherein to train the at least one object detection model, the at least one processor is to:
 train the at least one object detection model on higher epochs.   
     
     
         9 . The automatic damage detection and cost estimation system of  claim 1 , wherein to estimate an extent of damage to the damaged object, the at least one processor is to:
 determine an extent of damage to each of the repairable parts by employing image comparison techniques; and   estimate the extent of damage in terms of damage percentage by calculating a ratio of the damaged parts and the repairable parts to a total number of the various parts identified from the damaged object.   
     
     
         10 . The automatic damage detection and cost estimation system of  claim 9 , wherein to output the estimated extent of damage the at least one processor is to:
 generate a pie chart showing the damage percentage.   
     
     
         11 . The automatic damage detection and cost estimation system of  claim 1 , wherein to estimate the cost for restoring the damaged object, the at least one processor is to:
 retrieve a cost estimation for restoring the damaged object from one or more of a local database and external databases using scripts.   
     
     
         12 . A method of automatically detecting damage and providing a cost estimate for a damaged object, the method comprising:
 receiving images of the damaged object associated with an object category;   providing the images to an Machine Learning (ML) parts identification model, the ML parts identification model includes at least one object detection model that identifies various parts and states of various parts of the damaged object from the images;   obtaining labels for the parts and the states of the parts as an output from the ML parts identification model, 
wherein the labels are obtained for non-damaged parts, damaged parts and repairable parts of the damaged object; 
 estimating an extent of damage to the damaged object based on the obtained labels for the damaged parts, the repairable parts and the non-damaged parts; 
 estimating a cost for restoring the damaged object; and 
 outputting the estimated extent of damage, portions of the images showing the damaged parts and the repairable parts, and the estimated cost of restoring the damaged object. 
 
     
     
         13 . The method of  claim 12 , where obtaining labels for the parts and the states of the parts further comprises:
 training the ML parts identification model that includes a faster region convolution neural network (R-CNN) on training data that includes black and white images of other damaged objects belonging to the object category of the damaged object where each of the images is labeled with respective parts and corresponding states of the respective parts as one of damaged, non-damaged and repairable.   
     
     
         14 . The method of  claim 12 , wherein identifying non-damaged parts, damaged parts and repairable parts of the damaged object by the ML parts identification model, further comprises:
 generating region proposal networks (RPNs) from the images of the damaged object for identifying parts of the damaged object and states of the parts of the damaged object; and   employing region of interest pooling (ROIP) that reshape regions from the RPNs.   
     
     
         15 . The method of  claim 14 , wherein identifying non-damaged parts, damaged parts and repairable parts of the damaged object by the at least one ML parts identification model, further comprises:
 providing the reshaped regions to a fully connected layer of the faster R-CNN that produces an output showing the portions of images with the damaged parts and the repairable parts.   
     
     
         16 . The method of  claim 12 , wherein estimating the cost for restoring the damaged object further comprises:
 executing a script that accesses a third-party data source for obtaining cost for replacement of each of the damaged parts and repairs for each of the repairable parts.   
     
     
         17 . A non-transitory processor-readable storage medium comprising machine-readable instructions that cause a processor to:
 provide images of a damaged object to at least one ML parts identification model, wherein the at least one ML parts identification model includes at least one object detection model and the at least one ML parts identification model identifies various parts and states of the various parts of the damaged object from the images;   obtain labels for the various parts and the states of the various parts, wherein the labels obtained include labels for intact parts, replaceable parts and repairable parts of the damaged object;   estimate an extent of damage to the damaged object, the estimation of the extent of damage is based on the labels for the damaged parts, the repairable parts and the non-damaged parts;   estimate a cost for restoring the damaged object; and   output the estimated extent of damage, portions of the images showing the damaged parts and the repairable parts, and the estimated cost of restoring the damaged object.   
     
     
         18 . The non-transitory processor-readable storage medium of  claim 17 , wherein the at least one object detection model includes a faster region convolution neural network (R-CNN). 
     
     
         19 . The non-transitory processor-readable storage medium of  claim 17 , wherein the at least one object detection model includes multiple object detection models and the non-transitory processor-readable storage medium further comprises instructions that cause the processor to:
 train the multiple object detection models on corresponding training data of specific object categories that includes the object category of the damaged object, wherein each of the multiple object detection models is trained for identifying various parts of damaged objects belonging to an object category of the specific object categories.   
     
     
         20 . The non-transitory processor-readable storage medium of  claim 19 , wherein the training data for each of the multiple object detection models includes images of different damaged objects of the specific object categories with labeled parts that are identified in the training data with qualifiers indicative of whether the part is intact and not damaged, whether the part is damaged but repairable, or whether the part is damaged and needs to be replaced.