Estimating color of vehicles on a roadway

Abstract

The present invention extends to methods, systems, and computer program products for estimating color of vehicles on a roadway. The color can be at least estimated from captured imagery, such as, frames of a video stream. The captured imagery can be captured by variably configured traffic cameras, other public cameras, private cameras (e.g., CCTV), or virtually any other camera, etc., which may have different operating conditions. Color confidence scores can be derived based on image characteristics.

What is claimed: 
     
         1 . A method comprising:
 detecting a vehicle in a camera image;   computing a bounding box around the vehicle detection;   calculating a vehicle confidence score that the vehicle detection indicates a specified vehicle type;   selecting another bounding box indicative of the specific vehicle type from one or more additional bounding boxes based on additional calculated vehicle confidence scores;   deriving a vehicle color from camera image pixel color values inside the selected other bounding box;   computing a color confidence score that the derived vehicle color is an actual vehicle color based on: the vehicle detection, the derived vehicle color, an image darkness level of the camera image, an image contrast of the camera image, and an image entropy of the camera image; and   sending the vehicle detection, the derived vehicle color, and the color confidence score to an entity in response to an entity request.   
     
     
         2 . The method of  claim 1 , wherein deriving a vehicle color from camera image pixel color values comprises:
 using a convolutional neural network to compute a first color and corresponding first color confidence score and a second color and corresponding second color confidence score;   determining that the first color confidence score is indicative of a higher confidence than the second color confidence score; and   matching the first color to the vehicle detection; and   wherein sending the vehicle detection, the derived vehicle color, and the color confidence score to an entity in response to an entity request comprises sending the vehicle detection, the first color, and the color confidence score to the entity.   
     
     
         3 . The method of  claim 1 , wherein deriving a vehicle color from camera image pixel color values inside the selected bounding box comprises:
 dividing the content of the selected bounding box into a plurality of variable sized clusters;   for each variable sized cluster included in the plurality of variable sized clusters:
 computing a mean color value of pixels contained in the variable sized cluster; and 
 determining what percentage of the bounding box is accounted for by the variable sized cluster; and 
   computing the vehicle color based on the mean color values and percentages.   
     
     
         4 . The method of  claim 1 , further comprising formulating the one or more additional bounding boxes; and
 wherein selecting the other bounding box comprises filtering out the other bounding box from the one or more additional bounding boxes based on the vehicle confidence score calculated for the selected bounding box.   
     
     
         5 . The method of  claim 1 , further comprising:
 receiving a video stream from a camera; and   accessing the camera image from the video stream.   
     
     
         6 . The method of  claim 1 , further comprising:
 receiving a cached video stream from a storage device; and   accessing the camera image from the cached video stream.   
     
     
         7 . The method of  claim 1 , wherein computing a color confidence score comprises computing the color confidence score in accordance with a tiered scoring module. 
     
     
         8 . The method of  claim 1 , further comprising calculating the image entropy including:
 computing a greyscale image histogram; and   computing pixel entropy values.   
     
     
         9 . A system comprising:
 a processor;   system memory coupled to the processor and storing instructions configured to cause the processor to:
 detect a vehicle in a camera image; 
 compute a bounding box around the vehicle detection; 
 calculate a vehicle confidence score that the detected vehicle is a specified vehicle type; 
 select another bounding box indicative of the specific vehicle type from one or more additional bounding boxes based on additional calculated vehicle confidence scores; 
 derive a vehicle color from camera image pixel color values inside the selected other bounding box; 
 compute a color confidence score that the derived vehicle color is an actual vehicle color based on: detecting the vehicle, the derived vehicle color, an image darkness level of the camera image, an image contrast of the camera image, and an image entropy of the camera image; and 
 send the vehicle detection, the derived vehicle color, and the color confidence score to an entity in response to an entity request. 
   
     
     
         10 . The system of  claim 9 , wherein instructions configured to derive a vehicle color from camera image pixel color values comprise instructions configured to:
 use a convolutional neural network to compute a first color and corresponding first color confidence score and a second color and corresponding second color confidence score;   determine that the first color confidence score is indicative of a higher confidence than the second color confidence score; and   match the first color to the vehicle detection; and   wherein instructions configured to send the vehicle detection, the derived vehicle color, and the color confidence score to an entity in response to an entity request comprise instructions configured to send the vehicle detection, the first color, and the color confidence score to the entity.   
     
     
         11 . The system of  claim 9 , wherein instructions configured to derive a vehicle color from camera image pixel color values inside the selected bounding box comprise instructions configured to:
 divide the content of the selected bounding box into a plurality of variable sized clusters;   for each variable sized cluster included in the plurality of variable sized clusters:
 compute a mean color value of pixels contained in the variable sized cluster; and 
 determine what percentage of the bounding box is accounted for by the variable sized cluster; and 
   compute the vehicle color based on the mean color values and percentages.   
     
     
         12 . The system of  claim 9 , further comprising instructions configured to formulate the one or more additional bounding boxes; and
 wherein instructions configured to select a bounding box indicative of a specific vehicle type comprise instructions configured to filter out the selected bounding box from the plurality of bounding boxes based on the confidence score calculated for the selected bounding box.   
     
     
         13 . The system of  claim 9 , further comprising instructions configured to:
 receive a video stream from a camera; and   access the camera image from the video stream.   
     
     
         14 . The system of  claim 9 , further comprising instructions configured to:
 receive a cached video stream from a storage device; and   access the camera image from the cached video stream.   
     
     
         15 . The system of  claim 9 , wherein instructions configured to compute a color confidence score comprise instructions configured to compute the color confidence score in accordance with a tiered scoring module. 
     
     
         16 . The system of  claim 9 , further comprising instructions configured to calculate the image entropy including instructions configured to:
 compute a greyscale image histogram; and   compute pixel entropy values.   
     
     
         17 . A system comprising:
 a processor;   system memory coupled to the processor and storing instructions configured to cause the processor to:
 receive a request from an entity to determine vehicle colors on a roadway; 
 query a camera database; 
 receive an indication of one or more cameras having a field of view including at least a portion of the roadway; 
 select a camera from among the one or more cameras; 
 access an image associated with the camera; 
 detect a vehicle in the image; 
 compute a bounding box around the detected vehicle; 
 calculate a vehicle confidence score that the detected vehicle is a specified vehicle type; 
 select another bounding box indicative of the specific vehicle type from one or more additional bounding boxes based on additional calculated vehicle confidence scores; 
 derive a vehicle color from camera image pixel color values inside the selected other bounding box; 
 compute a color confidence score that the derived vehicle color is an actual vehicle color based on: detecting the vehicle, the derived vehicle color, an image darkness level of the camera image, an image contrast of the camera image, and an image entropy of the camera image; and 
 send the vehicle detection, the derived vehicle color, and the color confidence score to an entity in response to an entity request. 
   
     
     
         18 . The system of  claim 17 , further comprising instructions configured to access a video stream from the camera; and
 wherein instructions configured to access an image associated with the camera comprise instructions configured to access the camera image from the video stream.   
     
     
         19 . The system of  claim 18 , wherein instructions configured to access a video stream from the camera comprise instructions configured to access a live video stream from the camera 
     
     
         20 . The system of  claim 17 , further comprising instructions configured to access a cached video stream from a storage device; and
 wherein instructions configured to access an image associated with the camera comprise instructions configured to access the camera image from the cached video stream.