Machine vision and robotic installation systems and methods

Abstract

Machine vision methods and systems determine if an object within a work field has one or more predetermined features. Methods comprise capturing image data of the work field, applying a filter to the image data, in which the filter comprises an aspect corresponding to a presumed feature, and based at least in part on the applying, determining if the object has the presumed feature. Systems comprise a camera system configured to capture image data of the work field, and a controller communicatively coupled to the camera system and programmed to apply a filter to the image data, and based at least in part on applying the filter, determine if the object has the specific feature. Robotic installation methods and systems that utilize machine vision methods and systems also are disclosed.

1 . An automated machine vision method for determining if an object within a work field has one or more predetermined features, the automated machine vision method comprising:
 capturing image data of the work field from a camera system;   following the capturing the image data, binary thresholding the image data based on a presumed feature of the one or more predetermined features;   responsive to the binary thresholding, identifying one or more groups of pixels as candidates for the presumed feature;   following the identifying, applying a filter to the identified one or more groups of pixels to create filtered data, wherein the filter comprises an aspect corresponding to the presumed feature; and   based at least in part on the applying, determining if the object has the presumed feature.   
     
     
         2 . The automated machine vision method of  claim 1 , wherein the presumed feature is a specific color. 
     
     
         3 . The automated machine vision method of  claim 1 , wherein the presumed feature is a specific shape. 
     
     
         4 . The automated machine vision method of  claim 1 , wherein the presumed feature is a specific size. 
     
     
         5 . The automated machine vision method of  claim 1 , wherein the presumed feature is specific indicia. 
     
     
         6 . The automated machine vision method of  claim 1 , wherein the presumed feature is a specific texture. 
     
     
         7 . The automated machine vision method of  claim 1 , wherein at least one of:
 (i) the camera system is positioned in a known position and orientation relative to the work field; and   (ii) the automated machine vision method further comprises determining a relative orientation between the camera system and the work field.   
     
     
         8 . The automated machine vision method of  claim 1 , further comprising following the capturing the image data and prior to the applying the filter:
 transforming the image data into HSV domain; and   subtracting, from the image data, data corresponding to portions of the work field that are not the object based on a known color of the portions of the work field that are not the object.   
     
     
         9 . The automated machine vision method of  claim 1 , wherein the presumed feature corresponds to an expected or desired object to be within the work field based on a database associated with the work field. 
     
     
         10 . The automated machine vision method of  claim 1 , further comprising:
 noise filtering the image data following the binary thresholding;   wherein the identifying the one or more groups of pixels is further responsive to the noise filtering.   
     
     
         11 . The automated machine vision method of  claim 1 , wherein the filter is a Gabor filter. 
     
     
         12 . The automated machine vision method of  claim 1 , wherein the filtered data comprises one or more blobs of pixels that are candidates for being representative of the object, and wherein the automated machine vision method further comprises:
 following the applying the filter, analyzing the one or more blobs of pixels to determine presence of one or more blob features, wherein the one or more blob features comprise one or more of blob area, blob eccentricity, blob dimensions, blob brightness, blob correlation, and blob homogeneity.   
     
     
         13 . The automated machine vision method of  claim 12 , wherein the one or more blob features are associated with the one or more predetermined features in a database associated with the work field. 
     
     
         14 . The automated machine vision method of  claim 1 , wherein the camera system is a stereo camera system, wherein the image data comprises two images, and wherein the automated machine vision method further comprises:
 during the capturing the image data, projecting a light texture on the work field;   creating a point cloud of the filtered data, wherein the creating the point cloud comprises generating a disparity map from the two images based on the light texture;   selecting pixels associated with the object; and   comparing the pixels associated with the object to a computer model of an expected or desired object from a database associated with the work field.   
     
     
         15 . The automated machine vision method of  claim 1 , wherein the object comprises a fastener. 
     
     
         16 . A robotic installation method, comprising:
 performing the automated machine vision method of  claim 1 , wherein the camera system is mounted to, mounted with, or mouthed as an end effector of a robotic arm;   based on the determining, instructing the robotic arm to install a component in a predetermined configuration relative to the object; and   installing, using the robotic arm, the component in the predetermined configuration relative to the object.   
     
     
         17 . The robotic installation method of  claim 16 , wherein the object and the component comprise a fastener pair. 
     
     
         18 . A machine vision system for determining if an object within a work field has one or more predetermined features, the machine vision system comprising:
 a camera system configured to capture image data of the work field; and   a controller communicatively coupled to the camera system to receive the image data from the camera system, wherein the controller comprises non-transitory computer-readable media having computer-readable instructions that, when executed, cause the controller to:
 binary threshold the image data based on a presumed feature of the one or more predetermined features; 
 responsive to binary thresholding the image data, identify one or more groups of pixels as candidates for the presumed feature; 
 following the identification of the one or more groups of pixels, apply a filter to the identified one or more groups of pixels to create filtered data, wherein the filter comprises an aspect corresponding to the presumed feature; and 
 based at least in part on application of the filter, determine if the object has the presumed feature. 
   
     
     
         19 . The machine vision system of  claim 18 , wherein the presumed feature comprises one or more of a specific color, a specific shape, a specific size, specific indicia, or a specific texture. 
     
     
         20 . A robotic installation system, comprising:
 the machine vision system of  claim 18 ; and   a robotic arm, wherein the camera system is mounted to, mounted with, or mounted as an end effector of the robotic arm, and wherein the robotic arm is configured to install a component in a predetermined configuration relative to the object based in part on a determination by the controller that the object has the presumed feature;   wherein the computer-readable instructions, when executed, further cause the controller to instruct the robotic arm to install the component in the predetermined configuration relative to the object.