Invariant features for computer vision

Abstract

Technology is described for determining and using invariant features for computer vision. A local orientation may be determined for each depth pixel in a subset of the depth pixels in a depth map. The local orientation may an in-plane orientation, an out-out-plane orientation or both. A local coordinate system is determined for each of the depth pixels in the subset based on the local orientation of the corresponding depth pixel. A feature region is defined relative to the local coordinate system for each of the depth pixels in the subset. The feature region for each of the depth pixels in the subset is transformed from the local coordinate system to an image coordinate system of the depth map. The transformed feature regions are used to process the depth map.

1 . A method, comprising:
 accessing a depth map that includes a plurality of depth pixels, the depth map is associated with an image coordinate system having a plane;   estimating a local orientation for each depth pixel in a subset of the depth pixels, the local orientation is one or both of an in-plane orientation and an out-out-plane orientation relative to the plane of the image coordinate system;   defining a local coordinate system for each of the depth pixels in the subset, each local coordinate system is based on the local orientation of the corresponding depth pixel;   defining a feature region relative to the local coordinate system for each of the depth pixels in the subset;   transforming the feature region for each of the depth pixels in the subset from the local coordinate system to the image coordinate system; and   using the transformed feature regions to process the depth map.   
     
     
         2 . The method of  claim 1 , wherein the using the transformed feature regions to process the depth map includes:
 classifying features from the depth map based on the transformed feature regions.   
     
     
         3 . The method of  claim 1 , wherein the using the transformed feature regions to process the depth map includes:
 training a machine learning classifier based on the transformed feature regions.   
     
     
         4 . The method of  claim 1 , wherein the estimating a local orientation includes estimating an in-plane rotation, the in-plane rotation is a rotation invariant angle at a point of interest in the depth map. 
     
     
         5 . The method of  claim 1 , wherein the estimating a local orientation includes estimating an out-of-plane rotation, the out-of-plane rotation is defined by a surface normal to an object in the depth map at a test depth pixel. 
     
     
         6 . The method of  claim 5 , wherein the defining a local coordinate system for each of the depth pixels in the subset includes defining a z-axis of the local coordinate system as being the surface normal. 
     
     
         7 . The method of  claim 1 , wherein the defining a local coordinate system for each of the depth pixels in the subset includes:
 determining a vector in 3D space from a test depth pixel to a point of interest;   determining a surface normal to an object in the depth map; and   establishing an x-axis that is the component of the vector in 3D space that is orthogonal to the surface normal.   
     
     
         8 . The method of  claim 7 , wherein the defining a local coordinate system for each of the depth pixels in the subset includes establishing a y-axis as the cross product of the z-axis and the x-axis. 
     
     
         9 . A system comprising:
 a depth camera for generating depth maps that includes a plurality of depth pixels, each pixel having a depth value, each depth map is associated with a 2D image coordinate system;   logic coupled to the depth camera, the logic is operable to:   access a depth map from the depth camera, the depth map is associated with an image coordinate system having a plane;   estimate a local orientation for each depth pixel in a subset of the depth pixels, the local orientation includes one or both of an in-plane orientation that is in the plane of the 2D image coordinate system and an out-out-plane orientation that is out-of-the plane of the 2D image coordinate system;   define a local 3D coordinate system for each of the depth pixels in the subset, each local 3D coordinate system is based on the local orientation of the corresponding depth pixel;   define a feature region relative to the local coordinate system for each of the depth pixels in the subset;   transform the feature region for each of the depth pixels in the subset from the local 3D coordinate system to the 2D image coordinate system; and   identify an object in the depth map based on the transformed feature regions.   
     
     
         10 . The system of  claim 9 , wherein the estimating a local orientation includes estimating an in-plane rotation, determining the in-plane rotation includes:
 determining a closest point between the test depth pixel and a reference line of the object; and   determining a rotation invariant angle for the closest point.   
     
     
         11 . The system of  claim 9 , wherein the estimating a local orientation includes estimating an out-of-plane rotation, the out-of-plane rotation is defined by a surface normal to an object in the depth map at a test depth pixel. 
     
     
         12 . The system of  claim 11 , wherein the defining a local coordinate system for each of the depth pixels in the subset includes defining a z-axis of the local coordinate system as being the surface normal. 
     
     
         13 . The system of  claim 9 , wherein the defining a local coordinate system for each of the depth pixels in the subset includes:
 determining a vector in 3D space from a test depth pixel to a point of interest;   determining a surface normal to an object in the depth map; and   establishing an x-axis that is the component of the vector in 3D space that is orthogonal to the surface normal.   
     
     
         14 . The system of  claim 9 , wherein the defining a local coordinate system for each of the depth pixels in the subset includes establishing a y-axis as the cross product of the z-axis and the x-axis. 
     
     
         15 . A computer readable storage medium having instructions stored thereon which, when executed on a processor, cause the processor to perform the steps of:
 accessing a depth map that includes an array of depth pixels, each depth pixel has a depth value, the depth map is associated with a 2D image coordinate system;   estimating a local orientation for each depth pixel in a subset of the depth pixels, the local orientation includes in-plane orientation that is in the plane of the 2D image coordinate system and an out-of-plane orientation that is out-of-the plane of the 2D image coordinate system;   determining a 3D model for the depth map, the model includes a plurality of 3D points that are based on the depth pixels, each of the points has a corresponding depth pixel;   defining a local 3D coordinate system for each of the plurality of points, each local 3D coordinate system is based on the position and the local orientation of the corresponding depth pixel;   defining feature test points relative to the local coordinate system for each of the points;   transforming the feature test points from the local 3D coordinate system to the 2D image coordinate system for each of the feature test points; and   identifying an object in the depth map based on the transformed feature test points.   
     
     
         16 . The computer readable storage medium of  claim 15 , wherein the transforming the feature test points includes rotating the feature test points using a rotation matrix R=[{right arrow over (r 1 )}, {right arrow over (r 2 )}, {right arrow over (r 3 )}], where {right arrow over (r 3 )} is a unitized surface normal to the object, {right arrow over (r 1 )} is the component of an in-plane rotation-variant vector that is orthogonal to the surface normal, and {right arrow over (r 2 )} is the cross product between {right arrow over (r 3 )} and {right arrow over (r 1 )}. 
     
     
         17 . The computer readable storage medium of  claim 15 , wherein the in-plane rotation is a rotation invariant angle at a reference point to the test depth pixel. 
     
     
         18 . The computer readable storage medium of  claim 15 , wherein the out-of-plane rotation is defined by a surface normal to the object at the test depth pixel, the defining a local coordinate system for each of the plurality of points includes defining a z-axis of the local coordinate system as being the surface normal. 
     
     
         19 . The computer readable storage medium of  claim 15 , wherein the defining a local coordinate system for each of the plurality of points includes:
 determining a vector in 3D space from a first of the points to a point of interest;   determining a surface normal to the object; and   establishing an x-axis that is the component of the test feature vector that is orthogonal to the surface normal.   
     
     
         20 . The computer readable storage medium of  claim 19 , wherein the defining a local coordinate system for each of the plurality of points includes establishing a y-axis as the cross product of the z-axis and the x-axis.