Machine Learning Systems and Methods for Using an Orthogonality Heuristic to Identify an Ignored Labeling Target

Abstract

Among a great deal of other disclosure and scope, systems and methods are enclosed that enable efficient assessment of the currently known manifolds within a problem space. A set of labeled vectors is identified as well as a set of unlabeled vectors. An angular based comparison is made between each unlabeled vector and each labeled vector. If the smallest angle between a given unlabeled vector and any of the labeled vectors is deemed satisfactory, such as when the angle is small and acute, the vector is deemed not crucial to obtain information regarding. However, if the smallest between a given unlabeled vector and any of the labeled vectors is deemed large, such as when the angle is orthogonal to the labeled set, then the given vector possesses vital information pivotal to learning our problem space. All such vectors are ranked, with the unlabeled vectors with the largest angles to our labeled set sent to our oracle first in order to improve our labeled set of vectors.

What is claimed is: 
     
         1 . A method for identifying an ignored labeling target, the method comprising:
 receiving, by a processing resource, a set of vectors including at least an unlabelled vector, a first labelled vector, and a second labelled vector;   calculating, by the processing resource, a first angle between the unlabelled vector and the first labelled vector, and a second angle between the unlabelled vector and the second labelled vector; and   using, by the processing resource, a combination of the first angle and the second angle to determine a labeling value of the unlabelled vector.   
     
     
         2 . The method of  claim 1 , wherein using the combination of the first angle and the second angle to determine a labeling value of the unlabelled vector includes:
 determining, by the processing resource, that the first angle is less than the second angle; and   identifying, by the processing resource, the first angle as a minimum angle based at least in part on determining that the first angle is less than the second angle.   
     
     
         3 . The method of  claim 2 , wherein using the combination of the first angle and the second angle to determine a labeling value of the unlabelled vector further includes:
 comparing, by the processing resource, the minimum angle with a threshold value.   
     
     
         4 . The method of  claim 3 , wherein using the combination of the first angle and the second angle to determine a labeling value of the unlabelled vector further includes:
 identifying, by the processing resource, the unlabelled vector as a high value labeling target where the minimum angle exceeds the threshold value.   
     
     
         5 . The method of  claim 3 , wherein the threshold value is user programmable. 
     
     
         6 . The method of  claim 1 , the method further comprising:
 using, by the processing resource, the labeling value of the unlabelled vector along with the result of at least one other heuristic to rank the unlabelled vector relative to other unlabelled vectors.   
     
     
         7 . The method of  claim 6 , wherein the at least one other heuristic is selected from a group consisting of: a Shannon's entropy heuristic, a confidence based heuristic, a distance from decision hyperplane heuristic, an information density heuristic, a perturbation heuristic, an expected gradient length heuristic, and a consensus based heuristic. 
     
     
         8 . A system for identifying an ignored labeling target, the system comprising:
 a processing resource;   a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the processing resource to:
 receive a set of vectors including at least an unlabelled vector, a first labelled vector, and a second labelled vector; 
 calculate a first angle between the unlabelled vector and the first labelled vector, and a second angle between the unlabelled vector and the second labelled vector; and 
 use a combination of the first angle and the second angle to determine a labeling value of the unlabelled vector. 
   
     
     
         9 . The system of  claim 1 , wherein the instructions that when executed by the processing resource cause the processing resource to use the combination of the first angle and the second angle to determine the labeling value of the unlabelled vector include instructions that cause the processing resource to:
 determine that the first angle is less than the second angle; and   identify the first angle as a minimum angle based at least in part on determining that the first angle is less than the second angle.   
     
     
         10 . The system of  claim 9 , wherein the instructions that when executed by the processing resource cause the processing resource to the combination of the first angle and the second angle to determine a labeling value of the unlabelled vector further include instructions that cause the processing resource to:
 compare the minimum angle with a threshold value.   
     
     
         11 . The system of  claim 10 , wherein the instructions that when executed by the processing resource cause the processing resource to the combination of the first angle and the second angle to determine a labeling value of the unlabelled vector further include instructions that cause the processing resource to:
 identify the unlabelled vector as a high value labeling target where the minimum angle exceeds the threshold value.   
     
     
         12 . The system of  claim 10 , wherein the threshold value is user programmable. 
     
     
         13 . The system of  claim 8 , wherein the instructions that when executed by the processing resource further cause the processing resource to:
 using the labeling value of the unlabelled vector along with the result of at least one other heuristic to rank the unlabelled vector relative to other unlabelled vectors.   
     
     
         14 . The system of  claim 13 , wherein the at least one other heuristic is selected from a group consisting of: a Shannon's entropy heuristic, a confidence based heuristic, a distance from decision hyperplane heuristic, an information density heuristic, a perturbation heuristic, an expected gradient length heuristic, and a consensus based heuristic. 
     
     
         15 . A non-transitory computer-readable storage medium embodying a set of instructions, which when executed by one or more processing resources of a computer system, causes the one or more processing resources to:
 receive a set of vectors including at least an unlabelled vector, a first labelled vector, and a second labelled vector;   calculate a first angle between the unlabelled vector and the first labelled vector, and a second angle between the unlabelled vector and the second labelled vector; and   use a combination of the first angle and the second angle to determine a labeling value of the unlabelled vector.   
     
     
         16 . The non-transitory computer-readable storage medium of  claim 15 , wherein the instructions that when executed by the one or more processing resources of the computer system cause the one or more processing resources to use the combination of the first angle and the second angle to determine the labeling value of the unlabelled vector include instructions that cause the processing resource to:
 determine that the first angle is less than the second angle; and   identify the first angle as a minimum angle based at least in part on determining that the first angle is less than the second angle.   
     
     
         17 . The non-transitory computer-readable storage medium of  claim 16 , wherein the instructions that when executed by the one or more processing resources of the computer system cause the one or more processing resources to the combination of the first angle and the second angle to determine a labeling value of the unlabelled vector further include instructions that cause the processing resource to:
 compare the minimum angle with a threshold value.   
     
     
         18 . The non-transitory computer-readable storage medium of  claim 17 , wherein the instructions that when executed by the one or more processing resources of the computer system cause the one or more processing resources to the combination of the first angle and the second angle to determine a labeling value of the unlabelled vector further include instructions that cause the processing resource to:
 identify the unlabelled vector as a high value labeling target where the minimum angle exceeds the threshold value.   
     
     
         19 . The non-transitory computer-readable storage medium of  claim 17 , wherein the threshold value is user programmable. 
     
     
         20 . The non-transitory computer-readable storage medium of  claim 15 , wherein the instructions that when executed by the one or more processing resources of the computer system further cause the one or more processing resources to:
 using the labeling value of the unlabelled vector along with the result of at least one other heuristic to rank the unlabelled vector relative to other unlabelled vectors.   
     
     
         21 . The non-transitory computer-readable storage medium of  claim 20 , wherein the at least one other heuristic is selected from a group consisting of: a Shannon's entropy heuristic, a confidence based heuristic, a distance from decision hyperplane heuristic, an information density heuristic, a perturbation heuristic, an expected gradient length heuristic, and a consensus based heuristic.