Machine Learning Systems and Methods For Exponentially Scaled Regression for Spatial Based Model Emphasis

Abstract

Among a great deal of other disclosure and scope, systems and methods are disclosed in relation to training regression machine learning models. In one of many possible implementations, a region of particular interest is identified where it is important for the target model to be very accurate within the region even at the expense of accuracy outside the region. The operator then tunes the loss function hyperparameters in order to correctly fit the region of interest and importance dropoff desired for the problem space. The loss function generated is easily differentiable and scales the importance of the training example based on its distance from the region of interest. The custom loss function is plugged into one of multiple training algorithms such as the gradient descent algorithm Adam and can be used to train our target model as before.

What is claimed is: 
     
         1 . A method for training a mathematical model using spatial emphasis, the method comprising:
 receiving, by a processing resource, a set of vectors to be ranked;   applying, by the processing resource, a mathematical model to the set of vectors to be ranked to yield a set of predicted vectors;   using, by the processing resource, a spatial emphasis value, the set of vectors to be ranked, and the set of predicted vectors in a scaling function to enhance a region of interest within a range expected for the set of vectors to yield a tuned scaling function; and   
       training, by the processing resource, the mathematical model on the tuned scaling function. 
     
     
         2 . The method of  claim 1 , wherein the mathematical model is a neural network model. 
     
     
         3 . The method of  claim 1 , wherein the scaling function is a function of:
 the spatial emphasis value;   an expected label for each of the set of vectors to be ranked; and   a label predicted by the vector ranking model for each of the set of vectors to be ranked.   
     
     
         4 . The method of  claim 3 , wherein the spatial emphasis value is one. 
     
     
         5 . The method of  claim 3 , wherein the scaling function is further a function of:
 a weight decay tuning value.   
     
     
         6 . The method of  claim 5 , wherein the method further comprises:
 determining, by the processing resource, the weight decay tuning value using Tree Parzen Estimation.   
     
     
         7 . The method of  claim 1 , wherein the scaling function includes a combination of only exponent, square, and linear functions. 
     
     
         8 . The method of  claim 1 , wherein the scaling function is an exponential loss function. 
     
     
         9 . The method of  claim 1 , wherein the scaling function is: 
       
         
           
             
               
                 1 
                 N 
               
               ⁢ 
               
                 
                   ∑ 
                   i 
                 
                   
                 
                   
                     exp 
                     ( 
                     
                       - 
                       
                         
                           
                             ( 
                             
                               
                                 y 
                                 i 
                               
                               - 
                               
                                 region 
                                 interest 
                               
                             
                             ) 
                           
                           2 
                         
                         
                           2 
                           ⁢ 
                           
                             τ 
                             2 
                           
                         
                       
                     
                     ) 
                   
                   ⁢ 
                   
                     
                       ( 
                       
                         
                           y 
                           i 
                         
                         - 
                         
                           
                             y 
                             ^ 
                           
                           i 
                         
                       
                       ) 
                     
                     2 
                   
                 
               
             
           
         
       
       wherein regioninterest is the spatial emphasis value, wherein yi is the label that the vector ranking model should have provided, is the label predicted by the vector ranking model, N is the number of vectors in the set of vectors, i is a counter from 1 to N, and τ is a weight decay tuning value. 
     
     
         10 . A system for training a mathematical model using spatial emphasis, the system comprising:
 a processing resource;   a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the processing resource to:   receive a set of vectors to be ranked;   apply a mathematical model to the set of vectors to be ranked to yield a set of predicted vectors;   use a spatial emphasis value, the set of vectors to be ranked, and the set of predicted vectors in a scaling function to enhance a region of interest within a range expected for the set of vectors to yield a tuned scaling function; and   train the mathematical model on the tuned scaling function.   
     
     
         11 . The system of  claim 10 , wherein the mathematical model is a neural network model. 
     
     
         12 . The system of  claim 10 , wherein the scaling function is a function of:
 the spatial emphasis value;   an expected label for each of the set of vectors to be ranked; and   a label predicted by the vector ranking model for each of the set of vectors to be ranked.   
     
     
         13 . The system of  claim 12 , wherein the spatial emphasis value is one. 
     
     
         14 . The system of  claim 12 , wherein the scaling function is further a function of:
 a weight decay tuning value.   
     
     
         15 . The system of  claim 14 , wherein the instructions that when executed by the processing resource further cause the processing resource to:
 determine the weight decay tuning value using Tree Parzen Estimation.   
     
     
         16 . The system of  claim 10 , wherein the scaling function includes a combination of only exponent, square, and linear functions. 
     
     
         17 . The system of  claim 10 , wherein the scaling function is: 
       
         
           
             
               
                 1 
                 N 
               
               ⁢ 
               
                 
                   ∑ 
                   i 
                 
                   
                 
                   
                     exp 
                     ( 
                     
                       - 
                       
                         
                           
                             ( 
                             
                               
                                 y 
                                 i 
                               
                               - 
                               
                                 region 
                                 interest 
                               
                             
                             ) 
                           
                           2 
                         
                         
                           2 
                           ⁢ 
                           
                             τ 
                             2 
                           
                         
                       
                     
                     ) 
                   
                   ⁢ 
                   
                     
                       ( 
                       
                         
                           y 
                           i 
                         
                         - 
                         
                           
                             y 
                             ^ 
                           
                           i 
                         
                       
                       ) 
                     
                     2 
                   
                 
               
             
           
         
         wherein regioninterest is the spatial emphasis value, wherein yi is the label that the vector ranking model should have provided, is the label predicted by the vector ranking model, N is the number of vectors in the set of vectors, i is a counter from 1 to N, and τ is a weight decay tuning value. 
       
     
     
         18 . A non-transitory computer-readable storage medium embodying a set of instructions, which when executed by one or more processing resources of a computer system, causes the one or more processing resources to:
 receive a set of vectors to be ranked;   apply a mathematical model to the set of vectors to be ranked to yield a set of predicted vectors;   use a spatial emphasis value, the set of vectors to be ranked, and the set of predicted vectors in a scaling function to enhance a region of interest within a range expected for the set of vectors to yield a tuned scaling function; and   train the mathematical model on the tuned scaling function.   
     
     
         19 . The non-transitory computer readable medium of  claim 18 , wherein the scaling function is: 
       
         
           
             
               
                 1 
                 N 
               
               ⁢ 
               
                 
                   ∑ 
                   i 
                 
                   
                 
                   
                     exp 
                     ( 
                     
                       - 
                       
                         
                           
                             ( 
                             
                               
                                 y 
                                 i 
                               
                               - 
                               
                                 region 
                                 interest 
                               
                             
                             ) 
                           
                           2 
                         
                         
                           2 
                           ⁢ 
                           
                             τ 
                             2 
                           
                         
                       
                     
                     ) 
                   
                   ⁢ 
                   
                     
                       ( 
                       
                         
                           y 
                           i 
                         
                         - 
                         
                           
                             y 
                             ^ 
                           
                           i 
                         
                       
                       ) 
                     
                     2 
                   
                 
               
             
           
         
         wherein regioninterest is the spatial emphasis value, wherein yi is the label that the vector ranking model should have provided, is the label predicted by the vector ranking model, N is the number of vectors in the set of vectors, i is a counter from 1 to N, and τ is a weight decay tuning value. 
       
     
     
         20 . The non-transitory computer readable medium of  claim 18 , wherein the scaling function is a function of:
 the spatial emphasis value;   an expected label for each of the set of vectors to be ranked; and   a label predicted by the vector ranking model for each of the set of vectors to be ranked.