Incremental time window procedure for selecting training samples for a supervised learning algorithm

Abstract

Disclosed herein are system, method, and computer program product embodiments for generating labels for training a machine learning mode using an incremental time window process. The described process may be used in a recurrence detection system. A dataset may be analyzed using incremental split dates to divide the dataset into an analysis portion and a holdout portion. The analysis portion may be analyzed to determine input features related to a predicted recurrence in the dataset. The holdout portion may be tested against the analysis portion and the input features to generate a label. The label may indicate whether or not the holdout portion confirms the prediction. The testing of the holdout portion against the analysis portion may be repeated by incrementally using different split dates and multiple separate analysis portions and holdout portions to generate multiple labels and corresponding input features.

What is claimed is: 
     
         1 . A method, comprising:
 accessing, by an application server, a plurality of transactions associated with a first entity and a second entity, wherein the first entity is a customer of a plurality of customers and the second entity is a merchant of a plurality of merchants;   sorting, by the application server, the plurality of transactions into a plurality of transaction series, wherein each transaction series of the plurality of transaction series corresponds to a test dataset of a plurality of test datasets used for training a machine learning model for identifying an estimated recurrence period and a recurring transaction series of the plurality of transaction series, and wherein each test dataset comprises transactions between the first entity and the second entity;   splitting, by the application server, each test dataset into a first subset of transactions and a corresponding second subset of transactions;   analyzing, by the application server, transactions in the first subset of transactions to determine the estimated recurrence period;   based on the determined estimated recurrence period, determining, by the application server, whether one or more transaction dates of transactions in the corresponding second subset of the first subset of transactions matches the estimated recurrence period, wherein the one or more transaction dates are later in time of a last transaction date in the first subset of transactions;   in response to determining a number of the one or more transaction dates of transactions in the corresponding second subset exceeding a configured threshold, generating, by the application server, a target label using the estimated recurrence period; and   training, by the application server, the machine learning model using the generated target label.   
     
     
         2 . The method of  claim 1 , wherein the transactions in the first subset of transactions occur earlier in time than transactions in the corresponding second subset of transactions. 
     
     
         3 . The method of  claim 1 , wherein analyzing the transactions in the first subset of transactions to determine the estimated recurrence period comprises:
 converting transactions dates of the transactions in the first subset of transactions into ordinal transaction dates;   determining a degree of a periodic pattern for phase spaces based on the ordinal transaction dates, wherein each phase space of the phase spaces corresponds to one of a weekly, a biweekly, a monthly, a bimonthly, a quarterly, a semi-annually, and a yearly period;   determining a closest period based on the degree of the periodic pattern for the phase spaces, wherein the closest period is one of the weekly, the biweekly, the monthly, the bimonthly, the quarterly, the semi-annually, and the yearly period,   wherein the closest period is the estimated recurrence period.   
     
     
         4 . The method of  claim 1 , wherein determining whether the one or more transaction dates of transactions in the corresponding second subset matches the estimated recurrence period comprises:
 adding a length of a closest period to a transaction date of a chronologically last transaction in the first subset of transactions to determine a first future transaction date; and   adding the length of the closest period to the first future transaction date to determine a second future transaction date.   
     
     
         5 . The method of  claim 1 , wherein determining whether the one or more transaction dates of transactions in the corresponding second subset matches the estimated recurrence period comprises:
 determining a last phase offset, wherein the last phase offset is a difference between a mean phase angle of transactions in the first subset of transactions and a phase angle of a chronologically last transaction in the first subset of transactions;   adding days corresponding to the last phase offset to a transaction date of the chronologically last transaction in the first subset of transactions to determine a first future transaction date; and   adding the days corresponding to the last phase offset to the first future transaction date to determine a second future transaction date.   
     
     
         6 . The method of  claim 1 , wherein generating the target label using the estimated recurrence period comprises:
 finding a first future transaction in the corresponding second subset having a transaction date on a first future transaction date;   finding a second future transaction in the corresponding second subset having a transaction date on a second future transaction date; and   in response to finding the first future transaction and the second future transaction in the corresponding second subset, marking the test dataset as recurring at the determined estimated recurrence period.   
     
     
         7 . The method of  claim 1 , wherein generating the target label-using the estimated recurrence period comprises:
 finding a first future transaction in the corresponding second subset having a transaction date that matches with a date within a threshold of days of a first future transaction date;   finding a second future transaction in the corresponding second subset having a transaction date that matches with a date within a threshold of days of a second future transaction date; and   in response to finding the first future transaction and the second future transaction, marking the test dataset as recurring at the determined estimated recurrence period.   
     
     
         8 . A system, comprising:
 a memory for storing instructions; and   a processor, communicatively coupled to the memory, configured to execute the instructions, the instructions causing the processor to:
 access a plurality of transactions associated with a first entity and a second entity, wherein the first entity is a customer of a plurality of customers and the second entity is a merchant of a plurality of merchants; 
 sort the plurality of transactions into a plurality of transaction series, wherein each transaction series of the plurality of transaction series corresponds to a test dataset of a plurality of test datasets used for training a machine learning model for identifying an estimated recurrence period and a recurring transaction series of the plurality of transaction series, and wherein each test dataset comprises transactions between the first entity and the second entity; 
 split each test dataset into a first subset of transactions and a corresponding second subset of transactions; 
 analyze transactions in the first subset of transactions to determine the estimated recurrence period; 
 based on the determined estimated recurrence period, determine whether one or more transaction dates of transactions in the corresponding second subset of the first subset of transactions matches the estimated recurrence period, wherein the one or more transaction dates are later in time of a last transaction date in the first subset of transactions; 
 in response to determining a number of the one or more transaction dates of transactions in the corresponding second subset exceeding a configured threshold, generate a target label using the estimated recurrence period; and 
 training the machine learning model using the generated target label. 
   
     
     
         9 . The system of  claim 8 , wherein the transactions in the first subset of transactions occur earlier in time than transactions in the corresponding second subset of transactions. 
     
     
         10 . The system of  claim 8 , wherein to analyze the transactions in the first subset of transactions to determine the estimated recurrence period, the processor is further configured to:
 convert transactions dates of the transactions in the first subset of transactions into ordinal transaction dates;   determine a degree of a periodic pattern for phase spaces based on the ordinal transaction dates, wherein each phase space of the phase spaces corresponds to one of a weekly, a biweekly, a monthly, a bimonthly, a quarterly, a semi-annually, and a yearly period;   determine a closest period based on the degree of the periodic pattern for the phase spaces, wherein the closest period is one of the weekly, the biweekly, the monthly, the bimonthly, the quarterly, the semi-annually, and the yearly period,   wherein the closest period is the estimated recurrence period.   
     
     
         11 . The system of  claim 8 , wherein to determine whether the one or more transaction dates of transactions in the corresponding second subset matches the estimated recurrence period, the processor is further configured to:
 add a length of a closest period to a transaction date of a chronologically last transaction in the first subset of transactions to determine a first future transaction date; and   add the length of the closest period to the first future transaction date to determine a second future transaction date.   
     
     
         12 . The system of  claim 8 , wherein to determine whether the one or more transaction dates of transactions in the corresponding second subset matches the estimated recurrence period, the processor is further configured to:
 determine a last phase offset, wherein the last phase offset is a difference between a mean phase angle of transactions in the first subset of transactions and a phase angle of a chronologically last transaction in the first subset of transactions;   add days corresponding to the last phase offset to a transaction date of the chronologically last transaction in the first subset of transactions to determine a first future transaction date; and   add the days corresponding to the last phase offset to the first future transaction date to determine a second future transaction date.   
     
     
         13 . The system of  claim 8 , wherein the first subset of transactions and the corresponding second subset of transactions comprises at least three transactions. 
     
     
         14 . The system of  claim 8 , wherein a transaction date of each of the one or more transactions matches with a date within a threshold number of days of each of the one or more predicted transaction dates. 
     
     
         15 . A non-transitory, tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
 accessing a plurality of transactions including a first entity and a second entity, wherein the first entity is a customer of a plurality of customers and the second entity is a merchant of a plurality of merchants;   sorting the plurality of transactions into a plurality of transaction series, determine wherein each transaction series of the plurality of transaction series corresponds to a test dataset of a plurality of test datasets used for training a machine learning model for identifying an estimated recurrence period and a recurring transaction series of the plurality of transaction series, and wherein each test dataset comprises transactions between the first entity and the second entity;   splitting each test dataset into a first subset of transactions and a corresponding second subset of transactions;   analyzing transactions in the first subset of transactions to determine the estimated recurrence period;   based on the determined estimated recurrence period of each test dataset, determine whether one or more transaction dates of transactions in the corresponding second subset of the first subset of transactions matches the estimated recurrence period, wherein the one or more transaction dates are later in time of a last transaction date in the first subset of transactions;   in response to determining a number of the one or more transaction dates of transactions in the corresponding second subset exceeding a configured threshold, generating a target label using the estimated recurrence period; and   training the machine learning model using the generated target label.   
     
     
         16 . The non-transitory, tangible computer-readable device of  claim 15 , wherein the first subset of transactions and the corresponding second subset of transactions comprises at least three transactions. 
     
     
         17 . The non-transitory, tangible computer-readable device of  claim 15 , wherein transactions in the first subset of transactions occur earlier in time than transactions in the corresponding second subset of transactions. 
     
     
         18 . The non-transitory, tangible computer-readable device of  claim 15 , wherein a transaction date of each of the one or more transaction dates matches with a date within a threshold number of days of each of the one or more predicted transaction dates. 
     
     
         19 . The non-transitory, tangible computer-readable device of  claim 15 , wherein to analyze the transactions in the first subset of transactions to determine the estimated recurrence period, the operations further comprise:
 converting transactions dates of the one or more transaction dates in the first subset of transactions into ordinal transaction dates;   determining a degree of a periodic pattern for phase spaces based on the ordinal transaction dates, wherein each phase space of the phase spaces corresponds to one of a weekly, a biweekly, a monthly, a bimonthly, a quarterly, a semi-annually, and a yearly period;   determining a closest period based on the degree of the periodic pattern for the phase spaces, wherein the closest period is one of the weekly, the biweekly, the monthly, the bimonthly, the quarterly, the semi-annually, and the yearly period,   wherein the closest period is the estimated recurrence period.   
     
     
         20 . The non-transitory, tangible computer-readable device of  claim 15 , wherein to determine whether the one or more transaction dates of transactions in the corresponding second subset matches the estimated recurrence period, the operations further comprise:
 adding a length of a closest period to a transaction date of a chronologically last transaction in the first subset of transactions to determine a first future transaction date; and   adding the length of the closest period to the first future transaction date to determine a second future transaction date.