Systems and Techniques for Intelligent A/B Testing of Marketing Campaigns

Abstract

Systems and methods for testing two or more pieces of marketing communication content that intelligently selects test recipients sets to get effective results in a timely manner. One embodiment involves identifying a category of a marketing campaign and identifying potential test recipients who are interested in the category based on interactions by each respective potential test recipient with prior marketing communications associated with the category. The embodiment further involves selecting a first subset and a second subset of the potential test recipients and sending the first marketing communication content to the first subset and sending the second marketing communication content to the second subset. The embodiment further involves assessing responsiveness of recipients of the first subset and assessing responsiveness of recipients of the second subset.

1 . A method comprising:
 identifying a category of a marketing campaign, wherein a first marketing communication content and a second marketing communication content relate to the category;   identifying, by a processing device, potential test recipients who are interested in the category based on interactions by each respective potential test recipient with prior marketing communications associated with the category, the potential test recipients identified from a set of potential recipients;   sending the first marketing communication content to a first subset of the potential test recipients and sending the second marketing communication content to a second subset of the potential test recipients; and   assessing responsiveness of recipients of the first subset and assessing responsiveness of recipients of the second subset.   
     
     
         2 . The method of  claim 1  further comprising selecting the first marketing communication content or the second marketing communication content as A/B test winning content based on comparing the responsiveness of the first subset and the responsiveness of the second subset. 
     
     
         3 . The method of  claim 2  further comprising sending the A/B test winning content to additional recipients in the set of recipients. 
     
     
         4 . The method of  claim 1  further comprising:
 sending a marketing communication comprising at least one of the first marketing communication content and the second marketing communication content to additional recipients in the set of recipients; and 
 storing category-specific responsiveness information for the additional recipients based on responsiveness of each of the additional recipients to the marketing communication, wherein the category-specific responsiveness information is used in identifying future potential test recipients for future marketing communications. 
 
     
     
         5 . The method of  claim 1  further comprising:
 determining a first responsiveness score for the first marketing communication content based on:
 the responsiveness of the recipients of the first subset to the first marketing communication content; and 
 a first average responsiveness of the recipients of the first subset to prior marketing communications related to the category; 
 
 determining a second responsiveness score for the second marketing communication content based on:
 the responsiveness of the recipients of the second subset to the second marketing communication content; and 
 a second average responsiveness of the recipients of the second subset to prior marketing communications related to the category. 
 
 
     
     
         6 . The method of  claim 5  further comprising selecting a winning marketing communication based on the first responsiveness score and the second responsiveness score. 
     
     
         7 . The method of  claim 1  further comprising:
 identifying an average responsiveness to prior marketing communications related to the category; and 
 providing a notification based on determining that the responsiveness of the recipients of the first subset is less than the average responsiveness. 
 
     
     
         8 . The method of  claim 1  further comprising:
 identifying an average responsiveness to prior marketing communications related to the category; and 
 providing a notification based on determining that the responsiveness of the recipients of the first subset is less than the average responsiveness and that the responsiveness of the recipients of the second subset is less than the average responsiveness. 
 
     
     
         9 . The method of  claim 8  wherein the average responsiveness is an average of responsiveness by potential recipients interested in the category. 
     
     
         10 . The method of  claim 1  wherein selecting the first subset and the second subset is based at least in part on how quickly each of the potential test recipients interacted with marketing communications in the past. 
     
     
         11 . The method of  claim 1  wherein identifying the category of the marketing campaign comprises performing an automated process on text associated with the campaign to identify the category from a set of predefined categories. 
     
     
         12 . The method of  claim 1  wherein identifying the category of the marketing campaign comprises receiving input selecting a category from a set of predefined categories. 
     
     
         13 . The method of  claim 1  further comprising accessing a database to identify the interactions by each respective potential test recipient with prior marketing communications associated with the category, wherein the database is updated with category-specific responsiveness information following marketing communications. 
     
     
         14 . The method of  claim 1  wherein the first marketing communication content and second marketing communication content are sent via email or text message. 
     
     
         15 . The method of  claim 1  wherein the first marketing communication content and second marketing communication content are sent via social media or direct mail. 
     
     
         16 . A method comprising:
 identifying, by a processing device, potential test recipients who are quick responders based on timing of interactions by each respective potential test recipient with prior marketing communications, the potential test recipients identified from a set of potential recipients;   sending the first marketing communication content to a first subset of the potential test recipients and sending the second marketing communication content to a second subset of the potential test recipients; and   assessing responsiveness of recipients of the first subset and assessing responsiveness of recipients of the second subset.   
     
     
         17 . The method of  claim 16  wherein identifying potential test recipients who are quick responders comprises identifying recipients whose average response time to the prior marketing communications is less than a threshold response time. 
     
     
         18 . The method of  claim 17  further comprising determining the threshold response time based on input received from a user. 
     
     
         19 . The method of  claim 18  wherein identifying the potential test recipients who are quick responders comprises identifying test recipients who are both interested in a marketing campaign category and whose average response time to the prior marketing communications is less than a threshold response time. 
     
     
         20 . A system comprising:
 a processing device; and   a non-transitory computer-readable medium communicatively coupled to the processing device,   wherein the processing device is configured to execute instructions to perform operations comprising:
 identifying a category of a marketing campaign, wherein a first marketing communication content and a second marketing communication content relate to the category; 
 identifying, by a processing device, potential test recipients who are interested in the category based on interactions by each respective potential test recipient with prior marketing communications associated with the category, the potential test recipients identified from a set of potential recipients; 
 sending the first marketing communication content to a first subset of the potential test recipients and sending the second marketing communication content to a second subset of the potential test recipients; and 
 assessing responsiveness of recipients of the first subset and assessing responsiveness of recipients of the second subset.