Event-triggered microsurvey customization and delivery system

Abstract

A real-time survey system can monitor real-time event and interaction data generated as a user interacts with an application. Using the event data in combination with static user attributes, the survey system can identify a subset of microsurveys a user is eligible to receive or participate in. Identifying eligibility can include applying one or more filters based on user attributes as well as detecting a triggering event that signals real-time relevance of the survey to the user's actions. If the user is eligible for multiple surveys, the survey system can select a single survey to send to the user. Surveys can be presented to users directly using the real-time survey system instead of via an alternate delivery method, such as email. After collecting survey data, survey system can automatically analyze survey responses, including by performing machine learning based thematic analysis on freeform text responses.

What is claimed is: 
     
         1 . A method comprising:
 generating a set of training data comprising historical survey responses each labeled with a response topic and a response intent;   training a neural network to identify topics and intents of survey responses from users using the generated set of training data;   monitoring, in real-time, interactions by a user with an application running on a user device;   selecting a microsurvey from a set of microsurveys stored by a microsurvey database based on one or more monitored interactions by the user and one or more characteristics of the user;   modifying an interface of the application running on the user device to include the selected microsurvey, the display of the selected microsurvey comprising an interactive graphical element that, upon completion of the microsurvey by the user, is configured to modify the interface of the application to remove the selected microsurvey; and   applying the neural network to a response to the microsurvey from the user to identify a set of topics and a set of intents associated with the response.   
     
     
         2 . The method of  claim 1 , further comprising:
 responsive to modifying the interface of the application to include the selected microsurvey for display, collecting a result of the microsurvey comprising one or more question responses.   
     
     
         3 . The method of  claim 2 , further comprising:
 aggregating the result of the microsurvey with a plurality of other microsurvey results;   analyzing the set of microsurvey results using thematic analysis; and   grouping the microsurvey results based on theme based on the thematic analysis.   
     
     
         4 . The method of  claim 3 , further comprising:
 transmitting the grouped set of microsurvey results to an online system associated with the application.   
     
     
         5 . The method of  claim 1 , wherein selecting a microsurvey comprises:
 applying a leaky bucket algorithm to select a microsurvey from the set of microsurveys.   
     
     
         6 . The method of  claim 5 , wherein the leaky bucket algorithm is a cross-leaky bucket algorithm. 
     
     
         7 . The method of  claim 1 , wherein selecting a microsurvey comprises:
 selecting a time within the user's session of the application to display the microsurvey.   
     
     
         8 . A non-transitory computer-readable storage medium comprising instructions which, when executed by a processor, cause the processor to perform the steps of:
 generating a set of training data comprising historical survey responses each labeled with a response topic and a response intent;   training a neural network to identify topics and intents of survey responses from users using the generated set of training data;   monitoring, in real-time, interactions by a user with an application running on a user device;   selecting a microsurvey from a set of microsurveys stored by a microsurvey database based on one or more monitored interactions by the user and one or more characteristics of the user;   modifying an interface of the application running on the user device to include the selected microsurvey, the display of the selected microsurvey comprising an interactive graphical element that, upon completion of the microsurvey by the user, is configured to modify the interface of the application to remove the selected microsurvey; and   applying the neural network to a response to the microsurvey from the user to identify a set of topics and a set of intents associated with the response.   
     
     
         9 . The non-transitory computer-readable storage medium of  claim 8 , wherein the instructions, when executed, further cause the processor to perform the steps of:
 responsive to modifying the interface of the application to include the selected microsurvey for display, collecting a result of the microsurvey comprising one or more question responses.   
     
     
         10 . The non-transitory computer-readable storage medium of  claim 9 , wherein the instructions, when executed, further cause the processor to perform the steps of:
 aggregating the result of the microsurvey with a plurality of other microsurvey results;   analyzing the set of microsurvey results using thematic analysis; and   grouping the microsurvey results based on theme based on the thematic analysis.   
     
     
         11 . The non-transitory computer-readable storage medium of  claim 10 , wherein the instructions, when executed, further cause the processor to perform the steps of:
 transmitting the grouped set of microsurvey results to an online system associated with the application.   
     
     
         12 . The non-transitory computer-readable storage medium of  claim 8 , wherein selecting a microsurvey comprises:
 applying a leaky bucket algorithm to select a microsurvey from the set of microsurveys.   
     
     
         13 . The non-transitory computer-readable storage medium of  claim 12 , wherein the leaky bucket algorithm is a cross-leaky bucket algorithm. 
     
     
         14 . The non-transitory computer-readable storage medium of  claim 8 , wherein selecting a microsurvey comprises:
 selecting a time within the user's session of the application to display the microsurvey.   
     
     
         15 . A system comprising a hardware processor and a non-transitory computer-readable storage medium storing executable instructions that when executed by the hardware processor, cause the hardware processor to perform steps comprising:
 generating a set of training data comprising historical survey responses each labeled with a response topic and a response intent;   training a neural network to identify topics and intents of survey responses from users using the generated set of training data;   monitoring, in real-time, interactions by a user with an application running on a user device;   selecting a microsurvey from a set of microsurveys stored by a microsurvey database based on one or more monitored interactions by the user and one or more characteristics of the user;   modifying an interface of the application running on the user device to include the selected microsurvey, the display of the selected microsurvey comprising an interactive graphical element that, upon completion of the microsurvey by the user, is configured to modify the interface of the application to remove the selected microsurvey; and   applying the neural network to a response to the microsurvey from the user to identify a set of topics and a set of intents associated with the response.   
     
     
         16 . The system of  claim 15 , wherein the instructions, when executed, further cause the hardware processor to perform the steps of:
 responsive to modifying the interface of the application to include the selected microsurvey for display, collecting a result of the microsurvey comprising one or more question responses.   
     
     
         17 . The system of  claim 16 , wherein the instructions, when executed, further cause the hardware processor to perform the steps of:
 aggregating the result of the microsurvey with a plurality of other microsurvey results;   analyzing the set of microsurvey results using thematic analysis; and   grouping the microsurvey results based on theme based on the thematic analysis.   
     
     
         18 . The system of  claim 17 , wherein the instructions, when executed, further cause the hardware processor to perform the steps of:
 transmitting the grouped set of microsurvey results to an online system associated with the application.   
     
     
         19 . The system of  claim 15 , wherein selecting a microsurvey comprises:
 applying a leaky bucket algorithm to select a microsurvey from the set of microsurveys.   
     
     
         20 . The system of  claim 15 , wherein selecting a microsurvey comprises:
 selecting a time within the user's session of the application to display the microsurvey.