Latency reduction in feedback-based system performance determination

Abstract

The present disclosure is directed to a technique to reduce latency in feedback-based system performance determination. A system receives, from an application developer device, indications of an in-application event and a first input value for an application content delivery profile. The system receives, via an interface from an application developed by an application developer and executed by a computing device remote from the data processing system and different from the application developer device, a ping indicative of an occurrence of the in-application event on the computing device. The system merges data from the ping with internal data determined by the data processing system to generate merged data. The system determines a predicted performance for the in-application event and provides an indication of the predicted performance. The system configures, responsive to the indication of the predicted performance, the application content delivery profile with a second input value.

1 .- 20 . (canceled) 
     
     
         21 . A system to reduce latency for configuration updates, comprising:
 a data processing system comprising one or more processors and memory to:
 receive, from an application developer device, one or more indications of an in-application event comprising an audio interaction and a first input value for an application content delivery profile; 
 receive, via an interface from an application developed by an application developer and executed by a mobile computing device remote from the data processing system and different from the application developer device, an indication of an occurrence of the in-application event comprising the audio interaction on the mobile computing device; 
 determine a number of installations of the application on other mobile computing devices; 
 generate a first merged data set that merges performance signals for the in-application event across the other mobile computing devices on which the application is installed; 
 identify a plurality of applications that are similar to the application based on at least one characteristic of the application; 
 generate a second merged data set that merges performance signals for in-application events comprising audio interactions associated with installations of the plurality of applications; 
 generate a model that combines the first merged data set with the second merged data set using a machine learning technique; 
 determine, using the model, a predicted performance for the in-application event comprising the audio interaction based on the first input value; 
 provide, via the interface, an indication of the predicted performance determined based on the model and the first input value; and 
 configure, responsive to the indication of the predicted performance, the application content delivery profile with a second input value, execution by the data processing system of the application content delivery profile configured with the second input value to cause the data processing system to select content with an indication to install the application. 
   
     
     
         22 . The system of  claim 21 , comprising:
 the data processing system to identify the in-application event comprising the audio interaction based on audio input detected by a sensor of the mobile computing device.   
     
     
         23 . The system of  claim 21 , comprising:
 the data processing system to determine the in-application event comprising the audio interaction based on speech detected by a sensor of the mobile computing device.   
     
     
         24 . The system of  claim 21 , comprising:
 the data processing system comprising a digital assistant.   
     
     
         25 . The system of  claim 21 , wherein the mobile computing device comprises a digital assistant. 
     
     
         26 . The system of  claim 21 , comprising:
 the data processing system to determine the predicted performance for the in-application event comprising the audio interaction based on a regression model.   
     
     
         27 . The system of  claim 21 , comprising:
 the data processing system to determine the predicted performance for the in-application event comprising the audio interaction based on indications of occurrences of in-application events associated with the plurality of applications.   
     
     
         28 . The system of  claim 21 , comprising the data processing system to:
 determine a first predicted performance for the in-application event comprising the audio interaction based on a multivariate Poisson regression model; and   calibrate the first predicted performance based on a second predicted performance of the plurality of applications to determine the predicted performance.   
     
     
         29 . The system of  claim 21 , comprising the data processing system to generate the model with:
 data received from a first plurality of computing devices that installed the application;   data received from a second plurality of computing devices that installed, responsive to a content item delivered by the data processing system, one or more applications different from the application, the one or more applications matching a characteristic of the application; and   data received from a third plurality of computing devices that installed, responsive to an organic search, the one or more applications different from the application, the one or more applications matching the characteristic of the application.   
     
     
         30 . The system of  claim 21 , comprising the data processing system to:
 provide a software development kit to the application developer device that configured the application to transmit an indication to the data processing system responsive to the occurrence of the in-application event.   
     
     
         31 . A method of reducing latency for configuration updates, comprising:
 receiving, by a data processing system comprising one or more processors and memory, from an application developer device, one or more indications of an in-application event comprising an audio interaction and a first input value for an application content delivery profile;   receiving, by the data processing system via an interface from an application developed by an application developer and executed by a mobile computing device remote from the data processing system and different from the application developer device, an indication of an occurrence of the in-application event comprising the audio interaction on the mobile computing device;   determining, by the data processing system, a number of installations of the application on other mobile computing devices;   generating, by the data processing system, a first merged data set that merges performance signals for the in-application event across the other mobile computing devices on which the application is installed;   identifying, by the data processing system, a plurality of applications that are similar to the application based on at least one characteristic of the application;   generating, by the data processing system, a second merged data set that merges performance signals for in-application events comprising audio interactions associated with installations of the plurality of applications;   generating, by the data processing system, a model that combines the first merged data set with the second merged data set using a machine learning technique;   determining, by the data processing system using the model, a predicted performance for the in-application event comprising the audio interaction based on the first input value;   providing, by the data processing system via the interface, an indication of the predicted performance determined based on the model and the first input value; and   configuring, by the data processing system responsive to the indication of the predicted performance, the application content delivery profile with a second input value, execution by the data processing system of the application content delivery profile configured with the second input value to cause the data processing system to select content with an indication to install the application.   
     
     
         32 . The method of  claim 31 , comprising:
 identifying, by the data processing system, the in-application event comprising the audio interaction based on audio input detected by a sensor of the mobile computing device.   
     
     
         33 . The method of  claim 31 , comprising:
 determining, by the data processing system, the in-application event comprising the audio interaction based on speech detected by a sensor of the mobile computing device.   
     
     
         34 . The method of  claim 31 , wherein the data processing system comprises a digital assistant. 
     
     
         35 . The method of  claim 31 , wherein the mobile computing device comprises a digital assistant. 
     
     
         36 . The method of  claim 31 , comprising:
 determining, by the data processing system, the predicted performance for the in-application event comprising the audio interaction based on a regression model.   
     
     
         37 . The method of  claim 31 , comprising:
 determining, by the data processing system, the predicted performance for the in-application event comprising the audio interaction based on indications of occurrences of in-application events associated with the plurality of applications.   
     
     
         38 . The method of  claim 31 , comprising:
 determining, by the data processing system, a first predicted performance for the in-application event comprising the audio interaction based on a multivariate Poisson regression model; and   calibrating, by the data processing system, the first predicted performance based on a second predicted performance of the plurality of applications to determine the predicted performance.   
     
     
         39 . The method of  claim 31 , comprising generating the model with:
 data received from a first plurality of computing devices that installed the application;   data received from a second plurality of computing devices that installed, responsive to a content item delivered by the data processing system, one or more applications different from the application, the one or more applications matching a characteristic of the application; and   data received from a third plurality of computing devices that installed, responsive to an organic search, the one or more applications different from the application, the one or more applications matching the characteristic of the application.   
     
     
         40 . The method of  claim 31 , comprising:
 providing, by the data processing system, a software development kit to the application developer device that configured the application to transmit an indication to the data processing system responsive to the occurrence of the in-application event.