Annotate Apps with Entities by Fusing Heterogeneous Signals

Abstract

A system and method of annotating an application, including obtaining input signals associated with a target application, wherein the input signals are obtained from a plurality of sources, obtaining first annotation data from the obtained input signals, generating second annotation data in a machine-understandable form based on the first annotation data, and associating the second annotation data with the target application.

1 . A computer-implemented method, comprising:
 obtaining, using at least one processing circuit, input signals associated with a target application, wherein the input signals are obtained from a plurality of sources;   generating first annotation data from the input signals;   generating second annotation data in a machine-understandable form based on the first annotation data and a fusion model; and   associating the second annotation data with the target application.   
     
     
         2 . The method of  claim 1 , further comprising:
 obtaining verification reference data of the first annotation data and the second annotation data; and   adjusting the fusion model based on the verification reference data.   
     
     
         3 . The method of  claim 2 , wherein the plurality of input signals includes signal data representing one or more secondary applications which have been co-clicked with the target application. 
     
     
         4 . The method of  claim 3 , further comprising generating at least a portion of the first annotation data by propagating existing annotations from the one or more secondary applications. 
     
     
         5 . The method of  claim 2 , wherein the input signals include signal data representing search queries which have triggered downloads of the target application, web-based documents that include mentions of the target application, and meta data of the target application. 
     
     
         6 . The method of  claim 5 , further comprising:
 mapping text phrases of the search queries, meta data, and web-based documents to entities; and   generating at least a portion of the first annotation data based on the entities.   
     
     
         7 . The method of  claim 6 , wherein the verification reference data is obtained via human evaluation of a sampling of the first annotation data and the second annotation data. 
     
     
         8 . The method of  claim 7 , wherein the fusion model weights the input signals based on the human evaluations, and
 wherein generating the second annotation data further comprises applying a linear weighting model to the first annotation data according to the fusion model weighting of the input signals.   
     
     
         9 . The method of  claim 7 , further comprising:
 computing statistical comparison data based on the human evaluations and the second annotation data; and   providing the statistical comparison data in an interface to monitor the second annotation data.   
     
     
         10 . The method of  claim 9 , wherein the statistical comparison data includes at least one of precision, recall, coverage, or difference. 
     
     
         11 . A system, comprising: a storage device; a memory that stores computer executable components; and a processor that executes the following computer executable components stored in the memory:
 a document annotation component to receive a first input signal representing meta data associated with a target application and a second input signal representing web-based documents which mention the target application, and to generate first annotation data by mapping phrases of the web-based document and phrases of the target application meta data to predetermined entities;   a query annotation component to receive a third input signal representing queries which have triggered downloads of the target application and to generate first annotation data by mapping phrases from the queries to predetermined entities;   a smear annotation component to receive a fourth input signal representing existing annotations from secondary applications which are co-clicked pairs with the target application and to generate first annotation data based on the existing annotations;   an verification component to receive a human evaluation of samples of the first annotation data and the second annotation data; and   a fusion component to generate second annotation data based on the first annotation data by weighting the first, second, third and fourth input signals based on the human evaluation, and to associate the second annotation data with the target application.   
     
     
         12 . A system, comprising: a storage device; a memory that stores computer executable components; and a processor that executes the following computer executable components stored in the memory:
 one or more input signal components to obtain input signals associated with a target application, wherein the input signals are obtained from a plurality of sources;   an annotation component to generate first annotation data from the input signals;   a fusing component to generate second annotation data based on the first annotation data by applying a weighting value to one or more of the plurality of sources based on a fusion model, and to associate the second annotation data with the target application; and   a verification component to obtain verification reference data regarding the first annotation data and the second annotation data and to adjust the fusion model based on the verification reference data.   
     
     
         13 . The system of  claim 12 , wherein the verification component obtains the verification reference data based on a human evaluation of a sampling of the first annotation data and the second annotation data. 
     
     
         14 . The system of  claim 13 , wherein the input signals include data representing one or more secondary applications which have been co-clicked with the target application. 
     
     
         15 . The system of  claim 14 , wherein the annotation component generates the first annotation data by propagating existing annotation data from the one or more secondary applications. 
     
     
         16 . The system of  claim 13 , wherein the input signals include search queries which have triggered downloads of the target application, web-based documents that include mentions of the target application, and meta data of the target application. 
     
     
         17 . The system of  claim 16 , wherein the annotation component maps text phrases of the search queries, web-based documents, and meta data to entities and generates the second annotation data based on the entities. 
     
     
         18 . The system of  claim 13 , wherein the verification component weights the input signals based on the human evaluations, and
 wherein the annotation component generates the second annotation data from the first annotation data by applying a linear model to the weighted input signals.   
     
     
         19 . The system of  claim 13 , further comprising an output unit to compute statistical comparison data based on the human evaluations and the second annotation data, and provide the statistical comparison data in an interface. 
     
     
         20 . The system of  claim 19 , wherein the statistical comparison data includes at least one of precision, recall, coverage, or difference.