System and method for dynamic data filtering

Abstract

Methods, systems, and computer programs, for generating dynamic data filters. In one aspect, a data processing system is disclosed that is configured to perform operations that include receiving a data source agnostic information request (DSAIR), determining, based on the DSAIR, (i) one or more source selection keys and (ii) a data dictionary, identifying, by the data processing system, one or more data sources related to the received DSAIR using the one or more source selection keys, generating a dynamic data filter using one or more of the plurality of terms of the data dictionary, filtering, using the dynamic data filter, information in the one or more data sources that were identified using the source selection keys, and providing the filtered information from the one or more data sources for output by the user device.

1 . A data processing system for dynamic data filtering, the data processing system including one or more processors and one or more computer storage devices storing instructions that, when executed by the one or more processors, causes the one or more processors to perform operations, the operations comprising:
 receiving, by the data processing system that includes one or more computers hosting a data mining server and from a user device using an interface provided by the data mining service, a data source agnostic information request;   determining, by the data processing system and based on the data source agnostic information request, (i) one or more source selection keys and (ii) a data dictionary, wherein the data dictionary comprises a plurality of terms generated based on the received data source agnostic information request;   identifying, by the data processing system, one or more data sources related to the received data source agnostic information request using the one or more source selection keys;   generating, by the data processing system, a dynamic data filter using one or more of the plurality of terms of the data dictionary;   filtering, by the data processing system and using the dynamic data filter, information in the one or more data sources that were identified using the source selection keys; and   providing, by the data processing system, the filtered information from the one or more data sources for output by the user device.   
     
     
         2 . The data processing system of  claim 1 , wherein the data source agnostic information request comprises a request for information that is independent of the schema used to organize data by the one or more data sources. 
     
     
         3 . The data processing system of  claim 1 , wherein determining the data dictionary comprising a plurality of terms generated based on the received data source agnostic information request comprises:
 obtaining, by a parsing engine of the data processing system, a set of one or more terms of the data source agnostic information request that each correspond to a keyed data attribute of one the data sources.   
     
     
         4 . The data processing system of  claim 1 , wherein generating a dynamic data filter using one or more of the plurality of terms of the data dictionary comprises:
 obtaining, by the data processing system, one or more information filter terms from a library of predetermined information filter terms using the terms of data dictionary; and   generating, the data processing system, a dynamic data filter using (i) the one or more of the plurality of terms of the data dictionary and (ii) the obtained one or more information filter terms from the library of predetermined information filter terms.   
     
     
         5 . The data processing system of  claim 1 , the operations further comprising:
 generating, by the data processing system, a searchable index that indexes the filtered information.   
     
     
         6 . The data processing system of  claim 1 , the operations further comprising:
 periodically generating, by the data mining service and subsequent to the providing of the filtered information responsive to the received data source agnostic information request, alerts that include information that satisfies the received data source agnostic information request; and   providing, by the data mining service and to the user device, the periodically generated alerts to the user device.   
     
     
         7 . The data processing system of  claim 6 , wherein each alert of the periodically generated alerts include updated information that is responsive to the received data source agnostic information request. 
     
     
         8 . A method for dynamic data filtering comprising:
 receiving, by a data processing system that includes one or more computers hosting a data mining server and from a user device using an interface provided by the data mining service, a data source agnostic information request;   determining, by the data processing system and based on the data source agnostic information request, (i) one or more source selection keys and (ii) a data dictionary, wherein the data dictionary comprises a plurality of terms generated based on the received data source agnostic information request;   identifying, by the data processing system, one or more data sources related to the received data source agnostic information request using the one or more source selection keys;   generating, by the data processing system, a dynamic data filter using one or more of the plurality of terms of the data dictionary;   filtering, by the data processing system and using the dynamic data filter, information in the one or more data sources that were identified using the source selection keys; and   providing, by the data processing system, the filtered information from the one or more data sources for output by the user device.   
     
     
         9 . The method of  claim 8 , wherein the data source agnostic information request comprises a request for information that is independent of the schema used to organize data by the one or more data sources. 
     
     
         10 . The method of  claim 8 , wherein determining the data dictionary comprising a plurality of terms generated based on the received data source agnostic information request comprises:
 obtaining, by a parsing engine of the data processing system, a set of one or more terms of the data source agnostic information request that each correspond to a keyed data attribute of one the data sources.   
     
     
         11 . The method of  claim 8 , wherein generating a dynamic data filter using one or more of the plurality of terms of the data dictionary comprises:
 obtaining, by the data processing system, one or more information filter terms from a library of predetermined information filter terms using the terms of data dictionary; and   generating, the data processing system, a dynamic data filter using (i) the one or more of the plurality of terms of the data dictionary and (ii) the obtained one or more information filter terms from the library of predetermined information filter terms.   
     
     
         12 . The method of  claim 8 , the method further comprising:
 generating, by the data processing system, a searchable index that indexes the filtered information.   
     
     
         13 . The method of  claim 8 , the method further comprising:
 periodically generating, by the data mining service and subsequent to the providing of the filtered information responsive to the received data source agnostic information request, alerts that include information that satisfies the received data source agnostic information request; and   providing, by the data mining service and to the user device, the periodically generated alerts to the user device.   
     
     
         14 . The method of  claim 13 , wherein each alert of the periodically generated alerts include updated information that is responsive to the received data source agnostic information request. 
     
     
         15 . A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations for dynamic data filtering comprising:
 receiving, by the data processing system that includes one or more computers hosting a data mining server and from a user device using an interface provided by the data mining service, a data source agnostic information request;   determining, by the data processing system and based on the data source agnostic information request, (i) one or more source selection keys and (ii) a data dictionary, wherein the data dictionary comprises a plurality of terms generated based on the received data source agnostic information request;   identifying, by the data processing system, one or more data sources related to the received data source agnostic information request using the one or more source selection keys;   generating, by the data processing system, a dynamic data filter using one or more of the plurality of terms of the data dictionary;   filtering, by the data processing system and using the dynamic data filter, information in the one or more data sources that were identified using the source selection keys; and   providing, by the data processing system, the filtered information from the one or more data sources for output by the user device.   
     
     
         16 . The non-transitory computer-readable medium of  claim 15 , wherein the data source agnostic information request comprises a request for information that is independent of the schema used to organize data by the one or more data sources. 
     
     
         17 . The non-transitory computer-readable medium of  claim 15 , wherein determining the data dictionary comprising a plurality of terms generated based on the received data source agnostic information request comprises:
 obtaining, by a parsing engine of the data processing system, a set of one or more terms of the data source agnostic information request that each correspond to a keyed data attribute of one the data sources.   
     
     
         18 . The non-transitory computer-readable medium of  claim 15 , wherein generating a dynamic data filter using one or more of the plurality of terms of the data dictionary comprises:
 obtaining, by the data processing system, one or more information filter terms from a library of predetermined information filter terms using the terms of data dictionary; and   generating, the data processing system, a dynamic data filter using (i) the one or more of the plurality of terms of the data dictionary and (ii) the obtained one or more information filter terms from the library of predetermined information filter terms.   
     
     
         19 . The non-transitory computer-readable medium of  claim 15 , the operations further comprising:
 generating, by the data processing system, a searchable index that indexes the filtered information.   
     
     
         20 . The non-transitory computer-readable medium of  claim 15 , the operations further comprising:
 periodically generating, by the data mining service and subsequent to the providing of the filtered information responsive to the received data source agnostic information request, alerts that include information that satisfies the received data source agnostic information request; and   providing, by the data mining service and to the user device, the periodically generated alerts to the user device.