Transmitting Card Objects Based On Categorized And Scraped Data

Abstract

A server configured to receive data corresponding with a web page and/or a native application, the data comprising a uniform resource locator (URL); compare the URL included in the data with URL patterns stored in a category data store; determine a data category of the data based on the comparison; determine a data scraper based on the data category, the data scraper comprising a scraper configuration that enables the data scraper to extract information from the web page and/or the native application; scrape the web page and/or the native application by extracting information from the web page and/or the native application using the data scraper; store the extracted information in one or more data stores; used the stored information to provide search results and/or card objects.

What is claimed is: 
     
         1 . A server comprising:
 a network communication device;   a storage device comprising:
 a category data store that stores category records corresponding with data categories, each category record comprising:
 a category identifier (ID) that identifies the category record; and 
 a uniform resource locator (URL) pattern associated with the data category; 
 
 an entity data store that stores entity records; and 
 a card data store that stores card records; and 
   a processing device that executes computer-readable instructions that, when executed by the processing device, cause the processing device to:
 receive data corresponding with a web page and/or a native application, the data comprising a uniform resource locator (URL); 
 compare the URL included in the data with the URL patterns stored in the category data store; 
 determine a data category of the data based on the comparison of the URL with the URL patterns; 
 determine a data scraper based on the data category of the data, the data scraper comprising a scraper configuration that enables the data scraper to extract information from the web page and/or the native application; 
 scrape the web page and/or the native application by extracting information from the web page and/or the native application using the data scraper; 
 populate one or more entity records in the entity data store and/or one or more card records in the card data store based on the scraped data; 
 receive a card request, the card request comprising one or more search terms and/or contextual data; 
 determine a card record from the card data store based on the card request; 
 generate a card object based on the determined card record; and 
 transmit the card object via the network communication device. 
   
     
     
         2 . The server of  claim 1 , wherein comparing the URL with the URL patterns comprises using an inverted index that indexes the URL patterns to the category records. 
     
     
         3 . The server of  claim 1 , wherein comparing the URL with the URL patterns comprises, for each URL pattern, determining a Levenshtein distance between the URL and the URL pattern. 
     
     
         4 . The server of  claim 3 , wherein determining the data category comprises identifying the URL pattern with the lowest Levenshtein distance and selecting the category ID associated therewith. 
     
     
         5 . The server of  claim 1 , wherein determining the data category comprises identifying the URL pattern that matches the URL and selecting the category ID associated therewith. 
     
     
         6 . The server of  claim 1 , wherein:
 the storage device further comprises a scraper data store that stores scraper configuration files comprising:
 a file ID that identifies the scraper configuration file; 
 a category ID that identifies the data category associated with the scraper configuration file; and 
 scraping parameters; and 
   determining the data scraper comprises retrieving the scraper configuration file associated with the data category of the data.   
     
     
         7 . The server of  claim 1 , wherein determining the card record comprises performing a search of the card data store and/or the entity data store based on the search terms and/or the contextual data in the card request. 
     
     
         8 . The server of  claim 1 , wherein generating the card object comprises:
 creating a data container; and   writing information from the determined card record to the data container.   
     
     
         9 . A server comprising:
 a network communication device;   a storage device comprising:
 a category data store that stores category records corresponding with data categories, each category record comprising:
 a category identifier (ID) that identifies the category record; and 
 a structural pattern associated with the data category, the structural pattern indicating the structure of a web page and/or a native application; 
 
 an entity data store that stores entity records; and 
 a card data store that stores card records; and 
   a processing device that executes computer-readable instructions that, when executed by the processing device, cause the processing device to:
 receive data corresponding with a web page and/or a native application, the data comprising metadata that indicates a structure of the web page and/or the native application; 
 compare the structure indicated by the metadata with the structural patterns stored in the category data store; 
 determine a data category of the data based on the comparison of the metadata with the structural patterns; 
 determine a data scraper based on the data category of the data, the data scraper comprising a scraper configuration that enables the data scraper to extract information from the web page and/or the native application; 
 scrape the web page and/or the native application for scraped data by extracting information from the web page and/or the native application using the data scraper; 
 populate one or more entity records in the entity data store and/or one or more card records in the card data store based on the scraped data; 
 receive a card request, the card request comprising one or more search terms and/or contextual data; 
 determine a card record from the card data store based on the card request; 
 generate a card object based on the determined card record; and 
 transmit the card object via the network communication device. 
   
     
     
         10 . The server of  claim 9 , wherein the metadata indicating the structure of the web page and/or the native application comprises metatags that refer to graphical user interface (GUI) elements displayed by the web page and/or the native application, the GUI elements comprising data fields, images, labels, buttons and/or drop-down menus. 
     
     
         11 . The server of  claim 9 , wherein comparing the structure with the structural patterns comprises using an inverted index that indexes the structural patterns to the category records. 
     
     
         12 . The server of  claim 9 , wherein comparing the structure with the structural patterns comprises, for each structural pattern, determining a Levenshtein distance between the structure and the structural pattern. 
     
     
         13 . The server of  claim 12 , wherein determining the data category comprises identifying the structural pattern with the smallest Levenshtein distance and selecting the category ID associated therewith. 
     
     
         14 . The server of  claim 9 , wherein determining the data category comprises identifying the structural pattern that matches the structure and selecting the category ID associated therewith. 
     
     
         15 . A server comprising:
 a network communication device;   a storage device comprising:
 a category data store that stores category records corresponding with data categories, each category record comprising:
 a category identifier (ID) that identifies the category record; and 
 metatag patterns associated with the data category, the metatag patterns indicating content displayed by a web page and/or a native application; and 
 
 an entity data store that stores entity records; and 
   a processing device that executes computer-readable instructions that, when executed by the processing device, cause the processing device to:
 receive data corresponding with a web page and/or a native application, the data comprising metadata that includes metatag pairs, each metatag pair comprising a tag label and a value interposed between the metatag pair; 
 compare the tag labels with the metatag patterns stored in the category data store; 
 determine a data category of the data based on the comparison of the metadata with the metatag patterns; 
 determine a data scraper based on the data category of the data, the data scraper comprising a scraper configuration that enables the data scraper to extract information from the web page and/or the native application; 
 scrape the web page and/or the native application by extracting information from the web page and/or the native application using the data scraper; 
 populate one or more entity records in the entity data store based on the scraped data; 
 receive a search request, the search request comprising one or more search terms and/or contextual data; 
 determine one or more entity records from the entity data store based on the search request; 
 generate a result object based on the determined entity records; and 
 transmit the result object via the network communication device. 
   
     
     
         16 . The server of  claim 15 , wherein comparing the tag labels with the metatag patterns comprises using an inverted index that indexes the metatag patterns to the category records. 
     
     
         17 . The server of  claim 15 , wherein determining the data category comprises identifying the metatag pattern that matches the tag labels and selecting the category ID associated therewith. 
     
     
         18 . The server of  claim 15 , wherein the metatag patterns include vocabulary from a schema that is used to markup data. 
     
     
         19 . The server of  claim 15 , wherein the metatag patterns include uniform resource locators (URLs) that identify entity types, relationships between entities and/or actions.