Automated Monitoring and Verification of Internet Based Advertising

Abstract

Method for automatically monitoring and verifying advertising content during a campaign, delivered over a data network. Accordingly, advertisers submit a list of sites, on which the advertising content should be placed according to desired insertion order. Mapping crawlers visit these sites and locate pages with advertisements that belong to required sections, pages that do not belong to the required sections or pages with high probability for incidents. A list of pages to visit per every site is generated and autonomous or plug-in visual crawlers are allowed to visit the list of pages, according to predetermined site visiting plan. A crawlers' manager allocates the pages between visual crawlers, for obtaining adequate incident coverage and load on the visual crawlers. An incident identifier compares the insertion orders with the delivery data and whenever an insertion order and its corresponding delivery data do not match, an incident report is generated.

1 . A method for automatically monitoring multimedia content displayed over a data network, comprising the steps:
 a) user initiated uploading via said data network, by means of a user interface or a software interface of a processor enabled device, of content related information according to a predetermined agreement and to a list of sites or of sections per site, to which the content should or should not be provided according to said agreement;   b) distributing, by means of at least one crawler manager server to a plurality of web crawlers including at least one mapping crawler and at least one visual crawler, crawling tasks with respect to specified web pages that need to be crawled;   c) activating, by means of said at least one crawler manager server, said plurality of web crawlers so that they will visit said specified web pages and perform said crawling tasks, according to a predetermined site visiting plan, and will extract visual content related delivery data, contextual delivery data, or metadata related delivery data therefrom; and   d) storing said extracted delivery data and data associated with said plurality of web crawlers in a plurality of databases,
 wherein said at least one visual crawler is used to render a web-page graphically and to generate a hierarchical representation of said page based on a HTML text of said page. 
   
     
     
         2 . A method according to  claim 1 , further comprising activating a tracking process for tracking web pages containing the content related information. 
     
     
         3 . A method according to  claim 2 , wherein the tracking process uses the delivery data for analysis and extraction of the URL of the visited site, in which the content has been displayed. 
     
     
         4 . A method according to  claim 2 , wherein the tracking process uses the delivery data for detecting the tag ID of the displayed content. 
     
     
         5 . A method according to  claim 1 , wherein the visiting plan includes how many times per day should each page be visited and the start and end date of a campaign. 
     
     
         6 . A method according to  claim 1 , further comprising the steps of:
 a) comparing, by means of at least one incident generator server, the predetermined agreement with the extracted delivery data; and   b) generating an incident whenever one of predetermined agreements and its corresponding delivery data do not match.   
     
     
         7 . A method according to  claim 1 , wherein agreement information is modified at any time point. 
     
     
         8 . A method according to  claim 1 , wherein the at least one mapping crawler is used to identify content server key values and content categories associated with each page, for creating a site map related to content networks, content servers, or a network of sites. 
     
     
         9 . A method according to  claim 8 , wherein the site map includes a number of times each page is linked and parameters representing the weight of the page. 
     
     
         10 . A method according to  claim 1 , wherein the at least one visual crawler is also used to identify interstitials. 
     
     
         11 . A method according to  claim 1 , wherein the at least one visual crawler performs:
 Session Crawling;   Cookie Crawling;   Contextual Crawling; or   Classification Crawling.   
     
     
         12 . A method according to  claim 1 , wherein the at least one crawler manager server is used to:
 a) intermediate and arbitrate between one or more of the plurality of databases and running crawlers; and   b) retrieve sites or pages that needed to be crawled from said one or more of the plurality of databases and allocate them to different crawlers.   
     
     
         13 . A method according to  claim 1 , wherein the crawler is an autonomous crawler or a plug-in crawler. 
     
     
         14 . A method according to  claim 1 , wherein content is recognized according to:
 HTML tags   Flash tags   JavaScript; or   IFrames is which other content is embedded.   
     
     
         15 . A method according to  claim 6 , wherein the incident that is generated is selected from the group consisting of a Competitive Collision incident, a Frequency incident, a Missing Targeting incident, a Placement not found incident, a Sponsorship not enforced incident, a Wrong content incident, a Day time incident, an Out of channel incident, a Wrong date incident, an Out of inclusion site incident, and an Excluded site incident. 
     
     
         16 . A method according to  claim 6 , wherein the incident that is generated is a fold incident, a clutter incident, a fraud incident, a content hijacking incident, or an inappropriate content incident. 
     
     
         17 . A method according to  claim 2 , further comprising delivering a actual URL or site name by the tracking process and then extracting said URL or site name to produce an origin URL. 
     
     
         18 . A data processing system for automatically monitoring multimedia content displayed over a data network said data processing system comprising:
 a) at least one content server for storing, delivering and uploading content according to a predetermined agreement via said data network;   b) a plurality of web crawlers including at least one mapping crawler and at least one visual crawler, for extracting visual content related information from specified web pages according to a predetermined site visiting plan;   c) at least one mediator server for distributing, to said plurality of web crawlers, crawling tasks with respect to web pages that need to be crawled and for determining a status of each of said plurality of web crawlers; and   d) a plurality of databases in which is stored said extracted visual content related information and data associated with said plurality of web crawlers,
 wherein said at least one visual crawler is used to render a web-page graphically and to generate a hierarchical representation of said page based on a HTML text of said page. 
   
     
     
         19 . A method for automatically monitoring multimedia content displayed over a data network, comprising the steps:
 a) user initiated uploading via said data network, by means of a user interface or a software interface of a processor enabled device, of content related information according to a predetermined agreement and to a list of sites or of sections per site, to which the content should or should not be provided according to said agreement;   b) distributing, by means of at least one crawler manager server to a plurality of web crawlers including at least one mapping crawler and at least one visual crawler, crawling tasks with respect to specified web pages that need to be crawled;   c) activating, by means of said at least one crawler manager server, said plurality of web crawlers so that they will visit said specified web pages and perform said crawling tasks, according to a predetermined site visiting plan, and will extract visual content related delivery data, contextual delivery data, or metadata related delivery data therefrom; and   d) storing said extracted delivery data and data associated with said plurality of web crawlers in a plurality of databases,
 wherein said at least one visual crawler is used to render a web-page graphically and to generate a hierarchical representation of said page based on a HTML text of said page. 
 wherein said at least one visual crawler is used to render a web-page graphically, to identify media types that are displayed on said page, and to check if a HTML tag or a JavaScript tag of said page has certain signatures that define the media as being representative of said content related information. 
   
     
     
         20 . A data processing system for automatically monitoring multimedia content displayed over a data network said data processing system comprising:
 a) at least one content server for storing, delivering and uploading content according to a predetermined agreement via said data network;   b) a plurality of web crawlers including at least one mapping crawler and at least one visual crawler, for extracting visual content related information from specified web pages according to a predetermined site visiting plan;   c) at least one mediator server for distributing, to said plurality of web crawlers, crawling tasks with respect to web pages that need to be crawled and for determining a status of each of said plurality of web crawlers; and   d) a plurality of databases in which is stored said extracted visual content related information and data associated with said plurality of web crawlers,
 wherein said at least one visual crawler is used to render a web-page graphically, to identify media types that are displayed on said page, and to check if a HTML tag or a JavaScript tag of said page has certain signatures that define the media as being representative of said content related information.