Advertisement campaign simulator

Abstract

A shadow ad can be evaluated by receiving an ad request, identifying at least one shadow ad and at least one actual ad based on the received ad request, generating an ad ranking by analyzing one or more criteria associated with the identified at least one shadow ad and the at least one actual ad, and selecting one or more of the identified at least one shadow ad and the at least one actual ad based on the ad ranking. Further, the at least one shadow ad can be associated with one or more campaign parameters corresponding to a shadow ad campaign. Additionally, the ad request can be received from an online advertising system.

1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method comprising:
 identifying a shadow content item, the shadow content item not being eligible for an auction for selecting content items to be served in response to a received request for content;   receiving a request for content;   evaluating the inventory of content items to identify candidate content items in the inventory to serve responsive to the request, including ranking the candidate content items and selecting a highest-ranking candidate content item;   determining a relative ranking that ranks the highest-ranking candidate content item against the shadow content item based at least in part on the request;   determining a measure of performance of the shadow content item based at least in part on the relative ranking between the highest-ranking candidate content item and the shadow content item;   determining that the measure of performance of the shadow content item did not meet a performance goal; and   in response to determining that the measure of performance of the shadow content item did not meet a performance goal, automatically adjusting a parameter associated with the shadow content item.   
     
     
         22 . The method of claim  1 , further comprising:
 executing a shadow campaign according to one or more parameters associated with the shadow campaign, including determining one or more relative rankings that rank the shadow content item against one or more candidate content items.   
     
     
         23 . The method of claim  2 , wherein the performance goal for the shadow content item comprises a minimum click-through rate or a minimum conversion rate for a default bid over the shadow campaign. 
     
     
         24 . The method of claim  2 , further comprising:
 executing the shadow campaign according to the adjusted parameter of the shadow content item, including determining one or more relative rankings that rank the shadow content item against one or more candidate content items according to the adjusted parameter of the shadow content item;   determining a second measure of performance of the shadow content item based at least in part on the one or more relative rankings;   determining that the second measure of performance of the shadow content item meets the performance goal; and   saving the adjusted parameter in association with the shadow content item.   
     
     
         25 . The method of claim  4 , further comprising:
 receiving a request to execute the shadow campaign as an actual campaign; and   executing the shadow campaign as an actual campaign using the adjusted parameter of the shadow content item, including converting the shadow content item into candidate content item that is eligible to be served in response to requests for content.   
     
     
         26 . The method of claim  1 , wherein determining the relative ranking comprises conducting a shadow auction based on a first bid associated with the shadow content item and a second bid associated with the highest-ranking candidate content item. 
     
     
         27 . The method of claim  6 , wherein the shadow auction is conducted in response to the request for content on a content page. 
     
     
         28 . The method of claim  1 , further comprising:
 determining that the shadow content item would be served responsive to the request for content; and   in response to determining that the shadow content item would be served responsive to the request for content, recording a charge in shadow campaign data associated with the shadow content item.   
     
     
         29 . The method of claim  1 , wherein automatically adjusting the parameter associated with the shadow content item comprises increasing a bid amount associated with the shadow content item. 
     
     
         30 . A system comprising:
 one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:   identifying a shadow content item, the shadow content item not being eligible for an auction for selecting content items to be served in response to a received request for content;   receiving a request for content;   evaluating the inventory of content items to identify candidate content items in the inventory to serve responsive to the request, including ranking the candidate content items and selecting a highest-ranking candidate content item;   determining a relative ranking that ranks the highest-ranking candidate content item against the shadow content item based at least in part on the request;   determining a measure of performance of the shadow content item based at least in part on the relative ranking between the highest-ranking candidate content item and the shadow content item;   determining that the measure of performance of the shadow content item did not meet a performance goal; and   in response to determining that the measure of performance of the shadow content item did not meet a performance goal, automatically adjusting a parameter associated with the shadow content item.   
     
     
         31 . The method of  claim 30 , wherein the operations further comprise
 executing a shadow campaign according to one or more parameters associated with the shadow campaign, including determining one or more relative rankings that rank the shadow content item against one or more candidate content items.   
     
     
         32 . The system of  claim 31 , wherein the performance goal for the shadow content item comprises a minimum click-through rate or a minimum conversion rate for a default bid over the shadow campaign. 
     
     
         33 . The system of  claim 31 , wherein the operations further comprise:
 executing the shadow campaign according to the adjusted parameter of the shadow content item, including determining one or more relative rankings that rank the shadow content item against one or more candidate content items according to the adjusted parameter of the shadow content item;   determining a second measure of performance of the shadow content item based at least in part on the one or more relative rankings;   determining that the second measure of performance of the shadow content item meets the performance goal; and   saving the adjusted parameter in association with the shadow content item.   
     
     
         34 . The system of  claim 33 , wherein the operations further comprise:
 receiving a request to execute the shadow campaign as an actual campaign; and   executing the shadow campaign as an actual campaign using the adjusted parameter of the shadow content item, including converting the shadow content item into candidate content item that is eligible to be served in response to requests for content.   
     
     
         35 . The system of  claim 30 , wherein determining the relative ranking comprises conducting a shadow auction based on a first bid associated with the shadow content item and a second bid associated with the highest-ranking candidate content item. 
     
     
         36 . The system of  claim 35 , wherein the shadow auction is conducted in response to the request for content on a content page. 
     
     
         37 . The system of  claim 30 , wherein the operations further comprise:
 determining that the shadow content item would be served responsive to the request for content; and   in response to determining that the shadow content item would be served responsive to the request for content, recording a charge in shadow campaign data associated with the shadow content item.   
     
     
         38 . The system of  claim 30 , wherein automatically adjusting the parameter associated with the shadow content item comprises increasing a bid amount associated with the shadow content item. 
     
     
         39 . A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
 identifying a shadow content item, the shadow content item not being eligible for an auction for selecting content items to be served in response to a received request for content;   receiving a request for content;   evaluating the inventory of content items to identify candidate content items in the inventory to serve responsive to the request, including ranking the candidate content items and selecting a highest-ranking candidate content item;   determining a relative ranking that ranks the highest-ranking candidate content item against the shadow content item based at least in part on the request;   determining a measure of performance of the shadow content item based at least in part on the relative ranking between the highest-ranking candidate content item and the shadow content item;   determining that the measure of performance of the shadow content item did not meet a performance goal; and   in response to determining that the measure of performance of the shadow content item did not meet a performance goal, automatically adjusting a parameter associated with the shadow content item.   
     
     
         40 . The computer program product of  claim 39 , wherein the operations further comprise:
 executing a shadow campaign according to one or more parameters associated with the shadow campaign, including determining one or more relative rankings that rank the shadow content item against one or more candidate content items.