Image Recognition Based Content Item Selection

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for content presentation. In one aspect, a method includes receiving a request for a content item to be presented to a user; for a collection of content items, identifying one or more candidate content items including using image keywords derived from images associated with the user to identify the one or more candidate content items; selecting a content item responsive to the content request from the candidate content items; and sending the selected content item for presentation to the user.

What is claimed is: 
     
         1 .- 27 . (canceled) 
     
     
         28 . A computing system, the system comprising:
 one or more processors; and   one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:   receiving, via an application executed by a computing device connected to a network, an image captured by a camera of the computing device;   processing the image captured by the camera to determine one or more keywords associated with the image;   determining a plurality of content items responsive to the image capture by the camera based at least in part on the one or more keywords, wherein the plurality of content items comprise one or more sponsored content items; and   providing the plurality of content items to the computing device for display in a mobile application.   
     
     
         29 . The system of  claim 28 , wherein processing the image captured by the camera to determine the one or more keywords comprises:
 identifying, via optical character recognition, text in the image captured by the camera.   
     
     
         30 . The system of  claim 28 , wherein processing the image captured by the camera to determine the one or more keywords comprises:
 determining one or more existing objects in an image database are associated with the image captured by the camera; and   processing metadata associated with the one or more existing objects to obtain the one or more keywords.   
     
     
         31 . The system of  claim 28 , wherein processing the image captured by the camera to determine the one or more keywords comprises:
 determining a matching image that matches the image captured by the camera;   obtaining a search history log associated with the matching image;   processing the search history log to determine one or more search queries associated with the matching image; and   determining the one or more keywords based on one or more search terms in the one or more search queries.   
     
     
         32 . The system of  claim 31 , wherein processing the search history log to determine the one or more search queries associated with the matching image comprises:
 processing the search history log to determine the matching image was previously provided as a search result responsive to the one or more search queries.   
     
     
         33 . The system of  claim 31 , wherein processing the search history log to determine the one or more search queries associated with the matching image comprises:
 processing the search history log to determine the matching image was previously selected after being provided as a search result responsive to the one or more search queries.   
     
     
         34 . The system of  claim 28 , wherein the operations further comprise:
 expanding the one or more keywords to include categories;   determining a score for each of the plurality of content items based on matching the one or more keywords for each of the plurality of content items with the one or more keywords including the categories; and   ranking the plurality of content items based on the scores.   
     
     
         35 . The system of  claim 28 , wherein the computing device is a mobile computing device with the camera integrated into the mobile computing device. 
     
     
         36 . The system of  claim 28 , wherein processing the image captured by the camera to determine the one or more keywords comprises:
 processing the image with an image analysis module of a search system.   
     
     
         37 . The system of  claim 28 , wherein the one or more sponsored content items are determined by:
 obtaining an ad request for an ad to be presented to a user associated with the computing device;   identifying one or more candidate ads of a collection of ads for presentation to the user, wherein identifying the one or more candidate ads includes scoring each candidate ad based at least in part on a degree of match between the respective ad and the one or more keywords; and   selecting an ad responsive to the ad request from the candidate ads based on the scores.   
     
     
         38 . The system of  claim 37 , where selecting an ad responsive to the ad request includes determining a similarity between one or more keywords associated with each ad of the collection of ads and the image keywords, and wherein determining the similarity include calculating a similarity score for each ad, and wherein selecting the ad responsive to the ad request includes using a combination of the similarity score and one or more other scores for each. 
     
     
         39 . A computer-implemented method, the method comprising:
 receiving, via an application executed by a computing device connected to a network and by a data processing system including one or more processors and memory, an image captured by a camera of the computing device;   processing, by the data processing system, the image captured by the camera to determine one or more keywords;   determining, by the data processing system, a plurality of search results responsive to the image capture by the camera based at least in part on the one or more keywords;   determining, by the data processing system, one or more ads based on at least a subset of the plurality of search results; and   providing, by the data processing system, the plurality of search results and the one or more ads to the computing device.   
     
     
         40 . The computer-implemented method of  claim 39 , further comprising:
 receiving a request for an ad to be presented to a user associated with the computing device;   for a collection of ads, identifying one or more candidate ads for presentation to the user, wherein identifying the one or more candidate ads includes scoring each candidate ad based at least in part on a degree of match between the respective ad and the one or more keywords; and   wherein the one or more ads are determined based at least in part on the scores of the candidate ads.   
     
     
         41 . The computer-implemented method of  claim 39 , further comprising:
 processing the image captured by the camera to deter an intent of a user associated with the computing device.   
     
     
         42 . The computer-implemented method of  claim 41 , wherein the plurality of search results are determined based at least in part on the intent of the user. 
     
     
         43 . The computer-implemented method of  claim 39 , wherein the plurality of search results are determined by a search system. 
     
     
         44 . The computer-implemented method of  claim 39 , wherein the one or more ads are determined by a search system. 
     
     
         45 . One or more non-transitory computer-readable media comprising instructions that when executed by one or more computing devices cause the one or more computing devices to perform operations comprising:
 receiving, via an application executed by a computing device connected to a network, an image captured by a camera of the computing device;   processing the image captured by the camera to determine one or more keywords associated with the image captured by the camera;   determining a plurality of search results responsive to the image capture by the camera based at least in part on the one or more keywords;   determining one or more ads based on at least one of the image captured by the camera or a subset of the plurality of search results; and   providing the plurality of search results and the one or more ads to the computing device.   
     
     
         46 . The one or more non-transitory computer-readable media of  claim 45 , wherein the one or more ads comprise an image ad with embedded information associated with a link and machine executable instructions. 
     
     
         47 . The one or more non-transitory computer-readable media of  claim 45 , wherein providing the plurality of search results and the one or more ads to the computing device comprises:
 providing the plurality of search results and the one or more ads for display in a mobile application associated with a search system.