Intelligent online personal assistant with multi-turn dialog based on visual search

Abstract

Systems, methods, and computer program products for identifying a relevant candidate product in an electronic marketplace. Embodiments perform a visual similarity comparison between candidate product image visual content and input query image visual content, process formal and informal natural language user inputs, and coordinate aggregated past user interactions with the marketplace stored in a knowledge graph. Visually similar items and their corresponding product categories, aspects, and aspect values can determine suggested candidate products without discernible delay during a multi-turn user dialog. The user can then refine the search for the most relevant items available for purchase by providing responses to machine-generated prompts that are based on the initial search results from visual, voice, and/or text inputs. An intelligent online personal assistant can thus guide a user to the most relevant candidate product more efficiently than existing search tools.

What is claimed is: 
     
         1 . A method comprising:
 generating a plurality of candidate product image signatures for a plurality of candidate product images;   generating an input query image signature for an input query image;   calculating a plurality of visual similarity measures between the plurality of candidate product image signatures and the input query image signature; and   outputting a ranked list of the plurality of candidate product images based on the plurality of visual similarity measures.   
     
     
         2 . The method of  claim 1 , further comprising:
 storing, in a database, a product image signature of the plurality of candidate product image signatures in association with a product identification number associated with a candidate product image of the plurality of candidate product images.   
     
     
         3 . The method of  claim 2 , wherein the storing is performed for a new product listed in an electronic marketplace. 
     
     
         4 . The method of  claim 1 , further comprising:
 storing, in a database, a product image signature of the plurality of candidate product image signatures in association with a uniform resource locator associated with a candidate product image of the plurality of candidate product images.   
     
     
         5 . The method of  claim 4 , wherein the storing is performed for a new product listed in an electronic marketplace. 
     
     
         6 . The method of  claim 1 , wherein the plurality of candidate product image signatures semantically represent the plurality of candidate product images. 
     
     
         7 . The method of  claim 1 , wherein the input query image is associated with a product listed in an electronic marketplace. 
     
     
         8 . A non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:
 generating a plurality of candidate product image signatures for a plurality of candidate product images;   generating an input query image signature for an input query image;   calculating a plurality of visual similarity measures between the plurality of candidate product image signatures and the input query image signature; and   outputting a ranked list of the plurality of candidate product images based on the plurality of visual similarity measures.   
     
     
         9 . The non-transitory computer-readable storage medium of  claim 8 , wherein the operations further comprise:
 storing, in a database, a product image signature of the plurality of candidate product image signatures in association with a product identification number associated with a candidate product image of the plurality of candidate product images.   
     
     
         10 . The non-transitory computer-readable storage medium of  claim 9 , wherein the storing is performed for a new product listed in an electronic marketplace. 
     
     
         11 . The non-transitory computer-readable storage medium of  claim 8 , wherein the operations further comprise:
 storing, in a database, a product image signature of the plurality of candidate product image signatures in association with a uniform resource locator associated with a candidate product image of the plurality of candidate product images.   
     
     
         12 . The non-transitory computer-readable storage medium of  claim 11 , wherein the storing is performed for a new product listed in an electronic marketplace. 
     
     
         13 . The non-transitory computer-readable storage medium of  claim 8 , wherein the plurality of candidate product image signatures semantically represent the plurality of candidate product images. 
     
     
         14 . The non-transitory computer-readable storage medium of  claim 8 , wherein the input query image is associated with a product listed in an electronic marketplace. 
     
     
         15 . A system comprising:
 a processor; and   a memory comprising instructions that, when executed by the processor, cause the processor to perform operations comprising:
 generating a plurality of candidate product image signatures for a plurality of candidate product images; 
 generating an input query image signature for an input query image; 
 calculating a plurality of visual similarity measures between the plurality of candidate product image signatures and the input query image signature; and 
 outputting a ranked list of the plurality of candidate product images based on the plurality of visual similarity measures. 
   
     
     
         16 . The system of  claim 15 , wherein the operations further comprise:
 storing, in a database, a product image signature of the plurality of candidate product image signatures in association with a product identification number associated with a candidate product image of the plurality of candidate product images.   
     
     
         17 . The system of  claim 16 , wherein the storing is performed for a new product listed in an electronic marketplace. 
     
     
         18 . The system of  claim 15 , wherein the operations further comprise:
 storing, in a database, for a new product listed in an electronic marketplace, a product image signature of the plurality of candidate product image signatures in association with a uniform resource locator associated with a candidate product image of the plurality of candidate product images.   
     
     
         19 . The system of  claim 18 , wherein the storing is performed for a new product listed in an electronic marketplace. 
     
     
         20 . The system of  claim 15 , wherein the plurality of candidate product image signatures semantically represent the plurality of candidate product images.