System and method to retrieve relevant inventory using sketch-based query

Abstract

In various example embodiments, a system and method for sketch based queries are presented. A sketch of a search item is received from a user device. An item attribute corresponding to a physical attribute of the search item is extracted from the sketch. Inventory items are identified based on the extracted item attribute. The identified inventory items are presented to the user. A modification to the sketch of the search item is received and the inventory items are updated based on the received modification.

1 . A method comprising:
 receiving a sketch of a search item, the sketch being received from a user device;   extracting an item attribute from the sketch, the item attribute corresponding to a physical attribute of the search item;   identifying inventory items based on the extracted item attribute;   causing display of the identified inventory items in a user interface of the user device;   receiving a modification to the sketch of the search item;   updating, using one or more processors, the inventory items based on the received the modification to the sketch; and   causing display of the updated inventory items in the user interface of the user device.   
     
     
         2 . The method of  claim 1 , wherein the receiving the sketch includes receiving an annotation of an existing image on the user device. 
     
     
         3 . The method of  claim 1 , wherein the receiving the sketch includes receiving an annotation of an image that is captured using a camera on the user device. 
     
     
         4 . The method of  claim 1 , further comprising:
 determining similarity levels between the sketch and each of the identified inventory items;   sorting the inventory items based on the determined similarity levels; and   causing display of the sorted inventory items in the user interface.   
     
     
         5 . The method of  claim 1 , further comprising:
 retrieving, from a database, images of the inventory items; and   extracting one or more image attributes from the retrieved images; and wherein the identifying includes comparing the extracted item attribute with the one or more extracted image attributes.   
     
     
         6 . The method of  claim 1 , further comprising:
 determining suggested attributes based on item attributes of the search item; and   providing the suggested attributes to the user device including causing display of the suggested attributes on the user device.   
     
     
         7 . The method of  claim 6 , further comprising:
 determining a user preference based on a purchase history stored in a user profile; and   
       wherein the determining the suggested attributes is further based on the determined user preference. 
     
     
         8 . The method of  claim 6 , further comprising:
 receiving a user selected attribute from the user device, the user selected attribute being from the suggested attributes provided to the user device; and   updating the inventory items based on the received user selected attribute.   
     
     
         9 . The method of  claim 1 , further comprising:
 determining an item category based on the received sketch; and   presenting the determined item category; and wherein the identifying is further based on the determined item category.   
     
     
         10 . The method of  claim 1 , wherein:
 the extracted item attribute is mapped to a description of the item attribute; and   the identifying the inventory items is based on the description of the item attribute.   
     
     
         11 . The method of  claim 1 , further comprising mapping the extracted item attribute to inventory features of the inventory items; and wherein the identifying the inventory items is based on the mapping. 
     
     
         12 . A system comprising: 
       one or more processors and executable instructions accessible on a computer-readable medium that, when executed, configure the one or more processors to at least:
 receive a sketch of a search item, the sketch being received from a user device; 
 extract an item attribute from the sketch, the item attribute corresponding to a physical attribute of the search item; 
 identify inventory items based on the extracted item attribute; 
 cause display of the identified inventory items in a user interface of the user device; 
 receive a modification to the sketch of the search item; 
 update the inventory items based on the received the modification to the sketch; and 
 cause display of the updated inventory items in the user interface of the user device. 
 
     
     
         13 . The system of  claim 12 , wherein the one or more processors are further configured to receive an annotation of an existing image on the user device. 
     
     
         14 . The system of  claim 12 , wherein the one or more processors are further configured to receive an annotation of an image that is captured using a camera on the user device. 
     
     
         15 . The system of  claim 12 , wherein the one or more processors are further configured to:
 determine similarity levels between the sketch and each of the identified inventory items;   sort the inventory items based on the determined similarity levels; and   cause display of the sorted inventory items in the user interface.   
     
     
         16 . The system of  claim 12 , wherein the one or more processors are further configured to:
 retrieve, from a database, images of the inventory items;   extract one or more image attributes from the retrieved images; and   compare the extracted item attribute with the one or more extracted image attributes.   
     
     
         17 . The system of  claim 12 , wherein the one or more processors are further configured to:
 determine suggested attributes based on item attributes of the search item; and   cause display of the suggested attributes on the user device.   
     
     
         18 . The system of  claim 12 , wherein the one or more processors are further configured to:
 determine a user preference based on a purchase history stored in a user profile; and   determine the suggested attributes further based on the determined user preference.   
     
     
         19 . The system of  claim 12 , wherein the one or more processors are further configured to:
 receive a user selected attribute from the user device, the user selected attribute being from the suggested attributes provided to the user device; and   update the inventory items based on the received user selected attribute.   
     
     
         20 . A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
 receiving a sketch of a search item, the sketch being received from a user device;   extracting an item attribute from the sketch, the item attribute corresponding to a physical attribute of the search item;   identifying inventory items based on the extracted item attribute;   causing display of the identified inventory items in a user interface of the user device;   receiving a modification to the sketch of the search item;   updating the inventory items based on the received the modification to the sketch; and   causing display of the updated inventory items in the user interface of the user device.