Sketch-based image retrieval using feedback and hierarchies

Abstract

A system, method and program product for implementing a sketch-based retrieval system. A system is disclosed that includes: an interface having a canvas for entering inputs including user generated strokes, and extracting features associated with the inputs; a hierarchy system for assigning an importance to the inputs; a matching system for retrieving matches from a database based on extracted features and importance associated with the extracted features; a feedback system that provides real-time feedback on the canvas to suggest improved inputs for the user; and a ranking system that ranks retrieved matches for output in the interface by incorporating user interaction history.

What is claimed is: 
     
         1 . A sketch-based retrieval system, comprising:
 an interface having a canvas for entering inputs including user generated strokes, and for extracting features associated with the inputs, wherein the inputs include different layers on the canvas;   a hierarchy system for assigning an importance to the inputs;   a matching system for retrieving matches from a database based on extracted features and importance associated with the extracted features, wherein the matching system is configured to display a set of ranked images on the canvas based upon the retrieved matches;   a feedback system that provides real-time feedback on the canvas to suggest improved inputs for the user, wherein the feedback system suggests the improved inputs by a display on the canvas proximate the entered inputs; and   a ranking system that ranks retrieved matches for output in the interface.   
     
     
         2 . The sketch-based retrieval system of  claim 1 , wherein the strokes are submitted via a touch screen. 
     
     
         3 . The sketch-based retrieval system of  claim 1 , wherein the display on the canvas proximate the entered inputs to suggest improved inputs include at least one of: dashed lines, blinking lines, or colored shapes. 
     
     
         4 . The sketch-based retrieval system of  claim 3 , wherein the hierarchy system can assign an importance to an individual layer. 
     
     
         5 . The sketch-based retrieval system of  claim 1 , wherein the feedback system highlights existing inputs as problematic. 
     
     
         6 . The sketch-based retrieval system of  claim 1 , wherein the feedback system displays suggested inputs. 
     
     
         7 . The sketch-based retrieval system of  claim 1 , further comprising a system for collecting and analyzing user interaction history to enhance matching and ranking. 
     
     
         8 . A computer program product stored on a computer readable storage medium, which when executed by a computing system, provides sketch-based retrieval, the program product comprising:
 program code for rendering a canvas for entering inputs including user generated strokes, and for extracting features associated with the inputs, wherein the inputs include different layers on the canvas;   program code for assigning an importance to the inputs;   program code for retrieving matches from a database based on extracted features and importance associated with the extracted features and displaying a set of ranked images on the canvas based upon the retrieved matches;   program code that provides real-time feedback on the canvas to suggest improved inputs for the user, wherein the real-time feedback is a display on the canvas proximate the entered inputs; and   program code that ranks retrieved matches for output.   
     
     
         9 . The computer program product of  claim 8 , wherein the strokes are submitted via a touch screen. 
     
     
         10 . The computer program product of  claim 8 , wherein the display on the canvas proximate the entered inputs includes at least one of: dashed lines, blinking lines, or colored shapes. 
     
     
         11 . The computer program product of  claim 10 , wherein an importance is assignable to an individual layer within the different layers. 
     
     
         12 . The computer program product of  claim 8 , wherein the real-time feedback highlights existing inputs as problematic. 
     
     
         13 . The computer program product of  claim 8 , wherein the real-time feedback displays suggested inputs. 
     
     
         14 . The computer program product of  claim 8 , further comprising program code for collecting and analyzing user interaction history to enhance matching and ranking. 
     
     
         15 . A computerized method of providing sketch-based retrieval, comprising:
 rendering a canvas for receiving inputs including user generated strokes, and for extracting features associated with the inputs, wherein the inputs include different layers on the canvas;   assigning an importance to the inputs;   retrieving matches from a database based on extracted features and importance associated with the extracted features;   displaying a set of ranked images on the canvas based upon the retrieved matches:   providing real-time feedback on the canvas to suggest improved inputs for the user, wherein the real-time feedback is a display on the canvas proximate the entered inputs; and   ranking retrieved matches for output in the interface.   
     
     
         16 . The computerized method of  claim 15 , wherein the strokes are submitted via a touch screen. 
     
     
         17 . The computerized method of  claim 15 , wherein the inputs include different layers on the canvas. 
     
     
         18 . The computerized method of  claim 17 , wherein the display on the canvas proximate the entered inputs includes at least one of: dashed lines, blinking lines, or colored shapes. 
     
     
         19 . The computerized method of  claim 15 , wherein the real-time feedback highlights existing inputs as problematic. 
     
     
         20 . The computerized method of  claim 15 , wherein the real-time feedback displays suggested inputs.