Handwriting-based predictive population of partial virtual keyboards

Abstract

A “Stroke Untangler” composes handwritten messages from handwritten strokes representing overlapping letters or partial letter segments are drawn on a touchscreen device or touch-sensitive surface. These overlapping strokes are automatically untangled and then segmented and combined into one or more letters, words, or phrases. Advantageously, segmentation and composition is performed without requiring user gestures, timeouts, or other inputs to delimit characters within words, and without using handwriting recognition-based techniques to guide untangling and composing of the overlapping strokes to form characters. In other words, the user draws multiple overlapping strokes. Those strokes are then automatically segmented and combined into one or more corresponding characters. Text recognition of the resulting characters is then performed. Further, the segmentation and combination is performed in real-time, thereby enabling real-time rendering of the resulting characters in a user interface window. A related drawing mode enables entry of drawings in combination with the handwritten characters.

What is claimed is: 
     
         1 . A system, comprising:
 a writing panel; and   a computing device configured to:
 receive an input of one or more user-entered strokes that are entered within extents of the writing panel; 
 determine one or more most likely character matches to the user-entered strokes; 
 populate and display a dynamic virtual keyboard with a subset of user-selectable keys, the subset comprising the one or more most likely character matches to the user-entered strokes. 
   
     
     
         2 . The system of  claim 1  wherein the one or more most likely matches are determined by automated handwriting-based recognition of the user-entered strokes. 
     
     
         3 . The system of  claim 1  wherein a numerical size of the subset of user-selectable keys varies based on the determination of most likely character matches to the user-entered strokes. 
     
     
         4 . The system of  claim 1  wherein one or more of the user-entered strokes are at least partially overlapping. 
     
     
         5 . The system of  claim 1  wherein the computing device is further configured to automatically update the display of the dynamic virtual keyboard with a new subset of user-selectable keys in response to receipt of one or more additional user-entered strokes. 
     
     
         6 . The system of  claim 1  wherein one or more of the user-selectable keys of the dynamic virtual keyboard are sized relative to their likelihood of match to the user-entered strokes, with keys corresponding to more likely matches being displayed as larger than keys corresponding to less likely matches. 
     
     
         7 . The system of  claim 1  wherein the computing device is further configured to automatically generate an input of a single character corresponding to one or more of the user-entered strokes when those strokes are determined to represent that single character beyond a threshold likelihood of match prior to user selection of one of the subset of user-selectable keys of the dynamic virtual keyboard. 
     
     
         8 . The system of  claim 1  wherein the writing panel further comprises a touch-sensitive surface for tracking and receiving the user-entered strokes. 
     
     
         9 . The system of  claim 1  wherein the writing panel further comprises a touch-sensitive display device for concurrently tracking and receiving the user-entered strokes and displaying the dynamic virtual keyboard. 
     
     
         10 . The system of  claim 9  wherein the display device is a wrist-worn display device. 
     
     
         11 . The system of  claim 1  wherein the writing panel comprises a virtual surface; and
 further comprising a tracking device to configured to track user motions relative to the virtual surface and to provide the tracked user motions as the input of one or more user-entered strokes. 
 
     
     
         12 . The system of  claim 1  wherein the computing device is further configured to render the user-selectable keys adjacent to a location of the writing panel where the user-entered strokes are entered. 
     
     
         13 . A computer-implemented process, comprising:
 applying a computer to perform automated process actions for:
 receiving one or more strokes from a writing panel; 
 performing handwriting-based recognition of the strokes to determine one or more most likely character matches to the strokes; 
 populate a dynamic virtual keyboard with a subset of user-selectable keys, the subset comprising the one or more most likely character matches to the strokes; and 
 render and display the dynamic virtual keyboard on a display device. 
   
     
     
         14 . The computer-implemented process of  claim 13  further comprising populating and displaying the dynamic virtual keyboard in real-time as the strokes are being entered in response to ongoing determination of most likely character matches to the strokes. 
     
     
         15 . The computer-implemented process of  claim 13  further comprising displaying a fading trail of each stroke on the writing panel in real-time as each stroke is received from the writing panel. 
     
     
         16 . The computer-implemented process of  claim 13  further comprising varying a numerical size of the subset of user-selectable keys based on the determination of most likely character matches to the strokes. 
     
     
         17 . The computer-implemented process of  claim 13  further comprising automatically updating the display of the dynamic virtual keyboard with a new subset of user-selectable keys in response to receipt of one or more additional strokes. 
     
     
         18 . The computer-implemented process of  claim 13  further comprising automatically sizing one or more of the user-selectable keys of the dynamic virtual keyboard relative to their likelihood of match to the strokes, with keys corresponding to more likely matches being displayed as larger than keys corresponding to less likely matches. 
     
     
         19 . The computer-implemented process of  claim 13  wherein:
 the writing panel further comprises a touch-sensitive display device for concurrently tracking and receiving the user-entered strokes and displaying the dynamic virtual keyboard; and 
 wherein the display device is a wrist-worn display device. 
 
     
     
         20 . A computer-readable storage device having computer executable instructions stored therein, said instructions causing a computing device to execute a method comprising:
 receiving a plurality of overlapping strokes on a touchscreen;   segmenting and compositing the overlapping strokes to form one or more human-readable characters prior to performing any handwriting-based recognition of any of the strokes or human-readable characters; and   performing handwriting-based recognition of the human-readable characters to generate corresponding machine-readable text.