Freehand system and method for creating, editing, and manipulating block diagrams

Abstract

A method of integrating freehand user input into a block diagram environment is disclosed. The freehand user input is a user's approximation of a diagram component or feature of a component which is received by the block diagram environment and compared to multiple patterns stored in a storage location. The storage location holds patterns of block diagram components and block diagram component features. The freehand user input may be displayed, superimposed on a block diagram being shown to the user. Upon the freehand user input being matched to one of the patterns representing a block diagram component or feature of a component, the freehand user input is replaced on the displayed block diagram with an electronic device drawn rendering of the matched diagram feature component or feature of a component. Partial matches of the user drawn input may result in a menu of choices being presented to the user for selection.

1 . In an electronic device holding a block diagram environment with at least one block diagram, said block diagram being a model of a system, said electronic device interfaced with a display surface displaying said block diagram, a method, comprising: 
 receiving freehand user input entered with a pointing device into said block diagram;    comparing said user input to at least one of a stored plurality of patterns of at least one of block diagram components and block diagram component features;    adding said at least one of said block diagram components and block diagram component features to said model, said adding occurring as a result of a matching comparison between said freehand user input and at least one of said plurality of patterns; and    rendering programmatically with the block diagram environment at least one of said block diagram components and said block diagram component features in said diagram.    
   
   
       2 . The method of  claim 1 , further comprising: 
 displaying said freehand user input in said diagram prior to said rendering.    
   
   
       3 . The method of  claim 2 , further comprising: 
 replacing said displayed freehand user input with a programmatic rendering of said matching pattern.    
   
   
       4 . The method of  claim 1  wherein said freehand user input is added to an existing programmatically rendered diagram component in said diagram.  
   
   
       5 . The method of  claim 4 , further comprising: 
 modifying said existing programmatically rendered pattern by programmatically rendering at least one additional feature based on matching said user input to a stored pattern.    
   
   
       6 . The method of  claim 1  wherein said freehand user input is one of a complete diagram component and complete diagram component feature.  
   
   
       7 . The method of  claim 1  wherein said freehand user input is one of a partial diagram component and partial diagram component feature.  
   
   
       8 . The method of  claim 1 , further comprising: 
 providing a choice of at least two of diagram components and diagram component features to said user prior to said rendering, said choice prompting a user for a selection of one of said at least two diagram components and diagram component features.    
   
   
       9 . The method of  claim 8 , further comprising: 
 rendering programmatically at least one of said diagram components and diagram component features based on said selection.    
   
   
       10 . The method of  claim 8  wherein the identity of the at least two diagram components and diagram component features appearing in said choice is based on the context of said diagram.  
   
   
       11 . The method of  claim 1  wherein said freehand user input is a symbol.  
   
   
       12 . The method of  claim 1  wherein said diagram is a block diagram.  
   
   
       13 . The method of  claim 1 , further comprising: 
 manipulating at least one of a diagram component and diagram component feature displayed in said diagram as a result of said user input.    
   
   
       14 . The method of  claim 1 , further comprising: 
 deleting at least one of a diagram component and diagram component feature displayed in said diagram as a result of said user input.    
   
   
       15 . The method of  claim 1 , further comprising: 
 providing diagramming tools to a user in said block diagram environment in response to said freehand user input.    
   
   
       16 . The method of  claim 15  wherein said tool is a zoomed-out viewer superimposed over at least a portion of said displayed diagram.  
   
   
       17 . The method of  claim 1 , further comprising: 
 providing at least one visually displayed symbol associated with an attribute type;    selecting said symbol associated with an attribute type with said pointing device;    subsequently selecting with said pointing device at least one of a diagram component and diagram component feature displayed in said diagram; and    assigning programmatically said attribute type associated with said symbol to the selected at least one of a diagram component and diagram component feature.    
   
   
       18 . The method of  claim 1 , further comprising: 
 displaying said freehand user input on said display surface;    superimposing the programmatically rendered pattern over the displayed freehand user input; and    removing said freehand user input from said display surface following a user indication of the correctness of the programmatically rendered at least one of said diagram component and diagram component feature.    
   
   
       19 . The method of  claim 1  wherein said comparing occurs only after said user selects an interpreted mode option for said block diagram environment.  
   
   
       20 . In an electronic device interfaced with a display surface, a system comprising: 
 a block diagram environment with at least one block diagram displayed on said display surface;    a pointing device interfaced with said electronic device, said pointing device used to transmit freehand input from a user to said block diagram displayed on said display surface; and    a storage location holding a plurality of block diagram component patterns and diagram component feature patterns, said user input being compared to said plurality of block diagram component patterns and diagram component feature patterns, said block diagram environment programmatically rendering one of a diagram component and diagram component feature in said block diagram based on a matching comparison between said freehand user input and at least one of said plurality of patterns.    
   
   
       21 . The system of  claim 20  wherein said rendering superimposes said programmatically rendered one of a diagram component and diagram component feature over said user input in said diagram displayed on said display surface.  
   
   
       22 . The system of  claim 20  wherein said user input represents one of a partial diagram component and partial diagram component feature.  
   
   
       23 . The system of  claim 20  wherein said user input represents one of a complete diagram component and complete diagram component feature.  
   
   
       24 . The system of  claim 20 , further comprising: 
 a menu of choices displayed to said user prior to the rendering of said one of a diagram component and diagram component feature, said choices representing possible diagram components and diagram component features matching said user input.    
   
   
       25 . The system of  claim 24  wherein a user selects one of the choices, the selected choice then being rendered as a matching comparison.  
   
   
       26 . The system of  claim 20  wherein said user input is of an additional feature to a previously rendered diagram component.  
   
   
       27 . The system of  claim 20  wherein said pointing device is one of a mouse, trackball, joystick, mousepad, and light pen.  
   
   
       28 . The system of  claim 20  wherein said block diagram environment is a block diagram environment and said block diagram is a block diagram.  
   
   
       29 . In an electronic device holding a block diagram environment with at least one block diagram, said electronic device interfaced with a display surface displaying said diagram, a medium holding executable steps for a method, said method comprising: 
 comparing freehand user input entered with a pointing device into said diagram to at least one of a stored plurality of patterns; and    rendering programmatically with the block diagram environment one of a diagram component and diagram component feature in said diagram, said diagram component based on a matching comparison between said freehand user input and at least one of said plurality of patterns.    
   
   
       30 . The medium of  claim 29  wherein said method further comprises: 
 displaying said freehand user input in said diagram prior to said rendering.    
   
   
       31 . The medium of  claim 30  wherein said method further comprises: 
 replacing said displayed freehand user input with said programmatically rendered one of a diagram component and diagram component feature.    
   
   
       32 . The medium of  claim 29  wherein said freehand user input is added to an existing electronic device drawn diagram component in said diagram.  
   
   
       33 . The medium of  claim 32  wherein said method further comprises: 
 modifying said existing programmatically rendered component by programmatically rendering at least one additional feature based on matching said user input to a stored pattern.    
   
   
       34 . The medium of  claim 29  wherein said freehand user input is one of a complete diagram component and complete diagram component feature.  
   
   
       35 . The medium of  claim 29  wherein said freehand user input is one of a partial diagram component and partial diagram component feature.  
   
   
       36 . The medium of  claim 29  wherein said method further comprises: 
 providing a choice of at least two diagram components and diagram component features to said user prior to said rendering, said choice prompting a user for a selection of one of said at least two diagram components and diagram component features.    
   
   
       37 . The medium of  claim 36 , wherein said method further comprises: 
 rendering one of said diagram components and diagram component features based on said selection.    
   
   
       38 . The medium of  claim 36  wherein the identity of the at least two of said diagram components and diagram component features appearing in said choice is based on the context of said diagram.  
   
   
       39 . The medium of  claim 29  wherein said freehand user input is a symbol.  
   
   
       40 . The medium of  claim 29  wherein said diagram is a block diagram.  
   
   
       41 . The medium of  claim 29  wherein said method further comprises: 
 manipulating at least one of a diagram component and diagram component feature displayed in said diagram as a result of said user input.    
   
   
       42 . The medium of  claim 29  wherein said method further comprises: 
 deleting at least one of a diagram component and diagram component feature displayed in said diagram as a result of said user input.    
   
   
       43 . The medium of  claim 29  wherein said method further comprises: 
 providing diagramming tools to a user in said block diagram environment in response to said freehand user input.    
   
   
       44 . The medium of  claim 43  wherein said tool is a zoomed-out viewer superimposed over at least a portion of said displayed diagram.  
   
   
       45 . The medium of  claim 29  wherein said method further comprises: 
 providing at least one visually displayed symbol associated with an attribute type;    selecting said symbol associated with an attribute type with said pointing device;    subsequently selecting with said pointing device at least one of a diagram component and diagram component feature displayed in said diagram; and    assigning programmatically said attribute type associated with said symbol to the selected at least one of a diagram component and diagram component feature.    
   
   
       46 . In an electronic device holding a block diagram environment with at least one block diagram, said block diagram being a model of a system, said electronic device interfaced with a display surface displaying said block diagram, a method, comprising: 
 receiving freehand user input entered with a pointing device into said block diagram;    analyzing said user input to identify said user input as at least one of a block diagram component and block diagram component feature;    adding said at least one of said block diagram components and block diagram component features to said model, said adding occurring as a result of said analysis; and    rendering programmatically at least one of said block diagram components and said block diagram component features in said diagram.    
   
   
       47 . The method of  claim 46  wherein said block diagram component is a block diagram subsystem.  
   
   
       48 . The method of  claim 46  wherein said analysis is performed using a neural net.  
   
   
       49 . In an electronic device holding a block diagram environment with at least one block diagram, said block diagram being a model of a system, said electronic device interfaced with a display surface displaying said block diagram, a method, comprising: 
 receiving freehand user input entered with a pointing device into said block diagram environment executing in a learning mode;    receiving a user indication associating said freehand user input with an action;    storing said user input and its associated action as one of a plurality of patterns;    receiving freehand user input entered with a pointing device into said block diagram environment executing in an interpretive mode;    comparing said user input to at least one of a stored plurality of patterns of at least one of block diagram components and block diagram component features; and    performing the associated action, said performing occurring as a result of a matching comparison between said freehand user input and at least one of said plurality of patterns.    
   
   
       50 . In an electronic device holding a block diagram environment with at least one block diagram, said block diagram being a model of a system, said electronic device interfaced with a display surface displaying said block diagram, a medium holding executable steps for a method, said method comprising: 
 receiving freehand user input entered with a pointing device into said block diagram environment executing in a learning mode;    receiving a user indication associating said freehand user input with an action;    storing said user input and its associated action as one of a plurality of patterns;    receiving freehand user input entered with a pointing device into said block diagram environment executing in an interpretive mode;    comparing said user input to at least one of a stored plurality of patterns of at least one of block diagram components and block diagram component features; and    performing the associated action, said performing occurring as a result of a matching comparison between said freehand user input and at least one of said plurality of patterns.    
   
   
       51 . In an electronic device interfaced with a display surface, a system comprising: 
 a software diagram environment with at least one software diagram displayed on said display surface;    a pointing device interfaced with said electronic device, said pointing device used to transmit freehand input from a user to said software diagram displayed on said display surface; and    a storage location holding a plurality of software diagram component patterns and diagram component feature patterns, said user input being compared to said plurality of software diagram component patterns and diagram component feature patterns, said software diagram environment programmatically rendering one of a diagram component and diagram component feature in said software diagram based on a matching comparison between said freehand user input and at least one of said plurality of patterns.    
   
   
       52 . The system of  claim 51  wherein said software diagram is one of a Stateflow® diagram and Unified Modeling Language (UML) diagrams.