Touch sensitive computing surface for interacting with physical surface devices

Abstract

A method including detecting a plurality of user inputs via a touch sensitive integrated display system; detecting a tangible device input from a tangible device separate from the touch sensitive integrated display system and configured to interact with the touch sensitive integrated display system; interpreting each user input in accordance with a location of the associated input on the touch sensitive integrated display system and a gesture of the associated input; interpreting each tangible device interaction; generating a display of a graphical image based on the interpretations; updating graphical objects of the graphical image displayed on the touch sensitive display based on the interpretation; determining whether to generate an output via the tangible device interacting with the touch sensitive display based on the interpretations; sending the output to the tangible device based on the determination; generating the received output; and displaying the updated graphical objects of the graphical image.

1 . A method for operating a touch sensitive integrated display system comprising:
 detecting a plurality of user inputs from at least one user via the touch sensitive integrated display system;   detecting a tangible device input from a tangible device via the touch sensitive integrated display system, wherein the tangible device is separate from the touch sensitive integrated display system and configured to interact with the touch sensitive integrated display system;   interpreting, using at least one processor, each user input in accordance with a location of the associated input on the touch sensitive integrated display system and a gesture of the associated input;   interpreting each tangible device interaction in accordance with a location of the associated interaction, a position of the tangible device, and a state of the tangible device;   generating a display of a graphical image based on the interpretations;   updating graphical objects of the graphical image displayed on the touch sensitive display based on the interpretation;   determining whether to generate an output via the tangible device interacting with the touch sensitive display based on the interpretations;   sending the output to the tangible device based on the determination;   generating the received output via the tangible device; and   displaying the updated graphical objects of the graphical image on the touch sensitive integrated display system.   
     
     
         2 . The method of  claim 1 , wherein the plurality of user inputs are associated with the at least one user contacting the touch sensitive integrated display system 
     
     
         3 . The method of  claim 2 , wherein the plurality of user inputs are detected via the touch sensitive integrated display system based on a capacitance of each user input. 
     
     
         4 . The method of  claim 1 , further comprising:
 displaying a menu of options on the touch sensitive integrated display system;   determining whether one of the user inputs corresponds to the menu of options; and   generating an output via the tangible device interacting with the touch sensitive display system based on the determination of whether the one of the user inputs corresponds to the menu of options.   
     
     
         5 . The method of  claim 1 , further comprising:
 manipulating the state of the tangible device; and   updating the graphical objects of the graphical image displayed on the touch sensitive display based on the manipulation of the portion of the tangible device.   
     
     
         6 . The method of  claim 1 , wherein the touch sensitive integrated display system comprises at least one additional input device that is different from the tangible device. 
     
     
         7 . The method of  claim 6 , wherein the additional input device is one of a knob, a game controller, a button, a track ball, a mouse, and a position sensor. 
     
     
         8 . The method of  claim 1 , wherein the touch sensitive integrated display system comprises at least one output device different from the tangible device. 
     
     
         9 . The method of  claim 8 , wherein the output device is one of a speaker, a light, a projector, an actuator, and a tactile feedback device. 
     
     
         10 . The method of  claim 1 , wherein the tangible device comprises at least one output component to produce the output from the touch sensitive integrated display system by changing the state of the tangible device. 
     
     
         11 . The method of  claim 10 , wherein the output component is one of a speaker, a display, an actuator, and a light. 
     
     
         12 . The method of  claim 1 , wherein the tangible device comprises at least one input device, and wherein the graphical objects of the graphical image are updated based on an input via the at least one input device of the tangible device. 
     
     
         13 . The method of  claim 12 , wherein the input device is one of a touch screen, a light sensor, a position sensor, a camera, a microphone, a movable joint, a knob, and a button. 
     
     
         14 . The method of  claim 13 , wherein the graphical objects of the graphical image displayed on the touch sensitive display are updated in response to an input received via the input device of the tangible device. 
     
     
         15 . The method of  claim 1 , further comprising:
 tracking the location, position, and state of the tangible device relative to the touch sensitive integrated display system;   updating the graphical objects displayed on the touch sensitive integrated display system based on the tracked location, position, and state; and   updating the state of the tangible device based on the tracked location and position.   
     
     
         16 . The method of  claim 15 , wherein the location and position of the tangible device are determined based on contact between the tangible device and the touch sensitive integrated display system. 
     
     
         17 . The method of  claim 15 , wherein the location and position, of the tangible device are determined based on wireless communication between the tangible device and the touch sensitive integrated display system. 
     
     
         18 . The method of  claim 1 , further comprising:
 detecting another tangible device input from another tangible device via the touch sensitive integrated display system, wherein the other tangible device is separate from the touch sensitive integrated display system.   
     
     
         19 . The method of  claim 18 , wherein the graphical objects displayed on the touch sensitive integrated display system is updated based on detecting the other tangible device input. 
     
     
         20 . A display system comprising:
 a tangible device; and   a touch sensitive integrated display system separate from the tangible device and configured to interact with the tangible device, wherein the touch sensitive integrated display system comprises at least one processor configured to:   detect a plurality of user inputs from at least one user via the touch sensitive integrated display system;   detect an interaction between the tangible device and the touch sensitive integrated display system;   interpret each user input in accordance with a location of the associated input on the touch sensitive integrated display system and a gesture of the associated input;   interpret each tangible device interaction in accordance with a location of the associated interaction, a position of the tangible device, and a state of the tangible device;   generate a display of a graphical image based on the interpretations;   update graphical objects of the graphical image displayed on the touch sensitive display based on the interpretations;   determine whether to generate an output via the tangible device interacting with the touch sensitive display based on the interpretations;   send the output to the tangible device based on the determination; and   display the updated graphical objects of the graphical image on the touch sensitive integrated display system,   display the generated display on the touch sensitive integrated display system, wherein the tangible device.   
     
     
         21 . The display system of  claim 20 , wherein the touch sensitive integrated display system comprises a two-dimensional surface configured to sense to touch and display two-dimensional graphics. 
     
     
         22 . The display system of  claim 20 , wherein the touch sensitive integrated display system comprises a two-dimensional surface configured to sense to touch and display three-dimensional graphics.