Synthetic representation of a surgical instrument

Abstract

A synthetic representation of a tool for display on a user interface of a robotic system. The synthetic representation may be used to show force on the tool, an actual position of the tool, or to show the location of the tool when out of a field of view. A three-dimensional pointer is also provided for a viewer in the surgeon console of a telesurgical system.

1 . A robotic surgical system, comprising:
 a tool for performing surgery;   data for providing a synthetic representation of the tool;   an image capturing device that captures an image of the tool;   a display;   a first component coupling the image capture device to the display so as to display an image of the tool in the display; and   a second component coupling the data to the display so as to display the synthetic representation of the tool simultaneously with the displayed image of the tool.   
     
     
         2 . The system of  claim 1 , wherein the display comprises a viewer of a surgeon console of the robotic surgical system. 
     
     
         3 . The system of  claim 1 , wherein the synthetic representation comprises a model of the tool. 
     
     
         4 . The system of  claim 1 , wherein the synthetic representation is displayed to appear overlaid at least in part on the image of the tool. 
     
     
         5 . The system of  claim 1 , wherein the synthetic representation represents a position of the tool, an orientation of the tool, or both. 
     
     
         6 . The system of  claim 1 : wherein the synthetic representation is displayed to appear outside a displayed image from a field of view of the image capture device. 
     
     
         7 . In a robotic surgical system comprising a surgical tool and an image capture device, a method comprising:
 capturing an image of the surgical tool within a field of view of the image capture device;   displaying the captured image in a surgical display; and   displaying a synthetic representation of the surgical tool in the surgical display.   
     
     
         8 . The method of  claim 7 , wherein the display comprises a viewer of a surgeon console of the robotic surgical system. 
     
     
         9 . The method of  claim 7 , wherein the synthetic representation comprises a model of the surgical tool. 
     
     
         10 . The method of  claim 9 , wherein displaying the synthetic representation of the surgical tool comprises displaying the synthetic representation of the surgical tool to appear at least in part overlaid on the displayed image of the surgical tool. 
     
     
         11 . The method of  claim 7 , wherein the synthetic representation represents a position of the surgical tool, an orientation of the surgical tool, or both. 
     
     
         12 . The method of  claim 7 , wherein the synthetic representation of the surgical tool is displayed to appear outside the displayed image of the surgical tool. 
     
     
         13 . A method of providing a visual representation of a position of a tool in a robotic system, the method comprising:
 displaying a first image comprising video feed of a field of view showing a region, with the region containing a tool; and   displaying a second image representing a position of the tool, an orientation of the tool, or both, superimposed on the first image.   
     
     
         14 . The method of  claim 13 , wherein the second image comprises a synthetic representation of the tool. 
     
     
         15 . The method of  claim 14 , wherein the synthetic representation of the tool comprises a three-dimensional model of the tool. 
     
     
         16 . The method of  claim 13 , further comprising, tracking movement of the tool, and responsive to a movement of the tool to a new location, changing the second image to match the new position of the tool, the orientation of the tool, or both. 
     
     
         17 . The method of  claim 14 , further comprising:
 displaying the second image as a non-three-dimensional model of the tool if at least a portion of the tool appears in the first image; and   displaying the second image as a three-dimensional model of the tool if the tool does not appear in the first image.   
     
     
         18 . The method of  claim 13 :
 displaying the second image only if the tool does not appear in the first image.   
     
     
         19 . The method of  claim 13 , further comprising altering the second image based upon the availability of the first image. 
     
     
         20 . A robotic system, comprising:
 a surgical tool;   data for providing a synthetic representation of the tool;   an image capturing device that captures an image within a field of view;   a display comprising a first portion in which the captured image is displayed and a second portion in which a window that represents space adjacent the captured image and outside the field of view is displayed; and   a first component coupling the data to the display so as to superimpose in the window at least part of the synthetic representation of the tool, wherein the synthetic representation comprises a graphical representation of a position of the tool, an orientation of the tool, or both.   
     
     
         21 . The system of  claim 20 , wherein the synthetic representation of the tool comprises a three-dimensional model of the tool. 
     
     
         22 . The system of  claim 21 , further comprising:
 a second component coupled to the tool;   wherein the second component generates information about movement of the tool to a new position, a new orientation, or both; and   wherein the second component is configured to change the position, the orientation, or both, of the displayed synthetic representation based upon the information.   
     
     
         23 . The system of  claim 22 , wherein the new position is in the field of view. 
     
     
         24 . A robotic surgical system, comprising:
 a surgical tool;   a display;   a synthetic rendering of the tool, wherein the synthetic rendering appears on the display;   a synthetic rendering of a pointing device that appears in three dimensions on the display; and   a master input device;   wherein in a first mode the master input device controls movement of the surgical tool, in a second mode the master input device controls movement on the display of the synthetic rendering of the surgical tool, and in a third mode the master input device controls movement on the display of the synthetic rendering of the pointing device.   
     
     
         25 . The system of  claim 24 :
 wherein the pointing device comprises a synthetic rendering of a surgical tool that appears to be three-dimensional on the display; and   wherein in the third mode the pointing device appears to move on the display in a plurality of degrees of freedom in response to corresponding movement of the master input device.   
     
     
         26 . The system of  claim 24 , further comprising:
 a menu that appears on the display;   wherein the pointing device moves and selects an item on the menu.