Stereo telestration for robotic surgery

Abstract

In one embodiment of the invention, a robotic surgical system includes a master control console having a stereo viewer to view stereo images; a surgical manipulator having a stereo endoscopic camera coupled to a robotic arm to generate the stereo images of a surgical site; a stereo telestration device coupled between the stereo endoscopic camera and the stereo viewer to mix telestration graphics and the stereo images of the surgical site together for viewing by the stereo viewer; and a telestration generator coupled to the stereo telestration device to generate the telestration graphics for overlay on the stereo images of the surgical site.

1 - 16 . (canceled) 
     
     
         17 . A method for a robotic surgical system comprising:
 receiving a mono-view of a telestration graphic;   generating a stereo-view of the telestration graphic from the mono-view of the telestration graphic; and   overlaying the stereo-view of the telestration graphic onto a stereo-view of a surgical site.   
     
     
         18 . (canceled) 
     
     
         19 . The method of  claim 17 , wherein the generating of the stereo-view of the telestration graphic includes determining a disparity map between pixels in left images and right images of the stereo view of the surgical site to position the stereo view of the telestration graphic at one or more depths in the stereo-view of the surgical site. 
     
     
         20 . The method of  claim 17 , wherein the generating of the stereo-view of the telestration graphic includes determining a depth map between left images and right images of the stereo view of the surgical site to position the stereo view of the telestration graphic over a continuum of depths in the stereo-view of the surgical site. 
     
     
         21 . The method of  claim 17 , wherein the overlaying of the stereo-view of the telestration graphic onto the stereo-view of the surgical site includes combining a left image of the telestration graphic with a left image of the surgical site, and combining a right image of the telestration graphic with a right image of the surgical site. 
     
     
         22 . The method of  claim 21 , wherein the combining of the left and right images of the telestration graphic with the left and right images of the surgical site are by mixing in response to a signal. 
     
     
         23 - 32 . (canceled) 
     
     
         33 . A method for telestrating on a three-dimensional image of a surgical site, comprising:
 receiving a telestration graphic input associated with one of a pair of stereoscopic images of an surgical site;   determining a corresponding telestration graphic input in the other of the pair of stereoscopic images using at least one of a disparity map and a depth map corresponding to the pair of stereoscopic images so that a three-dimensional view of the telestration graphic input is generated which is displayable as a contoured overlay to a three-dimensional view of the surgical site; and   displaying the three-dimensional view of the telestration graphic input as the contoured overlay to the three-dimensional view of the surgical site on a three-dimensional display.   
     
     
         34 . The method according to  claim 33 , further comprising:
 transmitting information for the one of the pair of stereoscopic images to a location prior to receiving the telestration graphic input from the location.   
     
     
         35 . The method according to  claim 34 , wherein the location is a computer operated by a mentor. 
     
     
         36 . The method according to  claim 34 , further comprising:
 receiving information for the pair of stereoscopic images prior to transmitting the information for the one of the pair of stereoscopic images to the location.   
     
     
         37 . The method according to  claim 36 , wherein the information for the pair of stereoscopic images is received from a stereoscopic endoscope. 
     
     
         38 . The method according to  claim 37 , wherein the pair of stereoscopic images comprises corresponding right and left camera views. 
     
     
         39 . The method according to  claim 33 , further comprising:
 displaying the three-dimensional view of the telestration graphic input as a non-destructive graphics overlay to the three-dimensional view of the surgical site.   
     
     
         40 . A medical robotic system providing three-dimensional telestration comprising:
 a stereo endoscopic camera;   a console having a three-dimensional display for displaying stereo images captured by the stereo endoscopic camera;   a telestration generator receiving at least one of a pair of stereo images captured by the stereo endoscopic camera to generate a two-dimensional telestration graphic input relative to the received stereo image; and   a stereo telestration system configured to receive the two-dimensional telestration graphic input from the telestration generator, determine a corresponding two-dimensional telestration graphic input in the other of the pair of stereoscopic images using at least one of a disparity map and a depth map corresponding to the pair of stereoscopic images so that a three-dimensional view of the telestration graphic input is generated which is displayable as a contoured overlay to a three-dimensional view of the surgical site on the three-dimensional display.   
     
     
         41 . The medical robotic system according to  claim 40 , wherein the telestration generator includes one or more of a digitizing tablet coupled to the stereo telestration system, a digitizing pen coupled to the digitizing tablet, a first keyboard coupled to the digitizing tablet, a computer coupled to the stereo telestration system, a three-dimensional input device coupled to the computer, a mouse coupled to the computer, and a second keyboard coupled to the computer. 
     
     
         42 . The medical robotic system according to  claim 41 , wherein the two-dimensional telestration graphic input is generated from input provided by a mentor operating the telestration generator.