Devices and methods for presenting and regulating auxiliary information on an image display of a telesurgical system to assist an operator in performing a surgical procedure

Abstract

A method of performing a surgical procedure on a patient is provided. The method typically comprises positioning a surgical work station of a robotically controlled surgical system and a patient on which a surgical procedure is to be performed in close proximity relative to each other, directing a field of view of an image capture device of the surgical work station at a surgical site on the patient, at which site the surgical procedure is to be performed, and introducing at least one robotically controlled surgical instrument on the surgical work station to the surgical site so that an end effector of the surgical instrument is within the field of view of the image capture device. The method typically further comprises displaying an image of the surgical site and the end effector on a display area of an image display at an operator control station of the surgical system. The image display is typically operatively connected to the image capture device so as to display, on the display area, the image of the surgical site and the end effector captured by the image capture device. The method can further include permitting an operator of the surgical system to manipulate a master control at the operator control station whilst viewing the image of the surgical site on the image display, the master control being operatively associated with the surgical instrument thereby to cause the end effector to move in response to manipulation of the master control so as to perform at least part of the surgical procedure on the patient at the surgical site. The method yet further typically comprises operatively linking the image display to a source of selectively accessible auxiliary information related to the surgical procedure to be performed, enabling the operator selectively to access the source of auxiliary information from the operator control station so as to forward the auxiliary information to the image display, causing the auxiliary information to be displayed across the display area of the image display in response to the operator selectively accessing the source of auxiliary information at the operator control station and enabling the operator selectively to move the auxiliary information when displayed on the image display relative to the image of the surgical site displayed on the image display. The master control is typically operatively linked with the source of auxiliary information, enabling the operator selectively to access the source of auxiliary information then including permitting the operator selectively to disassociate the master control from the surgical instrument and to use the master control to access the source of auxiliary information so as to enable the auxiliary information to be displayed on the display area of the image display.

1 - 55 . (canceled)  
     
     
         56 . A medical robotic system comprising: 
 an ultrasound transducer;    a robotic arm mechanically coupled to the ultrasound transducer; and    a processor configured to controllably cause the robotic arm to move the ultrasound transducer according to a control input.    
     
     
         57 . The medical robotic system according to  claim 56 , wherein the ultrasound transducer is disposed at a site within a patient.  
     
     
         58 . The medical robotic system according to  claim 57 , further comprising a master control device, wherein the control input is received from the master control device and is indicative of manual movement of the master control device.  
     
     
         59 . The medical robotic system according to  claim 57 , wherein the processor is configured to controllably cause the robotic arm to move the ultrasound transducer according to the control input during a minimally invasive surgical procedure.  
     
     
         60 . The medical robotic system according to  claim 56 , wherein the control input includes a pre-programmed sequence of stored instructions.  
     
     
         61 . The medical robotic system according to  claim 60 , wherein the stored instructions instruct the processor to cause the robotic arm to move the ultrasound transducer so that 2D ultrasound image slices captured by the ultrasound transducer provide information for the processor to generate a 3D computer model of at least part of a structure.  
     
     
         62 . The medical robotic system according to  claim 61 , wherein the stored instructions instruct the processor to cause the robotic arm to move the ultrasound transducer so as to sweep across a surgical site including the structure.  
     
     
         63 . The medical robotic system according to  claim 61 , wherein the stored instructions instruct the processor to cause the robotic arm to move the ultrasound transducer across a surface of the structure.  
     
     
         64 . The medical robotic system according to  claim 56 , further comprising: 
 an endoscope camera for capturing images of a surgical site;    wherein the processor is configured to display the images captured by the endoscope camera in a picture section of a viewer and an ultrasound image captured by the ultrasound transducer within a picture-in-picture section of the viewer.    
     
     
         65 . The medical robotic system according to  claim 56 , further comprising; 
 an elongated shaft; and    a wrist mechanism mechanically coupling the ultrasound transducer to the elongated shaft so as to allow angular movement of the ultrasound transducer relative to an axis running along a length of the elongated shaft.    
     
     
         66 . A method for generating a 3D ultrasound image of an anatomic structure during a minimally invasive surgical procedure, comprising: 
 causing a robotic arm to move an ultrasound transducer to capture a sequence of 2D ultrasound image slices of an anatomic structure; and    processing the captured 2D ultrasound image slices to generate a 3D ultrasound image of the anatomic structure.    
     
     
         67 . The method according to  claim 66 , wherein the causing of the robotic arm to move the ultrasound transducer comprises: 
 receiving a control input from a master control device; and    causing the robotic arm to robotically move the ultrasound transducer to capture the sequence of 2D ultrasound image slices of the anatomic structure in response to the control input.    
     
     
         68 . The method according to  claim 66 , wherein the causing of the robotic arm to move the ultrasound transducer comprises: causing the robotic arm to robotically move the ultrasound transducer to capture the sequence of 2D ultrasound image slices of the anatomic structure in response to a pre-programmed sequence of stored instructions.  
     
     
         69 . The method according to  claim 66 , wherein the causing of the robotic arm to move the ultrasound transducer comprises: causing the robotic arm to robotically move the ultrasound transducer to capture the sequence of 2D ultrasound image slices of the anatomic structure by sweeping across a surgical site including the anatomic structure.  
     
     
         70 . The method according to  claim 66 , wherein the causing of the robotic arm to move the ultrasound transducer comprises: causing the robotic arm to robotically move the ultrasound transducer to capture the sequence of 2D ultrasound image slices of the anatomic structure by moving the ultrasound transducer across a surface of the anatomic structure.  
     
     
         71 . The method according to  claim 66 , further comprising: displaying a camera image captured by an endoscope camera in a picture section of a viewer and a corresponding view of the 3D ultrasound image of the anatomic structure within a picture-in-picture section of the viewer.