Reconfigurable display in computer-assisted tele-operated surgery

Abstract

The technology described in this document can be embodied in a method that includes operating a surgical system to perform a surgical process, the surgical system including a display device, and receiving, at one or more processing devices, data from multiple data sources. The method also includes determining a current phase of the surgical process, and displaying, on the display device, visual representations corresponding to the data from a first set of the multiple data sources in a first arrangement within a display region of the display device. At least one of the first set of the multiple data sources and the first arrangement is associated with the current phase of the surgical process.

1 . A method comprising:
 operating a surgical system to perform a surgical process, the surgical system including a display device;   receiving, at one or more processing devices, data from multiple data sources;   determining a current phase of the surgical process; and   displaying, on the display device, visual representations corresponding to the data from a first set of the multiple sources in a first arrangement within a display region of the display device,   wherein at least one of the first set of the multiple data sources and the first arrangement is associated with the current phase of the surgical process.   
     
     
         2 . The method of  claim 1 , further comprising:
 determining a new phase of the surgical process; and   updating at least one of the first set of the multiple data sources and the first arrangement in response to determining the new phase of the surgical process.   
     
     
         3 . The method of  claim 2 , wherein updating the at least one of the first set of the multiple data sources and the first arrangement is based on a user preference record for a current user of the surgical system. 
     
     
         4 . The method of  claim 2 , wherein updating the at least one of the first set of the multiple data sources and the first arrangement is based on a predetermined safety profile for the surgical system. 
     
     
         5 . The method of  claim 1 , further comprising:
 receiving, via an input device, user-input indicative of adjustments to one or more of the visual representations corresponding to the data from the multiple data sources; and   updating the display device in accordance with the adjustments.   
     
     
         6 . (canceled) 
     
     
         7 . (canceled) 
     
     
         8 . The method of  claim 1 , wherein the multiple data sources comprise at least two of: an endoscope, an ultrasound imaging device, a computed tomography (CT) imaging device, a nuclear imaging device, a radiography imaging device, and a magnetic resonance imaging (MRI) device. 
     
     
         9 . (canceled) 
     
     
         10 . The method of  claim 1 , wherein determining the current phase is based on a user-input indicative of the current phase. 
     
     
         11 . (canceled) 
     
     
         12 . The method of  claim 1 , wherein data from one or more of the multiple data sources includes positional information with respect to a common reference frame, and wherein displaying the visual representations comprises overlaying a first visual representation on a second visual representation, wherein the first visual representation is registered with respect to the second visual representation based on the common reference frame. 
     
     
         13 . (canceled) 
     
     
         14 . The method of  claim 1 , wherein the first arrangement is determined based on a user profile loaded prior to commencement of the surgical process, the user profile identifying an individual performing the surgical process, and includes user-preferences of the individual regarding organization of the visual representations corresponding to the data from the multiple data sources during different phases of the surgical process. 
     
     
         15 .- 25 . (canceled) 
     
     
         26 . A surgical system comprising:
 a display device; and   one or more processing devices configured to:
 operate the surgical system to perform a surgical process; 
 receive data from multiple data sources, 
 determine a current phase of the surgical process, and 
 display, on the display device, visual representations corresponding to the data from a first set of the multiple data sources in a first arrangement within a display region of the display device, 
 wherein at least one of the first set of the multiple data sources and the first arrangement is associated with the current phase of the surgical process. 
   
     
     
         27 . The surgical system of  claim 26 , wherein the one or more processing devices are further configured to:
 determine a new phase of the surgical process; and   update at least one of the first set of the multiple data sources and the first arrangement in response to determining the new phase of the surgical process.   
     
     
         28 . The surgical system of  claim 27 , wherein updating the at least one of the first set of the multiple data sources and the first arrangement is based on a user preference record for a current user of the surgical system. 
     
     
         29 . The surgical system of  claim 27 , wherein updating the at least one of the first set of the multiple data sources and the first arrangement is based on a predetermined safety profile for the surgical system. 
     
     
         30 . The surgical system of  claim 26 , wherein the one or more processing devices are further configured to:
 receive, via an input device, user-input indicative of adjustments to one or more of the visual representations corresponding to the data from the multiple data sources; and   update the display device in accordance with the adjustments.   
     
     
         31 . (canceled) 
     
     
         32 . (canceled) 
     
     
         33 . The surgical system of  claim 26 , wherein the multiple data sources comprise at least two of: an endoscope, an ultrasound imaging device, a computed tomography (CT) imaging device, a nuclear imaging device, a radiography imaging device, and a magnetic resonance imaging (MRI) device. 
     
     
         34 . (canceled) 
     
     
         35 . The surgical system of  claim 26 , wherein the one or more processing devices are configured to determine the current phase based on a user-input indicative of the current phase or on an image analysis process executed on the data from at least one of the multiple data sources. 
     
     
         36 . (canceled) 
     
     
         37 . The surgical system of  claim 26 , wherein data from one or more of the multiple data sources includes positional information with respect to a common reference frame, and wherein displaying the visual representations comprises overlaying a first visual representation on a second visual representation, wherein the first visual representation is registered with respect to the second visual representation based on the common reference frame. 
     
     
         38 . (canceled) 
     
     
         39 . The surgical system of  claim 26 , wherein the first arrangement is determined based on a user profile loaded prior to commencement of the surgical process, the user profile identifying an individual performing the surgical process, and includes user-preferences of the individual regarding organization of the visual representations corresponding to the data from the multiple data sources during different phases of the surgical process. 
     
     
         40 .- 50 . (canceled) 
     
     
         51 . One or more machine-readable non-transitory storage devices encoded with machine-readable instructions configured to cause one or more processing devices to perform operations comprising:
 operating a surgical system to perform a surgical process, the surgical system including a display device;   receiving data from multiple data sources;   determining a current phase of the surgical process; and   displaying, on the display device, visual representations corresponding to the data from a first set of the multiple data sources in a first arrangement within a display region of the display device,   wherein at least one of the first set of the multiple data sources and the first arrangement is associated with the current phase of the surgical process.   
     
     
         52 . The one or more machine-readable non-transitory storage devices of  claim 51 , further comprising instructions for:
 determining a new phase of the surgical process; and   updating at least one of the first set of the multiple data sources and the first arrangement in response to determining the new phase of the surgical process.   
     
     
         53 .- 75 . (canceled)