Method and system for simulating surgical procedures

Abstract

A system and method for converting static/still medical images of a particular patient into dynamic and interactive images interacting with medical tools models including models of medical devices by coupling a model of tissue dynamics and tool characteristics to the patient specific imagery for simulating a medical procedure in an accurate and dynamic manner, where the simulation can be shared with others even in remote locations.

What is claimed is: 
     
         1 . A method of representing a medical procedure, comprising the steps of:
 providing a computer system;   providing a display connected to said computer system;   obtaining patient image information about the actual organs and other tissues of a particular patient for storing in said computer system;   generating, using specialized software executing on the computer system, realistic dynamic 3D models of the biological organs and other tissues of the particular patient by utilizing said image information about the actual organs and tissue of the particular patient;   storing the 3D models of the biological organs and other tissues of the particular patient on the computer system;   generating, using specialized software executing on the computer system accessing the stored 3D models of the biological organs and other tissues of the particular patient, dynamic 3D tissue images for display on said display, said 3D tissue images realistically representing the actual organs and other tissues of the particular patient in a manner indicative of the actual organs and other tissues of the particular patient;   accepting, using specialized software executing on the computer system, user inputs to a user interface provided by the computer system for the user to realistically interact with the 3D tissue images displayed on said display in a manner indicative of real user interactions with actual organs and other tissues of the particular patient for representing a medical treatment provided to the particular patient.   
     
     
         2 . The method of  claim 1 , further comprising the step of generating, using specialized software executing on the computer system based on said user inputs, a realistic 3D user tool image that is displayed on said display realistically interacting with the 3D tissue images displayed on said display. 
     
     
         3 . The method of  claim 2 , further comprising the step of providing realistic feedback to the user via the user interface, said feedback realistically representing an actual user tool interacting with actual organs and other tissues. 
     
     
         4 . The method of  claim 2 , wherein a force that is input by the user using the user interface to manipulate the user tool image causes the 3D tissue images to realistically interact by showing a dynamic mechanical reaction on blood vessels, organs. and other tissues displayed in said 3D tissue images such that said displayed blood vessels, organs, and other tissues are dynamically reshaped according to mechanical properties of actual blood vessels, organs, and other tissues of actual patients. 
     
     
         5 . The method of  claim 2 , wherein realistic interactions between said user tool image and said 3D tissue images are provided such that changes to surrounding organs are accurately reflected in said 3D tissue images in a manner similar to a corresponding real surgery tool providing a feedback force to the user similar to that during interactions with actual tissues in an actual patient surgery. 
     
     
         6 . The method of  claim 2 , wherein the user tool image displays mechanical properties of an actual surgical tool that supports tool open/close status, spring load, and/or spring force reflecting actual values that are applied on the patient during an actual surgery. 
     
     
         7 . The method of  claim 2 , wherein said user tool image is of a probe or implantable medical device. 
     
     
         8 . The method of  claim 1 , further comprising the step of filtering out a particular type of tissue from the display of said 3D tissue images at the request of the user. 
     
     
         9 . The method of  claim 1 , further comprising the step of filtering out a 3D corridor according to a desired location, orientation and dimension of the corridor at the request of the user. 
     
     
         10 . The method of  claim 1 , wherein said 3D tissue images realistically represent the actual organs and other tissues of the particular patient in a manner indicative of the actual organs and tissue of the particular patient in texture, shape, shadowing, and coloring. 
     
     
         11 . The method of  claim 1 , wherein said 3D tissue images are provided showing an appearance including shadowing and textures indicative of an actual organ and surrounding tissue. 
     
     
         12 . The method of  claim 1 , wherein the patient specific images include CT and/or MRI scans providing images of organs and other tissues of the particular patient. 
     
     
         13 . The method of  claim 1 , wherein the step of representing the medical treatment includes simulating a noninvasive procedure using a probe while enhancing visual characteristics of the 3D tissue images with correlation to the probe's location to provide realistic “life-like” rendering in the 3D tissue images displayed by the display. 
     
     
         14 . The method of  claim 1 , further comprising the step of using an input to the computer system for freezing a simulation and/or rotating an organ displayed in said 3D tissue images to observe an area of interest from different orientations and/or perspectives. 
     
     
         15 . The method of  claim 1 , further comprising the steps of:
 using an input to the computer system for marking a point in time in the representation of the medical treatment; and   using an input to the computer system for returning the representation of the medical treatment to that point in time for continuing the representation of the medical treatment along a different path.   
     
     
         16 . The method of  claim 1 , further comprising the step of using a the computer system for evaluating different treatment approaches for a selected phase of the representation of the medical treatment without restarting the entire representation of the medical treatment from the original starting point. 
     
     
         17 . The method of  claim 1 , wherein said representation of the medical treatment is a simulation of the medical treatment. 
     
     
         18 . The method of  claim 1 , wherein said representation of the medical treatment participates in an actual medical treatment being performed on the particular patient. 
     
     
         19 . A method of representing a medical procedure, comprising the steps of:
 providing a computer system;   providing a display connected to said computer system;   obtaining patient image information about the actual organs and other tissues of a particular patient for storing in said computer system;   generating, using specialized software executing on the computer system, realistic dynamic 3D models of the biological organs and other tissues of the particular patient by utilizing said image information about the actual organs and tissue of the particular patient;   storing the 3D models of the biological organs and other tissues of the particular patient on the computer system;   generating, using specialized software executing on the computer system accessing the stored 3D models of the biological organs and other tissues of the particular patient, dynamic 3D tissue images for display on said display, said 3D tissue images realistically representing the actual organs and other tissues of the particular patient in a manner indicative of the actual organs and tissue of the particular patient in texture, shape, shadowing, and coloring;   accepting, using specialized software executing on the computer system, user inputs to a user interface representing a real medical tool provided by the computer system for the user to realistically interact with the 3D tissue images displayed on said display in a manner indicative of real user interactions with actual organs and other tissues of the particular patient using said real medical tool for representing a medical treatment provided to the particular patient.   
     
     
         20 . The method of  claim 19 , wherein a force that is input by the user using the user interface causes the 3D tissue images to realistically interact by showing a dynamic mechanical reaction on blood vessels, organs, and other tissues displayed in said 3D tissue images such that said displayed blood vessels, organs, and other tissues are dynamically reshaped according to mechanical properties of actual blood vessels, organs, and other tissues of actual patients. 
     
     
         21 . The method of  claim 19 , further comprising the step of generating, using specialized software executing on the computer system based on said user inputs, a realistic 3D medical tool image representing said real medical tool, said 3D medical tool image being displayed on said display realistically interacting with the 3D tissue images displayed on said display. 
     
     
         22 . The method of  claim 21 , further comprising the step of providing realistic feedback to the user via the user interface, said feedback realistically representing the real medical tool interacting with actual organs and other tissues. 
     
     
         23 . The method of  claim 21 , wherein realistic interactions between said medical tool image and said 3D tissue images are provided such that changes to surrounding organs are accurately reflected in said 3D tissue images in a manner similar to a corresponding real surgery tool providing a feedback force to the user similar to that during interactions with tissues in an actual patient surgery. 
     
     
         24 . The method of  claim 21 , wherein the medical tool image displays mechanical properties of an actual surgical tool that supports tool open/close status, spring load, and/or spring force reflecting actual values that are applied on the patient during an actual surgery. 
     
     
         25 . The method of  claim 21 , wherein said medical tool image is of an implantable medical device. 
     
     
         25 . The method of  claim 19 , further comprising the step of filtering out a particular type of tissue from the display of said 3D tissue images at the request of the user. 
     
     
         26 . The method of  claim 19 , further comprising the step of filtering out a 3D corridor according to a desired location, orientation and dimension of the corridor at the request of the user. 
     
     
         27 . The method of  claim 19 , further comprising the step of using an input to the computer system for freezing a simulation and/or rotating an organ displayed in said 3D tissue images to observe an area of interest from different orientations and/or perspectives. 
     
     
         28 . The method of  claim 19 , further comprising the steps of:
 using an input to the computer system for marking a point in time in the representation of the medical treatment; and   using an input to the computer system for returning the representation of the medical treatment to that point in time for continuing the representation of the medical treatment along a different path.   
     
     
         29 . The method of  claim 19 , wherein said representation of the medical treatment is a simulation of the medical treatment. 
     
     
         30 . The method of  claim 19 , wherein said representation of the medical treatment participates in an actual medical treatment being performed on the particular patient. 
     
     
         31 . A method of representing a medical procedure, comprising the steps of:
 providing a computer system;   providing a display connected to said computer system;   obtaining patient image information about the actual organs and other tissues of a particular patient for storing in said computer system;   generating, using specialized software executing on the computer system, realistic dynamic 3D models of the biological organs and other tissues of the particular patient by utilizing said image information about the actual organs and tissue of the particular patient;   storing the 3D models of the biological organs and other tissues of the particular patient on the computer system;   generating, using specialized software executing on the computer system accessing the stored 3D models of the biological organs and other tissues of the particular patient, dynamic 3D tissue images for display on said display, said 3D tissue images realistically representing the actual organs and other tissues of the particular patient in a manner indicative of the actual organs and tissue of the particular patient in texture, shape, shadowing, and coloring;   accepting, using specialized software executing on the computer system, user inputs to a user interface representing a real medical tool provided by the computer system for the user to realistically interact with the 3D tissue images displayed on said display in a manner indicative of real user interactions with actual organs and other tissues of the particular patient using said real medical tool for simulating a medical treatment or supporting an actual medical treatment provided to the particular patient, wherein   said user inputs include inputs for rotating an organ displayed in said 3D images to observe an area of interest from different orientations and/or perspectives; and wherein   said user inputs include inputs for filtering out a 3D corridor according to a desired location, orientation and dimension of the corridor at the request of the user, and further wherein   said user inputs include inputs for filtering out a particular type of tissue from the display of said 3D tissue images.   
     
     
         32 . The method of  claim 31 , further comprising the step of generating, using specialized software executing on the computer system based on said user inputs, a realistic 3D medical tool image that is displayed on said display realistically interacting with the 3D tissue images displayed on said display, said 3D medical tool image being displayed on said display realistically interacting with the 3D tissue images displayed on said display.