Ultrasound image-based guidance of medical instruments or devices

Abstract

In an example, a system includes an ultrasound sensor configured to transmit ultrasound energy and receive ultrasound energy reflected in a region of a patient and one or more processors configured to generate a reference ultrasound image of the region of the patient based on a portion of the ultrasound energy that was received by the ultrasound sensor prior to a medical instrument or medical device causing obstruction in the received ultrasound energy, generate a live ultrasound image based on a current portion of the received ultrasound energy obtained by the ultrasound sensor, register the reference ultrasound image and the live ultrasound image, and control a display device to display the reference ultrasound image with at least a portion of the live ultrasound image.

What is claimed is: 
     
         1 . A system comprising:
 an ultrasound sensor configured to transmit ultrasound energy and receive ultrasound energy reflected in a region of a patient; and   one or more processors configured to:
 generate a plurality of reference ultrasound images of the region of the patient based on a portion of the ultrasound energy that was received by the ultrasound sensor prior to a medical instrument or medical device causing obstruction in the received ultrasound energy; 
 generate a live ultrasound image based on a current portion of the received ultrasound energy obtained by the ultrasound sensor; 
 select one of the plurality of reference ultrasound images based on at least one of correspondence with event data received by the one or more processors or a spatial orientation of the live ultrasound image; 
 register the reference ultrasound image and the live ultrasound image; and 
 control a display device to display the selected reference ultrasound image with at least a portion of the live ultrasound image. 
   
     
     
         2 . The system of  claim 1 , wherein the event data comprises cardiac event data. 
     
     
         3 . The system of  claim 2 , wherein the cardiac event data includes a phase of a cardiac cycle, and the selected reference ultrasound image corresponds to the phase of the cardiac cycle. 
     
     
         4 . The system of  claim 3 , wherein the one or more processors are configured to determine the phase of the cardiac cycle based on at least one of an ECG signal or the live ultrasound image. 
     
     
         5 . The system of  claim 1 , wherein each of the plurality of reference ultrasound images corresponds to one of a plurality of spatial orientations, and each of the plurality of spatial orientations includes at least one of a translation, rotation, or perspective of the live ultrasound image. 
     
     
         6 . The system of  claim 5 , wherein the one or more processors are configured to select one of the plurality of reference ultrasound images based on both the event data and the spatial orientations of the live ultrasound image. 
     
     
         7 . The system of  claim 1 , wherein the one or more processors are configured to:
 receive EM tracking data from an EM tracking system;   determine at least one of a position, orientation or trajectory of the medical instrument or medical device based on the EM tracking data; and   generate a representation of the medical instrument or medical device based on the determined position, orientation or trajectory.   
     
     
         8 . The system of  claim 1 , wherein the portion of the received ultrasound energy received by the ultrasound sensor prior to the medical instrument or medical device causing the obstruction was received when the medical instrument or medical device was at least partially positioned within the region of the patient. 
     
     
         9 . The system of  claim 1 , wherein the portion of the received ultrasound energy received by the ultrasound sensor prior to the medical instrument or medical device causing the obstruction was received before the medical instrument or medical device was at least partially positioned within the region of the patient. 
     
     
         10 . The system of  claim 1 , wherein the medical instrument or medical device comprises at least one of an implantable medical device, medical implant delivery device, therapy delivery device, surgical device, mechanical circulatory support device, coronary stent device, heart valve device, heart valve repair device, cardiac ablation device, cardiac lead device, drug delivery device, catheter delivery device, or endoscopic delivery device. 
     
     
         11 . The system of  claim 1 , wherein the medical instrument or medical device comprises a medical instrument configured for transcatheter heart valve repair or replacement. 
     
     
         12 . The system of  claim 1 , wherein the medical instrument or medical device further comprises a prosthetic heart valve. 
     
     
         13 . The system of  claim 12 , wherein the prosthetic heart valve comprises a prosthetic mitral valve. 
     
     
         14 . The system of  claim 1 , wherein the one or more processors are configured to control the display device to display the selected reference ultrasound image with a representation of the medical instrument or medical device. 
     
     
         15 . The system of  claim 1 , wherein the one or more processors are configured to identify a physiological landmark of the region of the patient based on the live ultrasound image and register the selected reference ultrasound image with the physiological landmark. 
     
     
         16 . The system of  claim 1 , wherein the one or more processors are configured to control the display device to display the at least a portion of the live ultrasound image with the selected reference ultrasound image such that the selected reference ultrasound image moves in registration with the at least a portion of the live ultrasound image. 
     
     
         17 . The system of  claim 1 , wherein the one or more processors are further configured to:
 determine that at least one of a position, orientation, or trajectory of the medical instrument or medical device has changed; and   control the display device to display at least one of an updated position, orientation, or trajectory of the medical instrument or medical device based on the determination that at least one of the position, orientation, or trajectory has changed.   
     
     
         18 . The system of  claim 1 , wherein the one or more processors are configured to control the display device to display the selected reference ultrasound image with a representation of the medical instrument or medical device. 
     
     
         19 . The system of  claim 1 , further comprising:
 an electromagnetic (EM) tracking system configured to collect EM tracking data representative of a position or orientation of each of the ultrasound sensor and the medical instrument or medical device relative to the region of the patient,   wherein the one or more processors are further configured to:
 receive the EM tracking data from the EM tracking system; 
 determine at least one of a position, orientation, or trajectory of the medical instrument or medical device based on the EM tracking data; and 
 control the display device to display at least one of the position, orientation, or trajectory of the medical instrument or medical device with the selected reference ultrasound image. 
   
     
     
         20 . The system of  claim 1 , wherein the selected reference ultrasound image comprises a motion picture covering a full-cycle of a moving anatomical structure. 
     
     
         21 . The system of  claim 20 , wherein moving anatomical structure is a heart. 
     
     
         22 . The system of  claim 1 , wherein the one or more processors are further configured to identify common anatomical structures in the selected reference ultrasound image and the live ultrasound image using image recognition prior to registering the selected reference ultrasound image and the live ultrasound image. 
     
     
         23 . A method comprising:
 transmitting ultrasound energy;   receiving ultrasound energy reflected in a region of a patient;   generating a plurality of reference ultrasound images of the region of the patient based on a portion of the received ultrasound energy that was received prior to a medical instrument or medical device causing obstruction in the received ultrasound energy;   generating a live ultrasound image based on a current portion of the received ultrasound energy;   selecting one of the plurality of reference ultrasound images based on at least one of correspondence with event data received by the one or more processors or a spatial orientation of the live ultrasound image;   registering the reference ultrasound image and the live ultrasound image; and   controlling a display device to display the selected reference ultrasound image with at least a portion of the live ultrasound image.   
     
     
         24 . The method of  claim 23 , wherein the event data comprises cardiac event data. 
     
     
         25 . The method of  claim 24 , wherein the cardiac event data includes a phase of a cardiac cycle, and the selected reference ultrasound image corresponds to the phase of the cardiac cycle. 
     
     
         26 . The method of  claim 25 , further comprising determining the phase of the cardiac cycle based on at least one of an ECG signal or the live ultrasound image. 
     
     
         27 . The method of  claim 23 , wherein each of the plurality of reference ultrasound images corresponds to one of a plurality of spatial orientations, and each of the plurality of spatial orientations includes at least one of a translation, rotation, or perspective of the live ultrasound image. 
     
     
         28 . The method of  claim 27 , wherein the selecting the one of the plurality of reference ultrasound images is based on both the event data and the spatial orientations of the live ultrasound image. 
     
     
         29 . The method of  claim 23 , further comprising:
 receiving EM tracking data from an EM tracking system;   determining at least one of a position, orientation or trajectory of the medical instrument or medical device based on the EM tracking data; and   generating the representation of the medical instrument or medical device based on the determined position, orientation or trajectory.   
     
     
         30 . The method of  claim 23 , wherein the portion of the received ultrasound energy received by the ultrasound sensor prior to the medical instrument or medical device causing the obstruction was received when the medical instrument or medical device was at least partially positioned within the region of the patient. 
     
     
         31 . The method of  claim 23 , wherein the portion of the received ultrasound energy received by the ultrasound sensor prior to the medical instrument or medical device causing the obstruction was received before the medical instrument or medical device was at least partially positioned within the region of the patient. 
     
     
         32 . The method of  claim 23 , wherein the medical instrument or medical device comprises at least one of an implantable medical device, medical implant delivery device, therapy delivery device, surgical device, mechanical circulatory support device, coronary stent device, heart valve device, heart valve repair device, cardiac ablation device, cardiac lead device, drug delivery device, catheter delivery device, or endoscopic delivery device. 
     
     
         33 . The method of  claim 23 , wherein the medical instrument or medical device comprises an instrument configured for transcatheter heart valve repair or replacement. 
     
     
         34 . The method of  claim 23 , wherein the medical instrument or medical device further comprises a prosthetic heart valve. 
     
     
         35 . The method of  claim 34 , wherein the prosthetic heart valve comprises a prosthetic mitral valve. 
     
     
         36 . The method of  claim 23 , further comprising controlling the display device to display the selected reference ultrasound image with the representation of the medical instrument or medical device. 
     
     
         37 . The method of  claim 23 , further comprising identifying a physiological landmark of the region of the patient based on the live ultrasound image and registering the selected reference ultrasound image with the physiological landmark. 
     
     
         38 . The method of  claim 23 , further comprising controlling the display device to display the at least a portion of the live ultrasound image with the selected reference ultrasound image such that the selected reference ultrasound image moves in registration with the at least a portion of the live ultrasound image. 
     
     
         39 . The method of  claim 23 , further comprising determining that at least one of a position, orientation, or trajectory of the medical instrument or medical device has changed; and
 controlling the display device to display at least one of an updated position, orientation, or trajectory of the medical instrument or medical device based on the determination that at least one of the position, orientation, or trajectory has changed.   
     
     
         40 . The method of  claim 23 , further comprising controlling the display device to display the selected reference ultrasound image with the representation of the medical instrument or medical device. 
     
     
         41 . The method of  claim 23 , further comprising:
 collecting EM tracking data representative of a position or orientation of each of the ultrasound sensor and the medical instrument or medical device relative to the region of the patient;   receiving the EM tracking data from the EM tracking system;   determining at least one of a position, orientation, or trajectory of the medical instrument or medical device based on the EM tracking data; and   controlling the display device to display at least one of the position, orientation, or trajectory of the medical instrument or medical device with the selected reference ultrasound image.   
     
     
         42 . The method of  claim 23 , wherein the selected reference ultrasound image comprises a motion picture covering a full-cycle of a moving anatomical structure. 
     
     
         43 . The method of  claim 42 , wherein moving anatomical structure is a heart. 
     
     
         44 . The method of  claim 23 , further comprising identifying common anatomical structures in the selected reference ultrasound image and the live ultrasound image using image recognition prior to registering the selected reference ultrasound image and the live ultrasound image. 
     
     
         45 . A non-transitory computer-readable medium comprising instructions, which when executed, cause one or more processors to:
 generate a plurality of reference ultrasound images of a region of a patient based on a portion of received ultrasound energy that was received prior to a medical instrument or medical device causing obstruction in the received ultrasound energy;   generate a live ultrasound image based on a current portion of the received ultrasound energy;   select one of the plurality of reference ultrasound images based on at least one of correspondence with event data received by the one or more processors or a spatial orientation of the live ultrasound image;   register the reference ultrasound image and the live ultrasound image; and   control a display device to display the selected reference ultrasound image with at least a portion of the live ultrasound image.