Extended reality systems with three-dimensional visualizations of medical image scan slices

Abstract

A navigated surgery system includes at least one processor that is operative to obtain a 2D medical image slice of anatomical structure of a patient. The operations further obtain a 3D graphical model of anatomical structure. The operations determine a pose of a virtual cross-sectional plane extending through the 3D graphical model of the anatomical structure that corresponds to the anatomical structure of the 2D medical image slice. The operations control the XR headset to display the 2D medical image slice of the anatomical structure of the patient, display the 3D graphical model of the anatomical structure, and display a graphical object oriented with the pose relative to the 3D graphical model of the anatomical structure.

1 . A method by a navigated surgery system, the method comprising:
 obtaining a three-dimensional (3D) graphical model of an internal anatomical structure of a patient;   tracking a tip of a surgical tool relative to the 3D graphical model;   selecting, based on a pose of the tracked surgical tool tip, a first two-dimensional (2D) image slice of a portion of the internal anatomical structure from a plurality of 2D image slices of the internal anatomical structure;   selecting, based on the pose of the tracked surgical tool tip, a second 2D image slice of the portion of the internal anatomical structure from the plurality of 2D image slices of the internal anatomical structure, wherein the second 2D image slice is perpendicular to the first 2D image slice and the tip of the tracked surgical tool is positioned at an intersection of the first 2D image slice and the second 2D image slice; and   displaying the first 2D image slice and the second 2D image slice as overlays on the 3D graphical model of the internal anatomical structure of the patient.   
     
     
         2 . The method of  claim 1 , wherein selecting the first 2D image slice comprises dynamically selecting the first 2D image slice based on movement of the tracked surgical tool tip, and selecting the second 2D image slice comprises dynamically selecting the second 2D image slice based on movement of the tracked surgical tool tip. 
     
     
         3 . The method of  claim 1 , wherein the first 2D image slice is an axial 2D image slice. 
     
     
         4 . The method of  claim 1 , wherein the second 2D image slice is a sagittal 2D image slice. 
     
     
         5 . The method of  claim 1 , wherein the plurality of 2D image slices of the internal anatomical structure together form a 3D volume of the internal anatomical structure. 
     
     
         6 . The method of  claim 1 , wherein the internal anatomical structure is the patient's spine. 
     
     
         7 . A method by a navigated surgery system, the method comprising:
 tracking a tip of a surgical tool relative to an internal anatomical structure of a patient;   selecting, based on a pose of the tracked surgical tool tip, a first two-dimensional (2D) image slice of a portion of the internal anatomical structure from a plurality of 2D image slices of the internal anatomical structure;   selecting, based on the pose of the tracked surgical tool tip, a second 2D image slice of the portion of the internal anatomical structure from the plurality of 2D image slices of the internal anatomical structure, wherein the second 2D image slice is perpendicular to the first 2D image slice and the tip of the tracked surgical tool is positioned at an intersection of the first 2D image slice and the second 2D image slice; and   displaying the first 2D image slice and the second 2D image slice as overlays on the patient through a see-through screen of a head mounted display.   
     
     
         8 . The method of  claim 7 , wherein selecting the first 2D image slice comprises dynamically selecting the first 2D image slice based on movement of the tracked surgical tool tip, and selecting the second 2D image slice comprises dynamically selecting the second 2D image slice based on movement of the tracked surgical tool tip. 
     
     
         9 . The method of  claim 7 , wherein the first 2D image slice is an axial 2D image slice. 
     
     
         10 . The method of  claim 7 , wherein the second 2D image slice is a sagittal 2D image slice. 
     
     
         11 . The method of  claim 7 , wherein the plurality of 2D image slices of the internal anatomical structure together form a 3D volume of the internal anatomical structure. 
     
     
         12 . The method of  claim 7 , wherein the internal anatomical structure is the patient's spine.