Augmented reality navigation systems for use with robotic surgical systems and methods of their use

Abstract

The present disclosure is directed to augmented reality navigation systems and methods of their use that, inter alia, address the need for systems and methods of robotic surgical system navigation with reduced distraction to surgeons. Augmented reality navigation systems disclosed herein enable a surgeon to maintain focus on a surgical site and/or surgical tool being used in a surgical procedure while obtaining a wide range of navigational information relevant to the procedure. Navigational information can appear in the augmented reality navigation system as being presented on virtual displays that sit in a natural field of view of a surgeon during a procedure. Navigational information can also appear to be overlaid over a patient's anatomy. Augmented reality navigation systems comprise a head mounted display comprising an at least partially transparent display screen, at least one detector connected to the head mounted display for identifying real-world features, and a computer subsystem.

What is claimed is: 
     
         1 . An augmented reality navigation system for use with a robotic surgical system, the system comprising:
 a head mounted display comprising an at least partially transparent display screen configured to display augmentation graphics which appear to a user to be superimposed on at least a portion of a natural field of view of the user;   at least one detector for identifying real-world features, the at least one detector connected to the head mounted display;   a surgical tool having markers and configured to be detected by at the at least one detector, wherein a representation of at least a portion of the surgical tool and/or a trajectory of the surgical tool is presented in the head mounted display,   wherein a detector input signal from the at least one detector corresponds to a field of view of the at least one detector and the field of view comprises at least a portion of anatomy of a patient during a surgical procedure,   wherein the detector input signal includes a relative location and/or orientation for each of one or more of the real-world features,   wherein the surgical tool is inserted into or connected to the robotic surgical system.   
     
     
         2 . The augmented reality navigation system of  claim 1 , wherein a camera system for detecting real-world features is electrically coupled to the head mounted display. 
     
     
         3 . The augmented reality navigation system of  claim 1 , wherein the head mounted display provides a representation of the surgical tool and a trajectory of the surgical tool overlaid on the anatomy of the patient. 
     
     
         4 . The augmented reality navigation system of  claim 1 , further includes a motion sensor connected to the head mounted display for outputting a motion signal based on measured motion of the head mounted display. 
     
     
         5 . The augmented reality navigation system of  claim 1 , wherein the robotic surgical system includes an arm, an end effector coupled to a first end of the arm and a base coupled to the second end of the arm, wherein the arm moves the end effector that is configured to receive the surgical tool to a trajectory selected by a user. 
     
     
         6 . The augmented reality navigation system of  claim 5 , wherein the robotic surgical system and the augmented reality navigation system includes a haptic feedback system to control the robotic arm within a selected trajectory. 
     
     
         7 . The augmented reality navigation system of  claim 6 , wherein the surgical tool includes a communicating with the haptic feedback system that constrains the surgical tool to the user selected trajectory. 
     
     
         8 . The augmented reality navigation system of  claim 1 , wherein the at least one detector comprises a detector with at least a minimum field of view of 40 degrees. 
     
     
         9 . The augmented reality navigation system of  claim 1 , wherein the display screen has a resolution of at least 1280×720 pixels. 
     
     
         10 . The augmented reality navigation system of  claim 1 , comprising a pointer tool for making surgical planning selections, wherein the pointer tool is configured to be detected by the at least one detector. 
     
     
         11 . The augmented reality navigation system of  claim 1 , wherein the robotic surgical system registers anatomy of a patient with the augmented reality navigation system, and an anatomical model of the patient based on medical image data. 
     
     
         12 . The augmented reality navigation system of  claim 1 , wherein the at least one detector comprises a video camera and transmits a video signal to the head mounted display to display augmentation graphics which appear to the user to be superimposed on at least a portion ao natural field of view of the user. 
     
     
         13 . The augmented reality navigation system of  claim 1 , wherein the surgical procedure comprises at least one of a spinal surgical procedure, an orthopedic surgical procedure, an orthopedic trauma surgical procedure, and a neurosurgical procedure. 
     
     
         14 . An augmented reality navigation system for use with a robotic surgical system, the system comprising:
 a head mounted display comprising an at least partially transparent display screen configured to display augmentation graphics which appear to a user to be superimposed on at least a portion of a natural field of view of the user;   at least one detector for identifying real-world features, the at least one detector connected to the head mounted display; and   a computer subsystem configured to generate and/or access a representation of at least a portion of a surgical tool and/or a trajectory of the surgical tool during a surgical procedure, modify at least a portion of the representation based on a relative position and/or orientation of one or more real-world features in a detector input signal received from the at least one detector, and display, on the display screen, surgical tool augmentation graphics based on the modified representation, wherein the surgical tool is inserted into or connected to the robotic surgical system.   
     
     
         15 . The augmented reality navigation system of  claim 14 , wherein the computer subsystem is configured to render a surgical tool augmentation graphic for each of a plurality of surgical tool trajectories, and display, on the display screen, the plurality of surgical tool augmentation graphics such that the surgical tool augmentation graphics appear overlaid on the anatomy of the patient and each of the trajectory augmentation graphics indicate a physical trajectory that could be followed during the surgical procedure. 
     
     
         16 . The augmented reality navigation system of  claim 14 , wherein
 the computer subsystem is configured to modify an anatomical model of a patient based on one or more relative location(s) and/or orientation(s) determined from the detected input signal, thereby forming an updated anatomical model, and   the computer subsystem is configured to display, on the display screen, anatomical model augmentation graphics corresponding to the updated anatomical model such that the updated anatomical model appears overlaid on the anatomy of the patient.   
     
     
         17 . The augmented reality navigation system of  claim 14 , comprising:
 a motion sensor connected to the head mounted display for outputting a motion signal based on measured motion of the head mounted display,   wherein the computer subsystem is configured to update the surgical tool augmentation graphics based on motion detected by the motion sensor.   
     
     
         18 . The augmented reality navigation system of  claim 14 , wherein the computer subsystem is configured to determine a selected trajectory based at least in part on a user input trajectory selection signal that selects the selected trajectory from a set of one or more planned trajectories, and automatically move a robotic arm and/or end effector of the robotic surgical system to be aligned with the selected trajectory. 
     
     
         19 . The augmented reality navigation system of  claim 18 , wherein the computer subsystem is configured to automatically move the robotic arm and/or end effector of the robotic surgical system along the trajectory. 
     
     
         20 . The augmented reality navigation system of  claim 18 , wherein the computer subsystem is configured to define a haptic feedback system that comprises the trajectory and constrains motion of a robotic arm and/or end effector such that motion of at least a portion of a surgical tool attached to the robotic arm and/or end effector is constrained to within the haptic feedback system.