Method and System for Image Registration Using an Intelligent Artificial Agent

Abstract

Methods and systems for image registration using an intelligent artificial agent are disclosed. In an intelligent artificial agent based registration method, a current state observation of an artificial agent is determined based on the medical images to be registered and current transformation parameters. Action-values are calculated for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model, such as a trained deep neural network (DNN). The actions correspond to predetermined adjustments of the transformation parameters. An action having a highest action-value is selected from the plurality of actions and the transformation parameters are adjusted by the predetermined adjustment corresponding to the selected action. The determining, calculating, and selecting steps are repeated for a plurality of iterations, and the medical images are registered using final transformation parameters resulting from the plurality of iterations.

1 . A method for intelligent artificial agent based registration of medical images, comprising:
 determining a current state observation of an artificial agent based on a plurality of medical images and current transformation parameters;   calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images, wherein the plurality of actions correspond to predetermined adjustments of the transformation parameters;   selecting an action having a highest action-value from the plurality of actions and adjusting the transformation parameters by the predetermined adjustment corresponding to the selected action;   repeating the determining, calculating, and selecting steps for a plurality of iterations; and   registering the plurality of medical images using final transformation parameters resulting from the plurality of iterations.   
     
     
         2 . The method of  claim 1 , wherein the plurality of medical images includes a first medical image and a second medical image and registering the plurality of medical images using final transformation parameters resulting from the plurality of iterations comprises:
 transforming the second medical image using the final transformation parameters to align the second medical image with the first medical image.   
     
     
         3 . The method of  claim 1 , wherein the plurality of medical images includes a first medical image and a second medical image and determining a current state observation of an artificial agent based on a plurality of medical images and current transformation parameters comprises:
 determining the current state observation by comparing the second medical image transformed by the current transformation parameters and the first medical image.   
     
     
         4 . The method of  claim 3 , wherein determining the current state observation by comparing the second medical image transformed by the current transformation parameters and the first medical image comprises:
 determining the current state observation by subtracting the second medical image transformed by the current transformation parameters from the first medical image.   
     
     
         5 . The method of  claim 1 , wherein the transformation is a rigid-body transformation and the plurality of actions comprises actions corresponding to predetermined adjustments of translation and rotation rigid-body transformation parameters. 
     
     
         6 . The method of  claim 5 , wherein the plurality of actions comprises respective actions corresponding to increasing and decreasing each of the translation and rotation rigid-body parameters by a constant or time-varying step size. 
     
     
         7 . The method of  claim 6 , wherein the transformation is a 3D rigid-body transformation and the rigid-body transformation parameters include three translation parameters and three rotation parameters. 
     
     
         8 . The method of  claim 6 , wherein the transformation is a 2D rigid-body transformation and the rigid-body transformation parameters include two translation parameters and one rotation parameter. 
     
     
         9 . The method of  claim 1 , wherein calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images comprises:
 calculating action-values for the plurality of actions available to the artificial agent based on the current state observation using a trained deep neural network (DNN).   
     
     
         10 . The method of  claim 9 , wherein the trained DNN is trained to predict action-values for the plurality of actions based on a plurality of training image pairs with known ground truth transformation parameters using reinforcement learning in which, for each training image pair, a reward for each action of the plurality of actions at a given state is used to train the DNN to learn an optimal registration policy. 
     
     
         11 . The method of  claim 10 , wherein the trained DNN is trained using supervised reinforcement learning, in which the reward for each action of the plurality of actions at a given state is directly calculated based on a decrease in a geodesic distance between the transformation parameters at the given a state and the ground truth transformation parameters for the training image pair resulting from applying the action. 
     
     
         12 . The method of  claim 11 , wherein the plurality of training image pairs includes training image pairs synthetically generated by artificially de-aligning other training image pairs using randomly generated perturbations to the ground truth transformation parameters of the other training image pairs. 
     
     
         13 . The method of  claim 11 , wherein the plurality of training image pairs includes training image pairs synthetically generated from other training image pairs to provide dense sampling close to the ground truth transformation parameters by co-deforming the other training image pairs by randomly generated affine transformations within a range of the ground truth transformation parameters. 
     
     
         14 . The method of  claim 11 , wherein the plurality of training image pairs includes training image pairs synthetically generated by artificially altering image appearance of other training image pairs using synthetic image generators. 
     
     
         15 . The method of  claim 1 , wherein the plurality of medical images includes a first medical image and a second medical image, the transformation is a dense deformation model that models deformations applied to the second medical image, the plurality of actions comprises actions corresponding to adjustments to parameters of the deformation model, and registering the plurality of medical images using final transformation parameters resulting from the plurality of iterations comprises:
 generating a dense deformation field corresponding to final parameters of the deformation model resulting from the plurality of iterations; and   warping the second medical image to register the second medical image with the first medical image by applying the dense deformation field corresponding to the final parameters of the deformation model.   
     
     
         16 . The method of  claim 1 , wherein calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images comprises:
 calculating action-values for each action in an action set including actions to adjust each of the transformation parameters individually by a respective step size, actions to adjust multiple transformation parameters simultaneously by their respective step sizes, and actions to optimize the respective step size for each of the transformation parameters.   
     
     
         17 . The method of  claim 1 , wherein calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images comprises:
 calculating the action-values for the plurality of actions corresponding to the predetermined adjustments of the transformation parameters and an action-value for a stop registration action based on the current state observation using the trained machine learning based model.   
     
     
         18 . The method of  claim 17 , wherein repeating the determining, calculating, and selecting steps for a plurality of iterations comprises:
 repeating the determining, calculating, and selecting steps until the action-value calculated by the trained DNN for the stop registration action is higher than the action-values calculated by the trained DNN for each of the plurality of actions corresponding to the predetermined adjustments of the transformation parameters.   
     
     
         19 . The method of  claim 1 , wherein repeating the determining, calculating, and selecting steps for a plurality of iterations comprises:
 repeating the determining, calculating, and selecting steps using a first image resolution of the plurality of medical images and a first machine learning based model trained using training images at the first image resolution until a first stop condition is reached; and   repeating the determining, calculating, and selecting steps using a second image resolution of the plurality of medical images and a second machine learning based model trained using training images at the second image resolution until a second stop condition is reached.   
     
     
         20 . An apparatus for intelligent artificial agent based registration of medical images, comprising:
 a processor; and   a memory storing computer program instructions, which when executed by the processor cause the processor to perform operations comprising:   determining a current state observation of an artificial agent based on a plurality of medical images and current transformation parameters;   calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images, wherein the plurality of actions correspond to predetermined adjustments of the transformation parameters;   selecting an action having a highest action-value from the plurality of actions and adjusting the transformation parameters by the predetermined adjustment corresponding to the selected action;   repeating the determining, calculating, and selecting operations for a plurality of iterations; and   registering the plurality of medical images using final transformation parameters resulting from the plurality of iterations.   
     
     
         21 . The apparatus of  claim 20 , wherein the plurality of medical images includes a first medical image and a second medical image and determining a current state observation of an artificial agent based on a plurality of medical images and current transformation parameters comprises:
 determining the current state observation by comparing the second medical image transformed by the current transformation parameters and the first medical image.   
     
     
         22 . The apparatus of  claim 20 , wherein the transformation is a rigid-body transformation and the plurality of actions comprises actions corresponding to predetermined adjustments of translation and rotation rigid-body transformation parameters. 
     
     
         23 . The apparatus of  claim 20 , wherein calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images comprises:
 calculating action-values for the plurality of actions available to the artificial agent based on the current state observation using a trained deep neural network (DNN).   
     
     
         24 . The apparatus of  claim 23 , wherein the trained DNN is trained to predict action-values for the plurality of actions based on a plurality of training image pairs with known ground truth transformation parameters using reinforcement learning in which, for each training image pair, a reward for each action of the plurality of actions at a given state is used to train the DNN to learn an optimal registration policy. 
     
     
         25 . The apparatus of  claim 24 , wherein the trained DNN is trained using supervised reinforcement learning, in which the reward for each action of the plurality of actions at a given state is directly calculated based on a decrease in a geodesic distance between the transformation parameters at the given a state and the ground truth transformation parameters for the training image pair resulting from applying the action. 
     
     
         26 . The apparatus of  claim 25 , wherein the plurality of training image pairs includes training image pairs synthetically generated by artificially de-aligning other training image pairs using randomly generated perturbations to the ground truth transformation parameters of the other training image pairs. 
     
     
         27 . The apparatus of  claim 25 , wherein the plurality of training image pairs includes training image pairs synthetically generated from other training image pairs to provide dense sampling close to the ground truth transformation parameters by co-deforming the other training image pairs by randomly generated affine transformations within a range of the ground truth transformation parameters. 
     
     
         28 . The apparatus of  claim 20 , wherein the plurality of medical images includes a first medical image and a second medical image, the transformation is a dense deformation model that models deformations applied to the second medical image, the plurality of actions comprises actions corresponding to adjustments to parameters of the deformation model, and registering the plurality of medical images using final transformation parameters resulting from the plurality of iterations comprises:
 generating a dense deformation field corresponding to final parameters of the deformation model resulting from the plurality of iterations; and   warping the second medical image to register the second medical image with the first medical image by applying the dense deformation field corresponding to the final parameters of the deformation model.   
     
     
         29 . The apparatus of  claim 20 , wherein repeating the determining, calculating, and selecting operations for a plurality of iterations comprises:
 repeating the determining, calculating, and selecting operations using a first image resolution of the plurality of medical images and a first machine learning based model trained using training images at the first image resolution until a first stop condition is reached; and   repeating the determining, calculating, and selecting operations using a second image resolution of the plurality of medical images and a second machine learning based model trained using training images at the second image resolution until a second stop condition is reached.   
     
     
         30 . A non-transitory computer readable medium storing computer program instructions for intelligent artificial agent based registration of medical images, the computer program instructions defining operations comprising:
 determining a current state observation of an artificial agent based on a plurality of medical images and current transformation parameters;   calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images, wherein the plurality of actions correspond to predetermined adjustments of the transformation parameters;   selecting an action having a highest action-value from the plurality of actions and adjusting the transformation parameters by the predetermined adjustment corresponding to the selected action;   repeating the determining, calculating, and selecting operations for a plurality of iterations; and   registering the plurality of medical images using final transformation parameters resulting from the plurality of iterations.   
     
     
         31 . The non-transitory computer readable medium of  claim 30 , wherein the plurality of medical images includes a first medical image and a second medical image and determining a current state observation of an artificial agent based on a plurality of medical images and current transformation parameters comprises:
 determining the current state observation by comparing the second medical image transformed by the current transformation parameters and the first medical image.   
     
     
         32 . The non-transitory computer readable medium of  claim 30 , wherein the transformation is a rigid-body transformation and the plurality of actions comprises actions corresponding to predetermined adjustments of translation and rotation rigid-body transformation parameters. 
     
     
         33 . The non-transitory computer readable medium of  claim 30 , wherein calculating action-values for a plurality of actions available to the artificial agent based on the current state observation using a machine learning based model trained based on a plurality of registered training images comprises:
 calculating action-values for the plurality of actions available to the artificial agent based on the current state observation using a trained deep neural network (DNN).   
     
     
         34 . The non-transitory computer readable medium of  claim 33 , wherein the trained DNN is trained to predict action-values for the plurality of actions based on a plurality of training image pairs with known ground truth transformation parameters using reinforcement learning in which, for each training image pair, a reward for each action of the plurality of actions at a given state is used to train the DNN to learn an optimal registration policy. 
     
     
         35 . The non-transitory computer readable medium of  claim 34 , wherein the trained DNN is trained using supervised reinforcement learning, in which the reward for each action of the plurality of actions at a given state is directly calculated based on a decrease in a geodesic distance between the transformation parameters at the given a state and the ground truth transformation parameters for the training image pair resulting from applying the action. 
     
     
         36 . The non-transitory computer readable medium of  claim 35 , wherein the plurality of training image pairs includes training image pairs synthetically generated by artificially de-aligning other training image pairs using randomly generated perturbations to the ground truth transformation parameters of the other training image pairs. 
     
     
         37 . The non-transitory computer readable medium of  claim 35 , wherein the plurality of training image pairs includes training image pairs synthetically generated from other training image pairs to provide dense sampling close to the ground truth transformation parameters by co-deforming the other training image pairs by randomly generated affine transformations within a range of the ground truth transformation parameters. 
     
     
         38 . The non-transitory computer readable medium of  claim 30 , wherein the plurality of medical images includes a first medical image and a second medical image, the transformation is a dense deformation model that models deformations applied to the second medical image, the plurality of actions comprises actions corresponding to adjustments to parameters of the deformation model, and registering the plurality of medical images using final transformation parameters resulting from the plurality of iterations comprises:
 generating a dense deformation field corresponding to final parameters of the deformation model resulting from the plurality of iterations; and   warping the second medical image to register the second medical image with the first medical image by applying the dense deformation field corresponding to the final parameters of the deformation model.   
     
     
         39 . The non-transitory computer readable medium of  claim 30 , wherein repeating the determining, calculating, and selecting operations for a plurality of iterations comprises:
 repeating the determining, calculating, and selecting operations using a first image resolution of the plurality of medical images and a first machine learning based model trained using training images at the first image resolution until a first stop condition is reached; and   repeating the determining, calculating, and selecting operations using a second image resolution of the plurality of medical images and a second machine learning based model trained using training images at the second image resolution until a second stop condition is reached.