Methods and Systems for Accelerated Rreading of a 3D Medical Volume

Abstract

Methods and apparatus for automatically visualizing 3D medical image data to provide accelerated reading of the 3D medical image data are disclosed. A 3D medical volume is received. A synopsis volume or a tapestry image is generated from the 3D medical volume. A synopsis volume is a spatially compact volume that is created from the 3D medical image volume and contains target anatomical structures related to a particular clinical task. A tapestry image is a single 2D image that visualizes a combination of multiple 2D views of one or more target anatomical objects related to a particular clinical task.

1 . A method for automatically visualizing 3D medical image data to provide accelerated reading of the 3D medical image data, comprising:
 receiving a 3D medical volume;   segmenting a plurality of target anatomical structures in the 3D medical image volume;   automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures; and   displaying the synopsis volume.   
     
     
         2 . The method of  claim 1 , wherein the segmented plurality of target anatomical structures are repositioned with respect to each other in the synopsis volume. 
     
     
         3 . The method of  claim 1 , wherein automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures comprises:
 defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures; and   generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume.   
     
     
         4 . The method of  claim 3 , wherein generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume comprises:
 horizontally or vertically aligning the plurality of regions of interest in the synopsis volume.   
     
     
         5 . The method of  claim 3 , wherein generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume comprises:
 determining an optimal alignment of the plurality of regions of interest to minimize a size of the synopsis volume.   
     
     
         6 . The method of  claim 3 , wherein defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures comprises:
 for each of the plurality of regions of interest, defining the region of interest as a set of voxels included in the corresponding segmented target anatomical structure.   
     
     
         7 . The method of  claim 3 , wherein defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures comprises, for each of the plurality of regions of interest:
 determining a bounding box enclosing the corresponding segmented target anatomical structure in the 3D medical volume, wherein the bounding box enclosing the corresponding segmented target anatomical structure defines the region of interest.   
     
     
         8 . The method of  claim 3 , wherein defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures comprises, for each of the plurality of regions of interest:
 determining a bounding box enclosing the corresponding segmented target anatomical structure in the 3D medical volume; and   expanding the bounding box, wherein the expanded bounding box defines the region of interest.   
     
     
         9 . The method of  claim 3 , wherein generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume comprises:
 re-positioning the plurality of regions of interest with respect to each other, while preserving voxel intensities within the plurality of regions of interest.   
     
     
         10 . The method of  claim 3 , wherein automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures further comprises:
 processing the synopsis volume to smooth boundaries between the aligned plurality of regions of interest in the synopsis volume.   
     
     
         11 . The method of  claim 3 , wherein the plurality of target anatomical structures comprises the liver and kidneys, and generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume comprises:
 horizontally aligning a first region of interest corresponding to the segmented liver and a second region of interest corresponding to the segmented kidneys in the synopsis volume.   
     
     
         12 . A method for automatically visualizing 3D medical image data to provide accelerated reading of the 3D medical image data, comprising:
 receiving a 3D medical volume;   detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume;   automatically generating a 2D tapestry image that visualizes a combination of the plurality of relevant 2D views of the at least one target anatomical object; and   displaying the 2D tapestry image.   
     
     
         13 . The method of  claim 12 , wherein detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume comprises:
 segmenting a target anatomical structure in the 3D medical volume;   generating a set of 2D views of the target anatomical structure by cropping regions of interest of the segmented target anatomical structure in a plurality of 2D slices of the 3D medical volume; and   selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure.   
     
     
         14 . The method of  claim 13 , wherein selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure comprises:
 selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure using a predetermined sampling pattern.   
     
     
         15 . The method of  claim 13 , wherein selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure comprises:
 detecting a substructure in each 2D view in the set of 2D views of the target anatomical structure;   calculating a score for each 2D view in the set of 2D views of the target anatomical structure based on an amount of the substructure detected; and   selecting the plurality of relevant 2D views by selecting a number of 2D views having highest scores from the set of 2D views of the target anatomical structure.   
     
     
         16 . The method of  claim 15 , wherein automatically generating a 2D tapestry image that visualizes a combination of the plurality of relevant 2D views of the at least one target anatomical object comprises:
 ordering the plurality of relevant 2D views in the 2D tapestry image based on the scores calculated for the plurality of relevant 2D views.   
     
     
         17 . The method of  claim 13 , wherein the target anatomical structure is the liver. 
     
     
         18 . The method of  claim 17 , wherein selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure comprises:
 detecting vessels in each 2D view in the set of 2D views of the liver using a trained vesselness classifier;   calculating a vesselness score for each 2D view in the set of 2D views of the liver based on vesselness probabilities of pixels in each 2D view calculated by the trained vesselness classifier; and   selecting the plurality of relevant 2D views by selecting a number of 2D views having highest vesselness scores from the set of 2D views of the liver.   
     
     
         19 . The method of  claim 12 , wherein detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume comprises:
 detecting a plurality of target anatomical objects in the 3D medical volume; and   extracting at least one relevant 2D view of each of the plurality of target anatomical objects detected in the 3D medical volume.   
     
     
         20 . The method of  claim 19 , wherein detecting a plurality of target anatomical objects in the 3D medical volume comprises:
 detecting a plurality of liver lesions in the 3D medical volume.   
     
     
         21 . The method of  claim 20 , wherein extracting at least one relevant 2D view of each of the plurality of target anatomical objects detected in the 3D medical volume comprises:
 extracting an axial view of each of the plurality of liver lesions detected in the 3D medical volume.   
     
     
         22 . The method of  claim 12 , further comprising:
 storing a geometrical relationship between each pixel in the 2D tapestry image and a corresponding originating voxel in the 3D medical volume.   
     
     
         23 . The method of  claim 22 , further comprising:
 displaying the 3D medical volume;   receiving a user selection of a pixel in the 2D tapestry image; and   in response to receiving the user selection of the pixel in the 2D tapestry image, adjusting the displayed 3D medical volume to automatically navigate to the corresponding originating voxel in the 3D medical volume.   
     
     
         24 . An apparatus for automatically visualizing 3D medical image data to provide accelerated reading of the 3D medical image data, comprising:
 means for receiving a 3D medical volume;   means for segmenting a plurality of target anatomical structures in the 3D medical image volume;   means for automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures; and   means for displaying the synopsis volume.   
     
     
         25 . The apparatus of  claim 24 , wherein the segmented plurality of target anatomical structures are repositioned with respect to each other in the synopsis volume. 
     
     
         26 . The apparatus of  claim 24 , wherein the means for automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures comprises:
 means for defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures; and   means for generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume.   
     
     
         27 . The apparatus of  claim 26 , wherein the plurality of target anatomical structures comprises the liver and kidneys, and the means for generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume comprises:
 means for horizontally aligning a first region of interest corresponding to the segmented liver and a second region of interest corresponding to the segmented kidneys in the synopsis volume.   
     
     
         28 . An apparatus for automatically visualizing 3D medical image data to provide accelerated reading of the 3D medical image data, comprising:
 means for receiving a 3D medical volume;   means for detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume;   means for automatically generating a 2D tapestry image that visualizes a combination of the plurality of relevant 2D views of the at least one target anatomical object; and   means for displaying the 2D tapestry image.   
     
     
         29 . The apparatus of  claim 28 , wherein the means for detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume comprises:
 means for segmenting a target anatomical structure in the 3D medical volume;   means for generating a set of 2D views of the target anatomical structure by cropping regions of interest of the segmented target anatomical structure in a plurality of 2D slices of the 3D medical volume; and   means for selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure.   
     
     
         30 . The apparatus of  claim 29 , wherein the target anatomical structure is the liver. 
     
     
         31 . The apparatus of  claim 28 , wherein the means for detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume comprises:
 means for detecting a plurality of target anatomical objects in the 3D medical volume; and   means for extracting at least one relevant 2D view of each of the plurality of target anatomical objects detected in the 3D medical volume.   
     
     
         32 . The apparatus of  claim 31 , wherein the means for detecting a plurality of target anatomical objects in the 3D medical volume comprises:
 means for detecting a plurality of liver lesions in the 3D medical volume.   
     
     
         33 . The apparatus of  claim 28 , further comprising:
 means for displaying the 3D medical volume;   means for receiving a user selection of a pixel in the 2D tapestry image; and   means for adjusting the displayed 3D medical volume in response the user selection of the pixel in the 2D tapestry image to automatically navigate to a corresponding originating voxel in the 3D medical volume.   
     
     
         34 . A non-transitory computer readable medium storing computer program instructions for automatically visualizing 3D medical image data to provide accelerated reading of the 3D medical image data, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
 receiving a 3D medical volume;   segmenting a plurality of target anatomical structures in the 3D medical image volume;   automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures; and   displaying the synopsis volume.   
     
     
         35 . The non-transitory computer readable medium of  claim 34 , wherein the segmented plurality of target anatomical structures are repositioned with respect to each other in the synopsis volume. 
     
     
         36 . The non-transitory computer readable medium of  claim 34 , wherein automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures comprises:
 defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures; and   generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume.   
     
     
         37 . The non-transitory computer readable medium of  claim 36 , wherein defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures comprises, for each of the plurality of regions of interest:
 determining a bounding box enclosing the corresponding segmented target anatomical structure in the 3D medical volume, wherein the bounding box enclosing the corresponding segmented target anatomical structure defines the region of interest.   
     
     
         38 . The non-transitory computer readable medium of  claim 36 , wherein defining a plurality of regions of interest in the 3D medical volume, each of the plurality of regions of interest corresponding to a respective one of the segmented plurality of target anatomical structures comprises, for each of the plurality of regions of interest:
 determining a bounding box enclosing the corresponding segmented target anatomical structure in the 3D medical volume; and   expanding the bounding box, wherein the expanded bounding box defines the region of interest.   
     
     
         39 . The non-transitory computer readable medium of  claim 36 , wherein generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume comprises:
 re-positioning the plurality of regions of interest with respect to each other, while preserving voxel intensities within the plurality of regions of interest.   
     
     
         40 . The non-transitory computer readable medium of  claim 36 , wherein automatically generating a synopsis volume that is a spatially compact volume including the segmented plurality of target anatomical structures further comprises:
 processing the synopsis volume to smooth boundaries between the aligned plurality of regions of interest in the synopsis volume.   
     
     
         41 . The non-transitory computer readable medium of  claim 36 , wherein the plurality of target anatomical structures comprises the liver and kidneys, and generating the synopsis volume by aligning the plurality of regions of interest into a spatially compact volume comprises:
 horizontally aligning a first region of interest corresponding to the segmented liver and a second region of interest corresponding to the segmented kidneys in the synopsis volume.   
     
     
         42 . A non-transitory computer readable medium storing computer program instructions for automatically visualizing 3D medical image data to provide accelerated reading of the 3D medical image data, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
 receiving a 3D medical volume;   detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume;   automatically generating a 2D tapestry image that visualizes a combination of the plurality of relevant 2D views of the at least one target anatomical object; and   displaying the 2D tapestry image.   
     
     
         43 . The non-transitory computer readable medium of  claim 42 , wherein detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume comprises:
 segmenting a target anatomical structure in the 3D medical volume;   generating a set of 2D views of the target anatomical structure by cropping regions of interest of the segmented target anatomical structure in a plurality of 2D slices of the 3D medical volume; and   selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure.   
     
     
         44 . The non-transitory computer readable medium of  claim 43 , wherein selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure comprises:
 detecting a substructure in each 2D view in the set of 2D views of the target anatomical structure;   calculating a score for each 2D view in the set of 2D views of the target anatomical structure based on an amount of the substructure detected; and   selecting the plurality of relevant 2D views by selecting a number of 2D views having highest scores from the set of 2D views of the target anatomical structure.   
     
     
         45 . The non-transitory computer readable medium of  claim 44 , wherein automatically generating a 2D tapestry image that visualizes a combination of the plurality of relevant 2D views of the at least one target anatomical object comprises:
 ordering the plurality of relevant 2D views in the 2D tapestry image based on the scores calculated for the plurality of relevant 2D views.   
     
     
         46 . The non-transitory computer readable medium of  claim 43 , wherein the target anatomical structure is the liver. 
     
     
         47 . The non-transitory computer readable medium of  claim 46 , wherein selecting the plurality of relevant 2D views from the set of 2D views of the target anatomical structure comprises:
 detecting vessels in each 2D view in the set of 2D views of the liver using a trained vesselness classifier;   calculating a vesselness score for each 2D view in the set of 2D views of the liver based on vesselness probabilities of pixels in each 2D view calculated by the trained vesselness classifier; and   selecting the plurality of relevant 2D views by selecting a number of 2D views having highest vesselness scores from the set of 2D views of the liver.   
     
     
         48 . The non-transitory computer readable medium of  claim 42 , wherein detecting a plurality of relevant 2D views of at least one target anatomical object in the 3D medical volume comprises:
 detecting a plurality of target anatomical objects in the 3D medical volume; and   extracting at least one relevant 2D view of each of the plurality of target anatomical objects detected in the 3D medical volume.   
     
     
         49 . The non-transitory computer readable medium of  claim 48 , wherein detecting a plurality of target anatomical objects in the 3D medical volume comprises:
 detecting a plurality of liver lesions in the 3D medical volume.   
     
     
         50 . The non-transitory computer readable medium of  claim 42 , wherein the operations further comprise:
 displaying the 3D medical volume;   receiving a user selection of a pixel in the 2D tapestry image; and   in response to receiving the user selection of the pixel in the 2D tapestry image, adjusting the displayed 3D medical volume to automatically navigate to a corresponding originating voxel in the 3D medical volume.