Method and system for imaging and analysis of anatomical features

Abstract

A method and system are provided for characterizing a portion of biological tissue. This invention comprises a smartphone and tablet deployable mobile medical application that uses device sensors, internet connectivity and cloud-based image processing to document and analyze physiological characteristics of hand arthritis. The application facilitates image capture and performs image processing that identifies hand fiduciary features and measures hand anatomical features to report and quantify the progress of arthritic disease.

1 . A system for measuring anatomical features of the hand comprising: an image capture device capable of capturing a color digital image of a splayed hand disposed in front of a known background object, a system processor for analyzing the digital image to locate anatomical features and background features, determining the dimensions of the anatomical features, and providing a report on the anatomical dimensions that can used to assess the condition of the hand, and a data repository for storing the information as an electronic health record. 
     
     
         2 . A method for measuring anatomical features of the hand comprising: capturing a color digital image of a splayed hand disposed in front of a known background, analyzing the digital image to locate anatomical features and background features, determining the dimensions of the anatomical features, and providing a report on the anatomical dimensions that can used to assess the condition of the hand. 
     
     
         3 . The method of  claim 2  where the condition being assessed is arthritis of the hand. 
     
     
         4 . The method of  claim 2  where the condition being assessed is at least one of osteoarthritis, rheumatoid arthritis, or inflammatory arthritis, of the hand. 
     
     
         5 . The method of  claim 2  where analyzing the image further comprises correcting the image for distortions in the image of the hand created by the image capture device. 
     
     
         6 . The method of  claim 5  where the distortion is at least one of spatial distortion or color balance distortion. 
     
     
         7 . The method of  claim 5  where the distortion is perspective distortion caused by the angle of the image capture device relative to the hand and background. 
     
     
         8 . The method of  claim 6  where the spatial distortion is corrected by measuring the shape of the known background in the image, comparing it to the known values for the background, and adjusting the image to correct the spatial distortion. 
     
     
         9 . The method of  claim 6  where the color balance distortion is corrected by measuring color of the known background in the image, comparing it to the known values for the background, and adjusting the image to correct the color distortion. 
     
     
         10 . The method of  claim 2  where analyzing the image further comprises segmenting the digital image data representing the hand from the background image data. 
     
     
         11 . The method of  claim 10  where the method of image segmentation further comprises conversion of the image from a red, green, blue color image to a luminance-chrominance color image. 
     
     
         12 . The method of  claim 11  where the method of image segmentation further comprises using K-means clustering of the chrominance information of the image to segment the hand pixels from the background. 
     
     
         13 . The method of  claim 10  where the segmented image of the hand is a binary image. 
     
     
         14 . The method of  claim 13  where the boundary pixels of the binary image of the hand are determined and are recorded as a sequential array of Cartesian coordinates that trace the boundary of the hand. 
     
     
         15 . The method of  claim 14  where the boundary pixels are further analyzed to determine the location of anatomical fiduciary points in the image. 
     
     
         16 . The method of  claim 15  where the fiduciary points are the location of the tips of the fingers in the image. 
     
     
         17 . The method of  claim 16  where the fiduciary points are the location of the base of the fingers in the image. 
     
     
         18 . The method of  claim 14  where the boundary pixels are further analyzed to determine the width of the fingers in the image. 
     
     
         19 . The method of  claim 14  where the boundary pixels are further analyzed to determine the centerline of the fingers in the image. 
     
     
         20 . The method of  claim 19  where the centerline of the fingers in the hand are analyzed to determine the amount of angular deviation at the joints of the hand. 
     
     
         21 . A system for assessment and monitoring of joint abnormalities in a subject caused by disease or injury, wherein said system captures and analyzes an image of an affected anatomical region of said subject where one or more joints are located, said system comprising:
 (a) a mobile device comprising a camera for capturing a digital image of said affected anatomical region and a reference object of known dimensions;   (b) a mobile application executable by said mobile device and configured to collect sensor data relating to said digital image; and   (c) processing software executable by a processor and configured to analyze said sensor data to determine anatomical measurements of said affected anatomical region relevant to said assessment and monitoring of joint abnormalities, said measurements comprising the dimensions of said one or more joints and the amount of angular deviation of anatomical structures at said one or more joints.   
     
     
         22 . The system as defined in  claim 21 , wherein said subject is a human being, said affected anatomical region is a hand or foot of said subject, and said joint abnormalities are caused by arthritis. 
     
     
         23 . The system as defined in  claim 21 , wherein said mobile device is a smartphone or tablet computer. 
     
     
         24 . The system as defined in  claim 23 , wherein said mobile device has Internet connectivity and wherein said processor is a cloud processing server. 
     
     
         25 . The system as defined in  claim 21 , wherein said sensor data comprises at least one of image data, camera data, ambient light sensor data, orientation sensor data, geographic location data, and time and date data. 
     
     
         26 . The system as defined in  claim 21 , wherein at least one of said mobile application and said processing software is configured to determine whether image capture conditions of said digital image are acceptable for further processing of said digital image. 
     
     
         27 . The system as defined in  claim 21 , wherein said mobile application is configured to provide an interface enabling a user of said mobile device to input biographical and environmental data relevant to said subject, wherein said biographical and environmental data comprises at least one of biographical information, experiential measures of pain and movement, medication use, weather and regional demographics. 
     
     
         28 . The system as defined in  claim 27 , comprising electronic health record software executable on a cloud database server for storing and retrieving said anatomical measurements and said biographical and environmental data specific to said subject. 
     
     
         29 . The system as defined in  claim 21 , wherein said processing software is configured to produce a report indicating the condition of said one or more joints based on said anatomical measurements. 
     
     
         30 . The system as defined in  claim 24 , comprising a cloud database server operable to store and retrieve reference measurements, wherein said processing software is configured to compare said anatomical measurements to said reference measurements. 
     
     
         31 . The system as defined in  claim 24  wherein said processing software executable on said cloud processing server is configured to perform operations to determine said anatomical measurements, said operations comprising:
 (a) analyzing said sensor data to make adjustments to said digital image, wherein said adjustments comprise correcting a distortion of said image selected from the group consisting of a spatial distortion, a color balance distortion, and a perspective distortion; 
 (b) segmenting a calibrated image of said affected anatomical region from a background of said digital image to provide a segmented image; 
 (c) determining a boundary of said affected anatomical region and anatomical fiduciary points in said segmented image; and 
 (d) calculating said anatomical measurements based on said boundary and said fiduciary points. 
 
     
     
         32 . A method for assessment and monitoring of joint abnormalities in a subject caused by disease or injury, comprising:
 (a) capturing a digital image of an affected anatomical region of said subject where one or more joints are located and a reference object of known dimensions, wherein said image is captured using a mobile device comprising a camera;   (b) collecting sensor data from said mobile device relating to said digital image; and   (c) analyzing said sensor data to determine anatomical measurements of said affected anatomical region relevant to said assessment and monitoring of joint abnormalities, said measurements comprising the dimensions of said one or more joints and the amount of angular deviation of anatomical structures at said one or more joints.   
     
     
         33 . The method of  claim 32 , wherein said subject is a human being, said affected anatomical region is a hand or foot of said subject, and wherein said joint abnormalities are caused by arthritis. 
     
     
         34 . The method of  claim 32 , wherein said mobile device is a smartphone or tablet computer and wherein said method comprises transmitting said sensor data from said mobile device to a cloud processing server. 
     
     
         35 . The method as defined in  claim 32 , wherein said sensor data comprises at least one of image data, camera data, ambient light sensor data, orientation sensor data, geographic location data, and time and date data. 
     
     
         36 . The method as defined in  claim 32 , comprising determining, prior to analyzing said sensor data to determine said anatomical measurements, whether image capture conditions of said digital image are acceptable for further processing, and, if not, capturing a replacement digital image of said affected anatomical region and said reference object. 
     
     
         37 . The method as defined in  claim 32 , comprising collecting biographical and environmental data relevant to said subject using said mobile device, wherein said biographical and environmental data comprises at least one of biographical information, experiential measures of pain and movement, medication use, weather and regional demographics. 
     
     
         38 . The method as defined in  claim 37 , comprising providing a cloud database server for storing and retrieving an electronic health record specific to said subject comprising said anatomical measurements and said biographical and environmental data. 
     
     
         39 . The method as defined in  claim 32 , comprising providing a report indicating the condition of said one or more joints based on said anatomical measurements. 
     
     
         40 . The method as defined in  claim 32 , comprising comparing said anatomical measurements to reference measurements. 
     
     
         41 . The method as defined in  claim 40 , wherein said reference measurements are prior measurements of said one or more joints of said subject, wherein said comparing enables monitoring of any changes to said one or more joints over time. 
     
     
         42 . The method as defined in  claim 37 , comprising comparing said anatomical measurements and said biographical and environmental data to historical reference data, wherein said historical reference data comprises prior anatomical measurements and biographical and environmental data specific to said subject, wherein said comparing enables monitoring of any changes to said one or more joints relative to any changes to said biographical and environmental data over time. 
     
     
         43 . The method as defined in  claim 42 , wherein said monitoring provides a measure of any correlations between said changes to said one or more joints and biographical or environmental factors, wherein said factors are selected from the group consisting of treatment received by said subject, medication received by said subject, physical activity of said subject, geographic location of said subject and weather conditions at said geographic location of said subject. 
     
     
         44 . The method as defined in  claim 32 , wherein said analyzing comprises:
 (a) analyzing said sensor data to make adjustments to said digital image, wherein said adjustments comprise correcting a distortion of said image selected from the group consisting of a spatial distortion, a color balance distortion, and a perspective distortion;   (b) segmenting a calibrated image of said affected anatomical region from a background of said digital image to provide a segmented image;   (c) determining in said segmented image a boundary of said affected anatomical region and anatomical fiduciary points within said boundary; and   (d) calculating said anatomical measurements based on said boundary and said fiduciary points.   
     
     
         45 . The method as defined in  claim 44 , wherein said perspective distortion is corrected by determining the angle of said camera relative to said affected anatomical region and said reference object when said image is captured and adjusting said image to correct said perspective distortion. 
     
     
         46 . The method as defined in  claim 44 , wherein said spatial distortion is corrected by measuring the size and/or shape of said reference object in said image to determine spatial measurements of said object, comparing said spatial measurements to known values for said object, and adjusting said image to correct said spatial distortion. 
     
     
         47 . The method as defined in  claim 44 , wherein said color balance distortion is corrected by measuring the color of said reference object to determine color measurements, comparing said color measurements to known values for said object, and adjusting said image to correct said color distortion. 
     
     
         48 . The method as defined in  claim 44 , wherein said segmenting comprises using K-means clustering of chrominance information for said image to segment pixels corresponding to said affected anatomical region from background image data. 
     
     
         49 . The method of  claim 32 , wherein said reference object is a paper sheet of known dimensions and color. 
     
     
         50 . The method of  claim 36 , wherein said determining whether said image capture conditions of said digital image are acceptable comprises analyzing said orientation sensor data to determine whether the orientation of said camera relative to said affected anatomical region is at a desirable angle. 
     
     
         51 . The method of  claim 36 , wherein said determining whether image capture conditions of said digital image are acceptable comprises analyzing said ambient light sensor data to determine whether illumination of said affected anatomical region is suitable. 
     
     
         52 . The method of  claim 36 , wherein said determining whether image capture conditions of said digital image are acceptable comprises determining if said reference object is within the field of view. 
     
     
         53 . The method of  claim 44 , wherein said affected anatomical region is a hand and wherein said calculating said anatomical measurements comprises determining the length and location of segments of a finger extending between said one or more joints and determining the relative angle of said segments, wherein determining said relative angle provides a measure of joint breakdown causing deviation of said segments away from an anatomically normal orientation. 
     
     
         54 . A system for assessment and monitoring of joint abnormalities in a subject caused by disease or injury, wherein said system captures and analyzes an image of an affected anatomical region of said subject where one or more joints are located, said system comprising:
 (a) a mobile device comprising a camera for capturing a single digital image of said affected anatomical region and a reference object of known dimensions;   (b) a mobile application executable by said mobile device and configured to collect sensor data relating to said single digital image; and   (c) processing software executable by a processor and configured to analyze said sensor data to determine anatomical measurements of said affected anatomical region relevant to said assessment and monitoring of joint abnormalities, said measurements comprising the dimensions of said one or more joints and the amount of angular deviation of anatomical structures at said one or more joints.   
     
     
         55 . A system for assessment and monitoring of joint abnormalities in a subject caused by disease or injury, wherein said system captures and analyzes an image of an affected anatomical region of said subject where one or more joints are located, said system comprising:
 (a) a mobile device comprising a camera for capturing a digital image of said affected anatomical region and a reference object of known dimensions positioned independently of said affected anatomical region;   (b) a mobile application executable by said mobile device and configured to collect sensor data relating to said digital image; and   (c) processing software executable by a processor and configured to analyze said sensor data to determine anatomical measurements of said affected anatomical region relevant to said assessment and monitoring of joint abnormalities, said measurements comprising the dimensions of said one or more joints and the amount of angular deviation of anatomical structures at said one or more joints.