Computer based object detection within a video or image

Abstract

Described herein are software and systems for analyzing videos and/or images. Software and systems described herein are configured in different embodiments to carry out different types of analyses. For example, in some embodiments, software and systems described herein are configured to locate an object of interest within a video and/or image.

What is claimed is: 
     
         1 . A computer-based method for identifying an object of interest or factor of interest within a video, the method comprising:
 (a) inputting the video comprising a plurality of frames into a software module;   (b) generating a feature map from a frame of the plurality of frames with the software module; and   (c) analyzing the feature map using a statistical technique thereby identifying the object of interest or factor of interest within the video.   
     
     
         2 . The method of  claim 1 , wherein the software module comprises a deep neural network. 
     
     
         3 . The method of  claim 2 , wherein the feature map comprises data from a hidden layer or an output layer of the deep neural network. 
     
     
         4 . The method of  claim 2 , wherein the deep neural network comprises at least one of VGG-19, ResNet, Inception, and MobileNet. 
     
     
         5 . The method of  claim 1 , wherein the factor of interest comprises at least one of a location of a pixel within the frame and an angle within the frame. 
     
     
         6 . The method of  claim 1 , wherein the statistical technique comprises Monte Carlo Sampling, and wherein the Monte Carlo sampling is used to generate sample locations of the object of interest within the feature map. 
     
     
         7 . The method of  claim 6 , wherein the statistical technique further comprises Bayesian modeling, and wherein the Bayesian modeling is used to model a change in a location of the object of interest within the frame to a different location of the object of interest within a different frame of the plurality of frames. 
     
     
         8 . The method of  claim 7 , comprising identifying a position of the object of interest within the frame relative to a different object of interest within the frame. 
     
     
         9 . The method of  claim 1 , wherein the factor of interest comprises an angle. 
     
     
         10 . The method of  claim 1 , wherein the object of interest comprises a joint of a body of an individual. 
     
     
         11 . The method of  claim 10 , wherein the joint comprises a shoulder, elbow, hip, knee, or ankle. 
     
     
         12 . The method of  claim 11 , wherein the video captures the individual within the frame. 
     
     
         13 . The method of  claim 12 , wherein the video captures a factor of interest from the frame to a different frame within the plurality of frames. 
     
     
         14 . The method of  claim 13 , wherein the factor of interest comprises a movement of a joint. 
     
     
         15 . The method of  claim 14 , wherein the movement of the joint is measured relative to a different joint of the body of the individual and is expressed as an angle. 
     
     
         16 . The method of  claim 15 , wherein the angle is used by a healthcare provider to evaluate the individual. 
     
     
         17 . A computer-based system for identifying an object of interest or a factor of interest within a video, the system comprising:
 (a) a processor;   (b) a non-transitory medium comprising a computer program configured to cause the processor to:
 (i) input the video comprising a plurality of frames into a software module; 
 (ii) generate a feature map using the software module; and 
 (iii) analyze the feature map using a statistical technique thereby identifying the object of interest or the factor of interest within the video. 
   
     
     
         18 . The system of  claim 17 , wherein the software module comprises a deep neural network. 
     
     
         19 . The system of  claim 18 , wherein the feature map comprises data from a hidden layer or an output layer of the deep neural network. 
     
     
         20 . The system of  claim 18 , wherein the deep neural network comprises at least one of VGG-19, ResNet, Inception, and MobileNet. 
     
     
         21 . The system of  claim 17 , wherein the factor of interest comprises at least one of a pixel within the frame and an angle within the frame. 
     
     
         22 . The system of  claim 18 , wherein the statistical technique comprises Monte Carlo sampling, and wherein the Monte Carlo sampling is used to generate sample locations of the object of interest within the feature map. 
     
     
         23 . The system of  claim 22 , wherein the statistical technique further comprises Bayesian modeling, and wherein the Bayesian modeling is used to model a change in a location of the object of interest within the frame to a different location of the object of interest within a different frame of the plurality of frames. 
     
     
         24 . The system of  claim 23 , comprising identifying a position of the object of interest within the frame relative to a different object of interest within the frame. 
     
     
         25 . The system of  claim 17 , wherein the factor of interest comprises an angle. 
     
     
         26 . The system of  claim 17 , wherein the object of interest comprises a joint of a body of an individual. 
     
     
         27 . The system of  claim 26 , wherein the joint comprises a shoulder, elbow, hip, knee, or ankle. 
     
     
         28 . The system of  claim 27 , wherein the video captures the individual within the frame. 
     
     
         29 . The system of  claim 28 , wherein the video captures a factor of interest from the frame to a different frame within the plurality of frames. 
     
     
         30 . The system of  claim 29 , wherein the factor of interest comprises a movement of a joint. 
     
     
         31 . The system of  claim 30 , wherein the movement of the joint is measured relative to a different joint of the body of the individual and is expressed as an angle. 
     
     
         32 . The system of  claim 31 , wherein the angle is used by a healthcare provider to evaluate the individual. 
     
     
         33 . A non-transitory medium comprising a computer program configured to:
 (i) input a video comprising a plurality of frames into a software module;   (ii) generate a feature map using the software module; and   (iii) analyze the feature map using a statistical technique thereby identifying the object of interest or the factor of interest within the video.   
     
     
         34 . The medium of  claim 33 , wherein the software module comprises a deep neural network. 
     
     
         35 . The medium of  claim 34 , wherein the feature map comprises data from a hidden layer or an output layer of the deep neural network. 
     
     
         36 . The medium of  claim 34 , wherein the deep neural network comprises at least one of VGG-19, ResNet, Inception, and MobileNet. 
     
     
         37 . The medium of  claim 33 , wherein the factor of interest comprises at least one of a pixel within the frame and an angle within the frame. 
     
     
         38 . The medium of  claim 33 , wherein the statistical technique comprises Monte Carlo sampling, and wherein the Monte Carlo sampling is used to sample the likelihood of the presence of the object of interest within the feature map. 
     
     
         39 . The medium of  claim 38 , wherein the statistical technique further comprises Bayesian modeling, and wherein the Bayesian modeling is used to generate sample locations of the object of interest within the frame to a different location of the object of interest within a different frame of the plurality of frames. 
     
     
         40 . The medium of  claim 39 , comprising identifying a position of the object of interest within the frame relative to a different object of interest within the frame. 
     
     
         41 . The medium of  claim 33 , wherein the feature of interest comprises an angle. 
     
     
         42 . The medium of  claim 33 , wherein the object of interest comprises a joint of a body of an individual. 
     
     
         43 . The medium of  claim 42 , wherein the joint comprises a shoulder, elbow, hip, knee, or ankle. 
     
     
         44 . The medium of  claim 43 , wherein the video captures the individual within the frame. 
     
     
         45 . The medium of  claim 44 , wherein the video captures a factor of interest from the frame to a different frame within the plurality of frames. 
     
     
         46 . The medium of  claim 45 , wherein the factor of interest comprises a movement of a joint. 
     
     
         47 . The medium of  claim 46 , wherein the movement of the joint is measured relative to a different joint of the body of the individual and is expressed as an angle. 
     
     
         48 . The medium of  claim 47 , wherein the angle is used by a healthcare provider to evaluate the individual.