Tracking articles of interest in monitored areas

Abstract

Systems, methods, and computer program products for monitoring an area. A data collection and analysis system includes a plurality of data collection devices. Each data collection device includes an imaging unit that captures images of the area. The system receives time-stamped images from the imaging units and identifies the articles of interest in the images. The system further determines a spatial location of each article of interest identified based at least in part on a location of the article of interest in the image and the location of the portion of the area in the image. Each article of interest is associated with its spatial location and the time stamp of the image in which it was identified in a database. In response to receiving a query associated with an article of interest at the database, the system displays a timeline for the article of interest.

What is claimed is: 
     
         1 . A system for monitoring an area, comprising:
 one or more imaging units each configured to capture images of at least a portion the area, an article of interest in the area, or both the at least a portion of the area and the article of interest;   one or more processors in communication with the one or more imaging units; and   a memory coupled to the one or more processors and including program code that, when executed by the one or more processors, causes the system to:   receive a first image of a plurality of images of the area, each of the plurality of images including a time stamp;   identify a first article of interest in the first image;   determine a spatial location of the first article of interest based at least in part on an image location of the first article of interest in the first image;   associate the first article of interest with the time stamp of the first image and the spatial location in a database; and   in response to receiving a query associated with the first article of interest at the database, display a timeline for the first article of interest.   
     
     
         2 . The system of  claim 1 , wherein the plurality of images includes both visible light images and non-visible light images. 
     
     
         3 . The system of  claim 2 , wherein the first image is a visible light image, the first article of interest corresponds to a personal item, and the program code further causes the system to:
 identify a second article of interest in the first image corresponding to a visual identifier tag that identifies a bin holding the first article of interest;   receive a second image of the plurality of images of the area, the second image being a non-visible light image;   identify a third article of interest in the second image corresponding to a non-visual identifier tag that identifies the bin holding the first article of interest; and   determine the first article of interest in the first image corresponds to the first article of interest in the second image based at least in part on both the visual identifier tag and the non-visual identifier tag being associated with the bin holding the first article of interest.   
     
     
         4 . The system of  claim 1 , wherein the program code further causes the system to:
 determine if the first article of interest has been previously identified in at least one other image of the plurality of images;   in response to the first article of interest having been previously identified in the at least one other image, associate the first article of interest in the first image with a first identifier previously associated with the first article of interest;   in response to the first article of interest not having been previously identified, generate a second identifier different from each identifier associated with one or more other previously identified articles of interest, and associate the first article of interest in the first image with the second identifier.   
     
     
         5 . The system of  claim 4 , wherein the program code causes the system to determine if the first article of interest has been previously identified in the at least one other image of the plurality of images by:
 calculating a first feature descriptor for the first article of interest;   determining if the first feature descriptor matches any previously calculated feature descriptor;   if the first feature descriptor matches any previously calculated feature descriptors, determining the first article of interest has been previously identified in the at least one other image; and   if the first feature descriptor does not match any previously calculated feature descriptors, determining the first article of interest has not been previously identified in the at least one other image.   
     
     
         6 . The system of  claim 1 , wherein the program code causes the system to identify the first article of interest in the first image by:
 calculating a feature descriptor for each of a plurality of image segments of the first image; and   classifying the feature descriptor of a first image segment of the plurality of image segments as belonging to a class of articles including the first article of interest.   
     
     
         7 . The system of  claim 6 , wherein the program code further causes the system to:
 display a bounding box around the first image segment in the first image; and   transmit the query associated with the first article of interest to the database in response to a user selecting the bounding box.   
     
     
         8 . A method of monitoring an area, comprising:
 receiving a first image of a plurality of images of the area, each of the plurality of images including a time stamp;   identifying a first article of interest in the first image;   determining a spatial location of the first article of interest based at least in part on an image location of the first article of interest in the first image;   associating the first article of interest with the time stamp of the first image and the spatial location in a database; and   in response to receiving a query associated with the first article of interest at the database, displaying a timeline for the first article of interest.   
     
     
         9 . The method of  claim 8 , wherein the plurality of images includes both visible light images and non-visible light images. 
     
     
         10 . The method of  claim 9 , wherein the first image is a visible light image, the first article of interest corresponds to a personal item, and further comprising:
 identifying a second article of interest in the first image corresponding to a visual identifier tag that identifies a bin holding the first article of interest;   receiving a second image of the plurality of images of the area, the second image being a non-visible light image;   identifying a third article of interest in the second image corresponding to a non-visual identifier tag that identifies the bin holding the first article of interest; and   determining the first article of interest in the first image corresponds to the first article of interest in the second image based at least in part on both the visual identifier tag and the non-visual identifier tag being associated with the bin holding the first article of interest.   
     
     
         11 . The method of  claim 8 , further comprising:
 determining if the first article of interest has been previously identified in at least one other image of the plurality of images;   in response to the first article of interest having been previously identified in the at least one other image, associating the first article of interest in the first image with a first identifier previously associated with the first article of interest;   in response to the first article of interest not having been previously identified, generating a second identifier different from each identifier associated with one or more other previously identified articles of interest, and associating the first article of interest in the first image with the second identifier.   
     
     
         12 . The method of  claim 11 , wherein determining if the first article of interest has been previously identified in the at least one other image of the plurality of images comprises:
 calculating a first feature descriptor for the first article of interest;   determining if the first feature descriptor matches any previously calculated feature descriptor;   if the first feature descriptor matches any previously calculated feature descriptors, determining the first article of interest has been previously identified in the at least one other image; and   if the first feature descriptor does not match any previously calculated feature descriptors, determining the first article of interest has not been previously identified in the at least one other image.   
     
     
         13 . The method of  claim 8 , wherein identifying the first article of interest in the first image comprises:
 calculating a feature descriptor for each of a plurality of image segments of the first image; and   classifying the feature descriptor of a first image segment of the plurality of image segments as belonging to a class of articles including the first article of interest.   
     
     
         14 . The method of  claim 13 , further comprising:
 displaying a bounding box around the first image segment in the first image; and   transmitting the query associated with the first article of interest to the database in response to a user selecting the bounding box.   
     
     
         15 . A computer program product for monitoring an area, comprising:
 a non-transitory computer-readable storage medium; and   program code stored on the non-transitory computer-readable storage medium that, when executed by one or more processors, causes the one or more processors to:   receive a first image of a plurality of images of the area, each of the plurality of images including a time stamp;   identify a first article of interest in the first image;   determine a spatial location of the first article of interest based at least in part on an image location of the first article of interest in the first image;   associate the first article of interest with the time stamp of the first image and the spatial location in a database; and   in response to receiving a query associated with the first article of interest at the database, display a timeline for the first article of interest.