Background maintenance of an image sequence

Abstract

An image background maintenance system and method for maintaining a background model of an image sequence by processing on multiple spatial scales. These multiple spatial scales include a pixel scale, a regional scale and a frame scale. The image sequence undergoes pixel processing that determines a current background model and provides an initial pixel assignment as either a background or a foreground pixel. Region processing further refines the initial pixel assignments by considering relationships between pixels and possibly reassigning pixels. Frame processing further refines the current background model by determining whether a substantial change has occurred in the actual background and, if so, providing a more accurate background model.

1. An image sequence background maintenance system for maintaining a background model of an image sequence containing a current frame and previous frames, comprising:
 a general-purpose computing device; 
 a computer-readable storage medium having stored and encoded thereon a computer program having program modules containing computer-executable instructions that are executable by the general-purpose computing device, the computer program further comprising: 
 a stored model module that contains a plurality of stored background models; and 
 a background model module that uses the current frame to determine which of a current background model and the plurality of stored background models is a best background model; 
 wherein the background model module determines the best background model by finding the background model having a lowest percentage of foreground pixels. 
 
   
   
     2. The image sequence background maintenance system of  claim 1 , wherein the stored model module further comprises a frequency module that determines the frequency between testing for replacement of the plurality of stored background models. 
   
   
     3. The image sequence background maintenance system of  claim 2 , wherein an update frequency of the plurality of stored background models is determined by a percentage of a number of frames required for a pixel to regain stationarity. 
   
   
     4. The image sequence background maintenance system of  claim 3 , wherein the update frequency has a period of approximately 10 seconds. 
   
   
     5. The image sequence background maintenance system of  claim 2 , wherein the frequency module determines whether there are enough stored background models. 
   
   
     6. The image sequence background maintenance system of  claim 5 , wherein the frequency module selects a background model such that the plurality of background models is maximally diverse. 
   
   
     7. The image sequence background maintenance system of  claim 2 , wherein the stored model module further comprises a benefit module that determines which of the plurality of stored background models should be replaced. 
   
   
     8. The image sequence background maintenance system of  claim 7 , wherein the benefit module replaces the plurality of stored background models so as to maintain a maximally diverse set of stored background models. 
   
   
     9. The image sequence background maintenance system of  claim 8 , wherein the benefit module uses a clustering technique to determine which of the plurality of stored background models to replace. 
   
   
     10. A computer-readable storage medium having stored and encoded thereon a computer program having computer-executable modules containing computer-executable instructions that are executable by a general-purpose computing device, the computer program comprising:
 a stored model module that contains a plurality of stored background models; and 
 a background model module that determines which of a current background model and the plurality of stored background models best represents a background of a current frame; 
 wherein the determination of the best representation of the current frame background is made by finding the background model containing a lowest percentage of foreground pixels. 
 
   
   
     11. The apparatus of  claim 10 , wherein the stored module comprises a frequency module that determines an update frequency of the plurality of stored background models. 
   
   
     12. The apparatus of  claim 11 , wherein the update frequency is determined by a percentage of a number of frames required for a pixel to regain stationarity. 
   
   
     13. The apparatus of  claim 11 , wherein the frequency module determines whether there are enough stored background models. 
   
   
     14. The apparatus of  claim 13 , wherein the determination is made by the frequency module to provide a maximally diverse set of stored background models. 
   
   
     15. The apparatus of  claim 11 , wherein the stored model module further comprises a benefit module that determines which of the plurality of stored background models to replace. 
   
   
     16. The apparatus of  claim 10 , wherein the foreground pixels are determined using a pixel processing system that provides a classification of a pixel as either a background pixel or a foreground pixel. 
   
   
     17. A computer-implemented method that is implemented on a general-purpose computer device for maintaining a background model of an image sequence, the image sequence having a current frame and a previous frame, comprising:
 using the general-purpose computer device to perform the following: 
 create a maximally diverse set of stored background models; 
 determine whether a current background model adequately represents an actual background of the current frame by finding a background model containing a lowest percentage of foreground pixels; and 
 substitute one of the set of stored background models in place of the current background model if the current background model does not provide adequate representation.