Pulsed Laser-Based Firearm Training System, and Method for Facilitating Firearm Training Using Detection of Laser Pulse Impingement of Projected Target Images

Abstract

The invention provides a method for automatic calibration and subsequent correlation of the position of a pulsed laser on a projected image in a system having a projector for projecting images onto a surface and a camera for sensing laser pulses on the surface. The automatic calibration method sets the camera exposure, allowing the system to operate in normal room-lighting conditions, and correlates camera pixel positions to projected image pixel positions by use of projected calibration images formed by sets of horizontal and vertical lines, with automatic calibration completing in less than 5 seconds. After calibration, the system determines two-dimensional camera pixel centroids of laser beam pulses on the projection surface to sub-pixel accuracy, and the calibration data is used to correlate the camera pixel centroids to the exact positions of the laser pulses on the projected image.

1 . A method for automatic calibration of a laser-based firearm training system having a projector for projecting images onto a surface and a camera for reading images projected onto the surface, comprising:
 projecting a first image comprised of horizontal and vertical lines upon the surface;   reading the first image;   adjusting an exposure setting of the camera based on the first image so as to ensure pixel values below saturation;   projecting a second image comprising a black color upon the surface;   reading the second image;   calculating a background light level value for each pixel in the second image;   projecting a third image comprising a pattern of horizontal lines upon the surface;   reading the third image;   calculating a horizontal calibration value for each pixel in the third image, wherein the resulting horizontal calibration value for each pixel is reduced by the corresponding background light level value;   projecting a fourth image comprising a pattern of vertical lines upon the surface;   reading the fourth image;   calculating a vertical calibration value for each pixel in the fourth image, wherein the resulting vertical calibration value for each pixel is reduced by the corresponding background light level value;   calculating a centroid line for each vertical and horizontal line and calculating a second order line least squares fit for each centroid line, thereby generating three coefficients that describe each vertical and horizontal line; and   generating a data map for mapping a pixel from an image read from the surface to a pixel in an image projected into the surface, wherein the data map is based on the three coefficients that describe each vertical and horizontal line.   
   
   
       2 . The method of  claim 1 , wherein the first image is set to the color white. 
   
   
       3 . The method of  claim 1 , wherein the first image is set to the color red. 
   
   
       4 . The method of  claim 1 , wherein the first step of calculating further comprises:
 storing the background light level value for each pixel in a two dimensional array.   
   
   
       5 . The method of  claim 4 , wherein the second step of calculating further comprises:
 storing the horizontal calibration value for each pixel in a two dimensional array.   
   
   
       6 . The method of  claim 5 , wherein the third step of calculating further comprises:
 storing the vertical calibration value for each pixel in a two dimensional array.   
   
   
       7 . A method for automatic detection of a laser pulse in a laser-based firearm training system having a projector for projecting images onto a surface and a camera for reading images projected onto the surface, comprising:
 reading a first projected image into a first data structure;   searching for a laser pulse in the image by searching for pixels in the first data structure with a pixel value exceeding a predefined threshold;   selecting a set of pixels around a pixel having a pixel value exceeding a predefined threshold;   calculating a centroid of the laser pulse in the first data structure by performing a two dimensional centroid calculation upon the set of pixels; and   calculating a pixel location of the laser pulse in the first image based on the centroid.   
   
   
       8 . The method of  claim 7 , wherein the set of pixels comprises a set of 16×16 pixels centered at a pixel having a pixel value exceeding a predefined threshold. 
   
   
       9 . The method of  claim 8 , wherein the centroid is a two-dimensional centroid. 
   
   
       10 . The method of  claim 9 , wherein the second step of calculating further comprises:
 calculating an X-direction centroid.   
   
   
       11 . The method of  claim 10 , wherein the step of calculating an X-direction centroid further comprises:
 multiplying each pixel in the set of pixels by an X-direction coordinate of the pixel.   
   
   
       12 . The method of  claim 11 , wherein the step of calculating an X-direction centroid further comprises:
 summing the product of multiplying each pixel in the set of pixels by an X-direction coordinate of the pixel so as to produce a sum.   
   
   
       13 . The method of  claim 11 , wherein the step of calculating an X-direction centroid further comprises:
 dividing the sum by the sum of all pixel values in the set of pixels.   
   
   
       14 . The method of  claim 9 , wherein the second step of calculating further comprises:
 calculating a Y-direction centroid.   
   
   
       15 . The method of  claim 14 , wherein the step of calculating a Y-direction centroid further comprises:
 multiplying each pixel in the set of pixels by a Y-direction coordinate of the pixel.   
   
   
       16 . The method of  claim 15 , wherein the step of calculating a Y-direction centroid further comprises:
 summing the product of multiplying each pixel in the set of pixels by a Y-direction coordinate of the pixel so as to produce a sum; and   dividing the sum by the sum of all pixel values in the set of pixels.   
   
   
       17 . A firearm laser training system operable to detect the location of a projected laser beam pulse upon a projected target image is provided, said system comprising:
 a projection means operable to project a target image upon a surface;   a video camera means operable to scan said target image to produce scanned images of said target image, including impact locations of said laser beam pulse on said target image; and   a processing means operable to receive from said video camera means information associated with said impact locations detected by said video camera means, said processing means comprising:
 an automatic calibration module operable to correlate camera pixels to target pixels of the projected target image; 
 a detection module operable to determine a camera pixel position of the laser beam pulse; and 
 a shot correlation module operable to correlate the camera pixel position of the laser beam pulse to the projected target image, so as to determine the accuracy of the shot. 
   
   
   
       18 . The firearm laser training system of  claim 17 , wherein the automatic calibration module comprises:
 a means operable to project a first image comprised of horizontal and vertical lines upon the surface;   a means operable to read the first image;   a means operable to adjust an exposure setting of the camera based on the first image, so as to ensure pixel values below saturation;   a means operable to project a second image comprising a black color upon the surface;   a means operable to read the second image;   a means operable to calculate a background light level value for each pixel in the second image;   a means operable to project a third image comprising a pattern of horizontal lines upon the surface;   a means operable to read the third image;   a means operable to calculate a horizontal calibration value for each pixel in the third image, wherein the resulting horizontal calibration value for each pixel is reduced by the corresponding background light level value;   a means operable to project a fourth image comprising a pattern of vertical lines upon the surface;   a means operable to read the fourth image;   a means operable to calculate a vertical calibration value for each pixel in the fourth image, wherein the resulting vertical calibration value for each pixel is reduced by the corresponding background light level value;   a means operable to calculate a centroid line for each vertical and horizontal line and calculating a second order line least squares fit for each centroid line, thereby generating three coefficients that describe each vertical and horizontal line; and   a means operable to generate a data map for mapping a pixel from an image read from the surface to a pixel in an image projected into the surface, wherein the data map is based on the three coefficients that describe each vertical and horizontal line.   
   
   
       19 . The firearm laser training system of  claim 18 , wherein the first image is set to the color white. 
   
   
       20 . The firearm laser training system of  claim 18 , wherein the first image is set to the color red. 
   
   
       21 . The firearm laser training system of  claim 18 , wherein the means operable to calculate the background light level for each pixel in the second image further comprises:
 a means operable to store the background light level value for each pixel in a two dimensional array.   
   
   
       22 . The firearm laser training system of  claim 18 , wherein the means operable to calculate a horizontal calibration value for each pixel in the third image further comprises:
 a means operable to store the horizontal calibration value for each pixel in a two dimensional array.   
   
   
       23 . The firearm laser training system of  claim 18 , wherein the means operable to calculate a vertical calibration value for each pixel in the fourth image further comprises:
 a means operable to store the vertical calibration value for each pixel in a two dimensional array.