Supervised machine learning algorithm application for image cropping and skew rectification

Abstract

Systems and methods here may be used for pre-processing images, including using a computer for receiving a pixelated image of a paper document of an original size, downscaling the received pixelated image, employing a neural network algorithm to the downscaled image to identify four corners of the paper document in the received pixelated image, re-enlarging the downscaled image to the original size, identifying each of four corners of the paper document in the pixelated image, determining a quadrilateral composed of lines that intersect at four angles at the four corners of the paper document in the pixelated image, defining a projective plane of the pixelated image, and determining an inverse transformation of the pixelated image to transform the projective plane quadrilateral into a right angled rectangle.

What is claimed is: 
     
       1. A method for pre-processing images, comprising:
 by a computer with a processor and a memory, 
 receiving a pixelated image of a paper document of an original size; 
 downscaling the received pixelated image; 
 employing a neural network model with the downscaled image to identify four corners of the paper document in the received pixelated image, wherein employing the neural network model to identify the four corners of the paper document includes creation of four heat maps, each heat map predicting a location for each of the four corners; 
 re-enlarging the downscaled image to the original size; 
 identifying each of the four corners of the paper document in the pixelated image; 
 determining a quadrilateral composed of lines that intersect at four angles at the four corners of the paper document in the pixelated image; 
 defining a projective plane of the pixelated image; and 
 determining an inverse transformation of the pixelated image to transform the projective plane quadrilateral into a right angled rectangle. 
 
     
     
       2. The method of  claim 1  wherein re-enlarging the downscaled image includes using a bilinear interpolation to obtain the original size. 
     
     
       3. The method of  claim 1  wherein the neural network model is a stacked hourglass neural network. 
     
     
       4. The method of  claim 1  further comprising, by the computer, cropping any part of the pixelated image outside the right angled rectangle. 
     
     
       5. The method of  claim 1  further comprising, determining if the four angles fall within a pre-determined tolerance for corners. 
     
     
       6. The method of  claim 5  further comprising determining if the four angles fall within the pre-determined tolerance for the four corners. 
     
     
       7. The method of  claim 1  wherein receiving a pixelated image of a paper document of an original size is from a mobile client device by a network. 
     
     
       8. The method of  claim 1  wherein the identifying each of the four corners, is by identifying each pixel with a highest predicted probability for keypoint occurrence for each corner. 
     
     
       9. The method of  claim 1  further comprising, by the computer, extracting text using Optical Character Recognition from the transformed right angled rectangle pixelated image. 
     
     
       10. A system for pre-processing images, comprising:
 a computer with a processor and a memory, configured to, 
 receive a pixelated image of a paper document of an original size; 
 downscale the received pixelated image; 
 employ a neural network algorithm to the downscaled pixelated image to identify four corners of the paper document in the received pixelated image, wherein employing the neural network algorithm to identify the four corners of the paper document includes creation of four heat maps, each heat map predicting a location for each of the four corners; 
 re-enlarge the downscaled pixelated image to the original size; 
 identify each of the four corners of the paper document in the pixelated image; 
 determine a quadrilateral composed of lines that intersect at four angles at the four corners of the paper document in the pixelated image; 
 define a projective plane of the pixelated image; and 
 determine an inverse transformation of the pixelated image to transform the projective plane quadrilateral into a right angled rectangle. 
 
     
     
       11. The system of  claim 10  wherein re-enlarging the downscaled pixelated image includes using a bilinear interpolation to obtain the original size. 
     
     
       12. The system of  claim 10  wherein the neural network algorithm is a stacked hourglass neural network. 
     
     
       13. The system of  claim 10  further comprising, by the computer, cropping any part of the pixelated image outside the right angled rectangle. 
     
     
       14. The system of  claim 10  further comprising, determining if the four angles fall within a pre-determined tolerance for corners. 
     
     
       15. The system of  claim 14  further comprising determining if the four angles fall within the pre-determined tolerance for the four corners. 
     
     
       16. The system of  claim 10  wherein receiving a pixelated image of a paper document of an original size is from a mobile client device by a network. 
     
     
       17. The system of  claim 10  wherein the identifying each of the four corners, is by identifying each pixel with a highest predicted probability for keypoint occurrence for each corner. 
     
     
       18. The system of  claim 10  further comprising, by the computer, extracting text using Optical Character Recognition from the transformed right angled rectangle pixelated image.