On-device artificial intelligence systems and methods for document auto-rotation

Abstract

An auto-rotation module having a single-layer neural network on a user device can convert a document image to a monochrome image having black and white pixels and segment the monochrome image into bounding boxes, each bounding box defining a connected segment of black pixels in the monochrome image. The auto-rotation module can determine textual snippets from the bounding boxes and prepare them into input images for the single-layer neural network. The single-layer neural network is trained to process each input image, recognize a correct orientation, and output a set of results for each input image. Each result indicates a probability associated with a particular orientation. The auto-rotation module can examine the results, determine what degree of rotation is needed to achieve a correct orientation of the document image, and automatically rotate the document image by the degree of rotation needed to achieve the correct orientation of the document image.

What is claimed is: 
     
       1. A method for document auto-rotation, the method comprising:
 converting, by an auto-rotation module on a user device, a document image to a monochrome image, the converting comprising determining connected segments of pixels in the monochrome image; 
 segmenting, by the auto-rotation module, the monochrome image to produce bounding boxes, each bounding box corresponding to one of the connected segments; 
 determining, by the auto-rotation module utilizing the bounding boxes, a plurality of textual snippets in the monochrome image; 
 preparing, by the auto-rotation module, the plurality of textual snippets for input to a single-layer neural network; 
 processing, by the auto-rotation module, the plurality of textual snippets using the single-layer neural network, wherein the single-layer neural network is trained to process an input textual snippet, recognize an orientation of the input textual snippet, and output probability values each indicating a degree of rotation or number of turns needed to correct the orientation of the input textual snippet; 
 determining, by the auto-rotation module based on probability values in the outputs from the single-layer neural network that meet or exceed a threshold, a degree of rotation or number of turns needed to rotate the document image; and 
 automatically rotating, by the auto-rotation module, the document image by the degree of rotation or number of turns determined based on the probability values in the outputs from the single-layer neural network that meet or exceed the threshold. 
 
     
     
       2. The method according to  claim 1 , wherein the converting comprises performing an adaptive binarization on the document image in which, for every pixel in the document image, a neighborhood of pixels is examined so as to separate characters in the document image from background of the document image or from one another. 
     
     
       3. The method according to  claim 1 , wherein the bounding boxes overlap one another and wherein the connected segments of pixels do not overlap with one another. 
     
     
       4. The method according to  claim 1 , further comprising:
 determining, utilizing the bounding boxes, any non-textual snippets in the monochrome image; and 
 removing any bounding boxes that represent non-textual snippets in the monochrome image. 
 
     
     
       5. The method according to  claim 1 , further comprising:
 determining a width-to-height ratio of a bounding box; 
 determining whether the width-to-height ratio of the bounding box is outside of a predetermined range; and 
 responsive to the width-to-height ratio of the bounding box being outside of the predetermined range, removing the bounding box that represents a non-textual snippet. 
 
     
     
       6. The method according to  claim 1 , further comprising:
 randomly selecting textual snippets from the bounding boxes; and 
 testing each of the textual snippets randomly selected from the bounding boxes using the single-layer neural network. 
 
     
     
       7. The method according to  claim 1 , wherein the single-layer neural network is configured for outputting a probability value indicating that a zero degree of rotation or zero number of turns is needed to correct the orientation of the input textual snippet, a probability value indicating that a 90 degree of rotation or a single right or left turn is needed to correct the orientation of the input textual snippet, a probability value indicating that a 180 degree of rotation is or two right or left turns are needed to correct the orientation of the input textual snippet, and a probability value indicating that a 270 degree of rotation is or three right or left turns are needed to correct the orientation of the input textual snippet. 
     
     
       8. A system for document auto-rotation, the system comprising:
 a processor; 
 a non-transitory computer-readable medium; and 
 stored instructions translatable by the processor for:
 converting, by an auto-rotation module on a user device, a document image to a monochrome image, the converting comprising determining connected segments of pixels in the monochrome image; 
 segmenting, by the auto-rotation module, the monochrome image to produce bounding boxes, each bounding box corresponding to one of the connected segments; 
 determining, by the auto-rotation module utilizing the bounding boxes, a plurality of textual snippets in the monochrome image; 
 preparing, by the auto-rotation module, the plurality of textual snippets for input to a single-layer neural network; 
 processing, by the auto-rotation module, the plurality of textual snippets using the single-layer neural network, wherein the single-layer neural network is trained to process an input textual snippet, recognize an orientation of the input textual snippet, and output probability values each indicating a degree of rotation or number of turns needed to correct the orientation of the input textual snippet; 
 determining, by the auto-rotation module based on probability values in the outputs from the single-layer neural network that meet or exceed a threshold, a degree of rotation or number of turns needed to rotate the document image; and 
 automatically rotating, by the auto-rotation module, the document image by the degree of rotation or number of turns determined based on the probability values in the outputs from the single-layer neural network that meet or exceed the threshold. 
 
 
     
     
       9. The system according to  claim 8 , wherein the converting comprises performing an adaptive binarization on the document image in which, for every pixel in the document image, a neighborhood of pixels is examined so as to separate characters in the document image from background of the document image or from one another. 
     
     
       10. The system according to  claim 8 , wherein the bounding boxes overlap one another and wherein the connected segments of pixels do not overlap with one another. 
     
     
       11. The system according to  claim 8 , further comprising:
 determining, utilizing the bounding boxes, any non-textual snippets in the monochrome image; and 
 removing any bounding boxes that represent non-textual snippets in the monochrome image. 
 
     
     
       12. The system according to  claim 8 , further comprising:
 determining a width-to-height ratio of a bounding box; 
 determining whether the width-to-height ratio of the bounding box is outside of a predetermined range; and 
 responsive to the width-to-height ratio of the bounding box being outside of the predetermined range, removing the bounding box that represents a non-textual snippet. 
 
     
     
       13. The system according to  claim 8 , further comprising:
 randomly selecting textual snippets from the bounding boxes; and 
 testing each of the textual snippets randomly selected from the bounding boxes using the single-layer neural network. 
 
     
     
       14. The system according to  claim 8 , wherein the single-layer neural network is configured for outputting a probability value indicating that a zero degree of rotation or zero number of turns is needed to correct the orientation of the input textual snippet, a probability value indicating that a 90 degree of rotation or a single right or left turn is needed to correct the orientation of the input textual snippet, a probability value indicating that a 180 degree of rotation is or two right or left turns are needed to correct the orientation of the input textual snippet, and a probability value indicating that a 270 degree of rotation is or three right or left turns are needed to correct the orientation of the input textual snippet. 
     
     
       15. A computer program product for document auto-rotation, the computer program product comprising a non-transitory computer-readable medium storing instructions translatable by a processor for:
 converting, by an auto-rotation module on a user device, a document image to a monochrome image, the converting comprising determining connected segments of pixels in the monochrome image; 
 segmenting, by the auto-rotation module, the monochrome image to produce bounding boxes, each bounding box corresponding to one of the connected segments; 
 determining, by the auto-rotation module utilizing the bounding boxes, a plurality of textual snippets in the monochrome image; 
 preparing, by the auto-rotation module, the plurality of textual snippets for input to a single-layer neural network; 
 processing, by the auto-rotation module, the plurality of textual snippets using the single-layer neural network, wherein the single-layer neural network is trained to process an input textual snippet, recognize an orientation of the input textual snippet, and output probability values each indicating a degree of rotation or number of turns needed to correct the orientation of the input textual snippet; 
 determining, by the auto-rotation module based on probability values in the outputs from the single-layer neural network that meet or exceed a threshold, a degree of rotation or number of turns needed to rotate the document image; and 
 automatically rotating, by the auto-rotation module, the document image by the degree of rotation or number of turns determined based on the probability values in the outputs from the single-layer neural network that meet or exceed the threshold. 
 
     
     
       16. The computer program product according to  claim 15 , wherein the converting comprises performing an adaptive binarization on the document image in which, for every pixel in the document image, a neighborhood of pixels is examined so as to separate characters in the document image from background of the document image or from one another. 
     
     
       17. The computer program product according to  claim 15 , wherein the bounding boxes overlap one another and wherein the connected segments of pixels do not overlap with one another. 
     
     
       18. The computer program product according to  claim 15 , further comprising:
 determining, utilizing the bounding boxes, any non-textual snippets in the monochrome image; and 
 removing any bounding boxes that represent non-textual snippets in the monochrome image. 
 
     
     
       19. The computer program product according to  claim 15 , further comprising:
 determining a width-to-height ratio of a bounding box; 
 determining whether the width-to-height ratio of the bounding box is outside of a predetermined range; and 
 responsive to the width-to-height ratio of the bounding box being outside of the predetermined range, removing the bounding box that represents a non-textual snippet. 
 
     
     
       20. The computer program product according to  claim 15 , further comprising:
 randomly selecting textual snippets from the bounding boxes; and 
 testing each of the textual snippets randomly selected from the bounding boxes using the single-layer neural network.