Character recognition and document interpretation method and system based on layout recognition

Abstract

A character recognition system includes: a character-related information extraction unit configured to include a deep learning model trained to extract character area information, inter-character space area information, interline scale information of each character, and orientation information of each character from an image including text; a word unit division recognition unit configured to obtain word division information obtained by dividing characters included in the image into word units based on the character area information and inter-character space area information; a text line recognition unit configured to recognize text lines in the image based on the character area information, interline scale information, and orientation information; a layout analysis unit configured to obtain layout information of the text included in the image based on the recognized text lines; and a character recognition unit configured to recognize each of the character included in the image and obtain text data in which the recognized characters are aligned based on the word division information and the layout information.

What is claimed is: 
     
         1 . A character recognition system including at least one computing device, the character recognition system comprising:
 a character-related information extraction unit configured to include a deep learning model trained to extract character area information, inter-character space area information, interline scale information of each character, and orientation information of each character from an image including text;   a word unit division recognition unit configured to obtain word division information obtained by dividing characters included in the image into word units based on the character area information and inter-character space area information;   a text line recognition unit configured to recognize text lines in the image based on the character area information, interline scale information, and orientation information;   a layout analysis unit configured to obtain layout information of the text included in the image based on the recognized text lines; and   a character recognition unit configured to recognize each of the character included in the image and obtain text data in which the recognized characters are aligned based on the word division information and the layout information.   
     
     
         2 . The character recognition system of  claim 1 , wherein the character area information comprises information about an area in which a character is inferred to be located from among areas in the image,
 the inter-character space area information comprises information obtained by inferring a space area existing between adjacent characters,   the interline scale information comprises information related to spacing between text lines determined at each position of the characters, and   the orientation information comprises information about an angle of a text line determined at each position of the characters.   
     
     
         3 . The character recognition system of  claim 1 , wherein the text line recognition unit is configured to:
 define respective element areas for characters determined based on the character area information; and   recognize the text lines in the image based on whether the defined element areas are connected or overlapped with each other.   
     
     
         4 . The character recognition system of  claim 3 , wherein the text line recognition unit is configured to:
 define a first element area corresponding to an initial element area for each of the characters determined based on the character area information; and   cluster the characters into a text line candidate set based on whether the defined first element areas are connected or overlapped with each other,   wherein a center position of the first element area corresponds to a center position of a corresponding character, and a rotation angle of the first element area corresponds to orientation information of the corresponding character.   
     
     
         5 . The character recognition system of  claim 4 , wherein the text line recognition unit is configured to:
 cluster the characters into a text line candidate set based on whether second element areas obtained by increasing a size of the first element areas are connected or overlapped with each other; and   recognize text lines for the characters included in the image based on a clustering result.   
     
     
         6 . The character recognition system of  claim 5 , wherein the text line recognition unit is configured to:
 when the clustering result satisfies a certain condition, cluster the characters into a text line candidate set based on whether third element areas obtained by increasing a size of the second element areas are connected or overlapped with each other; and   when the clustering result does not satisfy a certain condition, recognize each of clustered text line candidate sets based on the first element area as one text line.   
     
     
         7 . The character recognition system of  claim 6 , wherein the text line recognition unit is configured to:
 define a polynomial having a minimum approximation error with respect to coordinates of a center point of each of characters included in an identical text line candidate set; and   determine whether a relationship between the approximation error for the defined polynomial and an average value of interline scale information of each of the characters satisfies the certain condition.   
     
     
         8 . The character recognition system of  claim 1 , wherein the layout analysis unit is configured to:
 generate paragraph information obtained by dividing paragraphs of the text in the image based on spacing between the recognized text lines; and   generate line number information obtained by dividing line numbers based on a y-axis intercept and center coordinate of each of the recognized text lines,   wherein the layout information comprises the paragraph information and the line number information.   
     
     
         9 . A character recognition method comprising:
 extracting character-related information about a plurality of characters from an image including text composed of the plurality of characters;   obtaining word division information obtained by dividing the plurality of characters included in the image into word units based on the extracted character-related information;   recognizing text lines in the image based on the extracted character-related information;   obtaining layout information of the text included in the image based on the recognized text lines; and   recognizing each of the plurality of characters included in the image and obtaining text data in which the recognized characters are aligned based on the word division information and the layout information.   
     
     
         10 . The character recognition method of  claim 9 , wherein the character-related information comprises:
 character area information including information about an area in which characters are inferred to be located from among areas in the image;   inter-character space area information including information obtained by inferring a space area existing between adjacent characters;   interline scale information including information related to spacing between text lines determined at each position of the plurality of characters; and   orientation information including information about an angle a text line determined at each position of the plurality of characters.   
     
     
         11 . The character recognition method of  claim 10 , wherein the recognizing of the text lines comprises:
 defining respective element areas for characters determined based on the character area information; and   recognizing the text lines in the image based on whether the defined element areas are connected or overlapped with each other.   
     
     
         12 . The character recognition method of  claim 11 , wherein the recognizing of the text lines comprises:
 defining a first element area corresponding to an initial element area for each of the characters determined based on the character area information; and   clustering the characters into a text line candidate set based on whether the defined first element areas are connected or overlapped with each other,   wherein a center position of the first element area corresponds to a center position of a corresponding character, and a rotation angle of the first element area corresponds to orientation information of the corresponding character.   
     
     
         13 . The character recognition method of  claim 12 , wherein the recognizing of the text lines comprises:
 clustering the characters into a text line candidate set based on whether second element areas obtained by increasing a size of the first element areas are connected or overlapped with each other; and   recognizing text lines for the characters included in the image based on a clustering result.   
     
     
         14 . The character recognition method of  claim 13 , wherein the recognizing of the text lines comprises:
 when the clustering result satisfies a certain condition, clustering the characters into a text line candidate set based on whether third element areas obtained by increasing a size of the second element areas are connected or overlapped with each other; and,   when the clustering result does not satisfy a certain condition, recognizing each of clustered text line candidate sets based on the first element area as one text line.   
     
     
         15 . The character recognition method of  claim 14 , wherein the recognizing of the text lines for the characters included in the image based on the clustering result comprises:
 defining a polynomial having a minimum approximation error with respect to coordinates of a center point of each of characters included in an identical text line candidate set; and   determining whether a relationship between the approximation error for the defined polynomial and an average value of interline scale information of each of the characters satisfies the certain condition.   
     
     
         16 . The character recognition method of  claim 9 , wherein the obtaining of the layout information comprises:
 generating paragraph information obtained by dividing paragraphs of the text in the image based on spacing between the recognized text lines; and   generating line number information obtained by dividing line numbers based on a y-axis intercept and center coordinate of each of the recognized text lines.