Automatic translation of digital graphic novels

Abstract

Digital graphic novel content is received and features of the graphic novel content are identified. At least one of the identified features includes text. Contextual information corresponding to the feature or features that include text is generated based on the identified features. The contextual information is used to aid translation of the text included in the feature or features that include text.

What is claimed is: 
     
         1 . A method comprising:
 receiving, by at least one processor, digital graphic novel content;   determining, by the at least one processor, a plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles;   determining, by the at least one processor, from the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles, features of the digital graphic novel content that include a plurality of speech bubbles containing text;   generating, by the at least one processor, based on the features of the digital graphic novel content, an intended reading order of the plurality of speech bubbles; and   automatically translating, by the at least one processor, based at least in part on the intended reading order of the plurality of speech bubbles, from a first natural language to a second natural language, the text contained in the plurality of speech bubbles to produce translated text.   
     
     
         2 . The method of  claim 1  wherein automatically translating the text comprises:
 after extracting the text contained in the plurality of speech bubbles, compiling the text into a single piece of text based on the intended reading order; and 
 translating the single piece of text from the first natural language to the second natural language to produce the translated text. 
 
     
     
         3 . The method of  claim 1 , wherein automatically translating the text comprises applying a translation algorithm to the text contained in the plurality of speech bubbles that is tailored for a depicted character or object contained in the digital graphic novel content. 
     
     
         4 . The method of  claim 1 , further comprising:
 creating a packaged digital graphic novel including the digital graphic novel content and presentation metadata, the presentation metadata including the translated text and an indication of the at least one feature of the features of the digital graphic novel to which the translated text corresponds; and   providing, by the at least one processor, the packaged digital graphic novel to a reader device that is configured to present the digital graphic novel content in a manner in accordance with the presentation metadata.   
     
     
         5 . The method of  claim 4 , wherein the reader device is configured to display the translated text within the plurality of speech bubbles in place of the text contained in the plurality of speech bubbles. 
     
     
         6 . The method of  claim 1 , wherein determining the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles comprises:
 producing, by the at least one processor, a numerical map that represents an image extracted from the digital graphic novel content; and   responsive to inputting the numerical map into a first artificial neural network of a machine learning model, receiving, by the at least one processor, from the first artificial neural network, the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles.   
     
     
         7 . The method of  claim 6 , wherein determining the features of the digital graphic novel content that include the plurality of speech bubbles containing text comprises:
 responsive to inputting the plurality of candidate regions into a second artificial neural network of the machine learning model, receiving, by the at least one processor, from the second artificial neural network, the features of the digital graphic novel content that include the plurality of speech bubbles containing text.   
     
     
         8 . An electronic device comprising:
 at least one processor; and   a non-transitory computer-readable storage medium storing instructions that, when executed, configured the at least one processor to:
 receive digital graphic novel content; 
 determine a plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles; 
 determine, from the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles, features of the digital graphic novel content that include a plurality of speech bubbles containing text; 
 generate, based on the features of the digital graphic novel content, an intended reading order of the plurality of speech bubbles; and 
 automatically translate, based at least in part on the intended reading order of the plurality of speech bubbles, from a first natural language to a second natural language, the text contained in the plurality of speech bubbles to produce translated text. 
   
     
     
         9 . The electronic device of  claim 8 , wherein the instructions, when executed, configure the at least one processor to automatically translate the text by at least:
 after extracting the text contained in the plurality of speech bubbles, compiling the text into a single piece of text based on the intended reading order; and   translating the single piece of text from the first natural language to the second natural language to produce the translated text.   
     
     
         10 . The electronic device of  claim 8 , wherein the instructions, when executed, configure the at least one processor to automatically translate the text by at least applying a translation algorithm to the text contained in the plurality of speech bubbles that is tailored for a depicted character or object contained in the digital graphic novel content. 
     
     
         11 . The electronic device of  claim 8 , wherein the instructions, when executed, further configure the at least one processor to:
 create a packaged digital graphic novel including the digital graphic novel content and presentation metadata, the presentation metadata including the translated text and an indication of the at least one feature of the features of the digital graphic novel to which the translated text corresponds; and   provide the packaged digital graphic novel to a reader device that is configured to present the digital graphic novel content in a manner in accordance with the presentation metadata.   
     
     
         12 . The electronic device of  claim 11 , wherein the reader device is configured to display the translated text within the plurality of speech bubbles in place of the text contained in the plurality of speech bubbles. 
     
     
         13 . The electronic device of  claim 8 , wherein the instructions, when executed, configure the at least one processor to determine the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles by at least:
 producing a numerical map that represents an image extracted from the digital graphic novel content; and   responsive to inputting the numerical map into a first artificial neural network of a machine learning model, receiving, from the first artificial neural network, the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles.   
     
     
         14 . The electronic device of  claim 13 , wherein the instructions, when executed, configure the at least one processor to determine the features of the digital graphic novel content that include the plurality of speech bubbles containing text by at least:
 responsive to inputting the plurality of candidate regions into a second artificial neural network of the machine learning model, receiving, from the second artificial neural network, the features of the digital graphic novel content that include the plurality of speech bubbles containing text.   
     
     
         15 . A computer-readable storage medium storing instructions that, when executed, configured at least one processor to:
 receive digital graphic novel content;   determine a plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles;   determine, from the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles, features of the digital graphic novel content that include a plurality of speech bubbles containing text;   generate, based on the features of the digital graphic novel content, an intended reading order of the plurality of speech bubbles; and   automatically translate, based at least in part on the intended reading order of the plurality of speech bubbles, from a first natural language to a second natural language, the text contained in the plurality of speech bubbles to produce translated text.   
     
     
         16 . The computer-readable storage medium of  claim 15 , wherein the instructions, when executed, configure the at least one processor to automatically translate the text by at least applying a translation algorithm to the text contained in the plurality of speech bubbles that is tailored for a depicted character or object contained in the digital graphic novel content. 
     
     
         17 . The computer-readable storage medium of  claim 15 , wherein the instructions, when executed, further configure the at least one processor to:
 create a packaged digital graphic novel including the digital graphic novel content and presentation metadata, the presentation metadata including the translated text and an indication of the at least one feature of the features of the digital graphic novel to which the translated text corresponds; and   provide the packaged digital graphic novel to a reader device that is configured to present the digital graphic novel content in a manner in accordance with the presentation metadata.   
     
     
         18 . The computer-readable storage medium of  claim 15 , wherein the reader device is configured to display the translated text within the plurality of speech bubbles in place of the text contained in the plurality of speech bubbles. 
     
     
         19 . The computer-readable storage medium of  claim 15 , wherein the instructions, when executed, configure the at least one processor to determine the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles by at least:
 producing a numerical map that represents an image extracted from the digital graphic novel content; and   responsive to inputting the numerical map into a first artificial neural network of a machine learning model, receiving, from the first artificial neural network, the plurality of candidate regions of the digital graphic novel content that are likely to include speech bubbles.   
     
     
         20 . The computer-readable storage medium of  claim 19 , wherein the instructions, when executed, configure the at least one processor to determine the features of the digital graphic novel content that include the plurality of speech bubbles containing text by at least:
 responsive to inputting the plurality of candidate regions into a second artificial neural network of the machine learning model, receiving, from the second artificial neural network, the features of the digital graphic novel content that include the plurality of speech bubbles containing text.