Natural Language Based Computer Animation

Abstract

The present disclosure relates to systems, methods, and non-transitory computer-readable media that facilitate near real-time dynamic generation of graphical content based on audio, text, or gesture input. An example method includes receiving an input from an input device such as a microphone, a keyboard, or a camera. As such, the input may include text, speech, and/or a gesture. The method includes determining, based on the input, one or more command instructions. The method also includes determining, based on the one or more command instructions, a scene layout. Each of the one or more command instructions correspond to at least one element of the scene layout. The method also includes providing a rendered scene based on the determined scene layout.

What is claimed is: 
     
         1 . A method comprising:
 receiving an input from a brain-machine interface, wherein the input is indicative of one or more command instructions;   determining, based on the input, at least one vector object arranged at specific coordinates within a scene layout; and   providing a rendered scene based on the scene layout.   
     
     
         2 . The method of  claim 1 , wherein the brain-machine interface comprises at least one of: a visual neuroprosthetic, an audio neuroprosthetic, or a motor neuroprosthetic. 
     
     
         3 . The method of  claim 1 , wherein the at least one vector object is provided as at least a portion of a point cloud. 
     
     
         4 . The method of  claim 1 , wherein the at least one vector object is provided as a plurality of polygons. 
     
     
         5 . The method of  claim 1 , further comprising:
 categorizing each of the one or more command instructions into at least one of: an object command instruction, a language command instruction, or an environmental command instruction; and   storing each categorized command instruction into at least one corresponding database, wherein the corresponding database comprises at least one of an object database, a linguistic database, or an environmental database.   
     
     
         6 . The method of  claim 5 , wherein determining the scene layout comprises:
 determining, for each object command instruction, at least one object and at least one corresponding object location.   
     
     
         7 . The method of  claim 5 , wherein determining the scene layout comprises:
 determining, for each environmental command instruction, at least one environmental element and at least one corresponding environmental element location.   
     
     
         8 . The method of  claim 1 , wherein providing a rendered scene comprises rendering, using a dedicated graphics processor, the rendered scene based on the determined scene layout, wherein the dedicated graphics processor comprises a graphics processing unit (GPU). 
     
     
         9 . The method of  claim 1 , further comprising providing an output in an HTML-compatible format. 
     
     
         10 . The method of  claim 1 , further comprising providing an output, wherein the output comprises a format that is compatible with at least one of: an audio output unit, a haptic output unit, a smell output unit, a taste output unit, or a graphical output unit. 
     
     
         11 . The method of  claim 10 , wherein the graphical output unit comprises a graphics engine or a communication link to an output target. 
     
     
         12 . The method of  claim 10 , wherein the graphical output unit comprises a virtual reality display or an augmented reality display. 
     
     
         13 . The method of  claim 1 , wherein the method is performed in near real-time, wherein near real-time comprises at least one of determining the one or more command instructions at least once every 50 milliseconds or determining the scene layout at least once every 50 milliseconds. 
     
     
         14 . The method of  claim 1 , wherein one or more steps of the method are carried out by an intelligent agent or artificial intelligence construct that utilizes machine learning. 
     
     
         15 . A system comprising:
 an input device, wherein the input device comprises a brain-machine interface;   a render processing unit;   a display; and   a controller comprising at least one memory and at least one processor, wherein the controller executes instructions so as to carry out operations, the operations comprising:
 receiving, via the input device, input information indicative of one or more command instructions; 
 determining, based on the input, at least one vector object arranged at specific coordinates within a scene layout; 
 rendering, using the render processing unit, a rendered scene based on the determined scene layout; and 
 displaying, via the display, the rendered scene. 
   
     
     
         16 . The system of  claim 15 , wherein the brain-machine interface comprises at least one of: a visual neuroprosthetic, an audio neuroprosthetic, or a motor neuroprosthetic. 
     
     
         17 . The system of  claim 15 , further comprising:
 categorizing each command instruction into at least one of: an object command instruction, a language command instruction, or an environmental command instruction; and   storing each categorized command instruction into at least one corresponding database, wherein the corresponding database comprises at least one of an object database, a linguistic database, or an environmental database.   
     
     
         18 . The system of  claim 15 , wherein the display comprises at least one of: a virtual reality display or an augmented reality display. 
     
     
         19 . A non-transitory computer readable medium having stored therein instructions that, when executed by a computing device, cause the computing device to perform operations comprising:
 receiving an input, wherein the input is indicative of one or more command instructions;   determining, based on the input, at least one vector object arranged at specific coordinates within a scene layout;   providing a rendered scene based on the scene layout, wherein each command instruction corresponds to at least one element of the scene layout;   categorizing each command instruction into at least one of: an object command instruction, a language command instruction, or an environmental command instruction;   storing each categorized command instruction into at least one corresponding database, wherein the corresponding database comprises at least one of an object database, a linguistic database, or an environmental database; and   providing an output in an HTML-compatible format.   
     
     
         20 . The non-transitory computer readable medium of  claim 19 , wherein the input comprises a brain-machine interface, wherein the brain-machine interface comprises at least one of: a visual neuroprosthetic, an audio neuroprosthetic, or a motor neuroprosthetic.