Generative artificial intelligence model streaming

Abstract

An artificial intelligence system includes one or more generative artificial intelligence models (“generative model”) and a retrieval system. The artificial intelligence (“AI”) system receives a natural language input query. The retrieval system retrieves data objects associated with the input query. A generative model streams content in response to input query. The AI system formats output from the generative model and the data objects into formatted data. A user interface outputs the formatted data.

What is claimed is: 
     
         1 . A system comprising:
 a non-transitory data storage medium to store specific computer-executable instructions; and   a computer hardware processor in communication with the non-transitory data storage medium, wherein the computer hardware processor is to execute the specific computer-executable instructions to at least:
 receive an input query from a user computing device; 
 determine a question type for the input query; 
 determine a prompt based at least in part on the question type and the input query; 
 provide the prompt to a trained generative model; 
 receive, in a stream from the trained generative model, first content comprising a plurality of tokens, wherein a first token of the plurality of tokens is an intermediate token; 
 determine first markup content from the first content, wherein to determine the first markup content, the computer hardware processor is to execute the computer-executable instructions to at least:
 replace, in the plurality of tokens, the intermediate token with markup content; 
 
 transmit the first markup content to the user computing device at a first time; 
 receive, in the stream from the trained generative model, second content; and 
 transmit at least some of the second content to the user computing device at a second time. 
   
     
     
         2 . The system of  claim 1 , wherein the computer hardware processor executes further computer-executable instructions to at least:
 determine an item associated with the input query, wherein to determine the prompt, the computer hardware processor is to execute the further computer-executable instructions to at least:
 insert the item into a prompt template. 
   
     
     
         3 . The system of  claim 2 , wherein to determine the first markup content, the computer hardware processor is to execute additional computer-executable instructions to at least:
 insert, at a placeholder location in the plurality of tokens, a title associated with the item.   
     
     
         4 . The system of  claim 2 , wherein to determine the first markup content, the computer hardware processor is to execute additional computer-executable instructions to at least:
 insert, at a placeholder location in the plurality of tokens, a resource locator for an image associated with the item.   
     
     
         5 . The system of  claim 1 , wherein the computer hardware processor executes further computer-executable instructions to at least:
 determine a related question associated with the input query, wherein to determine the prompt, the computer hardware processor is to execute the further computer-executable instructions to at least:
 insert the related question into a prompt template. 
   
     
     
         6 . The system of  claim 5 , wherein to determine the first markup content, the computer hardware processor is to execute additional computer-executable instructions to at least:
 insert, at a placeholder location in the plurality of tokens, the related question.   
     
     
         7 . A computer-implemented method comprising:
 receiving an input query from a user computing device;   determining a question type for the input query;   determining first input based at least in part on the question type and the input query;   providing the first input to a trained generative model;   receiving, in a stream from the trained generative model, first content comprising a plurality of tokens, wherein a first token of the plurality of tokens is an intermediate token;   determining, from the first content, first output content in a first data format different from a data format of the first content, wherein determining the first output content further comprises:
 replacing, in the plurality of tokens, the intermediate token with content in the first data format; 
   transmitting the first output content to the user computing device at a first time;   receiving, in the stream from the trained generative model, second content; and   transmitting at least some of the second content to the user computing device at a second time.   
     
     
         8 . The computer-implemented method of  claim 7 , further comprising:
 retraining a pre-trained language model with machine learning and a training data set comprising a prompt and response pair, wherein the prompt and response pair comprises the intermediate token, wherein retraining the pre-trained language model further comprises:
 outputting the trained generative model. 
   
     
     
         9 . The computer-implemented method of  claim 7 , wherein determining the first input further comprises:
 selecting a prompt template associated with the question type; and   inserting, into the prompt template, at least some of the input query.   
     
     
         10 . The computer-implemented method of  claim 7 , further comprising:
 determining an item associated with the input query, wherein determining the first output content further comprises:
 inserting, into intermediate content, a placeholder token; and 
 replacing the placeholder token with metadata associated with the item. 
   
     
     
         11 . The computer-implemented method of  claim 10 , wherein the metadata comprises a resource locator for the item. 
     
     
         12 . The computer-implemented method of  claim 7 , further comprising:
 determining a user profile associated with the input query; and   determining a communication history associated with the user profile, wherein determining the first input further comprises:
 determining second input based at least in part on the communication history. 
   
     
     
         13 . The computer-implemented method of  claim 7 , further comprising:
 determining a user profile associated with the input query; and   determining a navigation history associated with the user profile, wherein determining the first input further comprises:
 determining second input based at least in part on the navigation history. 
   
     
     
         14 . A system comprising:
 a non-transitory data storage medium to store specific computer-executable instructions; and   a computer hardware processor in communication with the non-transitory data storage medium, wherein the computer hardware processor is to execute the specific computer-executable instructions to at least:
 receive an input query from a user computing device; 
 determine a question type for the input query; 
 determine first input based at least in part on the question type and the input query; 
 provide the first input to a trained generative model; 
 receive, in a stream from the trained generative model, first content comprising a plurality of tokens, wherein a first token of the plurality of tokens is an intermediate token; 
 determine, from the first content, first output content in a first data format different from a data format of the first content, wherein to determine the first output content, the computer hardware processor is to execute the computer-executable instructions to at least:
 replace, in the plurality of tokens, the intermediate token with content in the first data format; 
 
 transmit the first output content to the user computing device at a first time; 
 receive, in the stream from the trained generative model, second content; and 
 transmit at least some of the second content to the user computing device at a second time. 
   
     
     
         15 . The system of  claim 14 , wherein the first data format comprises a markup format or an object notation format. 
     
     
         16 . The system of  claim 14 , wherein the computer hardware processor executes further computer-executable instructions to at least:
 retrain a pre-trained language model with machine learning and a training data set comprising a prompt and response pair, wherein the prompt and response pair comprises the intermediate token, wherein to retrain the pre-trained language model, the computer hardware processor is to execute the further computer-executable instructions to at least:
 output the trained generative model. 
   
     
     
         17 . The system of  claim 14 , wherein the computer hardware processor executes further computer-executable instructions to at least:
 determine an item associated with the input query, wherein to determine the first input, the computer hardware processor is to execute the further computer-executable instructions to at least:
 determine second input based at least in part on the item. 
   
     
     
         18 . The system of  claim 17 , wherein to determine the first output content, the computer hardware processor is to execute additional computer-executable instructions to at least:
 insert, at a placeholder location in the plurality of tokens, a title associated with the item.   
     
     
         19 . The system of  claim 17 , wherein to determine the first output content, the computer hardware processor is to execute further computer-executable instructions to at least:
 insert, into intermediate content, a placeholder token; and   replace the placeholder token with metadata associated with the item.   
     
     
         20 . The system of  claim 14 , wherein the computer hardware processor executes further computer-executable instructions to at least:
 determine a related question associated with the input query, wherein to generate the first output content, the computer hardware processor is to execute the further computer-executable instructions to at least:   insert, at a placeholder location in the plurality of tokens, the related question.