Voice-based order processing

Abstract

A vehicle is detected at a drive thru terminal and an automated voice-based dialogue is established with a user in the vehicle and the terminal. An order is placed for a transaction using natural-language speech. The automated voice-based dialogue identifies commands, nouns associated with menu items, adjectives associated with modifying the menu items, prepositions associated with modifying the menu items, and exclamations associated with confirming the order and associated menu items. The automated speech is identified and processed based on a specific lexicon associated with a specific type of restaurant. The speech is translated into actions processed through an ordering system/interface as text-based instructions. Feedback for the order and guidance during the order are provided to the user through: speech played over a speaker associated with the terminal, text displayed on a display, and/or images associated with the ordered items.

1 . (canceled) 
     
     
         2 . A method, comprising:
 detecting a user in a vehicle at a drive thru terminal by image processing, by sensor-raised events, or by user-provided speech to a microphone;   greeting the user by automated speech played over a speaker and reproduced as text presented on a display of the drive thru terminal;   engaging the user in a natural language dialogue to start a session for a transaction with the user using a lexicon for automated speech processing that is specific to a menu and menu options associated with the drive thru terminal;   receiving an order from the user during the session based on the dialogue;   providing the order to order fulfillment processing for fulfillment; and   providing instructions to the user during the dialogue using the automated speech processing for the user to provide a payment to complete the transaction, the session, and the dialogue.   
     
     
         3 . The method of  claim 2 , wherein detecting further includes identifying a wake-up word or a wake-up phrase in the user-provided speech to the microphone. 
     
     
         4 . The method of  claim 2 , wherein detecting further includes detecting a vehicle adjacent to or in front of the drive thru terminal in an image captured by a camera associated with the drive thru terminal. 
     
     
         5 . The method of  claim 2 , wherein engaging further includes translating the user-provided speech during the dialogue into feedback text and presenting the feedback text on the display to confirm with the user the user-provided speech provided as responses during the dialogue. 
     
     
         6 . The method of  claim 5 , wherein translating further includes translating automated speech for the automated speech processing generated during the dialogue into automated text and presenting the automated text on the display to ensure the user hears the automated speech and can also read the automated speech during the dialogue. 
     
     
         7 . The method of  claim 2 , wherein receiving further include obtaining menu item images as spoken by the user during the dialogue and presenting the menu item images on the display as visual feedback to the user of a given menu item ordered with the order by the user. 
     
     
         8 . The method of  claim 2 , wherein receiving further includes initiating a remote voice call to an agent when the user-provided speech is unable to be translated for completing the order during the dialogue. 
     
     
         9 . The method of  claim 8 , wherein initiating further includes receiving the order details for the order from the agent. 
     
     
         10 . The method of  claim 2  further comprising, receiving the user-provided speech during the dialogue from a mobile device operated by the user. 
     
     
         11 . The method of  claim 10  further comprising, providing the automated speech generated during the dialogue to a speaker of the mobile. 
     
     
         12 . The method of  claim 2  further comprising, processing the method on a server remotely located from the drive thru terminal or processing the method on a Point-Of-Sale (POS) terminal located at a same establishment that the drive thru terminal is located. 
     
     
         13 . A method, comprising:
 automatically greeting a user detected in a vehicle adjacent to a drive thru terminal with a spoken automatically generated greeting;   obtaining a transaction identifier for a transaction from a Point-Of-Sale (POS) terminal;   configuring automated speech processing based on a lexicon associated with a menu and menu items available from the drive thru terminal;   initiating a session with the user using the transaction identifier;   engaging the user during the session in a natural language dialogue to receive order details from the user for an order the user-provided speech and automated generated speech using the automated speech processing;   instructing the user during the dialogue on where and how a payment for the order can be supplied to complete the order; and   submitting the order details with the transaction identifier to the POS terminal for order fulfillment and completion of the transaction.   
     
     
         14 . The method of  claim 13 , wherein engaging further includes obtaining menu item images as spoken by the user during the dialogue from the POS terminal and displaying the menu item images as visual feedback to the user on a display of the drive thru terminal. 
     
     
         15 . The method of  claim 13 , wherein engaging further includes translating the user-provided speech to user-identified text, translating the automated generated speech to feedback text, and presenting the user-identified text and the feedback text on a display of the drive thru terminal during the session as visual feedback to the user. 
     
     
         16 . The method of  claim 13 , wherein instructing further includes using the automated generated speech to instruct the user to pull a vehicle of the user ahead to a next location where the payment for the order will be collected from the user. 
     
     
         17 . The method of  claim 13 , wherein instructing further includes activating a card reader at the drive thru terminal to receive a payment card for the payment. 
     
     
         18 . The method of  claim 13 , wherein instructing further includes instructing the user to pull ahead to a next location to receive items associated with the order after processing the payment. 
     
     
         19 . The method of  claim 13 , wherein instructing further includes instructing the user to pull ahead to a next location to receive the items associated with the order from a locked bin that automatically unlocks when the items are placed in the bin after the order is fulfilled for collection by the user and after processing the payment. 
     
     
         20 . A system, comprising:
 a drive thru terminal comprising a display, a microphone, and a camera;   a Point-Of-Sale (POS) terminal interfaced to an order fulfillment system;   a server comprising a processor and a non-transitory computer-readable storage medium;   the non-transitory computer-readable storage medium comprises executable instruction;   the executable instructions when executed by the processor from the non-transitory computer-readable storage medium cause the processor to perform operations comprising:
 automatically initiating a voice-based dialogue with a user when a vehicle of the user is detected adjacent to the drive thru terminal based on images provided by the camera or when the user speaks a wake-up word or phrase into the microphone; 
 generating automated speech responses to user-provided speech during the voice-based dialogue; 
 obtaining order details for an order being placed by the user based on the user-provided speech during the voice-based dialogue; 
 providing the order details to the POS terminal for order fulfillment; and 
 instructing the user on how and where to provide a payment to pay for the order and complete the order. 
   
     
     
         21 . The system of  claim 20 , wherein the server is a cloud processing environment and the drive thru terminal is a thin client device.