Methods and systems for food preparation in a robotic cooking kitchen

Abstract

The present disclosure is directed to methods, computer program products, and computer systems for instructing a robot to prepare a food dish by replacing the human chef's movements and actions. Monitoring a human chef is carried out in an instrumented application-specific setting, a standardized robotic kitchen in this instance, and involves using sensors and computers to watch, monitor, record and interpret the motions and actions of the human chef, in order to develop a robot-executable set of commands robust to variations and changes in the environment, capable of allowing a robotic or automated system in a robotic kitchen to prepare the same dish to the standards and quality as the dish prepared by the human chef.

What is claimed and desired to be secured by Letters Patent of the United States is: 
     
         1 . A robotic kitchen system, comprising:
 one or more robotic arms;   one or more robotic end effectors coupled to the one or more robotic arms, each end effector coupled to a respective robotic arm;   at least one processor communicatively coupled to the one or more robotic arms, the at least one processor being operable to:   receive an electronic recipe file including a machine-executable sequential commands script from a source; and   control the one or more robotic arms and the one or more robotic end effectors to replicate one or more cooking operations by executing the machine-executable sequential commands script of the electronic recipe file, each corresponding arm and end effector performing a cooking operation upon receiving a command.   
     
     
         2 . The system of  claim 1 , further comprising one or more sensors configured to collect sensor data, each corresponding arm and end effector performing a cooking operation upon receiving a command based at least in part on the collected sensor data. 
     
     
         3 . The system of  claim 2 , wherein the sensor data for operating in an instrumented environment, comprising (a) detecting the presence or absence of one or more objects in a standard position in an instrumented environment, (b) identifying an object type, position and orientation for a standard object placed in a non-standard position, and (c) identifying an object type, shape, dimensions, positions and orientations for a non-standard object placed in a non-standard position. 
     
     
         4 . The system of  claim 1 , wherein the processor is computed to collect sensor data as feedback to confirm or make real-time adjustment to a current operation. 
     
     
         5 . The system of  claim 1 , wherein the processor is computed to collect sensor data as feedback on one or smart appliances, one or more tools, or one or more ingredients for command execution. 
     
     
         6 . The system of  claim 5 , wherein sensor data comprises object status data, appliance status data, temperature data, humidity data, object color data, and object changing-shape data. 
     
     
         7 . The system of  claim 1 , wherein each command in the machine-executable sequential commands script comprises at least one preprogrammed cooking operation. 
     
     
         8 . The system of  claim 1 , wherein each command in the machine-executable sequential commands script comprises at least one pretested cooking operation. 
     
     
         9 . The system of  claim 1 , wherein the machine-executable sequential commands script, executed by the processor, comprises at least one command for controlling for controlling one or more smart appliances. 
     
     
         10 . The system of  claim 1 , wherein each corresponding arm and end effector performs a cooking operation upon receiving a cooking command by a user. 
     
     
         11 . The system of  claim 1 , wherein each corresponding arm and end effector performs a cooking operation upon receiving a command from the electronic recipe file. 
     
     
         12 . The system of  claim 1 , wherein the electronic recipe file is prerecorded, or generated based on one or more recipe creator real-time commands, via a computer interface. 
     
     
         13 . The system of  claim 1 , wherein the command to perform a cooking operation comprises a specific type of cooking operation, at least one parameter of the specific cooking operation, and timing data associated with the specific cooking operation. 
     
     
         14 . The system of  claim 1 , wherein the robotic kitchen system operates in a first mode and a second mode: during the first mode, the processor recording the machine-executable sequential commands script; and during the second mode, the processor executing the machine-executable sequential commands script. 
     
     
         15 . A robotic kitchen system, comprising:
 a kitchen module defining an instrumented environment for which the robotic kitchen module operates within;   one or more robotic arms;   one or more robotic end effectors coupled to the one or more robotic arms, each end effector coupled to a respective robotic arm;   one or more sensors are configured to collect sensor data;   at least one processor communicatively coupled to the one or more robotic arms, the at least one processor being operable to:   receive an electronic recipe file including a machine-executable sequential command script from a source; and   control the one or more robotic arms and the one or more robotic end effectors within the kitchen module to replicate one or more food preparation operations by executing the machine-executable sequential command script of the electronic recipe file, each corresponding arm and end effector performing a cooking operation in response to receiving a command using at least in part on the collected sensor data.   
     
     
         16 . The system of  claim 15 , wherein the machine-executable sequential commands script, executed by the processor, comprises at least one command for controlling for controlling one or more smart appliances. 
     
     
         17 . The system of  claim 15 , wherein the robotic kitchen system is located outside the kitchen module, the kitchen module defining an instrumented environment, the robotic kitchen system operating within the instrumented environment. 
     
     
         18 . The kitchen system of  claim 15 , wherein the kitchen module comprises one or more actuators for repositioning the one or more robotic arms and the one or more robotic end effectors to perform a specific operation within a requested part of the kitchen module. 
     
     
         19 . The system of  claim 15 , wherein the robotic kitchen system operates in a first mode and a second mode: during the first mode, the processor recording the machine-executable sequential commands script; and during the second mode, the processor executing the machine-executable sequential commands script. 
     
     
         20 . The system of  claim 15 , wherein the processor adjusting parameters for the robotic instructions machine-executable sequential command script based on at least in part of the sensor data. 
     
     
         21 . A method for operating a robotic system, performed by at least one processor, comprising:
 executing an electronic recipe file that contains a food preparation recipe, the electronic recipe file including machine-executable sequential command script and timing data originated from a recipe creator; and   controlling at least one robotic apparatus in an instrumented environment for replicating the food preparation recipe by executing the machine-executable sequential command script of the electronic recipe file;   wherein machine-executable sequential command script comprises one or more commands, each command in the command script activating at a predetermined time with a specified time duration.   
     
     
         22 . The method of  claim 21 , wherein the controlling step comprises receiving sensor data in the instrumented environment as generated by one or more sensors, comprising (a) detecting the presence or absence of one or more objects in a standard position in an instrumented environment, (b) identifying an object type, position and orientation for a standard object placed in a non-standard position, and (c) identifying an object type, shape, dimensions, positions and orientations for a non-standard object placed in a non-standard position. 
     
     
         23 . The method of  claim 21 , wherein the instrumented environment comprises
 one or more predefined objects, including one or more ingredients, one or more appliances, one or more utensils, one or more containers, one or more equipment, or in any combination; and   predefined operation space dimensions on x-axis, y-axis and z-axis.   
     
     
         24 . The method of  claim 21 , wherein the instrumented environment comprises a standardized instrumented environment that includes one or more standardized objects, each standardized object in a respective standardized position in a respective standardized orientation. 
     
     
         25 . The method of  claim 21 , wherein the machine-executable sequential command script comprises a sequence of food preparation commands; and wherein the electronic recipe file is prerecorded, or generated based on one or more recipe creator real-time commands, via a computer interface. 
     
     
         26 . The method of  claim 22 , wherein the instrumented environment is used to create one or more single commands and execute the one or more single commands. 
     
     
         27 . The method of  claim 22 , wherein the instrumented environment is used to create machine-executable sequential command script and execute the machine-executable sequential command script. 
     
     
         28 . The method of  claim 21 , wherein the timing data comprises at least one predetermined start time of execution associated with at least one particular cooking operation. 
     
     
         29 . The method of  claim 21 , wherein the timing data comprises a predetermined start time of execution and duration for a specific cooking operation. 
     
     
         30 . A robotic kitchen system, comprising:
 one or more robotic arms;   one or more robotic end effectors coupled to the one or more robotic arms, each end effector coupled to a respective robotic arm;   one or more sensors are configured to collect sensor data;   at least one processor communicatively coupled to the one or more robotic arms, the at least one processor being operable to:   receive an electronic recipe file including a mini-manipulation machine-executable sequential command script from a source;   control the one or more robotic arms and the one or more robotic end effectors to replicate one or more cooking operations by executing the mini-manipulation machine-executable sequential commands script of the electronic recipe file, each corresponding arm and end effector performing a cooking operation upon receiving a command based at least in part on the collected sensor data.