Methods and systems for food preparation in a robotic cooking kitchen

Abstract

The present disclosure is directed to methods, computer program products, and computer systems for instructing a robot to prepare a food dish by replacing the human chef's movements and actions. Monitoring a human chef is carried out in an instrumented application-specific setting, a standardized robotic kitchen in this instance, and involves using sensors and computers to watch, monitor, record and interpret the motions and actions of the human chef, in order to develop a robot-executable set of commands robust to variations and changes in the environment, capable of allowing a robotic or automated system in a robotic kitchen to prepare the same dish to the standards and quality as the dish prepared by the human chef.

1 .- 138 . (canceled) 
     
     
         139 . A robotic kitchen system comprising:
 a robotic apparatus including:
 one or more robotic arms; 
 one or more robotic end effectors coupled to the one or more robotic arms, the one or more robotic end effectors including at least one of:
 (i) one or more robotic hands including one or more fingers, 
 (ii) one or more grippers including one or more fingers, and 
 (iii) one or more holders; and 
 
   at least one processor communicatively coupled to the robotic apparatus, the at least one processor being operable to:
 receive a file corresponding to a cooking recipe, the file including a machine-executable sequential command script and being generated based on a combination of chef studio sensor data measured by one or more sensors in a chef studio system; and 
 control the robotic apparatus to replicate the cooking recipe by executing the machine-executable sequential command script of the file. 
   
     
     
         140 . The robotic kitchen system of  claim 139 ,
 wherein the robotic apparatus further includes at least one of:
 (i) one or more wrists corresponding to each of the one or more robotic end effectors, each of the one or more wrists being operable to couple the respective one or more robotic end effectors to the one or more robotic arms, and each of the one or more wrists being movable along one or more degrees of freedom, and 
 (ii) one or more palms corresponding to each of the one or more hands, the one or more palms being coupled to the respective one or more fingers. 
   
     
     
         141 . The robotic kitchen system of  claim 139 , wherein the one or more wrists form part of a respective one of the one or more end effectors. 
     
     
         142 . The robotic kitchen of  claim 140 , wherein the robotic apparatus includes the at least one or more wrists and the at least one or more palms. 
     
     
         143 . The robotic kitchen system of  claim 140 , wherein the robotic apparatus further includes one or more sensors, the one or more sensors being included in at least one of:
 (i) the one or more robotic arms,   (ii) the one or more robotic end effectors,   (iii) the one or more wrists, and   (iv) the one or more palms.   
     
     
         144 . The robotic kitchen system of  claim 143 , wherein the one or more sensors include a camera. 
     
     
         145 . A kitchen module comprising:
 the robotic kitchen system of  claim 140 .   
     
     
         146 . The kitchen module of  claim 145 ,
 wherein the robotic apparatus further includes a torso movable along one or more degrees of freedom, and   wherein at least one end of each of the one or more robotic arms is connected to the torso.   
     
     
         147 . The kitchen module of  claim 146 , wherein the torso is rotatable about one or more axes. 
     
     
         148 . The kitchen module of  claim 145 , further comprising a computer-controllable actuator system including one or more actuators, at least one of the one or more actuators being connected to the robotic apparatus,
 wherein the one or more actuators are configured to enable the movement of at least a portion of the robotic apparatus along one or more axes.   
     
     
         149 . The kitchen module of  claim 148 , wherein each of the one or more axes are different from one another. 
     
     
         150 . The kitchen module of  claim 145 , further comprising the chef studio system. 
     
     
         151 . The kitchen module of  claim 145 , further comprising:
 a safety screen; and   a hood portion configured to receive and store at least a portion of the robotic apparatus,   wherein the processor is further configured to:
 cause the at least a portion of the robotic apparatus to be extracted into and stored in the hood portion to transition the cooking module from a robotic cooking mode to a manual cooking mode. 
   
     
     
         152 . The kitchen module of  claim 145 , further comprising a plurality of kitchen module sensors configured to collect kitchen module sensor data during the replication of the cooking recipe. 
     
     
         153 . The kitchen module of  claim 152 , wherein the processor is further operable to:
 determine the accuracy of the replication of the cooking recipe based on at least a portion of the respective file and at least a portion of the collected kitchen module sensor data.   
     
     
         154 . The kitchen module of  claim 153 , wherein the accuracy of the replication of the cooking recipe is based on a comparison of a result of executing the cooking recipe with the chef studio system versus the result of executing the machine-executable sequential command script with the robotic apparatus. 
     
     
         155 . The kitchen module of  claim 152 ,
 wherein the replicating of the cooking recipe is configured such that the executing the machine-executable sequential command script achieves a set of one or more functional results corresponding to the cooking recipe.   
     
     
         156 . The kitchen module of  claim 154 ,
 wherein the determination of the accuracy of the replication of the cooking recipe is performed during the executing of the machine-executable sequential command script of the file, and   wherein the processor is further operable to make real-time adjustments to the file based on the determination.   
     
     
         157 . The kitchen module of  claim 156 ,
 wherein at least one of the one or more robotic end effectors includes a glove.   
     
     
         158 . The kitchen module of  claim 157 , wherein at least one of the kitchen module sensors is embedded in the glove corresponding to the one of the one or more robotic end effectors. 
     
     
         159 . The kitchen module of  claim 145 ,
 wherein the kitchen module is a standardized kitchen module including one or more of standardized equipment, appliances, utensils, tools, handles, and containers,   wherein characteristics of the standardized kitchen module are predefined, and   wherein the standardized kitchen module is configured to perform standardized operations that are pre-programmed and pre-tested.   
     
     
         160 . The kitchen module of  claim 159 ,
 wherein one or more of the standardized equipment, appliances, utensils, tools, handles and containers are smart equipment, smart appliances, smart utensils, smart tools, smart handles and smart containers operable to communicate with and be controlled by the robotic kitchen system.   
     
     
         161 . The kitchen module of  claim 145 ,
 wherein, if the kitchen module differs from a chef studio module corresponding to the chef studio system, the processor is further operable to:
 modify one or more commands of the machine-executable sequential command script to replicate the cooking recipe in the kitchen module, the modifications of the one or more commands based on the differences between the kitchen module and the chef studio module. 
   
     
     
         162 . The robotic kitchen system of  claim 139 , further comprising:
 at least one memory communicatively coupled to the at least one processor, the at least one memory being operable to store a recipe script database including a plurality of available files corresponding to respective cooking recipes, each of the available files including respective machine-executable sequential command scripts,   wherein the received file is received from the at least one memory.   
     
     
         163 . The robotic kitchen system of  claim 162 ,
 wherein the recipe script database further includes, for each of the plurality of available files, one or more of raw data and abstracted data corresponding to the respective machine-executable sequential command scripts.   
     
     
         164 . A kitchen module comprising:
 the robotic kitchen system of  claim 162 ,   wherein the machine-executable sequential command scripts of the plurality of available files are pre-programmed and pre-tested.   
     
     
         165 . The kitchen module of  claim 164 ,
 wherein the robotic kitchen system is operable to self-learn during the executing of the machine-executable sequential command scripts, and   wherein the self-learning includes updating the machine executable sequential command scripts.   
     
     
         166 . The kitchen module of  claim 164 , wherein the pre-programing or pre-testing of the machine-executable sequential command scripts are specifically performed for execution by the kitchen module. 
     
     
         167 . The robotic kitchen system of  claim 164 ,
 wherein the raw data is received from the chef studio system and includes the chef studio sensor data measured by the one or more sensors in the chef studio system, and   wherein the processor is further operable to generate the file by translating at least a portion of the raw data into the respective machine-executable sequential command script.   
     
     
         168 . The robotic kitchen system of  claim 139 ,
 wherein the machine-executable sequential command script includes a plurality of commands,   wherein at least one of the plurality of commands includes a plurality of functions performed simultaneously by different ones of the one or more robotic end effectors.   
     
     
         169 . A robotic system comprising:
 a robotic apparatus comprising one or more robotic end effectors, at least one of the one or more robotic end effectors including one or more sensors,   wherein the one or more robotic end effectors are configured to (i) collect sensor data via the one or more sensors, and (ii) replicate a process recipe by executing a machine-executable sequential command script corresponding to the process recipe, based at least in part on the collected sensor data.   
     
     
         170 . The robotic kitchen system of  claim 169 , wherein the one or more sensors include a camera. 
     
     
         171 . A method for robotic replication of recipes, comprising:
 receiving a file corresponding to a cooking recipe, the file including a machine-executable sequential command script and being generated based on chef studio sensor data measured by one or more sensors in a chef studio system; and   controlling one or more robotic arms and robotic hands of a robotic apparatus to replicate the cooking recipe by executing the machine-executable sequential command script of the received file.   
     
     
         172 . The method of  claim 170 , wherein the robotic arms and the robotic hands of the robotic apparatus are further controlled by user-input entered via an interface communicatively coupled thereto. 
     
     
         173 . The method of  claim 171 , further comprising:
 generating the machine-executable sequential command script based on at least a portion of the chef studio sensor data,   wherein the machine-executable sequential command script is generated specifically for execution by a kitchen module different than the chef studio system.   
     
     
         174 . The method of  claim 171 , further comprising:
 collecting kitchen module sensor data during the replication of the cooking recipe; and   monitoring, in real-time, an accuracy of the replication of the cooking recipe by comparing at least a portion of the kitchen module sensor data to at least a portion of the chef studio sensor data.   
     
     
         175 . The method of  claim 174 , further comprising:
 self-learning, during the replication of the cooking recipe based on at least a portion of the kitchen module sensor data and/or the chef studio sensor data, the self-learning including updating the machine-executable sequential command script.