Modularized model interaction system and method

Abstract

A modularized model interaction system and method of use, including an orchestrator, a set of hardware modules each including a standard set of hardware submodules with hardware-specific logic, and a set of model modules each including a standard set of model submodules with model-specific logic. In operation, the orchestrator determines a standard set of submodule calls to the standard submodules of a given hardware module and model module to implement model interaction on hardware associated with the hardware module.

We claim: 
     
         1 . A method for modular model implementation, comprising, at an orchestration module:
 receiving a set of requests identifying a set of models;   for each model of the set of models, initializing an instance of the model using a model module, associated with the respective model, from a set of model modules; and   for each instance of a model of the set, executing a same series of standard submodules from the respective model module, wherein each standard submodule comprises standard model-specific logic, and wherein at least one standard submodule additionally comprises user-defined model-specific logic.   
     
     
         2 . The method of  claim 1 , wherein a model module of the set of model modules further comprises a callback submodule, wherein the callback submodule comprises user-defined logic. 
     
     
         3 . The method of  claim 1 , wherein the at least one standard submodule comprises a mechanism to override the standard model-specific logic and use the user-defined model-specific logic. 
     
     
         4 . The method of  claim 1 , wherein the model module further comprises a model submodule, wherein the model submodule comprises logic for a set of machine learning models. 
     
     
         5 . The method of  claim 1 , wherein the model module further comprises an optimizer submodule associated with a set of optimizers, wherein the optimizer submodule comprises optimizer-specific logic for each optimizer of the set of optimizers. 
     
     
         6 . The method of  claim 1 , wherein each request of the set of requests identifies a hardware type. 
     
     
         7 . The method of  claim 6 , wherein the hardware type comprises at least one of: a central processing unit (CPU), graphics processing unit (GPU), image processing unit (IPU), or tensor processing unit (TPU). 
     
     
         8 . The method of  claim 6 , further comprising, for each request, initializing an instance of the respective hardware type using a hardware module, associated with the respective hardware type, from a set of hardware modules, wherein initializing the instance of the respective hardware type comprises executing a series of standard hardware submodules from the respective hardware module. 
     
     
         9 . The method of  claim 8 , wherein each hardware module comprises the same series of standard hardware submodules, wherein each hardware module defines logic specific to the respective hardware type within the respective standard hardware submodules. 
     
     
         10 . The method of  claim 8 , wherein, for each request, the respective model module is executed on the respective instance of the hardware type. 
     
     
         11 . A system for modular model implementation, comprising:
 a set of model modules, wherein each model module of the set is associated with a different model and comprises a same series of standard submodules as the other model modules, wherein each standard submodule of a model module comprises:
 standard model-specific logic; and 
 a standard hook associated with user-defined model-specific logic; and 
   an orchestration module configured to:
 receive a set of requests identifying a set of models; 
 execute the model module associated with each model within the set of models, comprising, for each standard submodule of the respective model module:
 executing the user-defined model-specific logic when the standard hook is triggered; and 
 executing the standard model-specific logic when the standard hook is not triggered. 
 
   
     
     
         12 . The system of  claim 11 , wherein a model module further comprises a callback submodule, wherein the callback submodule further comprises user-defined logic. 
     
     
         13 . The system of  claim 12 , wherein the standard hook for the model module calls the callback submodule. 
     
     
         14 . The system of  claim 11 , wherein a model module of the set of model modules further comprises a model submodule, wherein the model submodule comprises logic for an ensemble of machine learning models. 
     
     
         15 . The system of  claim 11 , wherein a model module of the set of model modules further comprises an optimizer submodule comprising optimizer-specific logic for each optimizer of a set of optimizers. 
     
     
         16 . The system of  claim 11 , further comprising a set of data preparation modules. 
     
     
         17 . The system of  claim 11 , further comprising a set of hardware modules, each associated with a different hardware type and comprising logic specific to the respective hardware type. 
     
     
         18 . The system of  claim 17 , wherein each hardware module of the set of hardware modules comprises a same set of hardware submodules, wherein the same hardware submodule from different hardware modules comprises different logic specific to the respective hardware type. 
     
     
         19 . The system of  claim 17 , wherein each request identifies a model and a hardware type, wherein the model module for the model is executed using an instance of the hardware type initialized using the hardware module for the hardware type. 
     
     
         20 . The system of  claim 17 , wherein the hardware type comprises at least one of: a central processing unit (CPU), graphics processing unit (GPU), image processing unit (IPU), or tensor processing unit (TPU).