Pipeline management tool

Abstract

Systems, methods, and non-transitory computer readable media are provided for managing pipelines of operations on data. A system may access data and provide a set of functions for the data. The system may receive a user's selection of one or more functions from the set of functions. The system may generate a pipeline of operations for the data based on the user's selection. The pipeline of operations may include the function(s) selected by the user.

1 . A system comprising:
 one or more processors;   memory storing instructions that, when executed by the one or more processors, cause the system to perform:
 providing access to a set of functions, each function configured to perform one or more operations on data, wherein the set of functions comprise a first function and a second function that references or calls the first function; 
 receiving a modification of the first function; 
 providing options to confirm the modification of the first function, reverse the modification of the first function, or to store the modification as a new function; 
 receiving an indication of whether to confirm or reverse the modification of the first function, or to store the modification as a new function; and 
 selectively modify the first function and the second function according to the indication. 
   
     
     
         2 . The system of  claim 1 , wherein the selectively modifying of the first function and the second function comprises:
 in response to receiving an indication to store the modification as a new function, refraining from modifying the second function.   
     
     
         3 . The system of  claim 1 , wherein the modification of the first function comprises a change in code corresponding to the first function. 
     
     
         4 . The system of  claim 1 , wherein the modification of the first function comprises a change in a restriction of a data type of an argument for the first function. 
     
     
         5 . The system of  claim 4 , wherein the instructions further cause the system to perform:
 providing a listing of portions of the data as potential arguments based on the change in the restriction of the data type; and   ranking the potential arguments.   
     
     
         6 . The system of  claim 1 , wherein the modification of the first function comprises a change in a dataframe of the first function. 
     
     
         7 . The system of  claim 1 , wherein the instructions that, when executed by the one or more processors, cause the system to perform:
 communicating an impact of the modification of the first function on the second function, wherein the impact comprises a resulting change in data operations of the second function.   
     
     
         8 . The system of  claim 1 , wherein the providing access to the set of functions comprises suggesting the set of functions based on the data or a historical usage of the set of functions and the suggesting the set of functions comprises ranking the set of functions according to a historical likelihood of use for a data type of the data or a frequency of prior use of the set of functions. 
     
     
         9 . The system of  claim 8 , wherein suggesting the set of functions comprises suggesting one or more parameters for at least one function. 
     
     
         10 . The system of  claim 1 , wherein the data is first data, and wherein execution of the instructions further cause the system to perform:
 determining that second data is historically used with the first data by the first function at least a threshold number of times; and   suggesting, via the pipeline creation interface, that the second data be used as an argument of the first function.   
     
     
         11 . A method, comprising:
 providing access to a set of functions, each function configured to perform one or more operations on data, wherein the set of functions comprise a first function and a second function that references or calls the first function;   receiving a modification of the first function;   providing options to confirm the modification of the first function, reverse the modification of the first function, or to store the modification as a new function;   receiving an indication of whether to confirm or reverse the modification of the first function, or to store the modification as a new function; and   selectively modify the first function and the second function according to the indication.   
     
     
         12 . The method of  claim 11 , wherein the selectively modifying of the first function and the second function comprises:
 in response to receiving an indication to store the modification as a new function, refraining from modifying the second function.   
     
     
         13 . The method of  claim 11 , wherein the modification of the first function comprises a change in code corresponding to the first function. 
     
     
         14 . The method of  claim 11 , wherein the modification of the first function comprises a change in a restriction of a data type of an argument for the first function. 
     
     
         15 . The method of  claim 14 , further comprising:
 providing a listing of portions of the data as potential arguments based on the change in the restriction of the data type; and   ranking the potential arguments.   
     
     
         16 . The method of  claim 11 , wherein the modification of the first function comprises a change in a dataframe of the first function. 
     
     
         17 . The method of  claim 11 , further comprising:
 communicating an impact of the modification of the first function on the second function, wherein the impact comprises a resulting change in data operations of the second function.   
     
     
         18 . The method of  claim 11 , wherein the providing access to the set of functions comprises suggesting the set of functions based on the data or a historical usage of the set of functions and the suggesting the set of functions comprises ranking the set of functions according to a historical likelihood of use for a data type of the data or a frequency of prior use of the set of functions. 
     
     
         19 . The method of  claim 18 , wherein suggesting the set of functions comprises suggesting one or more parameters for at least one function. 
     
     
         20 . A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
 providing access to a set of functions, each function configured to perform one or more operations on data, wherein the set of functions comprise a first function and a second function that references or calls the first function;   receiving a modification of the first function;   providing options to confirm the modification of the first function, reverse the modification of the first function, or to store the modification as a new function;   receiving an indication of whether to confirm or reverse the modification of the first function, or to store the modification as a new function; and   selectively modify the first function and the second function according to the indication.