Multi-Language Support for Interfacing with Distributed Data

Abstract

A data analysis system stores in-memory representation of a distributed data structure across a plurality of processors of a parallel or distributed system. Client applications interact with the in-memory distributed data structure to process queries using the in-memory distributed data structure and to modify the in-memory distributed data structure. The data analysis system creates uniform resource identifier (URI) to identify each in-memory distributed data structure. The URI can be communicated from one client application to another application using communication mechanisms outside the data analysis system, for example, by email, thereby allowing other client devices to interact with a particular in-memory distributed data structure. The in-memory distributed data structure can be a machine learning model that is trained by one client device and executed by another client device. A client application can interact with the in-memory distributed data structure using different programming languages.

1 . A computer-implemented method, comprising:
 storing a document associated with a distributed data-frame representing an in-memory distributed data structure distributed across a plurality of processors;   sharing the document among a plurality of user accounts;   receiving a request from a first user account of the plurality of user accounts to designate at least a portion of the document for local editing;   receiving, from the first user account, a modification of the document as a part of the local editing;   receiving, from the first user account, to share the local editing to a new set of collaborators; and   propagating the modification of the document to the new set of collaborators, the propagating excluding the plurality of user accounts outside of the new set of collaborators.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the document comprises one or more charts visualizing data obtained by processing a query based on data stored in the distributed data-frame. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein sharing the document among the plurality of user accounts comprising:
 allowing the plurality of user accounts to edit the shared document, wherein an edit from one of the plurality of user account is propagated to the plurality of user accounts.   
     
     
         4 . The computer-implemented method of  claim 1 , further comprising:
 receiving a second modification of the document outside the portion of the document designated for local editing; and   propagating the second modification of the document to the plurality of user accounts include the user accounts outside of the new set of collaborators.   
     
     
         5 . The computer-implemented method of  claim 1 , further comprising:
 receiving a request to update data stored in the distributed data frame;   generating an updated distributed data frame based on the request;   providing a first chart obtained by a first query of the updated distributed data frame including the local editing to the new set of collaborators; and   providing a second chart obtained by a second query of the updated distributed data frame excluding the local editing to the plurality of user accounts outside of the new set of collaborators.   
     
     
         6 . The computer-implemented method of  claim 1 , further comprising:
 receiving a request for globally sharing the local editing of the portion of the document with the plurality of user accounts; and   propagating the modification of the document to the plurality of user accounts including the user accounts outside of the new set of collaborators.   
     
     
         7 . The computer-implemented method of  claim 1 , further comprising:
 making a copy of the portion of the document designated for local editing; and   responsive to receiving the modification of the document, performing the modification on the copy.   
     
     
         8 . The computer-implemented method of  claim 1 , wherein the in-memory distributed data structure is a machine learning model, and wherein a request to update data stored in the distributed data-frame comprises a request to train the machine learning model. 
     
     
         9 . The computer-implemented method of  claim 1 , further comprising:
 receiving a second modification of the document form a user account outside of the new set of collaborators;   propagating the second modification of the document to the plurality of user accounts including the new set of collaborators and other user accounts outside of the new set of collaborators.   
     
     
         10 . The computer-implemented method of  claim 1 , wherein each user account is associated with a client device. 
     
     
         11 . A non-transitory computer readable medium for storing computer code comprising instructions, when executed by one or more processors, causing the one or more processors to perform steps comprising:
 storing a document associated with a distributed data-frame representing an in-memory distributed data structure distributed;   sharing the document among a plurality of user accounts;   receiving a request from a first user account of the plurality of user accounts to designate at least a portion of the document for local editing;   receiving, from the first user account, a modification of the document as a part of the local editing;   receiving, from the first user account, to share the local editing to a new set of collaborators; and   propagating the modification of the document to the new set of collaborators, the propagating excluding the plurality of user accounts outside of the new set of collaborators.   
     
     
         12 . The non-transitory computer readable medium of  claim 11 , wherein the document comprises one or more charts visualizing data obtained by processing a query based on data stored in the distributed data-frame. 
     
     
         13 . The non-transitory computer readable medium of  claim 11 , wherein sharing the document among the plurality of user accounts comprising:
 allowing the plurality of user accounts to edit the shared document, wherein an edit from one of the plurality of user account is propagated to the plurality of user accounts.   
     
     
         14 . The non-transitory computer readable medium of  claim 11 , wherein the steps further comprise:
 receiving a second modification of the document outside the portion of the document designated for local editing; and   propagating the second modification of the document to the plurality of user accounts include the user accounts outside of the new set of collaborators.   
     
     
         15 . The non-transitory computer readable medium of  claim 11 , wherein the steps further comprise:
 receiving a request to update data stored in the distributed data frame;   generating an updated distributed data frame based on the request;   providing a first chart obtained by a first query of the updated distributed data frame including the local editing to the new set of collaborators; and   providing a second chart obtained by a second query of the updated distributed data frame excluding the local editing to the plurality of user accounts outside of the new set of collaborators.   
     
     
         16 . The non-transitory computer readable medium of  claim 11 , wherein the steps further comprise:
 receiving a request for globally sharing the local editing of the portion of the document with the plurality of user accounts; and   propagating the modification of the document to the plurality of user accounts including the user accounts outside of the new set of collaborators.   
     
     
         17 . The non-transitory computer readable medium of  claim 11 , wherein the steps further comprise:
 making a copy of the portion of the document designated for local editing; and   responsive to receiving the modification of the document, performing the modification on the copy.   
     
     
         18 . The non-transitory computer readable medium of  claim 11 , wherein the in-memory distributed data structure is a machine learning model, and wherein a request to update data stored in the distributed data-frame comprises a request to train the machine learning model. 
     
     
         19 . The non-transitory computer readable medium of  claim 11 , wherein the steps further comprise:
 receiving a second modification of the document form a user account outside of the new set of collaborators;   propagating the second modification of the document to the plurality of user accounts including the new set of collaborators and other user accounts outside of the new set of collaborators.   
     
     
         20 . The non-transitory computer readable medium of  claim 11 , wherein each user account is associated with a client device.