Scalable interactive display of distributed data

Abstract

A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a “divide and conquer” algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.

1 . A method comprising:
 receiving a query to search a plurality of event records that are stored across a plurality of computing devices;   submitting a sub-query derived from the received query to each of the plurality of computing devices, wherein each of the plurality of computing devices analyzes event records stored on that computing device, based on the sub-query, to generate one or more event references and to generate an analysis result;   receiving one or more event references from each of the plurality of computing devices, wherein each event reference includes a unique event record identifier and an event order value;   combining the one or more event references from each of the plurality of computing devices into a list of event references ordered by event order value;   providing the ordered list of event references in response to the query;   receiving a request for event records associated with a contiguous subset of the ordered list of event references;   requesting event records from one or more of the plurality of computing devices by sending each unique event record identifier included in the requested contiguous subset of the ordered list of event references to the corresponding computing device; and   providing the requested event records and the analysis result to a display device for display.   
     
     
         2 . The method of  claim 1 , wherein each event reference of the ordered list of event references includes an identifier of the computing device that provided the event reference, and wherein retrieving event records associated with an event reference includes retrieving an event record from the computing device associated with the event reference. 
     
     
         3 . The method of  claim 1 , wherein event records are analyzed by parsing a log file containing event records and returning event references for each event record satisfying the sub-query. 
     
     
         4 . The method of  claim 1 , wherein sub-queries are submitted to computing devices that match a criteria included in the query. 
     
     
         5 . The method of  claim 1 , wherein the event references are ordered based on an order indicated in the query. 
     
     
         6 . The method of  claim 1 , wherein the query includes a first and a second ordering indication, wherein event references include a first order value and a second order value, wherein event references are ordered first by the first ordering indication, and wherein any event references that have the same first order value are ordered by the second order value. 
     
     
         7 . The method of  claim 1 , wherein an event record includes raw event data, the unique event record identifier, and the event order value, and wherein the event references do not include raw event data. 
     
     
         8 . An apparatus comprising:
 a processor; and   a memory storing instructions that when executed by the processor cause actions to be performed, including:
 receiving one or more event references from each of a plurality of computing devices, wherein each event reference includes an identifier of a source computing device, an index into an array of event records stored on the source computing device, and an event order value; 
 combining the one or more event references from each of the plurality of computing devices into a list of event references ordered by event order value; 
 receiving a request for event records associated with a contiguous range of the ordered list of event references; 
 retrieving event records from one or more of the plurality of computing devices by sending the index to the corresponding source computing device for each event reference in the requested contiguous range of the ordered list of event references; and 
 providing the requested event records to a display device for display. 
   
     
     
         9 . The apparatus of  claim 8 , wherein the contiguous range of the ordered list of event references is selected from an interactive report displayed at the display device. 
     
     
         10 . The apparatus of  claim 8 , wherein the one or more event references are received in response to a query received from the display device. 
     
     
         11 . The apparatus of  claim 8 , wherein the plurality of computing devices are selected based on one or more of a form factor, an operating system, or a hardware component. 
     
     
         12 . The apparatus of  claim 8 , wherein each computing device orders the one or more event references based on the event order value. 
     
     
         13 . A non-transitory processor readable storage device storing instructions that cause a processor to perform actions, comprising:
 receiving one or more event references from each of a plurality of computing devices, wherein each event reference includes an identifier of a source computing device, an index into an array of event records stored on the source computing device, and an event order value;   receiving, in response to a query request, statistical information from each of the plurality of computing devices, where the statistical information is generated by each of the plurality of computing devices performing an analysis of event records stored on that computing device;   combining the one or more event references from each of the plurality of computing devices into a list of event references, ordered by event order value;   combining the statistical information from each of the computing devices into an aggregated statistical information;   receiving a request for event records associated with a portion of the aggregated statistical information;   retrieving the requested event records from the plurality of computing devices by sending, for each event reference associated with the portion of the aggregated statistical information, the index into the array of event records stored on the source computing device to the corresponding source computing device; and   providing the requested event records to a display device for display.   
     
     
         14 . The processor readable storage device of  claim 13 , wherein the statistical information includes an order statistic or an approximate order statistic. 
     
     
         15 . The processor readable storage device of  claim 13 , wherein the statistical information includes a count of the number of event records matching a criteria included in a query received from the display device. 
     
     
         16 . A system comprising:
 a plurality of computing devices storing event records;   a first computing device configured to perform actions comprising:
 receiving one or more event references from each of the plurality of computing devices, wherein each event reference includes a unique event record identifier and an event order value; 
 combining the one or more event references from each of the plurality of computing devices into a list of event references ordered by event order value; 
 receiving a request for event records associated with a contiguous range of ordered event references; 
 retrieving event records from one or more of the plurality of computing devices by sending each unique event record identifier included in the requested contiguous range of ordered event references to the corresponding computing device; and 
 providing the requested event records for display. 
   
     
     
         17 . The system of  claim 16 , wherein each event reference of the ordered list of event references includes an identifier of the computing device that provided the event reference, and wherein retrieving event records associated with an event reference includes retrieving an event record from the computing device associated with the event reference. 
     
     
         18 . The system of  claim 16 , the actions further comprising:
 parsing a log file containing event records to generate a statistical analysis; and   providing the statistical analysis event references for each event record that satisfy a query.   
     
     
         19 . The system of  claim 16 , wherein an event record includes data collected about a computing system and is stored on the same computing system. 
     
     
         20 . The system of  claim 16 , wherein the ordered list of event references is ordered by a first order value and a second order value, wherein event references are ordered first by the first order value, and wherein any event references that have the same first order value are ordered by the second order value.