Distributed storage of aggregated data

Abstract

Techniques are described for managing aggregation of data in a distributed manner, such as for a particular client based on specified configuration information. The described techniques may include storing aggregated data values for an OLAP cube or other data structure in a distributed manner, such as in some situations in a distributed hash table. The aggregated data values to be stored may be generated in various manners, such as by performing multi-stage data manipulation operations—for example, a map-reduce architecture may be used, with a first stage involving the use of one or more specified map functions to be performed, and with at least a second stage involving the use of one or more specified reduce functions to be performed.

1 - 29 . (canceled) 
     
     
         30 . A computer-implemented method comprising:
 generating, by one or more configured computing systems, a hash key to be used in conjunction with an aggregated data value corresponding to one or more multi-dimensional arrays, wherein the hash key comprises information representing one or more aggregation metrics associated with the aggregated data value; and   causing, by the one or more configured computing systems, the aggregated data value to be stored in a storage location that is determined based at least in part on the generated hash key.   
     
     
         31 . The computer-implemented method of  claim 30  wherein the aggregated data value is an aggregation of multiple other data values that are associated with a specified value for each of multiple data dimensions within the one or more multi-dimensional arrays, and wherein the generating of the hash key is further based at least in part on the specified values for the multiple data dimensions. 
     
     
         32 . The computer-implemented method of  claim 30  further comprising determining the storage location by supplying the generated hash key as input to a hash function and using output of the hash function to select a storage location within a distributed key-value storage structure stored across multiple network-accessible storage nodes. 
     
     
         33 . The computer-implemented method of  claim 30  wherein the causing of the aggregated data value to be stored in the storage location includes storing the aggregated data value in the storage location. 
     
     
         34 . The computer-implemented method of  claim 30  further comprising, after causing the aggregated data value to be stored in the storage location:
 receiving a request for information that includes the aggregated data value and that indicates at least one of the one or more aggregation metrics associated with the aggregated data value, and 
 using, in response to the request, the indicated at least one aggregation metric to obtain and provide the stored aggregated data value from the storage location. 
 
     
     
         35 . The computer-implemented method of  claim 34  wherein the using of the indicated at least one aggregation metric to obtain and provide the stored aggregated data value includes determining one or more hash keys corresponding to the indicated at least one aggregation metric, and using the determined one or more hash keys to retrieve the stored aggregated data value from the storage location. 
     
     
         36 . The computer-implemented method of  claim 34  wherein the received request further indicates a value for each of one or more specified data dimensions within the one or more multi-dimensional arrays, and wherein the using the indicated at least one aggregation metric to obtain and provide the stored aggregated data value further includes using the indicated values for at least one of the one or more specified data dimensions. 
     
     
         37 . A non-transitory computer-readable medium having stored contents that cause one or more computing systems to perform automated operations including at least:
 generating, by the one or more computing systems and based on an aggregation metric used to calculate an aggregated data value associated with one or more multi-dimensional data arrays, a hash key based on the aggregation metric and for use in conjunction with a distributed data storage structure having multiple network-accessible storage locations; and   causing, by the one or more computing systems, the aggregated data value to be stored in one of the multiple network-accessible storage locations that is determined based at least in part on the generated hash key.   
     
     
         38 . The non-transitory computer-readable medium of  claim 37  wherein the aggregated data value is based on multiple other data values that are associated with specified dimension category values for multiple dimension categories within the one or more multi-dimensional data arrays, and wherein the stored contents further cause the one or more computing systems to generate the hash key using the specified dimension category values for the multiple dimension categories. 
     
     
         39 . The non-transitory computer-readable medium of  claim 37  wherein the stored contents include software instructions that, when executed, further cause the one or more computing systems to determine the one network-accessible storage location by supplying the generated hash key as input to a hash function and by using output of the hash function to select the one network-accessible storage location within the distributed data storage structure, and to store the aggregated data value in the one network-accessible storage location. 
     
     
         40 . The non-transitory computer-readable medium of  claim 37  wherein the stored contents further cause the one or more computing systems to, after the aggregated data value is stored in the one network-accessible storage location:
 receive a request for information that indicates at least the aggregation metric used to calculate the aggregated data value, and 
 obtain, in response to the request and by using the indicated aggregation metric, the stored aggregated data value from the one network-accessible storage location, and provide the obtained stored aggregated data value. 
 
     
     
         41 . The non-transitory computer-readable medium of  claim 40  wherein the using of the indicated aggregation metric includes generating a new hash key corresponding to the indicated aggregation metric that matches the generated hash key used for storing the aggregated data value, and wherein the obtaining of the stored aggregated data value from the one network-accessible storage location includes using the generated new hash key to retrieve the stored aggregated data value from the one network-accessible storage location. 
     
     
         42 . The non-transitory computer-readable medium of  claim 41  wherein the received request further indicates values for specified dimension categories within the one or more multi-dimensional data arrays, and wherein the using of the indicated aggregation metric further includes using the indicated values for the specified dimension categories as part of the generating of the new hash key. 
     
     
         43 . The non-transitory computer-readable medium of  claim 37  wherein the one or more computing systems are part of an online data aggregation service, and wherein the distributed data storage structure is implemented on multiple storage nodes provided by the online data aggregation service for use in storing data for clients of the online data aggregation service. 
     
     
         44 . A system, comprising:
 one or more hardware processors of one or more computing systems; and   one or more memories with software instructions that, when executed by at least one of the one or more hardware processors, cause the at least one hardware processor to:
 generate, based at least in part on an aggregation metric used to calculate an aggregated data value as part of a multi-dimensional data array, a hash key based on the aggregation metric and for use with a distributed data storage structure for storing data values of the multi-dimensional data array; 
 determine, based at least in part on the generated hash key, one of multiple storage locations of the distributed data storage structure; and 
 initiate storage of the aggregated data value in the determined one storage location. 
   
     
     
         45 . The system of  claim 44  wherein the aggregated data value is generated from multiple other data values that are associated with a set of multiple dimension categories for the multi-dimensional data array, and wherein generating of the hash key is further based on the multiple dimension categories. 
     
     
         46 . The system of  claim 44  wherein determining the one storage location based at least in part on the generated hash key includes supplying the generated hash key as input to a hash function, and using, output of the hash function to select the one storage location from the multiple storage locations. 
     
     
         47 . The system of  claim 44  wherein the software instructions further cause the at least one hardware processor to, after the aggregated data value is stored in the determined one storage location:
 receive, from a requester, a request for stored aggregated data values that correspond to the aggregation metric and to multiple values for multiple dimensions of the multi-dimensional data array, and 
 obtain, in response to the request and by using the aggregation metric and the multiple values for the multiple dimensions, at least the stored aggregated data value from the determined one storage location, and provide the obtained at least stored aggregated data value to the requester. 
 
     
     
         48 . The system of  claim 47  wherein the using of the aggregation metric and the multiple values for the multiple dimensions includes generating a new hash key based on the aggregation metric and on the multiple values for the multiple dimensions, and wherein the obtaining of at least the stored aggregated data value includes using the generated new hash key to select the determined one storage location and retrieving at least the stored aggregated data value from the determined one storage location. 
     
     
         49 . The system of  claim 44  wherein the one or more computing systems are part of an online storage service, and wherein the distributed data storage structure is implemented on multiple storage nodes provided by the online storage service for use in storing data for clients of the online storage service.