System and method for characterizing data through a probabilistic data structure

Abstract

A system and method for resolving data through a probabilistic data structure can include initializing a B-field data structure, inserting a key-value element into the B-field data structure, selecting at least one key query, and looking up the value of a key lookup request through the B-field data structure.

I claim: 
     
         1 . A data storage system for storing and querying key-value elements comprising:
 a set of reference key-value elements;   a bit array in memory;   a data management system configured to initialize a bit array in memory;   the data management system further configured to iteratively store instances of the reference key-value elements in the bit array, and wherein each iteration of key-value storage further comprises process configuration with respect to the storing of each instance of the reference key-value elements to:
 apply a set of hash operators to the key of an instance thereby obtaining a set of indexes particular to the key, 
 determine insertion locations in the bit array according to the set of indexes, 
 execute an encoding operator on the value of the instance, thereby converting the value to a particular bit mask, and 
 activate a set of bits in the bit array through a bitwise operation of the bit mask ORed at the insertion locations; 
   a query interface configured to select a query key; and   a search engine configured to extract a result value for the query key from the bit array, with further configuration to:
 apply the set of hash operators to the query key and thereby obtaining query indexes, 
 determine query locations in the bit array according to the query indexes, 
 retrieve stored bit masks from the query locations, 
 apply a bitwise AND operation on the stored bit masks and thereby generate a combined bit mask, and 
 apply a decoding operation to the combined bit mask and thereby obtaining the result value. 
   
     
     
         2 . The system of  claim 1 , wherein the set of reference key-value elements are a set of reference domain name server key-values pairs. 
     
     
         3 . The system of  claim 1 , wherein the bit array in memory is hosted in an internet-accessible distributed computing environment. 
     
     
         4 . The system of  claim 3 , wherein the bit array in memory comprises multiple distinct memory blocks; and wherein the data management system configured to initialize the bit array in memory of the computing platform further comprises configuration to shard the bit array across a set of computing resources. 
     
     
         5 . The system of  claim 1 , wherein the data management system is further configured to detect an indeterminate error scenario of the bit array in response to the storage of an instance of a reference key-value element in the bit array; and, in response to detecting an indeterminate error scenario, store the reference key-value element into a secondary bit array. 
     
     
         6 . The system of  claim 5 , wherein the bit array is stored in a first memory location and the secondary bit array is stored in a second memory location that is distinct from the first memory location. 
     
     
         7 . The system of  claim 5 , wherein configuration to store the reference key-value element into the secondary bit array further comprises configuration to: apply secondary hash operators to the reference key and obtain a second set of indexes; determine secondary insertion locations according to the second set of indexes; and activate a set of bits in the bit array through bitwise an OR operation of the bit mask applied at secondary insertion locations of the secondary bit array; and wherein the secondary bit array overlaps the bit array in memory. 
     
     
         8 . The system of  claim 5 , wherein the data management system is further configured to: detect a second indeterminate error scenario of the secondary bit array in response to storage of the reference key-value element into the secondary bit array; and, in response to detection of the second indeterminate error scenario, store the reference key-value element into a second secondary bit array. 
     
     
         9 . The system of  claim 1 , wherein there is a minimized key for a set of keys in reference key-value elements; and wherein configuration to obtain the set of indexes further comprises configuration to obtain a first subset of indexes for the set of indexes through configuration to apply a first subset of hash operators to the key and obtain a second subset of indexes for the set of indexes by applying a second subset of hash operators to the minimized key. 
     
     
         10 . The system of  claim 1 , wherein the set of reference elements are a set of key values pairs where each key value pair maps a biological sequence fragment to a biological characterization value. 
     
     
         11 . A method for storing reference key-value elements in a data structure and then retrieving a result value using a query key comprising:
 initializing a bit array in memory of a computing platform;   iteratively storing instances of the reference key-value elements in the bit array, and wherein each iteration of the storing further comprises the following steps with respect to the storing of each instance of the reference key-value elements:
 applying a set of hash operators to the key of an instance thereby obtaining a set of indexes particular to the key, 
 determining insertion locations in the bit array according to the set of indexes, 
 executing an encoding operator on the value of the instance, thereby converting the value to a particular bit mask, and 
 activating a set of bits in the bit array through bitwise ORing the bit mask at the insertion locations; 
   obtaining a query key; and   extracting a result value for the query key from the bit array, by:
 applying the set of hash operators to the query key and thereby obtaining query indexes, 
 determining query locations in the bit array according to the query indexes; 
 retrieving stored bit masks from the query locations, 
 bitwise ANDing the stored bit masks into a combined bit mask, and 
 applying a decoding operation to the combined bit mask and thereby obtaining the result value. 
   
     
     
         12 . The method of  claim 11 , wherein the reference key-value elements are a set of reference domain name server key values pairs. 
     
     
         13 . The method of  claim 11 , wherein the bit array in memory is hosted in an internet-accessible distributed computing environment, and wherein the query key is obtained in association with a query to a web platform. 
     
     
         14 . The method of  claim 11 , wherein initializing the bit array comprises sharding the bit array across a set of computing resources. 
     
     
         15 . The method of  claim 11 , further comprising: detecting an indeterminate error scenario of the bit array in response to storing an instance of a reference key-value element in the bit array; and, in response to detecting an indeterminate error scenario, storing the reference key-value element into a secondary bit array. 
     
     
         16 . The method of  claim 15 , wherein the wherein the bit array is stored in a first memory location and the secondary bit array is stored in a second memory location that is distinct from the first memory location. 
     
     
         17 . The method of  claim 15 , wherein storing the reference key-value element into the secondary bit array further comprises: applying secondary hash operators to the reference key thereby obtaining a second set of indexes; determining secondary insertion locations according to the second set of indexes; and activating a set of bits in the bit array through bitwise ORing the bit mask at secondary insertion locations of the secondary bit array; and wherein the secondary bit array overlaps the bit array in memory. 
     
     
         18 . The method of  claim 15 , further comprising: detecting a second indeterminate error scenario of the secondary bit array in response to storing the reference key-value element into the secondary bit array; and, in response to detecting the second indeterminate error scenario, storing the reference key-value element into a second secondary bit array. 
     
     
         19 . The method of  claim 11 , further comprising identifying a minimized key for a set of keys in reference key-value elements; and wherein obtaining the set of indexes further comprises obtaining a first subset of indexes for the set of indexes by applying a first subset of hash operators to the key and obtaining a second subset of indexes for the set of indexes by applying a second subset of hash operators to the minimized key. 
     
     
         20 . The method of  claim 1 , wherein the set of reference elements are a set of key values pairs where each key value pair maps a biological sequence fragment to a biological characterization value.