Persistent data storage techniques

Abstract

A database is maintained that stores data persistently. Tasks are accepted from task sources. At least some of the tasks have competing requirements for use of regions of the database. Each of the regions includes data that is all either locked or not locked for writing at a given time. Each of the regions is associated with an available processor. For each of the tasks, jobs are defined each of which requires write access to regions that are to be accessed by no more than one of the processors. Jobs are distributed for concurrent execution by the associated processors.

What is claimed is: 
     
         1 . A data processing system comprising
 a transaction system comprising one or more servers and an application program comprising instructions executable by the transaction system to
 communicate with user devices; 
 receive transaction requests from the user devices; and 
 translate the requests to tasks or to job instructions, 
   an update stream processor and instructions executable by the update stream processor to:
 receive the tasks or job instructions; 
 execute two or more potentially conflicting instructions associated with the tasks or job instructions by acting on data in a stored database; 
 before one of the tasks has been executed, provide an acknowledgement to the application program indicating that the task is guaranteed to be executed; and 
 execute the potentially conflicting instructions in a manner to maintain consistency of the data in the database. 
   
     
     
         2 . The system of  claim 1  in which the update stream processor comprises one or more job creation processes and instructions executable by the job creation processes to create jobs comprising the two or more potentially conflicting job instructions. 
     
     
         3 . The system of  claim 1  in which the instructions executable by the transaction system are executable to make assignments to respective disjoint sets of data in the database. 
     
     
         4 . The system of  claim 3  in which the instructions executable by the update stream processor are executable to cause execution in a given one of the disjoint sets in serial and execute in different disjoint sets in parallel. 
     
     
         5 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to cause synchronization of instruction execution to guarantee execution of the task. 
     
     
         6 . The system of  claim 1  in which the instructions executable by the transaction system are executable to receive the transaction requests in a pre-defined format. 
     
     
         7 . The system of  claim 1  in which the instructions executable by the transaction system are executable to include instructions in the tasks. 
     
     
         8 . The system of  claim 1  in which the two or more potentially conflicting instructions point to data in the database. 
     
     
         9 . The system of  claim 1  comprising the stored database. 
     
     
         10 . The system of  claim 9  in which the stored database comprises a federated database. 
     
     
         11 . The system of  claim 9  in which the stored database comprises a distributed database. 
     
     
         12 . The system of  claim 9  in which the stored database is distributed among storage devices at different locations interconnected by a network. 
     
     
         13 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to assure execution of the task after a failure. 
     
     
         14 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to assure consistency of the data in the database after a failure. 
     
     
         15 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to maintain the consistency by maintaining a relationship that enables a process to reach specific data from other data in the stored database or vice versa. 
     
     
         16 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to maintain the consistency by maintaining information about temporarily incomplete states of the specific data and the other data. 
     
     
         17 . The system of  claim 16  in which the instructions executable by the update stream processor are executable to communicate information about temporarily incomplete states to processes that are to update at least one of the specific data and the other data. 
     
     
         18 . The system of  claim 16  in which the instructions executable by the update stream processor are executable to maintain the consistency by concurrent processing with respect to specific data and other data. 
     
     
         19 . The system of  claim 1  in which the transaction request is associated with a commercial transaction. 
     
     
         20 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to read data in the database while the potentially conflicting instructions are being executed and the consistency of data is being maintained. 
     
     
         21 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to store data of the database in memory. 
     
     
         22 . The system of  claim 21  in which the data stored in memory comprises an index. 
     
     
         23 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to maintain the consistency between data in two different storage devices at two different locations, data in two different non-conflicting regions, data in the database and data in an index, or data in two different physical clusters. 
     
     
         24 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to
 receive additional tasks or job instructions associated with additional potentially conflicting instructions having competing requirements for use of regions of the database, 
 associate each of the regions with an available processor, 
 generate jobs each of which requires write access only to regions that are to be accessed by no more than one of the processors, and 
 distribute the jobs for concurrent execution by the processors. 
 
     
     
         25 . The system of  claim 1  in which the instructions executable by the update stream processor are executable to match a new item of data to be added to the database with data stored in the database by spawning jobs for matching the data item to the data stored in the database, receiving replies from the matching jobs, collapsing the replies into a list of matching data, merging the matching data of the list, and spawning update jobs to update the matching data. 
     
     
         26 . The system of  claim 25  in which the new item of data is associated with a real world object or person. 
     
     
         27 . The system of  claim 1  in which the task comprises or is the basis of a job that is not to update data in the database. 
     
     
         28 . A data processing system comprising
 a transaction system and instructions executable by the transaction system to receive transaction requests from user devices and translate the requests to tasks,   an update stream processor and instructions executable by the update stream processor to:
 receive the tasks; 
 execute two or more potentially conflicting instructions associated with the tasks by acting on data in a stored database; 
 before one of the tasks has been executed, provide an acknowledgement to the indicating that the task is guaranteed to be executed; and 
 execute the potentially conflicting instructions in a manner to maintain consistency of the data in the database.