Method and apparatus for selective caching of transactions in a computer system

Abstract

A database manager selectively caches journal entries for certain database changes based on dynamic selection criteria. Preferably, a journal disk drive is provided for storing changes to the database. Database change entries are sequentially cached in a main memory journal buffer, a waiting journal write. When a buffer is filled, its buffer contents are written to the journal disk. However, certain change operations will cause the contents of the buffer to be written to the journal disk immediately, before filling the buffer. Certain database change entries selectively cause the buffer to be written, depending on dynamic selection criteria. Preferably, these selection criteria take into account both the estimated time to recover in the event of a system crash and the current level of activity on the system. Generally, the busier the system, the more data will be cached, and vice versa.

1 . A method for managing data in a computer system, comprising the steps of: 
 maintaining a structured database in at least one non-volatile storage device of said computer system, said structured database comprising a plurality of records directly accessible without sequential access;    making first changes to selective records of said structured database in a volatile memory of said computer system;    entering said first changes to selective records of said structured database in a journal buffer, said journal buffer being in said volatile memory;    selectively determining whether to write contents of said journal buffer to a non-volatile journal storage, said selectively determining step using at least one dynamic selection criterion to selectively determine whether to write contents of said journal buffer to said journal storage device when said journal buffer is not filled to maximum capacity;    writing contents of said journal buffer containing said first changes to selective records of said structured database to sequential locations of said non-volatilejournal storage responsive to the result of said selectively determining step, wherein said journal buffer enables said step of writing contents of said journal buffer containing said first changes to selective records to be performed concurrently with entering second changes to selective records of said structured database in said journal buffer; and    writing contents of said journal buffer containing said first changes to selective records of said structured database to non-sequential locations of said at least one non-volatile storage device corresponding to respective records of said structured database.    
   
   
       2 . The method for managing data of  claim 1 , wherein said non-volatile journal storage comprises at least one rotating magnetic hard disk drive storage device.  
   
   
       3 . The method for managing data of  claim 1 , wherein said selectively determining step comprises: 
 identifying a plurality of commit transactions, each commit transaction containing a respective discrete set of said data changes entered in said journal buffer;    identifying a respective final data change of each of a plurality of commit transactions;    triggering said selective determination whether to write contents of saidjournal buffer to a non-volatile journal storage responsive to identifying a respective final data change of each of a plurality of commit transactions.    
   
   
       4 . The method for managing data of  claim 3 , wherein some of said commit transactions are explicitly identified as commit transactions by a user, and some of said commit transactions are implicitly identified as commit transactions by a managing function executing on said computer system, said step of triggering said selective determination whether to write contents of said journal buffer to a non-volatile journal storage being performed responsive to identifying a respective final data change of an implicitly identified commit transaction.  
   
   
       5 . The method for managing data of  claim 1 , wherein said at least one dynamic selection criterion comprises a measurement of current activity level of at least one component of said computer system.  
   
   
       6 . The method for managing data of  claim 5 , wherein said at least one dynamic selection criterion comprises a measurement of current activity level of said non-volatile journal storage.  
   
   
       7 . A program product for managing data in a computer system, said program product comprising a plurality of processor executable instructions recorded on signal-bearing media, wherein said instructions, when executed by at least one processor of said computer system, cause the system to perform the steps of: 
 maintaining a structured database in at least one non-volatile storage device of said computer system, said structured database comprising a plurality of records directly accessible without sequential access;    making first changes to selective records of said structured database in a volatile memory of said computer system;    entering said first changes to selective records of said structured database in a journal buffer, said journal buffer being in said volatile memory;    selectively determining whether to write contents of said journal buffer to a non-volatile journal storage, said selectively determining step using at least one dynamic selection criterion to selectively determine whether to write contents of said journal buffer to said journal storage device when said journal buffer is not filled to maximum capacity;    responsive to the result of said selectively determining step, sending contents of said journal buffer containing said first changes to selective records of said structured database to non-volatile journal storage for writing to sequential locations of said non-volatile journal storage, wherein said journal buffer enables contents of said journal buffer containing said first changes to selective records to be sent to said non-volatile journal storage concurrently with entering second changes to selective records of said structured database in said journal buffer; and    sending contents of said journal buffer containing said first changes to selective records of said structured database to said at least one non-volatile storage device for writing to non-sequential locations of said at least one non-volatile storage device corresponding to respective records of said structured database.    
   
   
       8 . The program product of  claim 7 , wherein said non-volatile journal storage comprises at least one rotating magnetic hard disk drive storage device.  
   
   
       9 . The program product of  claim 7 , wherein said selectively determining step comprises: 
 identifying a plurality of commit transactions, each commit transaction containing a respective discrete set of said data changes entered in said journal buffer;    identifying a respective final data change of each of a plurality of commit transactions;    triggering said selective determination whether to write contents of said journal buffer to a non-volatile journal storage responsive to identifying a respective final data change of each of a plurality of commit transactions.    
   
   
       10 . The program product of  claim 9 , wherein some of said commit transactions are explicitly identified as commit transactions by a user, and some of said commit transactions are implicitly identified as commit transactions by a managing function executing on said computer system, said step of triggering said selective determination whether to write contents of said journal buffer to a non-volatile journal storage being performed responsive to identifying a respective final data change of an implicitly identified commit transaction.  
   
   
       11 . The program product of  claim 7 , wherein said at least one dynamic selection criterion comprises a measurement of current activity level of at least one component of said computer system.  
   
   
       12 . The program product of  claim 11 , wherein said at least one dynamic selection criterion comprises a measurement of current activity level of said non-volatile journal storage.  
   
   
       13 . A computer system, comprising: 
 at least one processing unit;    a non-volatile data storage comprising at least one non-volatile data storage device, a portion of said non-volatile data storage being used as a journal;    a volatile memory for storing instructions executable on said at least one processing unit and for storing a journal buffer containing changes to data stored in said non-volatile data storage;    a manager for managing changes to a structured database stored in said at least one non-volatile data storage device, said structured database having a plurality of records of a defined structure, said manager receiving changes to selective records of said structured database and temporarily storing said changes in said journal buffer;    wherein said manager selectively determines whether to write contents of said journal buffer to said journal portion of said non-volatile data storage, said manager using an least one dynamic selection criterion to selectively determine whether to write contents of said journal buffer to said journal when said journal buffer is not filled to maximum capacity,    wherein said manager causes journal buffer contents containing said first changes to be written to said journal portion of said non-volatile data storage responsive to the result of said selective determination, wherein said journal buffer enables journal buffer contents containing said first changes to be written to said journal portion of said non-volatile storage concurrently with storing second changes to selective records of said structured database in said journal buffer; and    wherein said manager causes journal buffer contents containing said first changes to be written to non-sequential locations of said at least one non-volatile storage device, said non-sequential locations corresponding to records of said structured database to which said first changes correspond.    
   
   
       14 . The computer system of  claim 13 , wherein said non-volatile data storage comprises at least one rotating magnetic hard disk drive storage device.  
   
   
       15 . The computer system of  claim 13 , wherein said non-volatile data storage comprises a plurality of independently accessible data storage devices, and wherein said journal portion of said non-volatile data storage is a subset of said plurality of independently accessible data storage devices.  
   
   
       16 . The computer system of  claim 13 , wherein said manager selectively determining whether to write contents of said journal buffer to said journal portion by: (a) identifying a plurality of commit transactions, each commit transaction containing a respective discrete set of said data changes entered in said journal buffer; (b) identifying a respective final data change of each of a plurality of commit transactions; and (c) triggering said selective determination whether to write contents of said journal buffer to a non-volatile journal storage responsive to identifying a respective final data change of each of a plurality of commit transactions.  
   
   
       17 . The computer system of  claim 16 , wherein some of said commit transactions are explicitly identified as commit transactions by a user, and some of said commit transactions are implicitly identified as commit transactions by a managing function executing on said computer system, wherein triggering said selective determination whether to write contents of said journal buffer to a non-volatile journal storage is performed responsive to identifying a respective final data change of an implicitly identified commit transaction.