System and method of aggregating and consolidating security event data

Abstract

A method and system are provided that enable the processing of security event data is provided. In a first version, instructions for processing security event data are software encoded in separate modules. The software is organized into discrete modules and executed by an information technology system. The software as executed identifies the computational engines of the information technology available for processing the security event data and assigns modules to specific computational engines. A plurality of events stored in a buffer are processed sequentially through two or more modules. The results of each processing of an event by a module are recorded in an extended event structure and made accessible to a successive module. The location of the buffer storing an event is available for overwriting after the event has been fully processed.

1 . In an information technology system, the information technology system having at least one computational engine available for processing security event data, a method for processing security event data comprising: 
 a. software encoding machine-readable instructions to execute the security event processing, the instructions organized into modules;    b. providing the instructions to the information technology system;    c. determining the number of computational engines available for security event processing;    d. tasking each available computational engine with executing at least one module; and    e. processing a plurality of events.    
   
   
       2 . The method of  claim 1 , wherein each module is executed exclusively by only one computational engine.  
   
   
       3 . The method of  claim 1 , wherein information generated in the execution of at least one module is written into an extended event structure.  
   
   
       4 . The method of  claim 3 , wherein the information stored in the extended event structure is accessed in the execution of at least one additional module.  
   
   
       5 . The method of  claim 1 , wherein at least one event is stored on a main event buffer of the information technology system.  
   
   
       6 . The method of  claim 5 , wherein the main event buffer is a circular buffer.  
   
   
       7 . The system of  claim 5 , wherein the at least one event is overwritten in the main event buffer after the at least one event is processed.  
   
   
       8 . The method of  claim 5 , wherein the least one event is stored on a secondary memory after processing.  
   
   
       9 . The method of  claim 8 , wherein the secondary memory comprises a data storage disk.  
   
   
       10 . The method of  claim 1 , wherein at least one event is received by the information technology system via an electronic communications network.  
   
   
       11 . The method of  claim 1 , wherein at least one additional event is received by the information technology system via the Internet.  
   
   
       12 . In an information technology system, the information technology system having at least a first and a second computational engine, a method for processing security events, comprising: 
 a. providing a plurality of events to the information technology system, each event having an event type;    b. identifying all events having an identical event type designator; and    c. processing each event sequentially through at least two threads, each thread comprising at least one module and each thread executed by a separate computational engine.    
   
   
       13 . The method of  claim 12 , wherein information generated in the execution of at least one module is stored in an extended event structure.  
   
   
       14 . The method of  claim 12 , wherein the information stored in the extended event structure is applied in the execution of at least one module.  
   
   
       15 . The method of  claim 12 , wherein each event includes information related to a flow event.  
   
   
       16 . The method of  claim 12 , wherein each event further comprises a source dimensional data field storing an electronic message source address.  
   
   
       17 . The method of  claim 12 , wherein each event further comprises a destination dimensional data field storing an electronic message destination address.  
   
   
       18 . The method of  claim 12 , wherein each event further comprises an address dimensional data field storing a value selected from the group consisting of an IP address, a MAC address, and an Ethernet address.  
   
   
       19 . The method of  claim 12 , wherein at least one event is received via an electronic communications network.  
   
   
       20 . A computer-readable medium on which are stored a plurality of computer-executable instructions for performing steps (a)-(e), as recited in  claim 1 .  
   
   
       21 . In an information technology system, a method for sequentially processing an event through a sequentially ordered series of stages, the sequentially ordered series of stages to be applied in a pre-established sequence to the event in order from lower to higher, comprising: 
 a. associating a stage index with an event, and the stage index for indicating a next higher ordered stage;    b. examining the stage index to identify the next higher ordered stage; and    c. inhibiting the application of all stages other than the next higher ordered stage to the event    
   
   
       22 . The method of  claim 21 , wherein the method further comprises applying the next higher ordered stage to the event.  
   
   
       23 . The method of  claim 22 , wherein the method further comprises updating the stage index to identify the next higher ordered stage as the most recent stage applied to the event.  
   
   
       24 . The method of  claim 22 , wherein the method further comprises updating the stage index to indicate a following stage, the following stage to be the stage applied after the application of the next higher ordered stage and before all other stages of higher order than the next higher ordered stage.  
   
   
       25 . In an information technology system, the information technology having a secondary memory, a method for checkpointing information related to the processing of events by a sequentially ordered series of stages, comprising: 
 a. providing a checkpoint event, the checkpoint event for directing the information technology system to checkpoint information related to a stage applied to the checkpoint event;    b. processing the checkpoint event through the series of stages in the sequential order; and    c. checkpointing each stage when each stage is applied to the checkpoint event, wherein information associated with each stage is stored in the secondary memory.