Data stream protocol analysis using general purpose processors and filtering techniques

Abstract

Data stream protocol analysis using analysis processors. A network processor is connected with a distribution module that distributes network data to multiple memory buffers, each associated with an analysis processor, based at least on a status signal generated by the memory buffers. When the status signal is above a threshold level, the network data is distributed in a different manner or analyzed in a different manner. The analysis processors may begin performing less than a full protocol analysis and perform only selected protocol analysis tests. Some of the network data may be excluded by the network processor from network analysis. In another example, the same network data is sent to multiple analysis processors and each analysis processor performs different protocol analysis tests. Typically, network data corresponding to a particular transaction is sent to the same analysis processor.

1 . A method for performing protocol analysis on network data, the method comprising: 
 receiving network data at a network processor, wherein the network processor is connected with a plurality of memory buffers and each memory buffer is connected with an analysis processor;    assigning each packet a transaction identifier such that all packets associated with a particular transaction have the same transaction identifier;    routing each packet to at least two of the analysis processors based on the transaction identifier and on a status signal of each memory buffer; and    performing a first protocol analysis at a first of the at least two analysis processors and a second protocol analysis at a second of the at least two analysis processors.    
   
   
       2 . The method of  claim 1 , wherein routing each packet to at least two of the analysis processors based on the transaction identifier and on a status signal of each memory buffer further comprises monitoring a status signal of each memory buffer.  
   
   
       3 . The method of  claim 2 , wherein routing each packet to at least two of the analysis processors based on the transaction identifier and on a status signal of each memory buffer further comprises routing each packet to one of the analysis processors when the status signal of at least one memory buffer is below a threshold.  
   
   
       4 . The method of  claim 2 , wherein routing each packet to at least two of the analysis processors based on the transaction identifier and on a status signal of each memory buffer further comprises monitoring whether at least the status signal of at least one memory buffer is past a particular threshold level, the threshold level indicating a fullness of the memory buffer.  
   
   
       5 . The method of  claim 2 , wherein the first protocol analysis and the second protocol analysis are the same when the status signal is below a threshold level.  
   
   
       6 . The method of  claim 2 , wherein routing each packet to at least two of the analysis processors based on the transaction identifier and on a status signal of each memory buffer further comprises filtering each packet.  
   
   
       7 . The method of  claim 6 , wherein filtering each packet further comprises excluding certain packets having a predetermined type from analysis.  
   
   
       8 . The method of  claim 1 , further comprising combining results of the first protocol analysis with results of the second protocol analysis.  
   
   
       9 . A method for performing protocol analysis on network data, the method comprising: 
 receiving network data at a network processor, wherein the network processor is connected with a plurality of memory buffers and each memory buffer is connected with an analysis processor;    assigning each packet a transaction identifier such that all packets associated with a particular transaction have the same transaction identifier;    routing each packet to at least one of the analysis processors based at least on a status signal of each memory buffer; and    performing selected protocol analysis tests at each analysis processor for packets in the memory buffers associated with each analysis processor when the status signal is above a threshold level.    
   
   
       10 . The method of  claim 9 , wherein routing each packet to at least one of the analysis processors based at least on a status signal of each memory buffer further comprises routing each packet to at least two of the analysis processors when the status signal is below the threshold level.  
   
   
       11 . The method of  claim 9 , wherein performing selected protocol analysis tests at each analysis processor further comprises performing less than a full analysis for packets in the memory buffers associated with each analysis processor until the status signal is below the threshold level.  
   
   
       12 . The method of  claim 9 , wherein performing selected protocol analysis tests at each analysis processor further comprises excluding packets having a particular type from protocol analysis tests.  
   
   
       13 . The method of  claim 9 , wherein routing each packet to at least one of the analysis processors based at least on a status signal of each memory buffer further comprises filtering each packet when the status signal is above the threshold level.  
   
   
       14 . The method of  claim 9 , wherein performing selected protocol analysis tests at each analysis processor further comprises distributing different protocol analysis tests to different analysis processors.  
   
   
       15 . The method of  claim 9 , wherein routing each packet to at least one of the analysis processors based at least on a status signal of each memory buffer further comprises routing each packet having the same transaction identifier to the same analysis processor.  
   
   
       16 . The method of  claim 9 , wherein the selected protocol analysis tests have a higher priority than other protocol analysis tests.  
   
   
       17 . A system for performing protocol analysis on network data, the system comprising: 
 a plurality of memory buffers;    a distribution module connected with the plurality of memory buffers, wherein the distribution module distributes packets to the plurality of memory buffers based on at least one of a status signal generated by each memory buffer and a transaction identifier of each packet;    a network processor that processes the network data such that each packet in a particular transaction has the same transaction identifier; and    a plurality of analysis processors, each analysis processor connected with a particular memory buffer in the plurality of memory buffers, wherein each analysis processor performs protocol analysis tests that are selected based on whether the status signal is above or below the threshold level.    
   
   
       18 . The system of  claim 17 , wherein the distribution module directs the same network data to at least two of the plurality of analysis processors, wherein the at least two of the plurality of analysis processors perform different protocol analysis tests on the same network data.  
   
   
       19 . The system of  claim 17 , wherein each analysis processor performs less than a full protocol analysis for network data in the corresponding memory buffer when the status signal is above a threshold level.  
   
   
       20 . The system of  claim 17 , wherein the network processor further comprises a filter module that selectively removes packets of a certain type from being distributed by the distribution module.  
   
   
       21 . The system of  claim 20 , wherein the filter module filters portions of the packets.  
   
   
       22 . The system of  claim 21 , wherein the portion of the packets filtered include the payload.  
   
   
       23 . The system of  claim 17 , wherein the distribution module is one of a field programmable gate array and a demultiplexor and each memory buffer is a FIFO queue.