Method and apparatus for network packet capture distributed storage system

Abstract

This is invention comprises a method and apparatus for Infinite Network Packet Capture System (INPCS). The INPCS is a high performance data capture recorder capable of capturing and archiving all network traffic present on a single network or multiple networks. This device can be attached to Ethernet networks via copper or SX fiber via either a SPAN port ( 101 ) router configuration or via an optical splitter ( 102 ). By this method, multiple sources or network traffic including gigabit Ethernet switches ( 102 ) may provide parallelized data feeds to the capture appliance ( 104 ), effectively increasing collective data capture capacity. Multiple captured streams are merged into a consolidated time indexed capture stream to support asymmetrically routed network traffic as well as other merged streams for external consumption.

1 . A method of capturing data packets comprising of:
 connecting a capture device to a data communications path;   capturing data packets communicated along the data communications path;   persistently storing the captured data from the data packets in a predetermined combination of volatile and non-volatile storage media;   aggregating the persistently stored data packets into a slot of predetermined size;   annotating the aggregated data packets with persistent storage information;   storing the annotated data packets using an infinitely journaled, write-once, hierarchical file system;   reconstructing any corrupted data to ensure data accuracy of the persistently stored data; and   retrieving a predetermined portion of captured data and persistently stored annotations from the slot;   creating the slot of predetermined size to have a buffer of a predetermined size; and   managing the slot based on a least recently used cache to map the data in the slot to a non-volatile storage thereby creating a cache image of the captured data.   
   
   
       2 . A method capturing data packets comprising of connecting a capture appliance to data communications path;
 capturing data communicated along the data communications path;   replicating and persistently annotating the captured data in a predetermined combination of volatile and nonvolatile storage;   aggregating the captured data and persistent annotations in the volatile and non-volatile storage into a slot; and   storing the data in a non-volatile storage using an infinitely journaled, write-once, hierarchical file system.   
   
   
       3 . The method of  claim 2  wherein the data is aggregated into a slot by:
 creating the slot; and   managing the slot based on an least recently used cache.   
   
   
       4 . The method of  claim 3  wherein the least recently used cache maps the data in the slot to the non-volatile storage to create a cache image of the captured data across sectors of the non-volatile storage using striping and thereby allowing a controller simultaneously to write to a plurality of non-volatile storage devices. 
   
   
       5 . The method of  claim 4  wherein the data is copied from the slot to the volatile storage using a least recently used algorithm to allocate space in the volatile storage.