Packet Compression for Network Packet Traffic Analysis

Abstract

Methods of capturing and compressing trace data for use in network packet traffic analysis are described. In an embodiment, when a packet is received, two records of the packet are created and stored. One record is stored in a file associated with the source address of the packet and the other record is stored in a file associated with the destination address of the packet. Various packet compression techniques are described and one example compares a newly received packet to the previous packet which has been stored in the same file and sets bits in the record which denote whether fields in the newly received packet are the same as the corresponding fields in the previous packet.

1 . A method of capturing trace data for use in network packet traffic analysis, the method comprising:
 under control of one or more processors configured with executable instructions:   receiving a packet; and   storing a representation of the packet in a file, said representation and said file being associated with one of a source address of the packet and a destination address of the packet.   
     
     
         2 . A method according to  claim 1 , further comprising:
 storing a second representation of the packet in a second file, said second representation and said second file being associated with another of said source address of the packet and said destination address of the packet.   
     
     
         3 . A method according to  claim 2 , wherein the first representation and the second representation are the same. 
     
     
         4 . A method according to  claim 1 , wherein a representation of a packet comprises a truncated packet. 
     
     
         5 . A method according to  claim 1 , wherein a representation of a packet comprises values of specified fields. 
     
     
         6 . A method according to  claim 1 , wherein storing a representation of the packet in a file comprises:
 if the file associated with an address is empty, creating and storing the associated representation;   if the file associated with an address comprises a representation of a previous packet, creating the associated representation based on a comparison of said packet and said previous packet and storing the associated representation.   
     
     
         7 . A method according to  claim 6 , wherein creating the associated representation based on a comparison of said packet and said previous packet comprises:
 comparing a value of a first field in said packet with a value of said first field in said previous packet;   setting a bit in the representation identifying if the values are the same; and   repeating the comparing and setting steps for each field in a set of specified fields; and   appending the value of each field where the value of the field in said packet is not the same as the value of the field in the previous packet.   
     
     
         8 . A method according to  claim 1 , wherein storing a representation of the packet in a file comprises:
 creating a representation of the packet;   setting a flag in the representation indicating whether the address associated with the representation is the source address or the destination address;   mapping a value of a field in the representation based on said flag; and   storing the representation.   
     
     
         9 . A method according to  claim 1 , wherein each representation comprises at least one of a source address and a destination address and wherein storing two representations of the packet in separate files comprises, for each representation:
 replacing an address in the representation with an identifier;   storing a mapping between the address and the identifier in a separate file; and   storing the representation.   
     
     
         10 . A method according to  claim 1 , further comprising comparing the representation of the packet to a representation of a previously received packet which has been stored in the file and which sets bits which denote whether fields in the packet are the same as corresponding fields in the previously received packet. 
     
     
         11 . One or more tangible device-readable media with device-executable instructions for performing acts comprising:
 on receipt of a packet, creating a packet record for use in network packet traffic analysis, the packet record comprising a plurality of fields; and   discarding the packet.   
     
     
         12 . One or more tangible device-readable media according to  claim 11 , further comprising device-executable instructions for performing acts comprising:
 creating a first and second version of the packet record; and   storing the first version in a file associated with a source address of the packet; and   storing the second version of the packet record in a file associated with a destination address of the packet.   
     
     
         13 . One or more tangible device-readable media according to  claim 12 , wherein creating a first and second version of the packet record further comprises, for each version:
 setting a flag in the version of the packet record according to a direction of travel of the packet); and   mapping at least one of the plurality of fields based on said flag.   
     
     
         14 . One or more tangible device-readable media according to  claim 11 , further comprising device-executable instructions for performing act comprising:
 compressing each packet record based on a comparison of the packet and a packet previously stored in the file.   
     
     
         15 . One or more tangible device-readable media according to  claim 14 , wherein compressing each packet record based on a comparison of the packet and a packet previously stored in the file comprises:
 comparing the packet to a previous packet;   setting a flag in the packet record for each of the plurality of fields, the flag indicating whether a value of the field in the packet is different to a value of the field in the previous packet; and   for each field where the value of the field in the packet is different to a value of the field in the previous packet, appending the value of the field to the packet record.   
     
     
         16 . One or more tangible device-readable media according to  claim 14 , further comprising device-executable instructions for performing acts comprising:
 creating an instance of a record creation and compression method for each file;   passing a received packet to at least one of: an instance of the method corresponding to a source address of the packet and an instance of the method corresponding to a source address of the packet.   
     
     
         17 . One or more tangible device-readable media according to  claim 11 , further comprising device-executable instructions for performing acts comprising:
 replacing an address in the packet record with an identifier; and   storing a mapping between the address and the identifier in a dictionary.   
     
     
         18 . One or more tangible device-readable media with device-executable instructions for performing acts comprising:
 accessing a file comprising a plurality of compressed packet records, each compressed packet record comprising a flag byte;   reading a flag byte from the file; and   generating an uncompressed packet record comprising a packet time and a plurality of fields by:
 determining the packet time based on a timestamp within said flag byte; and 
 determining the plurality of fields based on a plurality of flags in the flag byte, each flag corresponding to one of the plurality of fields. 
   
     
     
         19 . One or more tangible device-readable media according to  claim 18 , wherein determining the plurality of fields based on a plurality of flags in the flag byte comprises:
 reading a first flag from the flag byte;   if said first flag is set, reading a value of a first field from the file;   if said first flag is not set, setting the value of the first field to a value of a first field in a previous uncompressed packet record; and   repeating the steps for each of the plurality of flags.   
     
     
         20 . One or more tangible device-readable media according to  claim 18 , wherein determining the packet time based on a timestamp within said flag byte comprises:
 examining the timestamp; and   if the timestamp is equal to a first value, reading a time from the file and setting the packet time to said time; and   if the timestamp is equal to one of a set of values, reading one or more bytes from the file and setting the packet time based on said one or more bytes and a packet time of a previous uncompressed packet record.