Systems and methods for detecting, identifying and categorizing intermediate nodes

Abstract

A system and method for obtaining node information from a variety of potential sources and storing the information in a logical repository, and a system and method for identifying and categorizing Intermediate Nodes using a combination of requesting and responding node information.

1 . A system for obtaining node data, the system comprising: at least one system node configured to obtain source node data of at least one source node, to apply at least one rule to the obtained source node data to extract candidate node data from the obtained source node data, and to send the candidate node data to a logical data repository. 
     
     
         2 . The system of  claim 1 , wherein the at least one system node comprises the logical data repository which stores the candidate node data. 
     
     
         3 . The system of  claim 1 , further comprising at least one other node external to the at least one system node, the at least one other node comprising the logical data repository which stores the candidate node data. 
     
     
         4 . The system of  claim 1 , wherein the at least one source node is a plurality of source nodes. 
     
     
         5 . The system of  claim 4 , further comprising a set of source node information including at least one Uniform Resource Identifier (URI), wherein the at least one URI points to a corresponding source node of the plurality of source nodes, and the at least one system node obtains the source node data for the corresponding source node based on the at least one URI. 
     
     
         6 . The system of  claim 1 , wherein the at least one rule comprises a plurality of rules that are selected from the group consisting of: regular expressions, publically available processing sources, WebKit sources and combinations of any of the foregoing. 
     
     
         7 . The system of  claim 6 , wherein the at least one system node iteratively applies at least a portion of the plurality of rules to the source node data of the at least one node to extract the candidate node data from the source node data. 
     
     
         8 . The system of  claim 6 , wherein the plurality of rules are configured to extract candidate node data having a plurality of different formats. 
     
     
         9 . The system of  claim 6 , wherein the plurality of rules comprise at least one regular expression command or at least one custom expression command. 
     
     
         10 . The system of  claim 9 , wherein the at least one custom expression command comprises a workflow process which is iteratively applied to the source node data of the at least one source node. 
     
     
         11 . The system of  claim 6 , further comprising a rule generation tool configured to create rules based on input from a user and store the created rules in the dynamic rule repository. 
     
     
         12 . The system of  claim 6 , wherein, after a first rule is applied which successfully obtains candidate node data from the source node data of the at least one source node, the at least one system node does not apply the remainder of the plurality of rules to the source node data of the at least one source node. 
     
     
         13 . The system of  claim 6 , wherein the at least one system node applies the plurality of rules by successively iterating through the plurality of rules in increasing complex logic to progressively filter content from the source node data until a desired level of the candidate node data is obtained. 
     
     
         14 . The system of  claim 1 , wherein the at least one source node is a plurality of source nodes of a given network, and the at least one system node is configured to apply the at least one rule to the source node data of each of the plurality of source nodes of the given network to extract to extract candidate node data for each said node of the given network, and send the candidate node data for each said node of the given network to the logical data repository. 
     
     
         15 . A method for obtaining and storing node data, the method comprising:
 obtaining, by at least one system node, source node data of at least one source node;   applying, by the at least one system node, at least one rule to the source node data to extract candidate node data from the source node data; and   sending, by the at least one node, the candidate node data to a logical data repository.   
     
     
         16 . The method of  claim 15 , further comprising storing, by the logical data repository, the candidate node data in the at least one system node. 
     
     
         17 . The method of  claim 15 , further comprising storing, by the logical data repository, the candidate node data in at least one other node external to the at least one system node. 
     
     
         18 . The method of  claim 15 , wherein the at least one source node is a plurality of source nodes. 
     
     
         19 . The method of  claim 18 , further comprising reading, by the at least one system node, a set of source node information including at least one Uniform Resource Identifier (URI), wherein the at least one URI points to a corresponding source node of the plurality of source nodes, and the source node data for the corresponding source node is obtained by the at least one system node based on the at least one URI. 
     
     
         20 . The method of  claim 15 , wherein the at least one rule is a plurality of rules stored by a dynamic rule repository.