Automatically detecting invalid events in a distributed computing environment

Abstract

Described herein is a system for automatically detecting invalid events in a distributed computing environment. The system for automatically detecting invalid events may include sub-systems and a learning engine. The learning engine may generate a rule set for each sub-system specifying circumstances under which an event is considered invalid specific to the sub-system using machine learning. Sub-systems may detect an invalid event being propagated through the distributed computing environment based on a set of rules specifying circumstances under which an event is considered invalid specific to the sub-system and/or metadata of the event.

1 - 20 .(canceled) 
     
     
         21 . A computer-implemented method, the method comprising:
 generating, by one or more computing devices, using a learning engine, a rule set for a first sub-system of a plurality of sub-systems, wherein the learning engine is trained using streams of data received from a plurality of data sources to generate rules for determining circumstances under which a given event is considered invalid at a respective sub-system;   receiving, by the one or more computing devices, a plurality of events, each event specified at least in part by a plurality of elements;   assigning, by the one or more computing devices, weights to each element of each event of the set of events such that a given weight of a given element is assigned based on the rule set;   for respective events in the plurality of events:
 determining, by the one or more computing devices, a score based on the weights assigned to each element of the respective event; 
   determining, by the one or more computing devices, based on the score, whether the respective event is considered invalid at the first sub-system;   transmitting, by the one or more computing devices, a first set of events determined to be valid, to a second sub-system; and   preventing, by the one or more computing devices, a second set of events determined to be invalid from being propagated to remaining sub-systems of the plurality of sub-systems.   
     
     
         22 . The method of  claim 21 , further comprising validating, by the one or more computing devices, each element according to the rule set. 
     
     
         23 . The method of  claim 21 , wherein the plurality of elements include: event type, origination location, origination date and time against the rules governing events of a specified type, originating at a specified location, and originating after a specified date and time. 
     
     
         24 . The method of  claim 21 , wherein the streams of data include one or more of: state information of each of the plurality of sub-systems, processing time information of each of the plurality of sub-systems, and load information of each of the plurality of sub-systems. 
     
     
         25 . The method of  claim 21 , wherein the second set of events determined to be invalid do not accord with a current state of the plurality of sub-systems. 
     
     
         26 . The method of  claim 21 , wherein the first sub-system is configured to execute a specified task in response to each occurrence of a respective event of the first set of events deemed to be valid. 
     
     
         27 . The method of  claim 21 , further comprising:
 transmitting, by the one or more computing devices, the first set of events and the second set of events to the learning engine; and   training, by the one or more computing devices, the learning engine to generate the rules for determining circumstances under which a given event is considered invalid at a respective sub-system using the first set of events and the second set of events.   
     
     
         28 . The method of  claim 27 , wherein each event of the first set of events includes a valid tag and each event of the second set of events includes an invalid tag and wherein the learning engine determines a validity of a respective event of the first set of events or the second set of events based on the valid tag or invalid tag of the respective event. 
     
     
         29 . A system comprising:
 a memory; and   a processor coupled to the memory, the processor configured to:   generate, using a learning engine, a rule set for a first sub-system of a plurality of sub-systems, wherein the learning engine is trained using streams of data received from a plurality of data sources to generate rules for determining circumstances under which a given event is considered invalid at a respective sub-system;
 receive each event specified at least in part by a plurality of elements; 
   assign weights to each element of each event of the set of events such that a given weight of a given element is assigned based on the rule set;   for respective events in the plurality of events:
 determine a score based on the weights assigned to each element of the respective event; 
   determine, based on the score, whether the respective event is considered invalid at the first sub-system;   transmit a first set of events determined to be valid, to a second sub-system; and   prevent a second set of events determined to be invalid from being propagated to remaining sub-systems of the plurality of sub-systems.   
     
     
         30 . The system of  claim 29 , wherein the processor is further configured to: validate each element according to the rule set. 
     
     
         31 . The system of  claim 29 , wherein the plurality of elements include: event type, origination location, origination date and time against the rules governing events of a specified type, originating at a specified location, and originating after a specified date and time. 
     
     
         32 . The system of  claim 29 , wherein the streams of data include one or more of: state information of each of the plurality of sub-systems, processing time information of each of the plurality of sub-systems, and load information of each of the plurality of sub-systems. 
     
     
         33 . The system of  claim 29 , wherein the second set of events determined to be invalid do not accord with a current state of the plurality of sub-systems. 
     
     
         34 . The system of  claim 29 , wherein the first sub-system is configured to execute a specified task in response to each occurrence of a respective event of the first set of events deemed to be valid. 
     
     
         35 . The system of  claim 29 , wherein the processor is further configured to:
 transmit the first set of events and the second set of events to the learning engine; and   train the learning engine to generate the rules for determining circumstances under which a given event is considered invalid at a respective sub-system using the first set of events and the second set of events.   
     
     
         36 . The system of  claim 35 , wherein each event of the first set of events includes a valid tag and each event of the second set of events includes an invalid tag and wherein the learning engine determines a validity of a respective event of the first set of events or the second set of events based on the valid tag or invalid tag of the respective event. 
     
     
         37 . A non-transitory computer-readable medium having instructions stored thereon, execution of which, by one or more processors of a device, cause the one or more processors to perform operations comprising:
 generating, using a learning engine, a rule set for a first sub-system of a plurality of sub-systems, wherein the learning engine is trained using streams of data received from a plurality of data sources to generate rules for determining circumstances under which a given event is considered invalid at a respective sub-system;   receiving a plurality of events, each event specified at least in part by a plurality of elements;   assigning weights to each element of each event of the set of events such that a given weight of a given element is assigned based on the rule set;   for respective events in the plurality of events:
 determining a score based on the weights assigned to each element of the respective event; 
   determining, based on the score, whether the respective event is considered invalid at the first sub-system;   transmitting a first set of events determined to be valid, to a second sub-system; and   preventing a second set of events determined to be invalid from being propagated to remaining sub-systems of the plurality of sub-systems.   
     
     
         38 . The non-transitory computer-readable medium of  claim 37 , the operations further comprising validating each element according to the rule set. 
     
     
         39 . The non-transitory computer-readable medium of  claim 37 , wherein the second set of events determined to be invalid do not accord with a current state of the plurality of sub-systems. 
     
     
         40 . The non-transitory computer-readable medium of  claim 37 , the operations further comprising:
 transmitting the first set of events and the second set of events to the learning engine; and   training the learning engine to generate the rules for determining circumstances under which a given event is considered invalid at a respective sub-system using the first set of events and the second set of events.