Dynamic reconstruction of cloud application traffic for security monitoring

Abstract

A system and a method are disclosed for dynamic reconstruction of cloud traffic for security monitoring. A catalog is maintained that includes application programming interface (API) data associated with one or more APIs of cloud applications. One or more collectors, that are each specific to a different API of the one or more APIs, collect network information describing communications associated with the one or more cloud applications used by devices of an organization, the network information including partial log data from some of the one or more collectors. A reconstruction of API network traffic is generated using at least the partial log data and the catalog. Security information is determined that identifies a security risk using the reconstruction of the API network traffic. The security information is provided to an administrator device of the organization.

What is claimed is: 
     
         1 . A method, performed in a security system comprising one or more processors and a non-transitory computer readable medium, the method comprising:
 maintaining a catalog that includes application programming interface (API) data associated with one or more application programming interfaces (APIs) of one or more cloud applications, the one or more APIs are used by one or more devices of an organization to access information from one or more cloud API servers;   collecting, using one or more collectors that are each specific to a different API of the one or more APIs, network information describing communications associated with the one or more cloud applications used by the one or more devices of the organization and the one or more cloud API servers, the network information including partial log data from some of the one or more collectors;   generating an enriched reconstruction of API network traffic using at least the partial log data and the catalog;   determining security information that identifies a security risk using the enriched reconstruction of the API network traffic; and   providing the security information to an administrator device of the organization.   
     
     
         2 . The method of  claim 1 , wherein maintaining the catalog that includes the API data associated with the one or more APIs of the one or more cloud applications, comprises:
 loading the API data for the one or more APIs into the catalog; and   extracting exposure information for each of the one or more APIs from the API data.   
     
     
         3 . The method of  claim 2 , wherein loading the API data for the one or more APIs into the catalog, comprises:
 retrieving API documentation for an API of the one or more APIs; and   converting the API documentation into a data structure used by the catalog to form a portion of the API data.   
     
     
         4 . The method of  claim 2 , wherein extracting the exposure information for each of the one or more APIs using in part the API data, comprises:
 analyzing the API data for exposures;   determining endpoints that correspond to the exposures; and   updating the API data with associations between the determined endpoints and the corresponding exposures.   
     
     
         5 . The method of  claim 1 , wherein maintaining the catalog that includes the API data associated with the one or more APIs of the one or more cloud applications, comprises:
 performing one or more API calls within a sample test instance of a cloud application of the one or more cloud applications in accordance with API documentation associated with the cloud application, to identify exposures associated with the one or more API calls;   determining endpoints that correspond to the exposures; and   updating the catalog with associations between the determined endpoints and the corresponding exposures.   
     
     
         6 . The method of  claim 1 , wherein generating the enriched reconstruction of API network traffic using at least the partial log data and the catalog, comprises:
 organizing requests and responses from the partial log data into corresponding tables;   identifying matches between the requests and the responses to create pairs that together form reconstructed API network traffic; and   associating the created pairs with respective endpoint entries in the catalog to form connection entries that make up the enriched reconstruction of API network traffic.   
     
     
         7 . The method of  claim 6 , further comprising:
 using a geo-location enricher to add geo-location information for IP address for at least some of the connection entries.   
     
     
         8 . The method of  claim 1 , wherein collecting, using the one or more collectors that are each specific to a different API of the one or more APIs, network information, further comprises:
 establishing a separate application instance for each collector, of the one or more collectors; and   performing, by each of the one or more collectors, an API call to collect a portion of the network information.   
     
     
         9 . A non-transitory computer readable storage medium comprising stored instructions, the instructions when executed by one or more processors of a security system cause the security system to:
 maintain a catalog that includes API data associated with one or more APIs of one or more cloud applications, the one or more APIs are used by one or more devices of an organization to access information from one or more cloud API servers;   collect, using one or more collectors that are each specific to a different API of the one or more APIs, network information descriptive of communications associated with the one or more cloud applications used by the one or more devices of the organization and the one or more cloud API servers, the network information including partial log data from some of the one or more collectors;   generate an enriched reconstruction of API network traffic using at least the partial log data and the catalog;   determine security information that identifies a security risk using the enriched reconstruction of API network traffic; and   provide the security information to an administrator device of the organization.   
     
     
         10 . The non-transitory computer readable storage medium of  claim 9 , wherein the instructions to maintain the catalog that includes the API data associated with the one or more APIs of the one or more cloud applications further comprises stored instruction that when executed cause the security system to:
 load the API data for the one or more APIs into the catalog; and   extract exposure information for each of the one or more APIs from the API data.   
     
     
         11 . The non-transitory computer readable storage medium of  claim 10 , wherein the instructions to load the API data for the one or more APIs into the catalog further comprises stored instructions that when executed cause the security system to:
 retrieve API documentation for an API of the one or more APIs; and   convert the API documentation into a data structure used by the catalog to form a portion of the API data.   
     
     
         12 . The non-transitory computer readable storage medium of  claim 10 , wherein the instructions to extract the exposure information for each of the one or more APIs using in part the API data further comprises stored instructions that when executed cause the security system to:
 analyze the API data for exposures;   determine endpoints that correspond to the exposures; and   update the API data with associations between the determined endpoints and the corresponding exposures.   
     
     
         13 . The non-transitory computer readable storage medium of  claim 9 , wherein the instructions to maintain the catalog that includes the API data associated with the one or more APIs of the one or more cloud applications further comprises stored instructions that when executed cause the security system to:
 perform one or more API calls within a sample test instance of a cloud application of the one or more cloud applications in accordance with API documentation associated with the cloud application, to identify exposures associated with the one or more API calls;   determine endpoints that correspond to the exposures; and   update the catalog with associations between the determined endpoints and the corresponding exposures.   
     
     
         14 . The non-transitory computer readable storage medium of  claim 9 , where the stored instructions to generate the enriched reconstruction of API network traffic using at least the partial log data and the catalog further comprises stored instructions that when executed cause the security system to:
 structure requests and responses from the partial log data into corresponding tables;   identify matches between the requests and the responses to create pairs that together form reconstructed API network traffic; and   generate connection entries that make up the enriched reconstruction of API network traffic by associating the created pairs with respective endpoint entries in the catalog.   
     
     
         15 . The non-transitory computer readable storage medium of  claim 14 , further comprising stored instructions that when executed cause the security system to:
 enable a geo-location enricher to add geo-location information for IP address for at least some of the connection entries.   
     
     
         16 . The non-transitory computer readable storage medium of  claim 9 , wherein the instructions to collect, using the one or more collectors that are each specific to a different API of the one or more APIs, network information further comprises stored instructions that when executed cause the security system to:
 establish a separate application instance for each collector, of the one or more collectors; and   perform, by each of the one or more collectors, an API call to collect a portion of the network information.   
     
     
         17 . A security system comprising:
 one or more processors; and   a non-transitory computer readable storage medium coupled to the one or more processors and comprising stored instructions that, when executed by the one or more processors, cause the security system to:   maintain a catalog that includes API data associated with one or more APIs of one or more cloud applications, the one or more APIs are used by one or more devices of an organization to access information from one or more cloud API servers,   collect, using one or more collectors that are each specific to a different API of the one or more APIs, network information describing communications associated with the one or more cloud applications used by the one or more devices of the organization and the one or more cloud API servers, the network information including partial log data from some of the one or more collectors,   generate an enriched reconstruction of API network traffic using at least the partial log data and the catalog,   determine security information that identifies a security risk using the enriched reconstruction of API network traffic, and provide the security information to an administrator device of the organization.   
     
     
         18 . The security system of  claim 17 , wherein the instructions to maintain the catalog that includes the API data associated with the one or more APIs of the one or more cloud applications further comprises stored instruction that when executed cause the security system to:
 perform one or more API calls within a sample test instance of a cloud application of the one or more cloud applications in accordance with API documentation associated with the cloud application, to identify exposures associated with the one or more API calls;   determine endpoints that correspond to the exposures; and   update the catalog with associations between the determined endpoints and the corresponding exposures.   
     
     
         19 . The security system of  claim 17 , wherein the instructions to generate the enriched reconstruction of API network traffic using at least the partial log data and the catalog further comprises stored instruction that when executed cause the security system to:
 structure requests and responses from the partial log data into corresponding tables;   identify matches between the requests and the responses to create pairs that together form reconstructed API network traffic; and   generate connection entries that make up the enriched reconstruction of API network traffic by associating the created pairs with respective endpoint entries in the catalog.   
     
     
         20 . The security system of  claim 17 , wherein the instructions to collect, using the one or more collectors that are each specific to a different API of the one or more APIs, network information further comprises stored instruction that when executed cause the security system to:
 establish a separate application instance for each collector, of the one or more collectors; and   perform, by each of the one or more collectors, an API call to collect a portion of the network information.